Nearly five miles up the winding two-lane country road, past rolling hills and dairy farms, is the tiny Chester A. Arthur Historic Site, proclaiming the spot where the nation's 21st president was born in a cottage.
Or was he?
Nearly 123 years after his death, doubts about his U.S. citizenship linger, thanks to lack of documentation and a political foe's claim that Arthur was really born in Canada - and was therefore ineligible for the White House, where he served from 1881 to 1885.
Long before "birthers" began questioning the citizenship of President Barack Obama, similar questions were raised about the early years of Arthur, an accidental president who ascended to the job after President James Garfield was assassinated.
"It's an old rumor that won't die, political slander," said John Dumville, who runs Vermont's historic sites and knows well the legend. "It's a fun story, and it comes up every year. People latch on to it and they've read about it somewhere and they want to know more."
The U.S. Constitution says you must be a "natural-born citizen" to serve as president. The issue has received renewed interest due to legions of Obama doubters who say his Hawaiian birth certificate is fake and that he was born in Kenya.
But the Arthur birthplace question came up before the Internet was around to spread such theories.
Known as Vermont's "other president" - Calvin Coolidge was born in Vermont - Arthur was the son of a Baptist minister whose first assignment was this small town (pop. 1,916) in the heart of northern Vermont dairy country.
He was born Oct. 5, 1829, but later in life, he lied about the year. Even his gravestone lists 1830, though Arthur family bibles at the Library of Congress in Washington say 1829.
The family moved often, and by 1835 had left Vermont. Arthur went on to become a teacher, lawyer and political operative, serving as quartermaster general for the state of New York during the Civil War and later Collector of the Port of New York, appointed by President Ulysses Grant.
The focus on his place of birth became an issue in the 1880 presidential campaign, when Arthur was tapped to be the running mate for Garfield.
Stories on President Obama's Birthplace Controversy
According to historical accounts, Republican bosses wanted him to provide proof of his birthplace, but he never did.
Democrats, meanwhile, hired a lawyer named Arthur Hinman who sought to discredit Arthur, claiming he was born in Dunham, Quebec, about 47 miles north of Fairfield. Hinman traveled to Vermont and Canada to research Arthur's past, eventually concluding that Arthur was born in Canada but appropriated the birth records of a baby brother who was born in Fairfield, but died as an infant.
He later incorporated the findings into a book titled "How A British Subject Became President of the United States."
Arthur, who served from 1881 to 1885, never publicly addressed the allegation.
Vermont officials hold fast to their claim on Arthur's birth, but have little to back it up.
The state of Vermont didn't begin receiving birth records until 1857, according to state archivist Gregory Sanford. The birth records at the Town of Fairfield go back no further, Town Clerk Amanda Forbes said.
Arthur biographer Thomas C. Reeves, a former University of Wisconsin history professor who wrote "Gentleman Boss" in 1975, debunked the born-in-Canada claim.
"This was a little campaign trick, in an era when politics were just as dirty as they are now," Reeves said in an interview. "It didn't threaten him in anyway. He was lying about his age, which complicated things. Like so many people, he just lopped a year off his life."
But the legend lingers, like a salacious rumor too juicy not to repeat.
In 1998, the Ottawa (Ont.) Citizen newspaper published a story asserting Arthur was born in his grandparents' home in Dunham but "probably" appropriated the birth records of the dead brother.
"The great impostor, the ultimate spoilsman, has never been defrocked. Not bad for a Canadian, eh?" said the newspaper, which called Arthur "our man in Washington."
At the Chester A. Arthur Historic Site, which draws about 400 visitors a year despite its remote setting, the topic is the first thing on people's lips, said caretaker Shirley Paradee.
"That's usually pretty much the most-asked question - if he was born here or in Canada," she said. "And I don't really have an answer for that because there isn't anything, any proof anywhere, where he was born."
If anything, the display boards inside a two-room replica of the parsonage where Arthur spent some of his early years fan the controversy. One contains a quote from a "J.H. Corey" who asserted in 1881: "I am positive C.A. Arthur was born in Canada."
Another reads, "Today, in an era when virtually every detail of a politician's life is open to public view, there is no concrete proof of the location of President Arthur's birthplace. Records and recollections lend strong support to the claim that Arthur was born in Fairfield, Vt."
For his part, Dumville says Vermont is proud to call Arthur a native son. Until someone proves otherwise.
"There's no way to prove he was not born in Vermont. It's a little boosterism for Vermont, having a U.S. president born here," Dumville said.