But former neighbors say the 73-year-old Austrian - accused of imprisoning his daughter for 24 years and fathering her seven children - ran his household like a dictator, and ordered people to leave the campground he owned if they didn't keep their areas clean.
Piece by piece, a picture is emerging of a shrewd liar and an obsessive tyrant.
"At home, he was clearly the lord of the manor. Even at his campground, he was very strict and his rules had to be followed," said Anton Graf, who rented him land along Austria's Mondsee lake.
"He was inflexible and had no sensitivity," Graf, 57, told The Associated Press. "You were sick, something happened, he didn't care ... There was a rule - and that was it."
Although authorities have clamped down hard on records, examples of Fritzl's double life are coming to light.
Fritzl was both a hard worker respected by his peers, and a fiercely private man whose life revolved around the home he ruled with an iron fist.
The mosaic of Fritzl now taking shape also points to an astonishingly agile criminal mind: He allegedly forged letters, concocted an elaborate but consistent cover story that his daughter Elisabeth had joined a cult, and even impersonated her in a phone call to his wife.
"Only a sharp and precise thinker could plan such a thing," said Reinhard Haller, an Austrian psychiatrist. "To organize so many births, supply so many alibis at the same time and create an atmosphere where no one dared ask questions, he had to be very lucid and intelligent."
Fritzl apparently complemented trickery with a heavy reliance on authoritarianism: To keep family and tenants from the windowless, soundproofed rooms he built for Elisabeth and three of the children, he menacingly banned them from the basement.
"He was obviously a tyrant," said Sigrun Rossmanith, who works with Austria's court system. "If they heard over and over that the cellar was taboo, then they didn't dare to check on anything. His word was like the word of God."
Fritzl was born in Amstetten on April 9, 1935, but little is known of his early life. Even his parents' names have been withheld by authorities, who say privacy laws prevent them from granting access to basic documents such as birth, marriage and death certificates.
A class photo from a school trip in 1951 - obtained by AP - shows a 16-year-old Fritzl as tall and handsome, with dark hair and a serious demeanor. A former classmate who gave his name only as Erich S. recalled Fritzl as "slightly different" teenager and remembers his unfashionable haircut.
Johann Kreitler, director of the high school Fritzl attended from 1947-51, said he graduated "with a positive record," and later attended a vocational school to become an electrician.
Fritz's later employers and colleagues say he gained their respect.
"If you put 50 men in a line, he would be one of the last who could ever be suspected of committing such a crime," said Herbert Katzengrueber, Amstetten's mayor.
Yet outside the workplace, there were warning signs.
Reports suggest Fritzl was arrested in the 1960s in Linz and may have served prison time. Police have declined to comment, saying records that old would have been erased under Austria's statutes of limitation. But the daughter of a former employer backed up the reports.
"He was hired even though he had a record," said Sigrid Reisinger, who heads the Amstetten construction material firm Zehetner and whose father employed Fritzl there from 1969-71. She said the alleged crime was of a sexual nature but did not recall details.
Calls to Rudolf Mayer, Fritzl's lawyer, went unanswered Friday.
After leaving Zehetner, Fritzl sold machines for a Germany company in Austria and was often on the road. He purchased an inn and campground that his wife Rosemarie - who police contend was unaware of the cellar dungeon - ran during summers from 1973 to 1996.
"One day he came to my door and told me that Elisabeth was not coming home any more, that she had left to join a cult," Graf said.
He said Fritzl delivered the story with such aplomb that no one was suspicious. Graf said Fritzl also told him how he "discovered" one of Elisabeth's children on his doorstep - and Graf said he never doubted the tale.
"He was so convincing of the sorrow he felt and the suffering of his family," Graf said. "Nobody had any clue."
Local authorities say Fritzl twice was suspected of arson. But Gerhard Neuhuber, an Unterach police official, said Fritzl was cleared in both cases - in 1974 and 1982 - because of lack of evidence.
During the second investigation, Fritzl was forced to spend a short time in prison, Neuhuber told the AP. Since the story of Fritzl's kidnapping and incest became known, police in Upper Austria have also been examining whether he might be linked to an unsolved murder nearby.
Yet Fritzl remained little-known in Unterach, where those who dealt with him say they saw little in his character that seemed exceptional or suspicious.
Graf said he sometimes met Fritzl for business dealings, and the pair would share a beer. "He told jokes, not always the cleanest," Graf said. "He laughed loud, a real boom."
Germany's Bild newspaper interviewed a man it identified only as Paul H., who claims he twice vacationed in Thailand with Fritzl, and obtained video showing Fritzl on the beach receiving a massage, eating supper and laughing.
"We sat out on the terrace and had a nice evening," it quoted the friend as saying.
When the friend visited Fritzl in Amstetten, and realized no one was allowed to go into the cellar, "we never thought anything of it," he told Bild.
Fritzl, who acquired a number of properties in the region, managed to persuade social workers he was fit to care for the three small children he allegedly moved upstairs from the cellar.
"There was no reason to suspect that something was wrong," said Josef Schloegl, head of the Amstetten district court. He said the house was visited about 20 times, and inspectors found nothing amiss.
Yet much about Fritzl remains a mystery - even to police, who say he clammed up this week.
He wasn't active in community or church groups. Even his fishing club says he was something of a question mark.
Fritzl paid his dues, and "there was never a problem with him," said club treasurer Reinhard Kern.
"Whether he actually went fishing or not, how am I to know? Maybe it was an alibi."