The employees were fired in April after speaking to a lawyer the town hired as a fact-finder to rout out chatterboxes.
They say questions about a close relationship between Town Administrator David Jodoin and a female employee, identified only as "A" in the lawyer's report, drifted into Town Hall sometime in March. They say they weren't the only ones who discussed the rumor, and dismissed it as untrue after briefly talking about it.
"We didn't start the rumor, nor did we say there was an affair going on," Joanne Drewniak said Tuesday. "We didn't have time like they think ... to sit around and just gossip. That is so untrue."
Drewniak worked for tax assessor Sandra Piper; both were fired. Also fired were code enforcement officer Michelle Bonsteel and her assistant, Jessica Skorupski.
"They're going after our character, and I have to say, I think we were darn good employees," Piper said.
The women now known as "The Hooksett Four" comprise about a quarter of the town's staff, including two entire departments. They are well-known in Hooksett, home to about 13,000 people.
"We're the first door they hit when they come in Town Hall," said Piper, who said she loved her job handling property tax bills.
"They leave our offices smiling," said Bonsteel, who described code enforcement as her passion.
The women say they were told that nothing they said to the lawyer would be held against them. Then they were fired. The ensuing publicity has given the rumor new life, the women say.
"It's still out there and it's rattling like a rattlesnake," Piper said.
Town officials and their lawyers aren't talking. Two town lawyers did not respond to phone messages left Tuesday. All nine members of the Town Council either declined to comment to The Associated Press or did not respond to voice messages. Jodoin declined an interview request by e-mail.
The fact-finding report said he was upset by the rumor because he is happily married with two children.
"He does not want his family life to be threatened and does not want his career to be tainted by a false accusation," fact-finder Lauren Irwin wrote.
The women say what upset them about the rumor was the perception of impropriety by the public and preferential treatment "A" was receiving, including higher pay than Drewniak and Skorupski despite having less seniority and experience.
"What an insult," Piper said. "Doesn't length of time and length of service count for anything in this community?"
Each woman had received positive work reviews and is appealing to get back her job. Skorupski's and Drewniak's firings were reviewed at a hearing last week; Piper's and Bonsteel's hearing is pending.
The woman are not optimistic, partly because their appeals are before the council that fired them. The council is to rule on Skorupski's and Drewniak's appeals on Friday.
"The Town Council has said that they will hear an appeal of their own ruling, so you know you don't feel overly warm and fuzzy about that," said their lawyer, B.J. Branch.
The women hope public pressure will help. Several hundred people have signed a petition seeking their reinstatement, and Piper said the women are planning to produce red T-shirts proclaiming "Rehire the Hooksett 4" with a bullseye printed over the heart.
At the Brick House drive-in on Hooksett Road, diner Claudette Burbank said she wishes the women well.
"I really don't think it's fair," Burbank, 61, said between bites of french fries. "We all know everybody gossips about their bosses."