Huckabee, who is considering a run for president in 2008, made the comment Wednesday on Don Imus' radio show after the host told him he looked "emaciated."
"I have just come out of six weeks at a concentration camp held by the Democrat Party of Arkansas in an undisclosed location, making a hostage tape," Huckabee said. "That's why I look that way."
A spokesman for the National Jewish Democratic Council said Huckabee should apologize.
"It's insensitive, and we think making light of the Holocaust isn't something any elected official should be doing," said David Goldenberg, deputy executive director of the Washington-based group.
Huckabee denied the comment was a reference to the Holocaust or a slight against Jews.
In a statement issued through his spokesman, he said he has visited Israel nine times and taken groups to Israel's Holocaust memorial, and "I have a great deal of respect for the Jewish people."
"I never identified a particular kind of camp, and to make such a far-reaching statement is laughable and is the type of allegation that makes people cynical," he added.
Huckabee, who now weighs around 185 pounds, has written a book about his weight loss and has emphasized health, diet and exercise as governor.
Callers to Imus' radio show, also telecast on MSNBC, were challenged to guess Huckabee's heaviest weight (290 pounds). The winner won $1,000.
Huckabee, who is prevented by term limits from running again and leaves office in January, has been visiting early presidential battleground states such as Iowa and New Hampshire.