"I don't make a habit out of supporting people who attack my wife and attack my family."
Trump's use of photos to poke fun at Heidi Cruz's appearance, and other derogatory comments about women, are also turning off women voters according to some Cruz supporters.
"His mannerism and his speech, he tears too many people down and against each other," one woman said.
A CBS News/New York Times poll in October found that 57 percent of registered women voters had an unfavorable view of Trump. Now, it's up to 63 percent.
In a head-to-head match-up with Hillary Clinton, Trump trails 50 to 40 percent. The reason is women, who give Clinton a whopping 20 point advantage over Trump -- 55 to 35 percent.
But it's a very different story among women who vote in Republican primaries. Trump is leading with 41 percent, followed by Cruz at 27 and John Kasich at 23 percent.
Many women at Trump's rallies have told us they like him precisely because he's not afraid to create controversy.
"I just think he's not politically correct and I think it's wonderful," one woman said.
"He is just straightforward and says it like it is and doesn't sugar coat things," said another.