"This might be overdoing it, but I don't want this stuff on me,'' he says. "That's what I'm going to do to protect my family."
It's wood that was treated with a powerful pesticide called CCA that includes arsenic. Under pressure, suppliers have agreed to stop making it.
But, as CBS News Correspondent Mark Strassmann reports, already 50 million American families, like Horseman's, share a potential health worry. Their wooden deck, fence and picnic table were all built with this wood.
His wife's suffering started just two weeks after her husband built their new sunroom with CCA treated wood.
"The worst was the tingling in my feet," says Cheryl Horseman. "It's totally miserable. Your hands are burning. Your feet are burning. Your toes are red."
Most adults don't notice immediate symptoms, but they're not the most at-risk group. Small children are. They touch the wood as they play on it or eat on it and then put their hands in their mouth along with any arsenic that seeped through and the potential health impact is startling.
"Individual risks for a child can easily approach a one in 100 chance of developing cancer in a lifetime," says Jane Houlihan, vice president of Environmental Working Group.
At the very least, experts suggest sealing your CCA wood every six months.
The Consumer Product Safety Commission is currently testing which sealant protects best.
"I think any sealant seems to be better than nothing," says David Cobb, a CPSC chemist. "There are some that perform much better than others."
Until the agency makes that recommendation, Eric Criss of the CPSC suggests making "sure your kids wash their hands after playing on this wood, and don't let them eat on this wood."
Horseman feels sick from it all in a different way.
"What have I done? I thought I made something beautiful but ended up making something that would hurt my family."
Prying away his guilt will take more than a crowbar.