When both he and their infant daughter died, Sarah, grief-stricken, visited a psychic.
The psychic told her she was cursed by the spirits of all the people killed by the Winchester Rifle, and in order to appease them, she had to move west and build a house for them.
So began the construction of the Winchester Mystery House in 1884 in San Jose, Calif.
Sarah was told that she had to build - and never stop building - a house for the spirits. She bought an eight-room farmhouse, and started to add room after room, until at one point the house was seven stories tall.
The top three fell off after the great San Francisco Earthquake. Sarah thought it was a sign from the beyond and did not rebuild those floors.
Not only was the Winchester Mystery House - all 160 rooms of it - supposedly built for the spirits, but they say it was built with the guidance of ghosts, too.
It's huge, with 2,000 doors to 160 rooms, and a whole mansion full of quirks.
"We call this the goofy staircase," says mansion employee Cheryl Hamilton. "There's 42 steps on here. And it makes seven complete turns. And it's about 100 feet of walking, but it's only going to go up nine feet to the second floor."
For 38 years, room after room was added, destroyed, and constructed again. There's a staircase that leads to the ceiling. And on the second floor, there's a door to nowhere.
General manager Shozo Kagoshima is skeptical that there are any ghosts.
However, employee Jack Stubbert says he saw a ghost 15 years ago in a hallway.
"And he turned around. And he looked at me. And he - you know, it was the type of image that was like, looking through a fog," Stubbert says.
It's said that Mrs. Winchester would come to the séance room every night and talk with the spirits of the people who had been killed by the Winchester rifle.
According to legend, the spirits gave Sarah Winchester building instructions. That might explain the 47 fireplaces, or the extensive use of art glass, or the reoccurring number 13.
"She used the number 13 wherever she could. Thirteen windows, 13 panels, 13 skylights, 13 bathrooms, 13 steps. Wherever she could incorporate the number 13, she used it," says Kagoshima.
Did Sarah Winchester know something we don't? Well, Jack Stubbert now thinks she did. He says he never believed in ghosts before he saw one.