Somehow, it evolved into much more than that, and today the house on the rock has to be seen to be believed, The Early Show correspondent Melinda Murphy reports for "The Best of America" series.
In the mid 1940s, Alex Jordan started building a hilltop retreat for himself, on a rock shelf in south central Wisconsin. It didn't take long before people realized this was not going to be an ordinary house.
Friend Betty Long says, "The house was like no house that was ever built. People kept coming to see what he was doing because at the time you could see it from the highway. And they would drop in on him and ... he let them."
Long has worked at The House on the Rock for more than 20 years. And she says, "They decided to start charging a little money, so they charged 50 cents. That first year, they took in over $5,000."
Jordan was more than a builder, he was also a collector. He packed his house to the gills with all sorts of things, weird and wonderful collections.
Long notes, "Alex collected everything and anything. It's a collection upon collections here."
They said he had to be eccentric; nobody in his right mind would do something like this.
Jordan didn't stop with The House on the Rock. Over the years, he added building after building filled with an ever-growing collection of oddities - everything from a miniature circus with more than a million individual figures, to the world's largest carousel, complete with 20,000 lights and 289 creatures.
Hands down, the most architecturally interesting feature is the infinity room, which juts out 218 feet over the Wisconsin valley. It has 3,064 windows, Long notes.
Many of the most bizarre displays, such as a 200-foot-long sea creature doing battle with a giant octopus, were literally dreamed up, Long says. "He always said that he could go to bed at night and wake up with a whole layout of what he was going to do with the building."
Some rooms are created from discarded odds and ends. Take the organ room. Sure, there are lots of organs, but the cool part is the pipes which used to be part of a waterworks plant in Madison, Wis.
Long says Jordan just bought stuff and kept it and then decided what to do with it later. "He might not have anything in mind for it. But if it was for sale and he heard about it, he was going to buy it," she says.
If you would like to visit and take the 2.5 mile tour, visit The House On The Rock Web site.