He said it is important to "create a system that can withstand their failure, so you can let them fail" and then control the damage.
The treasury secretary told Chris Wallace of Fox News that it is important to preserve "the competitive market that allows for innovation" – but said that doing so "doesn't work so well, as we learned, if you don't put in strong standards to protect consumers."
He said the administration was seeking to establish "sensibly designed stronger standards, enforced across the country" as well as "constrain excess leverage in the financial system."
Though unemployment remains "appallingly high" and "we're nowhere close to declaring victory," Geithner said he sees "tentative signs of growth" in the economy.
"We're only just beginning to see the signs of recovery," he said.
Geithner said the most important decision the administration has made so far on monetary policy "has been to come together early" with the rest of the world and "agree on a common strategy and execute it."
He said the "classic mistake" of waiting too long to act and then being tentative had thus been avoided.
The treasury secretary said he is "pretty optimistic" that regulatory reform will pass this year. He said it is important to act quickly because "if you really wait to act and change the rules until the memory of the crisis has faded, it's just much easier to fight reform."
The economy, he said, "came very close" to going under at the height of the financial crisis. A broad range of people, himself included, "really thought we were at the edge of the abyss," he said.
"It should not have ever gotten to that point," Geithner saud. He said it did so because too much risk was allowed to build up in the financial system at a time when monetary policy around the world was "very, very loose."
"We came into this, again, without anything like the basic tools, the basic arsenal you need to contain the damage," he said.
"I think that we had a financial crisis this acute in part because frankly we let banks choose their regulator," added Geithner.
More from the "First Draft of History" Conference:
Lindsey Graham Hits Glenn Beck and Bill O'Reilly
Petraeus: Afghanistan is not Iraq or Vietnam
Doctor: U.S. Needs to Control "Tsunami of Obesity"
McCain Pushes for More Troops for Afghanistan