The move provoked the most widespread protests in the Fed's history: Angry farmers blockaded the Fed with their tractors. "I think it's fair to say it was a difficult time," he said.
Three decades later, he's under attack again - this time from the banks.
"Some CEOs have been particularly critical of you," Mason said. "Jamie Dimon, the head of JP Morgan Chasem, said, 'Volcker by his own admission has said he doesn't understand capital markets. He has proven that to me.'"
"Well, unfortunately, I think that they proved some of that to me, too," Volcker responded. "Their own misunderstanding. How did they get in so much trouble?"
But if Volcker has put himself in the line of fire, the president has been an even bigger target.
"President Obama has taken a lot of heat from Wall Street - he's often accused of being hostile to Wall Street, waging class warfare. Do you think that's true?" asked Mason.
"No, I don't understand the depth of that feeling. I really don't," Volcker said. "This business that he's a great socialist and out to undermine the free enterprise system and so forth, I just think it has no connection with reality.
"How could you have a President of the United States taking office in the midst of a financial crisis and a deep recession and not be critical of the financial system? He would have been deaf, dumb and blind."
Volcker believes the banking system is in better shape now. But while there's more work to be done, he may not be the one to do it.
"Did you think you'd be back in the mix like this at 84?" Mason asked.
"No, no, no. And somehow I'm not so happy about it. I think it's time to get out of here!"
The banks will be glad to see the back of him. On the day we visited, Paul Volcker was thinking about his 65th college reunion next year.
"Are you looking forward to that?" Mason asked.
Volcker said he doesn't care about the reunion: "I was just looking forward to living that long!"