He's been labeled a "counterrevolutionary."
"A counterrevolutionary!" Scalia reacts. "Sounds exciting."
The critics say his aim is to undo Roe v. Wade and affirmative action, and to allow more religion in public life.
"The public sense of you is that [you] make your decisions based on your social beliefs," Stahl says, with Scalia shaking his head. "That is the perception."
"I'm a law-and-order guy. I mean, I confess I'm a social conservative, but it does not affect my views on cases," Scalia says."
His philosophy has occasionally led him to decisions he deplores, like his upholding the constitutionality of flag burning, as he told a group of students in Missouri.
"If it was up to me, I would have thrown this bearded, sandal-wearing flag burner into jail, but it was not up to me," Scalia told the students.
To Scalia, flag burning was protected by the founding fathers in the First Amendment, which is his only criterion, he says, under originalism.
"But do you respect that there is another way to look at this?" Stahl asks.
"You know the story of the Baptist preacher who was asked if he believed in total-immersion baptism? And he said, 'Believe in it? Why I've seen it done!' I have to say the same thing about your question. There must be other views because I've seen them," Scalia says.
"Yeah, but do you respect them? You don't, do you?" Stahl asks.
"I respect the people who have them, but I think those views are just flat out wrong," Scalia says.
He's talking about some of his fellow justices, like Ruth Bader Ginsburg, a liberal who is - and this never ceases to surprise people - one of Scalia's best friends, both on and off the court.
To Ginsburg, the Constitution evolves and should reflect changes in society; that going back to what was meant originally when they wrote, for instance, "We the People," makes little sense.
"Who were 'We the People' in 1787? You would not be among 'We the People.' African Americans would not be among the people," Ginsburg tells Stahl.
"Justice Ginsburg and you disagree…on lots of things. And yet you're such good friends," Stahl remarks.
"I attack ideas. I don't attack people. And some very good people have some very bad ideas," Scalia says. "And if you can't separate the two, you gotta get another day job. You don't want to be a judge. At least not a judge on a multi-member panel."