The court on Monday overturned a lower court decision that let Javaid Iqbal's lawsuit against the high-ranking officials proceed.
Iqbal, a Pakistani Muslim, spent nearly six months in solitary confinement in New York in 2002.
He had argued that while Ashcroft and Mueller did not single him out for mistreatment, they were responsible for a policy of confining detainees in highly-restrictive conditions because of their religious beliefs or race in the aftermath of the September 11, 2001 attacks.
But the government argued that there was nothing linking Mueller and Ashcroft to the abuses that happened to Iqbal.
"I'm actually surprised that the vote on the Court was as close as it was, 5-4, because the Justices have traditionally been very reluctant to allow lawsuits like this to proceed against government officials," said CBS news legal analyst Andrew Cohen.
"Here's a perfect example of a result that probably wouldn't be much different once Justice David Souter is replaced on the Court," Cohen. added. "He was in the dissent here; he would have allowed the lawsuit against former Bush officials to proceed at least a little further toward trial - and so, we presume, would his successor."
Despite the court's rejection, the case isn't over: Cohen said Iqbal can amend his complaint, but the legal hurdles he faces are still huge.
"The immunity that officials receive under law when they are acting in their formal capacity makes these lawsuits almost impossible to win," he said.
The case is Ashcroft v. Iqbal.
For more info: