The 5-4 ruling, affirming a Kansas inmate rehabilitation program, could spawn similar efforts in other states. Inmates, who want to keep their privileges, may be forced into therapy and be required to admit past wrongdoing that could be used in future prosecutions, the court held.
"Sex offenders are a serious threat in this nation," Justice Anthony M. Kennedy wrote for the majority.
He said states have an important interest in rehabilitating those inmates.
Also Monday, the court ruled that the Americans with Disabilities Act does not entitle people to jobs that might jeopardize their health and agreed to referee a congressional boundary dispute from Mississippi, a victory for Democrats.
The court rejected arguments from a man who argued he should be able to decide for himself whether to take the risk of working in an oil refinery, where chemicals might aggravate his liver ailment.
In the Kansas case, convicted rapist Robert Lile argued that the state violated his Fifth Amendment right against self-incrimination in 1994 when it decided to punish him for refusing to sign paperwork admitting guilt in the crime he was convicted of and refusing to fill out a form that listed his sexual history, including names of any other victims.
Among the privileges that now may be taken from Lile are rights to visitors, personal television, and work and recreation.
Kennedy said that "offering inmates minimal incentives to participate (in rehabilitation) does not amount to compelled self incrimination."
In a dissent, Justice John Paul Stevens said the court missed the point, and that Kansas was actually punishing inmates for not admitting crimes.
"This is truly a watershed case," he wrote.
The federal government also has a sex offender program, but unlike the one in Kansas, participation is optional. Kennedy said a ruling against Kansas in this case could have jeopardized the federal program.
The 10th U.S. Circuit Court of Appeals had sided with Lile, and said the state should consider giving rehabilitation participants immunity.
Lile is in prison for raping a young woman, but he maintains the sex was consensual.
Kansas contends that its program protects the public because sex offenders have a high likelihood of repeating their crimes. The Bush administration and other states supported Kansas.
Kansas lawyers argued that no one has been prosecuted with information divulged to counselors. But Kansas, unlike some other states with similar programs, does not give prisoners immunity for information they share.
Stevens, joined by the court's more liberal members, said that the majority's "policy judgment does not justify the evisceration of a constitutional right.
"We should consider that even members of the Star Chamber thought they were pursuing righteous ends," wrote Stevens, joined by Justices David H. Souter, Ruth Bader Ginsburg and Stephen Breyer.
The Star Chamber, a former English court that operated without a jury, became so disliked that the title has come to be associated with unfair judicial proceedings.
Justice Sandra Day O'Connor, who sided with Kansas, wrote separately to say that inmates may have a "more unpleasant" prison experience without having their constitutional rights violated.
"These changes in living conditions seem to me minor," she said.
Also voting with the majority were Chief Justice William H. Rehnquist and Justices Antonin Scalia and Clarence Thomas.
The decision came about five months after a defeat for Kansas and its aggressive sex predator laws. The court restricted state efforts to confine sex offenders after their prison terms expire. Justices said states must prove convicted sex offenders have serious problems controlling themselves before the civil commitments. That ruling was 7-2.
In other decisions Monday: