The study found MRI machines, which are used to take images of the brain, are more than 90 percent accurate at detecting deception, said Dr. Mark George, a distinguished professor of psychiatry, radiology and neurosciences.
That compares with polygraphs that range from 80 percent to "no better than chance" at finding the truth, George said.
His results are to be published this week in the journal Biological Psychiatry.
Software expected to be on the market next year could make it easier to tell if someone is a liar, which has implications for law enforcement.
Researchers at MUSC conducted the study using 60 healthy men. They offered some extra money if they could manage to trick the machine but none could.
"We had some of our study group try to dupe us, and they were unable," George said.
The MRI images show that more blood flows to parts of the brain associated with anxiety and impulse control when people lie. More blood also flows to the part of the brain handling multitasking because it is hard for people to keep track of lies they have told.
In the study, researchers had participants commit a mock theft. Then questions about the theft were projected onto a screen while they were inside the MRI machine. Participants pressed a button to respond to the yes or no questions.
The test won't work if people don't remain still in the MRI machine so a clear imagine of the brain can be recorded. And some people's brains don't seem to show the same changes while lying.
It's also not clear whether certain psychiatric conditions might change the test results.