Man versus Machine: Guess Who Won?

CAROUSEL -- Logan County officers survey and secure the home where five members of a family were found slain in their home as a search is under way for a suspect, in Beason, Ill., Tuesday, Sept. 22, 2009. (AP Photo/Seth Perlman)
AP Photo/Seth Perlman
It's not every day that people can get published in one of the world's leading scientific journals by playing computer games, but Foldit is no ordinary game. The brainchild of Seth Cooper from the University of Washington, Foldit taps into the collective efforts of tens of thousands computer gamers to solve scientific problems.

The goal of the game is to work out the complicated three-dimensional structures of different proteins. Proteins are feats of biological origami; they consist of long chains of amino acids that fold into very specific and complicated shapes. These shapes can reveal how proteins work but solving them is fiendishly challenging. To do it, scientists typically need to grow crystals of purified protein before bouncing X-rays off them. Foldit takes a different approach, using the collective efforts of causal gamers to do the hard work. And its best players can outperform software designed to do the same job. Best of all, you don't need a PhD to play Foldit, or any scientific training at all. The controls are intuitive, tutorial levels introduce the game's mechanics, colourful visuals provide hints, and the interface is explained in simple language. While protein scientists concern themselves with "rotating alpha-helices" and "fixing degrees of freedom", Foldit players simply 'tweak', 'freeze', 'wiggle' and 'shake' their on-screen shapes. It's telling that barely an eighth of the players work in science, and two-thirds of the top scorers have no biochemistry experience beyond high school.

Foldit's success relies on the fact that it doesn't shallowly flirt with interactivity - it's a true game. Cooper designed it to "attract the widest possible audience… and encourage prolonged engagement". For a start, that meant making it competitive. Players are scored based on the stability of the structures they end up with and a leader board shows how they rank against other gamers.

There's also a social side - gamers can chat on online forums, work in groups to solve puzzles and share solutions on a wiki. Of course, there's the thrill of contributing to genuine scientific research, but that motivates less than half of the community. The rest do it for the achievement, the social aspects and largely, because the game was fun and immersive.

And just like real game development, everything was tuned according to feedback from the players. Tools were added and refined, the difficulty of the tutorials was tweaked to stop frustrated beginners from leaving, and puzzles were matched to the skills of the players.

Foldit's origins lie within Rosetta, a piece of software designed to solve protein structures by simulating and testing thousands of different folds. Rosetta is an example of 'distributed computing', where volunteers run the program on their home computers when they don't need it. They effectively donate their computing power to speed up the laborious task of solving protein structures. But the volunteers wanted to use their biological computers - their brains - as well as their man-made ones. They suggested an interactive version of the programme and in May 2008, they got their wish with Foldit.

To test the usefulness of the approach, Cooper challenged his gamers with ten puzzles - proteins whose structures were solved but weren't on any public database. Even so, the Foldit players did themselves proud. They outperformed the original Rosetta software on five of the puzzles and matched it on three of them. And in the two cases where Rosetta actually beat the players, it still didn't get anywhere near the right answer.

The human gamers had many advantages that gave them the edge. The players used a far wider range of strategies than Rosetta does, and some specialize in the early, middle or late stages of the game. And they were also good at working out where to begin. In one game, when they were given ten possible starting structures, they picked the one that was closest to the actual answer. Armed with the same starting set, Rosetta couldn't do as well.

The Human Advantage

The humans also had the massive advantage of long-term planning. Rosetta uses a "rebuild and refine" method to gradually arrive at the most stable structure through tiny tweaks. But it won't investigate a potentially interesting move if it means making the protein temporarily less stable. The gamers had no such problems. They were quite happy to substantially unravel the protein because they knew that they could put it back together in an even better state.

"Computers are obviously superior in the ability to crunch numbers, but lack the intuitive sense of knowing when to crunch numbers, or which tool to best use at a certain point in the game," says CharlieFortsConscience, one of Foldit's gamers. "The experience and intuition part of the gameplay, for me, is being able to look fresh at a puzzle structure and note which areas 'don't look right'."

But even experience and intuition has its limits. Some puzzles proved too much for them. While some could solve the hardest of challenges - folding a full protein from its extended chain of amino acids - most struggled. Cooper thinks that he'll get the best outcomes by blending the best of both human creativity and computer power. If Rosetta can take them half-way, gamers can finish the job with aplomb.

Cooper was so impressed with the players' strategies that he thinks he can use them to design better algorithms for later versions of Rosetta. As he says, "Our results indicate that scientific advancement is possible if even a small fraction of the energy that goes into playing computer games can be channelled into scientific discovery." It's a perfect example of what Clay Shirky describes as the "cognitive surplus" - the free time and mental energy that's spent on leisure pursuits. Thanks to the Internet, this valuable resource could be channeled into something more productive.

In the meantime, the gamers themselves can smile at the fact that they've been published in the prestigious journal Nature. They're final authors on Cooper's new Nature paper, listed anonymously (as seems fitting) as the "Foldit players".

By Ed Yong
Reprinted with permission from Discover