Wyeth, Elan and Bapineuzumab -- How to Lie With (Drug) Statistics
In what is becoming a sadly common ritual, Wyeth and Elan are pressing forward with an expensive, large-scale "phase III" trial of a risky drug based on wishful thinking and shoddy statistical analysis.
The drug in question is bapineuzumab, a monoclonal antibody intended to treat Alzheimer's disease by clearing the protein beta amyloid from the brain. Beta amyloid is often -- but not always -- found clumped together in "plaques" throughout the brains of Alzheimer's patients. Conventional wisdom in the field holds that these plaques cause the disease, although that theory has never been proven, and some scientific critics believe that the plaques are either irrelevant or even possibly a defensive reaction by brain tissue to the disease's neural assault.
Earlier today, Wyeth and Elan disclosed detailed results of the drug's phase II trial, which found that bapineuzumab failed to improve cognitive function in a test of 234 Alzheimer's patients after 18 months of treatment. You could be forgiven for not gleaning that from the companies' joint press release, however, as Wyeth and Elan chose instead to highlight post-hoc analyses that purported to demonstrate the drug's efficacy in a subset of patients who don't have a gene variant called ApoE4, which increases the risk of Alzheimer's.
To put it bluntly, this is magical thinking on a truly impressive scale. A few points:
- Prospective measures of success are the only accurate way to judge trial results. Honest clinical trials require researchers to specify in advance what they're looking for -- and by that measure, the bapineuzumab trial was a failure.
- Post-hoc subgroup analyses amount to lying with statistics. By contrast, a post-hoc analysis involves mining the trial data in order to identify some group of patients who appeared to benefit from the drug. It's tantamount to moving the finish line after the race is over -- or, as FDA's Richard Pazdur memorably put it, firing an arrow into the wall and then drawing a target around it.
- Such findings in subgroups rarely hold up under further study. Or, as the old computer-science saying goes, "Garbage in, garbage out."
- Drug companies will do and say almost anything to boost the promise of a potential blockbuster. Wyeth and Elan don't expect data from the phase III trial in patients without the ApoE4 gene variant until 2010. A lot can happen in that time, including the possibility that the FDA will once again warm to the idea of approving drugs based on marginal evidence. It's like the old story in which a prisoner staves off execution by promising to teach the king's horse to sing within a year, reasoning: "A year is a long time. The king might die. The horse might die. I might die. And maybe the horse will learn to sing."
Update: As it turns out, Wall Street was disappointed by bapineuzumab anyway -- but mainly because the drug didn't show a dose effect (more wasn't better, even in the "subgroup") and because of side effects such as fluid build-ups in the brain. That apparent toxicity may or may not be serious -- unsurprisingly, the companies believe it can be managed -- but it's almost beside the point. Whenever companies start talking up after-the-fact subgroup results, it's time to hold onto your wallet.
Update redux: In the Pipeline's Derek Lowe offers some characteristically trenchant thoughts on the bapineuzumab trial.
MRI scan of an Alzheimer's brain via Wikimedia Commons
