Selective Publication of Antidepressant Trials and Its Influence on Efficacy

https://www.nejm.org/doi/full/10.1056/NEJMsa065779

 

Selective Publication of Antidepressant Trials and Its Influence on Apparent Efficacy

 

Abstract

BACKGROUND

Evidence-based medicine is valuable to the extent that the evidence base is complete and unbiased. Selective publication of clinical trials — and the outcomes within those trials — can lead to unrealistic estimates of drug effectiveness and alter the apparent risk–benefit ratio.

 

METHODS

We obtained reviews from the Food and Drug Administration (FDA) for studies of 12 antidepressant agents involving 12,564 patients. We conducted a systematic literature search to identify matching publications. For trials that were reported in the literature, we compared the published outcomes with the FDA outcomes. We also compared the effect size derived from the published reports with the effect size derived from the entire FDA data set.

 

RESULTS

Among 74 FDA-registered studies, 31%, accounting for 3449 study participants, were not published. Whether and how the studies were published were associated with the study outcome. A total of 37 studies viewed by the FDA as having positive results were published; 1 study viewed as positive was not published. Studies viewed by the FDA as having negative or questionable results were, with 3 exceptions, either not published (22 studies) or published in a way that, in our opinion, conveyed a positive outcome (11 studies). According to the published literature, it appeared that 94% of the trials conducted were positive. By contrast, the FDA analysis showed that 51% were positive. Separate meta-analyses of the FDA and journal data sets showed that the increase in effect size ranged from 11 to 69% for individual drugs and was 32% overall.

 

CONCLUSIONS

We cannot determine whether the bias observed resulted from a failure to submit manuscripts on the part of authors and sponsors, from decisions by journal editors and reviewers not to publish, or both. Selective reporting of clinical trial results may have adverse consequences for researchers, study participants, health care professionals, and patients.

 

________________________________

 

I've also included a link to a podcast with the principal author on this paper, Dr. Erick Turner, where he discusses his time at the FDA, selective publication in medicine, and his work, with Dr. Vinay Prasad. It's really worth listening to. The FDA failed spectacularly to protect consumers (ie do its job) by failing to disclose negative antidepressant trials, preventing patients and doctors from being able to properly weigh risks vs benefits. They continue to fail and will do so perpetually unless they extricate themselves from regulatory capture at the hands of the pharmaceutical industry. If you are thanking your lucky stars you don't live in America, you might want to check in on your respective regulatory agencies, many other countries are not much better.

 

 

I've also included a link to a podcast with the principal author on this paper, Dr. Erick Turner, where he discusses his time at the FDA, selective publication in medicine, and his work, with Dr. Vinay Prasad. It's really worth listening to. The FDA failed spectacularly to protect consumers (ie do its job) by failing to disclose negative antidepressant trials, preventing patients and doctors from being able to properly weigh risks vs benefits. They continue to fail and will do so perpetually unless they extricate themselves from regulatory capture at the hands of the pharmaceutical industry. If you are thanking your lucky stars you don't live in America, you might want to check in on your respective regulatory agencies, many other countries are not much better.

 

 

Very good podcast, thanks for sharing, DG.   It's certainly encouraging that there are many more doctors starting to speak up but I keep thinking of all the junk science that's already been published and cited.  How does one ever clean up that pre-existing chain, it's an enormous problem.

You're welcome, abcd. I have no idea how they are going to clean up all the trash out there, but I think a good start would be to ignore any study that is sponsored by a pharmaceutical company. It's kind of doubtful they contain any useful information, other than maybe adverse event rates, which you can double when trying to get an accurate estimate of them. Then maybe they can start eliminating studies by authors with a financial conflict of interest, whether funding was secured prior to the study conducted or afterwards (a sort of researcher capture, like the regulatory capture of the FDA). Until then we will just have to be skeptical of the literature and trust people like Healy, Prasad and Turner to sort through the dump that is academic medicine.