Bias, Spin, and Misreporting: Time for Full Access to Trial Protocols and Results


article has not abstract


Published in the journal: . PLoS Med 5(11): e230. doi:10.1371/journal.pmed.0050230
Category: Perspective
doi: 10.1371/journal.pmed.0050230

Summary

article has not abstract

Although randomized trials provide key guidance for how we practice medicine, trust in their published results has been eroded in recent years due to several high-profile cases of alleged data suppression, misrepresentation, and manipulation [1–5, 39]. While most publicized cases have involved pharmaceutical industry trials, accumulating empiric evidence has shown that selective reporting of results is a systemic problem afflicting all types of trials, including those with no commercial input [6]. These examples highlight the harmful potential impact of biased reporting on patient care, and the violation of ethical responsibilities of researchers and sponsors to disseminate results accurately and comprehensively.

Biased reporting arises when two main decisions are made based on the direction and statistical significance of the data—whether to publish the trial at all, and if so, which analyses and results to report in the publication. Strong evidence for the selective publication of positive trials has been available for decades [7,8]. More recent cohort studies have focused on the misreporting of trials within publications by comparing journal articles either with documents from regulatory agencies [9–12] or with trial protocols from research ethics committees [13–16], funding agencies [17], research groups [18,19], and journals [20]. These cohort studies identified major discrepancies—favorable results were often highlighted while unfavorable data were suppressed; definitions of primary outcomes were changed; and methods of statistical analysis were modified without explanation in the journal article.

Linked Research Article

This Perspective discusses the following new study published in PLoS Medicine:

Rising K, Bacchetti P, Bero L (2008) Reporting bias in drug trials submitted to the Food and Drug Administration: A review of publication and presentation. PLoS Med 5(11): e217. doi:10.1371/journal.pmed.0050217

Lisa Bero and colleagues review the publication status of all efficacy trials carried out in support of new drug approvals from 2001 and 2002, and find that a quarter of trials remain unpublished.

New Evidence

In a new study published in PLoS Medicine, Lisa Bero and colleagues make an important contribution to the growing body of evidence that the randomized trial literature is skewed towards reporting favorable results [9]. The researchers identified trials from 33 new drug applications (NDAs) for new molecular entities approved by the United States Food and Drug Administration (FDA) in 2001–2002, and compared information from FDA reviews with journal articles. By including all NDAs from a variety of specialty fields, their findings have broad generalizability to pharmaceutical trials.

Overall, a substantial amount of primary outcome data submitted to the FDA was found to be missing from the literature. One quarter of trials in their sample were unpublished—predominantly those with unfavorable results. Not only were data suppressed for the unpublished trials, but an additional quarter of primary outcomes were omitted from journal articles of published trials. These findings are consistent with two recent reviews of FDA documents and journal articles [10,21], one of which was published in PLoS Medicine in September 2008 [21].

Bero and colleagues also identified important discrepancies between the primary outcomes, statistical analyses, and conclusions presented in NDAs versus those reported in journal articles. The vast majority of discrepancies favored the sponsor's new drug, suggesting biased reporting. While it is possible that the FDA requested modifications to the sponsor's analyses, these amendments should be mentioned in the FDA's statistical review; should not involve altering primary outcomes without explanation in the publication; and would not be expected to favor the sponsor's drug as often as was found in this study.

Biased reporting of results from NDA trials is particularly concerning because these journal articles are the only peer-reviewed source of information on recently approved drugs for health care providers, who will have had limited clinical experience with these new treatments. There are also substantial cost implications if the efficacy is overestimated and the drugs overused, as new molecular entities are among the most expensive pharmaceuticals on the market [22].

The Need for Increased Transparency

Since the interests of patients are of utmost importance, it is difficult to justify why health care providers and policy makers should have access to only a biased subset of information that is substantially different from that which regulatory agencies have at their disposal. Bero and colleagues' study highlights the importance of public access to key documents that have traditionally been deemed confidential—regulatory agency submissions and trial protocols. Both types of documents have unique properties that complement each other.

Regulatory agency submissions represent the final description of how the trial was conducted and analyzed prior to journal publication. However, details from these submissions are not publicly available in most countries. Although summaries of FDA reviews are posted on the FDA Web site, their content and availability is variable, and sections are often redacted [9,21,23]. Furthermore, regulatory agency submissions are prepared by companies after data analysis and may themselves be subject to biased reporting. Finally, only devices, pharmaceuticals, and biological agents require regulatory approval in the United States and other countries, meaning that trials examining other types of interventions (e.g., surgery, education)—which constitute 20% of published randomized trials [24]—would be excluded from reviews of regulatory agency documents. Pharmaceutical trials conducted post-approval would also be missed.

On the other hand, protocols constitute the most comprehensive description of study design prior to trial inception. Their content therefore cannot be influenced by the study results. However, access to trial protocols is particularly difficult to obtain [25,26]. As with summaries of FDA reviews, their content is also highly variable and often lacks sufficient detail [13–18,20]. The SPIRIT initiative (Standard Protocol Items for Randomized Trials) aims to address these deficiencies by producing evidence-based recommendations for key information to include in a trial protocol [27].

Time for Action

It is clear that the trial literature is biased, facilitated in part by limited oversight and difficulty in accessing detailed trial documents. Ongoing progress in trial registration and results disclosure represents a key initial step towards ensuring public access to basic information on trial methods and results [28–33]. Several journals have also acted by publishing protocols and requiring their submission with manuscripts [34–36].

However, much remains to be done—not only to establish reliable, comprehensive registration and results disclosure processes worldwide, but also to start heeding the calls for increased access to full protocols and regulatory agency submissions [14,23,33,37,38]. As shown by recent examples and studies highlighted above, misreporting of trials can be difficult to detect without access to detailed documents beyond what is currently available on registries and results databases. Only with full transparency can the validity of a randomized trial be judged.

The time has come to tackle the challenge of making key trial documents public. It has taken decades for trial registration and results disclosure to be implemented; hopefully, for the sake of patients, public access to full protocols and regulatory agency submissions will come much sooner.


Zdroje

1. McHenryLBJureidiniJN

2008

Industry-sponsored ghostwriting in clinical trial reporting: A case study.

Account Res

15

152

167

2. JureidiniJNMcHenryLBMansfieldPR

2008

Clinical trials and drug promotion: Selective reporting of study 329.

Int J Risk Safety Med

20

73

81

3. PsatyBMKronmalRA

2008

Reporting mortality findings in trials of rofecoxib for Alzheimer disease or cognitive impairment: A case study based on documents from rofecoxib litigation.

JAMA

299

1813

1817

4. CurfmanGDMorrisseySDrazenJM

2006

Expression of concern reaffirmed.

N Engl J Med

354

1193

5. WhittingtonCJKendallTFonagyPCottrellDCotgroveA

2004

Selective serotonin reuptake inhibitors in childhood depression: Systematic review of published versus unpublished data.

Lancet

363

1341

1345

6. DwanKAltmanDGArnaizJABloomJChanA-W

2008

Systematic review of the empirical evidence of study publication bias and outcome reporting bias.

PLoS ONE

3

e3081

doi:10.1371/journal.pone.0003081

7. SongFEastwoodAJGilbodySDuleyLSuttonAJ

2000

Publication and related biases.

Health Technol Assess

4

1

115

8. DickersinK

1997

How important is publication bias? A synthesis of available data.

AIDS Educ Prev

9

15

21

9. RisingKBacchettiPBeroL

2008

Reporting bias in drug trials submitted to the Food and Drug Administration: A review of publication and presentation.

PLoS Med

5

e217

doi:10.1371/journal.pmed.0050217

10. TurnerEHMatthewsAMLinardatosETellRARosenthalR

2008

Selective publication of antidepressant trials and its influence on apparent efficacy.

N Engl J Med

358

252

260

11. MelanderHAhlqvist-RastadJMeijerGBeermannB

2003

Evidence b(i)ased medicine—Selective reporting from studies sponsored by pharmaceutical industry: Review of studies in new drug applications.

BMJ

326

1171

1173

12. HemminkiE

1980

Study of information submitted by drug companies to licensing authorities.

BMJ

280

833

836

13. ChanA-WHróbjartssonAJørgensenKJGøtzschePCAltmanDG

2008

Discrepancies in sample size calculations and data analyses reported in randomized trials: Comparison of publications with protocols.

BMJ

In press

14. ChanA-WHróbjartssonAHaahrMTGøtzschePCAltmanDG

2004

Empirical evidence for selective reporting of outcomes in randomized trials: Comparison of protocols to published articles.

JAMA

291

2457

2465

15. HahnSWilliamsonPRHuttonJL

2002

Investigation of within-study selective reporting in clinical research: Follow-up of applications submitted to a local research ethics committee.

J Eval Clin Pract

8

353

359

16. PildalJChanA-WHróbjartssonAForfangEAltmanDG

2005

Does unclear allocation concealment in trial publications reflect poor methods or poor reporting of adequate methods? Cohort study of trial protocols and corresponding published reports.

BMJ

330

1049

1052

17. ChanA-WKrleža-JericKSchmidIAltmanDG

2004

Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research.

CMAJ

171

735

740

18. ScharfOColevasAD

2006

Adverse event reporting in publications compared with sponsor database for cancer clinical trials.

J Clin Oncol

24

3933

3938

19. SoaresHPDanielsSKumarAClarkeMScottC

2004

Bad reporting does not mean bad methods for randomised trials: Observational study of randomised controlled trials performed by the Radiation Therapy Oncology Group.

BMJ

328

22

24

20. Al-MarzoukiSRobertsIEvansSMarshallT

2008

Selective reporting in clinical trials: Analysis of trial protocols accepted by The Lancet.

Lancet

372

201

21. LeeKBacchettiPSimI

2008

Publication of clinical trials supporting successful new drug applications: A literature analysis.

PLoS Med

5

e191

doi:10.1371/journal.pmed.0050191

22. MorganSGBassettKLWrightJMEvansRGBarerML

2005

“Breakthrough” drugs and growth in expenditure on prescription drugs in Canada.

BMJ

331

815

816

23. TurnerEH

2004

A taxpayer-funded clinical trials registry and results database.

PLoS Med

1

e60

doi:10.1371/journal.pmed.0010060

24. ChanA-WAltmanDG

2005

Epidemiology and reporting of randomised trials published in PubMed journals.

Lancet

365

1159

1162

25. ChanA-WUpshurRSinghJAGhersiDChapuisF

2006

Research protocols: Waiving confidentiality for the greater good.

BMJ

332

1086

1089

26. LuriePZieveA

2008

Sometimes the silence can be like the thunder: Access to pharmaceutical data at the FDA.

Law Contemporary Problems

69

85

97

27. ChanA-WTetzlaffJAltmanDGGøtzschePCHróbjartssonA

2008

The SPIRIT initiative: Defining Standard Protocol Items for Randomized Trials [conference abstract].

German J Evid Quality Health Care (suppl)

102

S27

28. World Health Organization

2008

International Clinical Trials Registry Platform.

Available: http://www.who.int/ictrp/en/. Accessed 20 October 2008

29. The PLoS Medicine Editors

2008

Next stop, don't block the doors: Opening up access to clinical trials results.

PLoS Med

5

e160

doi:10.1371/journal.pmed.0050160

30. United States Congress

2007

Food and Drug Administration Amendments Act of 2007, Title VIII, Section 801. Expanded clinical trial registry data bank.

Available: http://www.govtrack.us/congress/billtext.xpd?bill=h110-3580. Accessed 20 October 2008

31. SimIChanA-WGulmezogluAMEvansTPangT

2006

Clinical trial registration: Transparency is the watchword.

Lancet

367

1631

1633

32. DeAngelisCDDrazenJMFrizelleFAHaugCHoeyJ

2005

Is this clinical trial fully registered?: A statement from the International Committee of Medical Journal Editors.

JAMA

293

2927

2929

33. Krleža-JericKChanA-WDickersinKSimIGrimshawJ

2005

Principles for international registration of protocol information and results from human trials of health related interventions: Ottawa statement (part 1).

BMJ

330

956

958

34. PLoS Medicine

2008

Guidelines for authors.

Available: http://journals.plos.org/plosmedicine/guidelines.php#supporting. Accessed 20 October 2008

35. McNameeDJamesAKleinertS

2008

Protocol review at The Lancet.

Lancet

372

189

190

36. JonesGAbbasiK

2004

Trial protocols at the BMJ.

BMJ

329

1360

37. LassereMJohnsonK

2002

The power of the protocol.

Lancet

360

1620

1622

38. HawkeyCJ

2001

Journals should see original protocols for clinical trials.

BMJ

323

1309

39. MitkaM

2008

Controversies surround heart drug study: Questions about Vytorin and trial sponsors' conduct.

JAMA

299

885

887

Štítky
Interní lékařství

Článek vyšel v časopise

PLOS Medicine


2008 Číslo 11

Nejčtenější v tomto čísle

Tomuto tématu se dále věnují…


Kurzy

Zvyšte si kvalifikaci online z pohodlí domova

Diabetická neuropatie a její léčba
nový kurz
Autoři: MUDr. Michal Dubský, Ph.D.

Mnohočetný myelom: Úvodní léčba netransplantovatelných pacientů
Autoři:

Proč v prevenci zubního kazu kartáček nestačí?
Autoři: doc. MUDr. Romana Koberová – Ivančaková, CSc.

Hypersenzitivita dentinu jako příznak poškození tvrdých zubních tkání
Autoři: MUDr. Erika Lenčová, Ph.D.

Cesta pacienta s CHOPN
Autoři: doc. MUDr. Vladimír Koblížek, Ph.D.

Všechny kurzy
Kurzy Doporučená témata Časopisy
Přihlášení
Zapomenuté heslo

Nemáte účet?  Registrujte se

Zapomenuté heslo

Zadejte e-mailovou adresu se kterou jste vytvářel(a) účet, budou Vám na ni zaslány informace k nastavení nového hesla.

Přihlášení

Nemáte účet?  Registrujte se