Trial registration as a safeguard against outcome reporting bias and spin? A case study of randomized controlled trials of acupuncture

Autoři: Jiyoon Won aff001;  Seoyeon Kim aff001;  Inhu Bae aff003;  Hyangsook Lee aff001
Působiště autorů: Department of Korean Medical Science, Graduate School, Kyung Hee University, Dongdaemun-gu, Seoul, Republic of Korea aff001;  Acupuncture & Meridian Science Research Centre, College of Korean Medicine, Kyung Hee University, Dongdaemun-gu, Seoul, Republic of Korea aff002;  Kyung Hee University Korean Medicine Hospital, Dongdaemun-gu, Seoul, Republic of Korea aff003
Vyšlo v časopise: PLoS ONE 14(10)
Kategorie: Research Article


Background and objective

Trial registration is widely endorsed as it is considered not only to enhance transparency and quality of reporting but also to help safeguard against outcome reporting bias and probably spin, known as specific reporting that could distort the interpretation of results thus mislead readers. We planned to investigate the current registration status of recently published randomized controlled trials (RCTs) of acupuncture, outcome reporting bias in the prospectively registered trials, and the association between trial registration and presence of spin and methodological factors in acupuncture RCTs.


Acupuncture RCTs published in English in recent 5 years (January 2013 to December 2017) were searched in PubMed, Cochrane Central Register of Controlled Trials, and EMBASE. Trial registration records identified in the publications and trial registries were classified into prospectively registered, retrospectively registered, or unregistered. Primary outcomes were identified and the direction of the results was judged as statistically significant (positive) or statistically nonsignificant (negative). We compared registered and published primary outcomes to assess outcome reporting bias and assessed whether discrepancies favored statistically significant outcomes. Frequency and strategies of spin in published reports with statistically nonsignificant results for primary outcomes were then identified. We also analyzed whether the trial registration status was associated with spin and quality of methodological factors.


Of the 322 included RCTs, 41.9% (n = 135) were prospectively registered. Among 64 studies that were prospectively registered and specified primary outcomes, 25 trials had the discrepancies between the registered and published primary outcomes and 60% of them (15 trials) favored the statistically significant findings. Among 169 studies that specified primary outcomes, trial registration status was not associated with the direction of results, i.e., statistically significant or not. Spin was identified in 56.4% out of 78 studies with statistically nonsignificant primary outcomes and claiming efficacy with no consideration of statistically nonsignificant primary outcomes was the most common strategy for spin. Trial registration status was not statistically different between studies with and without spin.


While trial registration seemed to have improved over time, primary outcomes in registered records and publications were often inconsistent, tending to favor statistically significant findings and spin was common in studies with statistically nonsignificant primary outcomes. Journal editors and researchers in this field should be alerted to still prevalent reporting bias and spin.

Klíčová slova:

Asia – Clinical trials – Database searching – Drug therapy – Randomized controlled trials – Scientific publishing – Sleep – Acupuncture


1. Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869. doi: 10.1136/bmj.c869 20332511

2. Chan A, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291(20):2457–2465. doi: 10.1001/jama.291.20.2457 15161896

3. Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan A, Cronin E, et al. Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS one. 2008;3(8):e3081. doi: 10.1371/journal.pone.0003081 18769481

4. Dwan K, Gamble C, Williamson PR, Kirkham JJ. Systematic review of the empirical evidence of study publication bias and outcome reporting bias—an updated review. PLoS one. 2013;8(7):e66844. doi: 10.1371/journal.pone.0066844 23861749

5. Chan AW, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ. 2005;330(7494):753. doi: 10.1136/bmj.38356.424606.8F 15681569

6. Higgins J, Altman D, Sterne C. Chapter 8; Assessing the risk of bias in included studies. In: Higgins JPT, Churchill R, Chandler J, Cumpston MS (editors), Cochrane Handbook for Systematic Reviews of Interventions version 5.2.0 (updated June 2017), Cochrane, 2017.

7. Horton R. The rhetoric of research. BMJ. 1995;310(6985):985–987. doi: 10.1136/bmj.310.6985.985 7728037

8. Fletcher RH, Black B. Spin in scientific writing: scientific mischief and legal jeopardy. Med Law. 2007;26(3):511–525. 17970249

9. Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA. 2010;303(20):2058–2064. doi: 10.1001/jama.2010.651 20501928

10. Boutron I, Altman DG, Hopewell S, Vera-Badillo F, Tannock I, Ravaud P. Impact of spin in the abstracts of articles reporting results of randomized controlled trials in the field of cancer: the SPIIN randomized controlled trial. J Clin Oncol. 2014;32(36):4120–4126. doi: 10.1200/JCO.2014.56.7503 25403215

11. Song F, Parekh S, Hooper L, Loke YK, Ryder J, Sutton AJ, et al. Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess. 2010;14(8):iii, ix–xi, 1–193. doi: 10.3310/hta14080 20181324

12. Boutron I, Ravaud P. Misrepresentation and distortion of research in biomedical literature. Proc Natl Acad Sci U S A. 2018;115(11):2613–2619. doi: 10.1073/pnas.1710755115 29531025

13. Laine C, Horton R, DeAngelis CD, Drazen JM, Frizelle FA, Godlee F, et al. Clinical trial registration—looking back and moving ahead. N Z Med J. 2007;120(1256):U2586. 17589554

14. Pilot LR, Waldmann DR. Food and Drug Administration Modernization Act of 1997: medical device provisions. Food Drug Law J. 1998;53(2):267–295. 10346685

15. De Angelis C, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. Croat Med J. 2004;45(5):531–532. 15906482

16. Zarin DA, Tse T, Sheehan J. The proposed rule for US clinical trial registration and results submission. N Engl J Med 2015;372(2):174–180. doi: 10.1056/NEJMsr1414226 25539444

17. Viergever RF, Li K. Trends in global clinical trial registration: an analysis of numbers of registered clinical trials in different parts of the world from 2004 to 2013. BMJ Open. 2015;5(9):e008932. doi: 10.1136/bmjopen-2015-008932 26408831

18. Zarin DA, Tse T, Williams RJ, Rajakannan T. Update on Trial Registration 11 Years after the ICMJE Policy Was Established. N Engl J Med. 2017;376(4):383–391. doi: 10.1056/NEJMsr1601330 28121511

19. US National Institutes of Health. [cited 12 Oct 2018]

20. Harriman SL, Patel J. When are clinical trials registered? An analysis of prospective versus retrospective registration. Trials. 2016;17(1):187.

21. Dechartres A, Ravaud P, Atal I, Riveros C, Boutron I. Association between trial registration and treatment effect estimates: a meta-epidemiological study. BMC Med. 2016;14(1):100. doi: 10.1186/s12916-016-0639-x 27377062

22. Huić M, Marušić M, Marušić A. Completeness and changes in registered data and reporting bias of randomized controlled trials in ICMJE journals after trial registration policy. PLoS One 2011;6(9):e25258. doi: 10.1371/journal.pone.0025258 21957485

23. Mathieu S, Boutron I, Moher D, Altman DG, Ravaud P. Comparison of registered and published primary outcomes in randomized controlled trials. JAMA. 2009;302(9):977–984. doi: 10.1001/jama.2009.1242 19724045

24. Emdin C, Odutayo A, Hsiao A, Shakir M, Hopewell S, Rahimi K, et al. Association of cardiovascular trial registration with positive study findings: Epidemiological Study of Randomized Trials (ESORT). JAMA Intern Med. 2015;175(2):304–307. doi: 10.1001/jamainternmed.2014.6924 25545611

25. You B, Gan HK, Pond G, Chen EX. Consistency in the analysis and reporting of primary end points in oncology randomized controlled trials from registration to publication: a systematic review. J Clin Oncol. 2012;30(2):210–216. doi: 10.1200/JCO.2011.37.0890 22162583

26. Nankervis H, Baibergenova A, Williams HC, Thomas KS. Prospective registration and outcome-reporting bias in randomized controlled trials of eczema treatments: a systematic review. J Invest Dermatol. 2012;132(12):2727–2734. doi: 10.1038/jid.2012.231 22832489

27. Burke A, Upchurch DM, Dye C, Chyu L. Acupuncture use in the United States: Findings from the national health interview survey. J Altern Complement Med. 2006;12(7):639–648. doi: 10.1089/acm.2006.12.639 16970534

28. Hunt KJ, Coelho HF, Wider B, Perry R, Hung S, Terry R, et al. Complementary and alternative medicine use in England: results from a national survey. Int J Clin Pract. 2010;64(11):1496–1502. doi: 10.1111/j.1742-1241.2010.02484.x 20698902

29. Han J, Ho Y. Global trends and performances of acupuncture research. Neurosci Biobehav Rev. 2011;35(3):680–687. doi: 10.1016/j.neubiorev.2010.08.006 20800613

30. Liu JP, Han M, Li XX, Mu YJ, Lewith G, Wang YY, et al. Prospective registration, bias risk and outcome-reporting bias in randomised clinical trials of traditional Chinese medicine: an empirical methodological study. BMJ Open. 2013;3(7):e002968. doi: 10.1136/bmjopen-2013-002968 23864210

31. Jones CW, Keil LG, Holland WC, Caughey MC, Platts-Mills TF. Comparison of registered and published outcomes in randomized controlled trials: a systematic review. BMC Med. 2015;13(1):282.

32. Spieth PM, Kubasch AS, Penzlin AI, Illigens BM, Barlinn K, Siepmann T. Randomized controlled trials. Neuropsychiatr Dis Treat. 2016;12:1341–1349. doi: 10.2147/NDT.S101938 27354804

33. Juszczak E, Altman DG, Hopewell S, Schulz K. Reporting of multi-arm parallel group randomized trials: extensions of the CONSORT 2010 statement. JAMA. 2019;321(16):1610–1620. doi: 10.1001/jama.2019.3087 31012939

34. World Health Organization. Acupuncture: review and analysis of reports on controlled clinical trials. World Health Organization; 2002.

35. World Health Organization. [cited 24 Oct 2018] International clinical trial registry platform search portal. 2018;

36. Killeen S, Sourallous P, Hunter IA, Hartley JE, Grady HL. Registration rates, adequacy of registration, and a comparison of registered and published primary outcomes in randomized controlled trials published in surgery journals. Ann Surg. 2014;259(1):193–196. doi: 10.1097/SLA.0b013e318299d00b 23732270

37. International Committee of Medical Journal Editors. [cited 24 Oct 2018] Clinical trials registration.

38. International Committee of Medical Journal Editors. [cited 24 Oct 2018] FDAAA 801 and the Final Rule.

39. Vickers A, Goyal N, Harland R, Rees R. Do certain countries produce only positive results? A systematic review of controlled trials. Control Clin Trials. 1998;19(2):159. 9551280

40. Roest AM, De Jonge P, Williams CD, De Vries YA, Schoevers RA, Turner EH. Reporting bias in clinical trials investigating the efficacy of second-generation antidepressants in the treatment of anxiety disorders: a report of 2 meta-analyses. Jama Psychiatry. 2015;72(5):500–510. doi: 10.1001/jamapsychiatry.2015.15 25806940

41. Yavchitz A, Ravaud P, Altman DG, Moher D, Hróbjartsson A, Lasserson T, et al. A new classification of spin in systematic reviews and meta-analyses was developed and ranked according to the severity. J Clin Epidemiol. 2016;75:56–65. doi: 10.1016/j.jclinepi.2016.01.020 26845744

42. Su C, Han M, Ren J, Li W, Yue S, Hao Y, et al. Empirical evidence for outcome reporting bias in randomized clinical trials of acupuncture: comparison of registered records and subsequent publications. Trials. 2015;16(1):28.

43. Turrentine M. It’s all how you “spin” it: interpretive bias in research findings in the obstetrics and gynecology literature. Obstet Gynecol. 2017;129(2):239–242. doi: 10.1097/AOG.0000000000001818 28079777

44. van de Wetering Fleur T, Scholten RJ, Haring T, Clarke M, Hooft L. Trial registration numbers are underreported in biomedical publications. PLoS One 2012;7(11):e49599. doi: 10.1371/journal.pone.0049599 23166724

45. Kirkham JJ, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, et al. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ. 2010;340:c365. doi: 10.1136/bmj.c365 20156912

46. Papageorgiou SN, Xavier GM, Cobourne MT, Eliades T. Registered trials report less beneficial treatment effects than unregistered ones: a meta-epidemiological study in orthodontics. J Clin Epidemiol. 2018;100:44–52. doi: 10.1016/j.jclinepi.2018.04.017 29705094

47. Odutayo A, Emdin CA, Hsiao AJ, Shakir M, Copsey B, Dutton S, et al. Association between trial registration and positive study findings: cross sectional study (Epidemiological Study of Randomized Trials-ESORT). BMJ. 2017;356:j917. doi: 10.1136/bmj.j917 28292744

48. Ma B, Chen Z, Xu J, Wang Y, Chen K, Ke F, et al. Do the CONSORT and STRICTA checklists improve the reporting quality of acupuncture and moxibustion randomized controlled trials published in chinese journals? a systematic review and analysis of trends. PloS one. 2016;11(1):e0147244. doi: 10.1371/journal.pone.0147244 26808550

49. Zhao H, Zhang J, Yang F, Tan L. Improve the ethical review of clinical trials on traditional medicine: a cross-sectional study of clinical trials registration, ethical review, and informed consent in clinical trials of Traditional Chinese Medicine. Medicine. 2018;97(47):e13062. doi: 10.1097/MD.0000000000013062 30461608

50. Wager E, Williams P, Project Overcome failure to Publish nEgative fiNdings Consortium. "Hardly worth the effort"? Medical journals’ policies and their editors’ and publishers’ views on trial registration and publication bias: quantitative and qualitative study. BMJ. 2013;347:f5248. doi: 10.1136/bmj.f5248 24014339

51. Kaptchuk TJ. Effect of interpretive bias on research evidence. BMJ. 2003;326(7404):1453–1455. doi: 10.1136/bmj.326.7404.1453 12829562

52. Chiu K, Grundy Q, Bero L. ‘Spin’ in published biomedical literature: A methodological systematic review. PLoS biol. 2017;15(9):e2002173. doi: 10.1371/journal.pbio.2002173 28892482

53. Anand V, Scales DC, Parshuram CS, Kavanagh BP. Registration and design alterations of clinical trials in critical care: a cross-sectional observational study. Intensive Care Med. 2014;40(5):700–722. doi: 10.1007/s00134-014-3250-7 24737256

54. Mayo-Wilson E, Heyward J, Keyes A, Reynolds J, White S, Atri N, et al. Clinical trial registration and reporting: a survey of academic organizations in the United States. BMC Med. 2018;16(1):60. doi: 10.1186/s12916-018-1042-6 29716585

55. Lazarus C, Haneef R, Ravaud P, Hopewell S, Altman DG, Boutron I. Peer reviewers identified spin in manuscripts of nonrandomized studies assessing therapeutic interventions, but their impact on spin in abstract conclusions was limited. J Clin Epidemiol. 2016;77:44–51. doi: 10.1016/j.jclinepi.2016.04.012 27164274

56. Chauvin A, Moher D, Altman D, Schriger DL, Alam S, Hopewell S, et al. A protocol of a cross-sectional study evaluating an online tool for early career peer reviewers assessing reports of randomised controlled trials. BMJ Open. 2017;7(9):e017462. doi: 10.1136/bmjopen-2017-017462 28918414

Článek vyšel v časopise


2019 Číslo 10
Nejčtenější tento týden