Institutional differences in USMLE Step 1 and 2 CK performance: Cross-sectional study of 89 US allopathic medical schools


Autoři: Jesse Burk-Rafel aff001;  Ricardo W. Pulido aff002;  Yousef Elfanagely aff003;  Joseph C. Kolars aff004
Působiště autorů: Department of Internal Medicine, New York University Langone Health, New York, NY, United States of America aff001;  Department of Otolaryngology–Head and Neck Surgery, University of Washington, Seattle, WA, United States of America aff002;  Department of Internal Medicine, Brown University, Providence, RI, United States of America aff003;  Department of Internal Medicine, University of Michigan Medical School, Ann Arbor, MI, United States of America aff004
Vyšlo v časopise: PLoS ONE 14(11)
Kategorie: Research Article
doi: 10.1371/journal.pone.0224675

Souhrn

Introduction

The United States Medical Licensing Examination (USMLE) Step 1 and Step 2 Clinical Knowledge (CK) are important for trainee medical knowledge assessment and licensure, medical school program assessment, and residency program applicant screening. Little is known about how USMLE performance varies between institutions. This observational study attempts to identify institutions with above-predicted USMLE performance, which may indicate educational programs successful at promoting students’ medical knowledge.

Methods

Self-reported institution-level data was tabulated from publicly available US News and World Report and Association of American Medical Colleges publications for 131 US allopathic medical schools from 2012–2014. Bivariate and multiple linear regression were performed. The primary outcome was institutional mean USMLE Step 1 and Step 2 CK scores outside a 95% prediction interval (≥2 standard deviations above or below predicted) based on multiple regression accounting for students’ prior academic performance.

Results

Eighty-nine US medical schools (54 public, 35 private) reported complete USMLE scores over the three-year study period, representing over 39,000 examinees. Institutional mean grade point average (GPA) and Medical College Admission Test score (MCAT) achieved an adjusted R2 of 72% for Step 1 (standardized βMCAT 0.7, βGPA 0.2) and 41% for Step 2 CK (standardized βMCAT 0.5, βGPA 0.3) in multiple regression. Using this regression model, 5 institutions were identified with above-predicted institutional USMLE performance, while 3 institutions had below-predicted performance.

Conclusions

This exploratory study identified several US allopathic medical schools with significant above- or below-predicted USMLE performance. Although limited by self-reported data, the findings raise questions about inter-institutional USMLE performance parity, and thus, educational parity. Additional work is needed to determine the etiology and robustness of the observed performance differences.

Klíčová slova:

Academic skills – Medical education – Observational studies – Primary care – Schools – Standardized tests – Undergraduates – United States


Zdroje

1. Haist SA, Katsufrakis PJ, Dillon GF. The evolution of the United States Medical Licensing Examination (USMLE): Enhancing assessment of practice-related competencies. JAMA. 2013;310(21): 2245–2246. doi: 10.1001/jama.2013.282328 24302081

2. US Medical Licensing Examination, 2019 Bulletin of Information. National Board of Medical Examiners. [cited 2018 Sep 15]. Available from: https://www.usmle.org/pdfs/bulletin/2019bulletin.pdf.

3. US Medical Licensing Examination, 2018 Performance Data. National Board of Medical Examiners. [cited 2019 Jul 17]. Available from: https://www.usmle.org/performance-data/default.aspx#2018_step-1

4. Andriole DA, Jeffe DB. Prematriculation variables associated with suboptimal outcomes for the 1994–1999 cohort of US medical school matriculants. JAMA. 2010;304(11): 1212–1219. doi: 10.1001/jama.2010.1321 20841535

5. National Resident Matching Program, Data Release and Research Committee: Results of the 2018 NRMP Program Director Survey. National Resident Matching Program. Washington, DC. [cited 2018 Sep 15]. Available from: http://www.nrmp.org/wp-content/uploads/2018/07/NRMP-2018-Program-Director-Survey-for-WWW.pdf

6. Jayakumar KL, Lipoff JB. US Medical Licensing Examination Step 2 Clinical Knowledge score in dermatology resident selection. JAMA Dermatol. 2017;153(11): 1193–1194. doi: 10.1001/jamadermatol.2017.2924 28832874

7. Jeffe DB, Andriole DA. Factors associated with American Board of Medical Specialties member board certification among US medical school graduates. JAMA. 2011;306(9): 961–970. doi: 10.1001/jama.2011.1099 21900136

8. Donnon T, Paolucci EO, Violato C. The predictive validity of the MCAT for medical school performance and medical board licensing examinations: A meta-analysis of the published research. Acad Med. 2007;82(1): 100–106. doi: 10.1097/01.ACM.0000249878.25186.b7 17198300

9. Basco WT Jr, Way DP, Gilbert GE, Hudson A. Undergraduate institutional MCAT scores as predictors of USMLE Step 1 performance. Acad Med. 2002;77(10S): 13–16.

10. Burk-Rafel J, Santen SA, Purkiss J. Study behaviors and USMLE Step 1 performance: Implications of a student self-directed parallel curriculum. Acad Med. 2017;92(11S): 67–74.

11. Hecker K, Violato C. How much do differences in medical schools influence student performance? A longitudinal study employing hierarchical linear modeling. Teach Learn Med. 2008;20(2): 104–113. doi: 10.1080/10401330801991915 18444195

12. Hecker K, Violato C. Medical school curricula: Do curricular approaches affect competence in medicine? Fam Med. 2010;41(6): 420–426.

13. Ghaffari-Rafi A, Lee RE, Fang R, Miles DJ. Multivariable analysis of factors associated with USMLE scores across US medical schools. BMC Med Ed. 2019;19: 154.

14. von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for reporting observational studies. PLOS Med. 2007;4(10): e296. doi: 10.1371/journal.pmed.0040296 17941714

15. Best Medical Schools: Research. US News and World Report. 2008–2016 editions, online and print. [cited 2018 Sep 15]. Available from: http://grad-schools.usnews.rankingsandreviews.com/best-graduate-schools/top-medical-schools/research-rankings/ Subscription required.

16. Medical School Admission Requirements. Association of American Medical Colleges. Washington (DC): 2008–2012 editions.

17. FACTS Table 17. Association of American Medical Colleges. 2013. [cited 2016 Nov 6] Available by request from: https://www.aamc.org/data-reports/students-residents/report/facts

18. US Medical Licensing Examination, Performance Data. National Board of Medical Examiners. 2014–2016. Available from: https://www.usmle.org/performance-data/

19. USMLE Score Interpretation Guidelines. 2015 ed. Available from: https://www.usmle.org/pdfs/transcripts/USMLE_Step_Examination_Score_Interpretation_Guidelines.pdf

20. Altman N, Krzywinski. Regression diagnostics. Nat Meth. 2016;13: 385–386.

21. Werner J, Griebeler EM. New insights into non-avian dinosaur reproduction and their evolutionary and ecological implications: Linking fossil evidence to allometries of extant close relatives. PLOS ONE. 2013;8(8): e72862. doi: 10.1371/journal.pone.0072862 23991160

22. Banos JH, Pepin ME, Van Wagoner N. Class-wide access to a commercial Step 1 question bank during preclinical organ-based modules: a pilot project. Acad Med. 2018;93(3): 486–490. doi: 10.1097/ACM.0000000000001861 28817433

23. Daniel M, Fleming A, Grochowski C, Harnik V, Klimstra S, Morrison G, et al. Why not wait? Eight institutions share their experiences moving United States Medical Licensing Examination Step 1 after core clinical clerkships. Acad Med. 2017;92(11): 1515–1524. doi: 10.1097/ACM.0000000000001714 28422816

24. National Resident Matching Program, Data Release and Research Committee: Results of the 2018 NRMP Program Director Survey. National Resident Matching Program, Washington (DC): 2018. Available from: https://www.nrmp.org/wp-content/uploads/2018/07/NRMP-2018-Program-Director-Survey-for-WWW.pdf

25. Morse R, Martin E. Methodology: 2020 Best Medical Schools. Available from: https://www.usnews.com/education/best-graduate-schools/articles/medical-schools-methodology

26. Zhao X, Oppler S, Dunleavy D, Kroopnick M. Validity of four approaches of using repeaters' MCAT scores in medical school admissions to predict USMLE Step 1 total scores. Acad Med. 2010;85(10 Suppl): S64–67. doi: 10.1097/ACM.0b013e3181ed38fc 20881707

27. Jurich D, Daniel M, Paniagua M, Fleming A, Harnik V, Pock A, et al. Moving the United States Medical Licensing Examination Step 1 after core clerkships: An outcomes analysis. Acad Med. 2019;94(3): 371–377. doi: 10.1097/ACM.0000000000002458 30211755


Článek vyšel v časopise

PLOS One


2019 Číslo 11