A comparison of orthopaedic surgery and internal medicine perceptions of USMLE Step 1 pass/fail scoring

Mun et al. BMC Medical Education (2021) 21:255 https://doi.org/10.1186/s12909-021-02699-4 RESEARCH Open Access A comparison of orthopaedic surgery and internal medicine perceptions of USMLE Step 1 pass/fail scoring Frederick Mun1*, Alyssa R. Scott1, David Cui1, Erik B. Lehman2, Seong Ho Jeong3, Alia Chisty1,4, Paul J. Juliano1,5, William L. Hennrikus1,5 and Eileen F. Hennrikus1,4 Abstract Background: United States Medical Licensing Examination Step 1 will transition from numeric grading to pass/fail, sometime after January 2022. The aim of this study was to compare how program directors in orthopaedics and internal medicine perceive a pass/fail Step 1 will impact the residency application process. Methods: A 27-item survey was distributed through REDCap to 161 U.S. orthopaedic residency program directors and 548 U.S. internal medicine residency program directors. Program director emails were obtained from the American Medical Association’s Fellowship and Residency Electronic Interactive Database. Results: We received 58 (36.0%) orthopaedic and 125 (22.8%) internal medicine program director responses. The majority of both groups disagree with the change to pass/fail, and felt that the decision was not transparent. Both groups believe that the Step 2 Clinical Knowledge exam and clerkship grades will take on more importance. Compared to internal medicine PDs, orthopaedic PDs were significantly more likely to emphasize research, letters of recommendation from known faculty, Alpha Omega Alpha membership, leadership/extracurricular activities, audition elective rotations, and personal knowledge of the applicant. Both groups believe that allopathic students from less prestigious medical schools, osteopathic students, and international medical graduates will be disadvantaged. Orthopaedic and internal medicine program directors agree that medical schools should adopt a graded pre-clinical curriculum, and that there should be a cap on the number of residency applications a student can submit. Conclusion: Orthopaedic and internal medicine program directors disagree with the change of Step 1 to pass/fail. They also believe that this transition will make the match process more difficult, and disadvantage students from less highly-regarded medical schools. Both groups will rely more heavily on the Step 2 clinical knowledge exam score, but orthopaedics will place more importance on research, letters of recommendation, Alpha Omega Alpha membership, leadership/extracurricular activities, personal knowledge of the applicant, and audition electives. Keywords: USMLE, Step 1, Pass/fail scoring, Orthopaedic surgery, Internal medicine * Correspondence: 1 Penn State College of Medicine, Penn State Milton S. Hershey Medical Center, Hershey, PA, USA Full list of author information is available at the end of the article © The Author(s). 2021 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data. Mun et al. BMC Medical Education (2021) 21:255 Background The United States Medical Licensing Examinations (USMLE) currently consists of three numerically scored knowledge based exams. On February 12, 2020, the Federation of State Medical Boards (FSMB) and the National Board of Medical Examiners (NBME) announced that the first of these three exams, Step 1, would change from the three-digit numeric score to reporting a pass/fail outcome -- starting sometime after January 2022 [1]. This change occurred after extensive national discussion on potential ways to optimize the transition from undergraduate to graduate medical education [1, 2]. The unintended consequences of a scored Step 1 on medical students and residency programs were widely reported in 2008 by the Committee to Evaluate the USMLE Program (CEUP). CEUP recognized the diverse stakeholders affected by Step 1 score reporting and suggested a need to redesign USMLE [2]. Although a numerically scored Step 1 provided an objective measure for residency programs to assess applicants from different backgrounds and medical schools, the perceived overemphasis on this test was controversial. In March 2019, the Invitational Conference on USMLE Scoring (InCUS), sponsored by the Association of American Medical Colleges (AAMC), American Medical Association (AMA), NBME, FSMB, and Educational Commission for Foreign Medical Graduates (ECFMG), convened to review the USMLE’s practice of numeric score reporting. InCUS concluded that licensing examinations for undergraduate medical students must be reevaluated. FSMB and NBME took into consideration the recommendations from InCUS, and stated that the change to a pass/fail Step 1 would help residency programs to refocus on the main purpose of Step 1 -- to assess medical licensure eligibility [1, 2]. Step 1 is one of the most important factors used to screen applicants by internal medicine (IM) and orthopaedic surgery residency programs [1–3]. Some residency programs also place a high emphasis on other factors, such as grades in required clerkships, research experiences, extracurricular activities, letters of recommendation, and audition rotations. With the absence of a scored Step 1, all residency programs will need to reconsider the factors used to determine which applicants are selected. The aim of this study was to compare the perceptions among program directors (PDs) in orthopaedics and IM on the change of Step 1 from a graded to a pass/fail exam, and its impact on the residency application process. Some groups have studied PD opinions of the Step 1 pass/fail change; however, to the best of our knowledge, the current study is the first comprehensive survey to directly compare the impact on medicine and a surgical subspecialty [4–12]. We chose to study IM PDs because IM has historically been a popular specialty Page 2 of 8 that has had the most number of total applicants. We decided to study Orthopaedic PDs because orthopaedics has historically received some of the highest number of applicants per residency position [ (...truncated)