Validity of the Medical College Admission Test for predicting MD-PhD student outcomes

Revolutionizing Health Professions Admissions to Achieve an Inclusive Workforce

This article describes the "The Admissions Revolution: Bold Strategies for Diversifying the Healthcare Workforce" conference, which preceded the 2022 Beyond Flexner Alliance Conference and called for health professions institutions to boldly reimagine the admission process to diversify the health care workforce. Proposed strategies encompassed 4 key themes: admission metrics, aligning admission practices with institutional mission, community partnerships to fulfill social mission, and student support and retention. Transformation of the health professions admission process requires broad institutional and individual effort. Careful consideration and implementation of these practices will help institutions achieve greater workforce diversity and catalyze progress toward health equity.

Factors Leading to Successful Performance on U.S. National Licensure Exams for Medical Students: A Scoping Review

PURPOSE

To synthesize the evidence of the factors leading to successful performance on knowledge-based national licensure exams (NLEs) for medical students.

METHOD

The authors conducted a scoping review to summarize the peer-reviewed empiric literature that used United States Medical Licensing Examination (USMLE) Step 1 or Step 2 Clinical Knowledge or Comprehensive Osteopathic Medical Licensing Examination (COMLEX) Level 1 or Level 2 Cognitive Evaluation scores as outcomes. The authors searched PubMed and Scopus without date restrictions through April 30, 2021. Two reviewers independently screened and selected studies for inclusion. Data were summarized narratively and with descriptive statistics.

RESULTS

The authors screened 1,185 unique citations and included 233 full-text studies in their review. Of these, 201 (86%) were studies of USMLE exams, 31 (13%) were studies of COMLEX exams, and 1 (0.4%) reported on both. The authors classified 29 studies (12%) as informing NLE preparation, 163 (70%) as attempting to identify predictive variables, and 76 (33%) as using NLE scores for program evaluation. Preparation studies found that the number of practice test items, practice exam scores, and less time in dedicated preparation correlated with higher NLE scores. Use of other commercial resources or study strategies was not consistently associated with higher scores. Predictive studies found the strongest relationships between individuals' performance on past assessments and their NLE scores.

CONCLUSIONS

The factors leading to successful performance on knowledge-based NLEs align with well-known principles from the cognitive sciences. Learners build on existing foundations of knowledge (reflected in their prior academic performance) and are likely to learn more efficiently with testing and spaced learning over time. While commercial test preparation resources are ubiquitous, there is no evidence that a single resource gives students a competitive advantage on NLEs. Developing habits of regular and continuous learning is necessary for clinical practice and successful NLE performance.

Increasing Clinician-Scientist Workforce Diversity through the National Institute of General Medical Sciences’ Medical Scientist Training Program

The National Institute of General Medical Sciences Medical Scientist Training Program (MSTP) has been successful in producing clinician-scientists, with a majority of graduates pursuing research-related careers. However, there are a number of areas of continuing concern for the program. In particular, women and individuals from certain racial and ethnic backgrounds remain persistently underrepresented in MSTPs relative to the average college-aged U.S. population and to students receiving life sciences bachelor’s degrees. The authors, who include leaders of NIGMS, identify a number of challenges and opportunities for enhancing diversity, equity and inclusion in the MSTPs and suggest strategies for addressing them.

Why clinical training in China should improve: a cross-sectional study of MD graduates

BACKGROUND

China is experiencing major medical education reforms that include establishing national training standards, standards for health professionals, and advanced health delivery system requirements. Graduate medical education (GME) is being piloted as a merger of Doctor of Medicine (MD) with PhD programs to improve academic research and clinical training. However, the academic degree-centred system has led to a preoccupation with research rather than clinical training. Unfortunately, there is a shortage of quality information regarding the clinical training of MD graduates from Chinese medical schools. To fill this gap, this general investigation aims to provide the perspective of recent MD graduates in China for the different subspecialties of clinical training as experienced in different contexts.

METHODS

There were 432 MD graduates who participated in an online survey regarding their clinical training. Information collected included overall satisfaction, educational supervision, supervised learning events, curriculum coverage, local teaching, teamwork, educational governance, workload, supportiveness of the environment, feedback, clinical experience, patient safety, handovers, and reporting systems.

RESULTS

Only 37.4% reported satisfaction with the overall clinical training quality; 54.6% rated the informal and bedside quality as "good"; 64.4% reported they knew who provided clinical supervision; but only 35.5% rated the quality of clinical supervision as high; 51.8% reported that they judged senior physicians as "not competent"; 41.9% agreed that the staff treated each other respectfully; 97.4% admitted that they worked beyond the mandatory hours and claimed they were regularly short of sleep; 84.2% raised concerns about patient safety; 45.3% reported that they received regular informal feedback; 48.1% believed that their concerns about education and training would be addressed.

CONCLUSIONS

This study suggests that the quality of clinical training for MD graduates should be improved. While the overall satisfaction with the teaching quality was acceptable, the quality of many clinical training aspects scored poorly. A major problem seems an undue focus on research in MD/PhD training at the cost of the quality of clinical training, due to career perspectives that undervalue clinical competence. The findings of this study should benefit from a deeper investigation to understand the causes and possible remediation. Suggestions include defining subspecialties and training lengths; monitoring, evaluation, and integration SST with MD degree; providing funds or rewards for academic and clinical training; establishing supervising teams to guide clinical training; and establishing physician scientist task force to help overcome challenges.

Performance measures of racially underrepresented Ph.D. students in biomedical sciences: The UAMS IMSD Program Outcomes

The purpose of this study was to identify performance measures of racially underrepresented minority (RUM) Ph.D. trainees who needed additional training initiatives to assist with completing the UAMS biomedical science degree. A sample of 37 trainees in the 10-year NIH-NIGMS funded Initiative for Maximizing Student Development (IMSD) program at the University of Arkansas for Medical Sciences (UAMS) were examined. Descriptive statistics and correlations examined process measures (GRE scores, GPAs, etc.) and outcome measures (time-to-degree, publications, post-doctoral fellowship, etc.) While differences were found, there were no statistically significant differences between how these two groups (Historically Black Colleges and Universities (HBCUs) and Predominately White Institutions (PWIs)) of students performed over time as Ph.D. students. Graduates who scored lower on the verbal section of the GRE also had a higher final graduate school grade point average in graduates who received their undergraduate training from HBCUs. Of the graduates who received their undergraduate training from PWIs, graduates who scored lower on the quantitative section of the GRE had higher numbers of publications. These findings stimulate the need to 1) reduce reliance on the use of the GRE in admission committee decisions, 2) identify psychometrically valid indicators that tailored to assess outcome variables that are relevant to the careers of biomedical scientists, and 3) ensure the effective use of the tools in making admission decisions.

Assessment of Diversity Outcomes in American Medical School Admissions: Applying the Grutter Legitimacy Principles

In the last 30 years, except for female participation, the enrollment of Latinx, African Americans, Native Americans, Alaskan natives, and disadvantaged students in medical school has been constant; however, increasing enrollment of these minority populations is feasible, if admissions committees make two changes in approach. First, the traditional belief that matriculation merit is a linear function of past academic performance must be rejected. Second, once the threshold needed to complete medical school in four years and to pass licensing examinations at the first attempt has been met, all candidates are equally qualified, and matriculation decisions must be based, in part, on societal interests. In Grutter vs. Bollinger, the United States Supreme Court determined that graduate admission committees can and should consider societal interests. Each admission decision represents a substantial government investment in each student, as the Medicare Act directly subsidizes much of the cost of medical education. As Grutter explained, there is a societal interest in the public having confidence in, and access to, the medical school training that will prepare tomorrow’s medical, professional, and political leaders. Our analysis suggests that medical school admissions are biased towards academic achievement in matriculants, beyond acceptable thresholds for graduation and licensure. We believe medical schools must shift their admissions strategies and consider noncognitive factors in all candidates as determinative once minimum acceptable academic standards have been met.

A Study of the Validity of the New MCAT Exam

PURPOSE

To conduct a study of the validity of the new Medical College Admission Test (MCAT).

METHOD

Deidentified data for first- and second-year medical students (185 women, 54.3%; 156 men, 45.7%) who matriculated in 2016 and 2017 to the University of Minnesota Medical School-Twin Cities were included. Of those students, 220 (64.5%) had taken the new MCAT exam and 182 (53.4%) had taken the old MCAT exam (61 [17.9%] had taken both). The authors calculated descriptive statistics and Pearson product moment correlations (r) between new and old MCAT section scores. They conducteda regression analysis of MCAT section scores with Step 1 scores and with preclerkship course performance. They also conducted an exploratory factor analysis (principal component analysis with varimax rotation) of MCAT scores, undergraduate grade point average, Step 1 scores, and course performance.

RESULTS

The new MCAT exam section mean score percentiles ranged from 72 to 78 (mean composite score percentile of 80). The old MCAT exam section mean score percentiles ranged from 84 to 88 (mean composite score percentile of 83). The pattern of correlations among and between new and old MCAT exam section scores (range of r: 0.03-0.67; P < .01) provided evidence of both divergent and convergent validities. Backward multiple regression of new MCAT exam section scores and Step 1 scores resulted in a multiple R of .440; the same analysis with Human Behavior course performance as the dependent variable provided a similar solution with the expected sections of the new MCAT exam (multiple R = .502). The factor analysis resulted in 4 cohesive, theoretically meaningful factors: biomedical knowledge, basic science concepts, cognitive reasoning, and general achievement.

CONCLUSIONS

This study provided empirical evidence of multiple types of validity for the new MCAT exam.

Gender Disparities in Medical Student Research Awards: A 13-Year Study From the Yale School of Medicine

PURPOSE

The Liaison Committee on Medical Education mandates instruction in research conduct, and many U.S. medical schools require students to complete a research project. All Yale School of Medicine (YSM) graduating students submit a research thesis, and ~5% are awarded highest honors. Gender disparities exist in areas related to physician research productivity, including academic rank, research funding, and publications. The authors asked whether gender disparities exist for medical student research.

METHOD

The authors conducted a retrospective review of 1,120 theses submitted by graduating medical students from 2003 to 2015 at YSM and collected data on gender, mentoring, research type, sponsoring department, and other characteristics. Multivariate logistic regression modeling examined gender differences in medical student research awards.

RESULTS

Women authored 50.9% of theses, but earned only 30.9% of highest honors awards (OR 0.41; 95% CI: 0.23, 0.74). Among factors associated with increased receipt of highest honors that differed by gender, men were more likely than women to work with a mentor with a history of three or more thesis honorees, take a fifth year of study, secure competitive research funding, undertake an MD-master of health science degree, and conduct laboratory research (all P < .001). After adjustment for these factors, and for underrepresented in medicine status and sponsoring department, women remained less likely to receive highest honors (OR 0.51; 95% CI: 0.27, 0.98).

CONCLUSIONS

Women YSM students were less likely to receive highest honors for medical research. Gender disparities in postgraduate biomedical research success may start during undergraduate medical education.

Undergraduate Rigor Scores: Do They Predict Achievement in Veterinary School?

The relations between potential indicators of undergraduate rigor and subsequent achievement in professional school are not clear; some studies have shown that greater undergraduate selectivity is associated with greater achievement in medical science programs, while others have not. We sought to determine the extent to which indicators of undergraduate rigor were associated with achievement in veterinary school. Participants were graduates from three cohorts. The predictors were undergraduate GPA (UGPA), plus five rigor scores-degree or number of undergraduate credits, number of honors courses, number of withdrawals from or repeats of prerequisite science courses, number of part-time semesters, and ratio of community college credits to total college credits. The outcomes were the veterinary medicine cumulative GPA (CVM GPA), Qualifying Exam scores, and North American Veterinary Licensing Exam scores. Using correlations corrected for range restriction, we regressed each outcome on the five rigor scores and UGPA for each of the three graduating cohorts. In most cases, indicators of undergraduate rigor did not predict subsequent achievement in veterinary school; however, in two comparisons, number of honors courses taken as an undergraduate predicted subsequent achievement. UGPA, as expected, predicted CVM GPA. Admissions committees may want to reevaluate whether they include undergraduate rigor when considering admission to their programs, with the caveat that our findings are specific to our institution and are not generalizable.

Rescuing the physician-scientist workforce: the time for action is now

The 2014 NIH Physician-Scientist Workforce (PSW) Working Group report identified distressing trends among the small proportion of physicians who consider research to be their primary occupation. If unchecked, these trends will lead to a steep decline in the size of the workforce. They include high rates of attrition among young investigators, failure to maintain a robust and diverse pipeline, and a marked increase in the average age of physician-scientists, as older investigators have chosen to continue working and too few younger investigators have entered the workforce to replace them when they eventually retire. While the policy debates continue, here we propose four actions that can be implemented now. These include applying lessons from the MD-PhD training experience to postgraduate training, shortening the time to independence by at least 5 years, achieving greater diversity and numbers in training programs, and establishing Physician-Scientist Career Development offices at medical centers and universities. Rather than waiting for the federal government to solve our problems, we urge the academic community to address these goals by partnering with the NIH and national clinical specialty and medical organizations.