Predictive validity study of the medical college admission test using multiple regression and latent variable path analysis (LVPA)
Date
2005
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
The present study employed multiple regression and latent variable path analysis to identify and confirm the directional and predictive components that explain the greatest amount of variability in pre-clinical and clinical indices of success (i.e. United States Medical Licensure Examination (USMLE) Steps 1, 2, and 3) in medical education. There were three major purposes of the statistical analyses: 1) To re-visit and support previous studies documenting the predictive power of subscales of the Medical College Admission Test (MCAT) (i.e. biological science, physiological science), demographic variables ( e.g. ethnicity, age), and school variables (e.g. medical schools across the United States) in predicting performance on Steps 1, 2, and 3 of the USMLE; 1 2) To use a "theory strong" statistical tool (Structural Equation Modeling - SEM) to define the hypothesized developmental sequence of variables; and 3) To formulate a confirmatory model (Latent Variable Path Model- LVPA) to identify both the structural (i.e. ordering of variables) and directional pathways of independent variables historically and presently hypothesized to account for performance on Steps 1, 2, and 3 of the USMLE. Using the total sample as well as multisampling techniques (5% random sample of the total N) descriptive and inferential statistics were performed for Version 1 of the MCAT (1991). Results from the multiple regression revealed that with 3 predictors 33% of the variability was accounted for in Step 1 of the USMLE. The best predictor was the biological subscale of the MCAT (R2 = .28). For Step 2, 5 predictors accounted for 24% of the variability, with the biological science subscale of the MCAT again accounting for the most variance (R2 = .15). Lastly, 5 variables predicted performance on Step 3 of the USMLE, accounting for 30% of the variance in scores. The best predictor of Step 3 USMLE performance was the verbal reasoning subscale of the MCAT (R2 = .20). L VP A of Aptitude, Achievement and Performance The final stage of analysis performed was the LVPA employing Maximum Likelihood estimation (n = 24, 872) resulting in a Comparative Fit Index (CFI) of .928. Three latent variables measured by undergraduate grade point average (general achievement), subscales of the MCAT (aptitude for medicine), and Steps 1, 2, and 3 of the USMLE (performance in medicine) were identified. The path coefficients between these latent variables were moderate to high, ranging from .25 (general achievement and aptitude for medicine) to .4 7 between aptitude for medicine and performance in medicine. These results provide evidence for the predictive validity of the MCAT (with an incremental increase with undergraduate GP A) as well as for the overall L VP A model positing structural relationships between and among general achievement, aptitude for medicine, and performance in medicine. Nonetheless, it is argued that there has been an over reliance on cognitive measures (i.e. MCA T subscales) for admission to medical school with less importance placed on non-cognitive factors (i.e. judgment, humanistic qualities, empathy).4 A challenge to the medical profession, then, is to develop screening and selection methods that supplement the MCAT focusing on key personal characteristics and the complex nature of physician roles.
Description
Bibliography: p. 84-93
Keywords
Citation
Collin, V. T. (2005). Predictive validity study of the medical college admission test using multiple regression and latent variable path analysis (LVPA) (Doctoral thesis, University of Calgary, Calgary, Canada). Retrieved from https://prism.ucalgary.ca. doi:10.11575/PRISM/56