Item response analysis on an examination in anesthesiology for medical students in Taiwan: A comparison of one- and two-parameter logistic models

Background: Student examinations are an essential component of medical education and item analyses are important to assess test quality. Among miscellaneous psychometric theories used for test analyses, item response theory is more flexible and versatile than other theories. This study aimed to apply item response models to analyze an anesthesiology examination for medical and dental students.
Methods: This examination included 50 items that were administered to 170 5th- and 6th-year medical and dental students. One- and two-parameter logistic (1-PL and 2-PL) item response models were used to conduct item analyses of the examination. Fit statistics were examined to exclude misfit items and evaluate test reliability. Goodness-of-fit analyses were used to select the model having the better fit to data. Examinee's ability and item difficulty were estimated and then expressed on the common scale. Potentially differential items were detected using logistic regression.
Results: The goodness-of-fit analysis revealed that, in our case, the 1-PL model was more suitable for item response analyses. No misfit item was noted and the test reliability was 0.81 (1-PL model). The mean examinee's ability was set at 0 by definition [standard deviation (SD) = 0.61] and the mean item difficulty was -2.08 (SD = 1.93). There were 24 items with a difficulty level lower than the least able examinee, and three items had a difficulty level higher than the most able examinee. Four potentially differential items were identified.
Conclusion: Item response models are useful for medical test analyses and provide valuable information about model comparisons and identification of differential items other than test reliability, item difficulty, and examinee's ability. Copyright (c) 2013 Elsevier Taiwan LLC and the Chinese Medical Association. All rights reserved.