Advances in Physiology Education, Vol 271, Issue 6 55-S60, Copyright © 1996 by American Physiological Society
A comparison of methods for dealing with troublesome examination questions
J. M. Norton
University of New England, College of Osteopathic Medicine, Biddeford, Maine 04005, USA. jnorton@mailbox.une.edu
This study compared two methods for identifying troublesome examination
questions: a postexam student "challenge committee" process and a "question
citation" procedure allowing students to identify troublesome questions
during an exam. Challenged and cited questions on eight examinations given
to second-year medical students were compared with respect to the
percentage of correct responses (%CR) and the discrimination index (DI).
Challenged questions had a significantly lower mean value for %CR (53.2 +/-
22.5, n = 74) than unchallenged questions (84.7 +/- 15.4, n = 428) but the
two groups did not differ with respect to DI. For cited questions, the
number of citations was correlated with %CR (r = -0.5480, p < 0.001) but
not with DI. The group of questions that received three or more citations
(n = 145) included 67.6% of all challenged questions and 78.3% of
challenges that eventually led to a change in an answer key. A simple
citation procedure completed during an exam could replace a complicated,
post-exam challenge committee process and achieve the same goal of
identifying troublesome questions.
