Against Standardized Exams I

The path to become a doctor is fraught with standardized exams. From premedical courses to preparing for the MCAT to tests in the first two years of medical school to shelf exams during the clinical years to USMLE licensing exams, we keep the number 2 pencil business (and now the computerized testing industry) alive. In fact, selecting for doctors is akin to selecting for good test-takers, a skill set that is necessary for being a physician but has little real life application.

I don't think standardized exams accomplish what they're supposed to do; a person who does well on them may be a terrible a doctor, and a person who does poorly on them may turn out to be a fantastic physician. In other words, the sensitivity and specificity of standardized tests in determining whether someone should be a doctor is poor. They are neither necessary nor sufficient for establishing that someone has the skill set to take care of patients.

Fund of knowledge is important, but I think people overemphasize its centrality in clinical medicine. A certain proficiency in the basics is necessary, but when things get complex, doctors look it up or consult a specialist. There's no shame in turning to a textbook or the Internet; in fact, medicine changes so much that looking up references may yield newer findings that one didn't originally know. Standardized tests assess fund of knowledge well, but they must be calibrated to determining whether someone has the foundation to be a physician rather than how much esoteria one knows. Furthermore, standardized tests lack the ability to discriminate problem solving ability and clinical reasoning.