Boards II

I'm going to write about some problems I have with the USMLE, but this is actually not meant to be a rant (I apologize if that's what it turns out to be). There are a number of intrinsic problems with standardized tests as methods of evaluation of whether someone is qualified to be a physician. The questions have to be written such that there is one "best answer," but clearly, this is not the case in real life. On the test, a sickle cell person with osteomyelitis always has Salmonella. My reaction as a test taker is immediate; where's the bubble to fill in? But in real life, there's a 20% chance it's another bug. That's going to get someone someday.

I recently did a question with a stem: "A 40 year old man with recent contact with commercial sex workers presents with..." What? The first few words out of real life patients usually isn't, "I courted a prostitute last night" (though I may be wrong). This question doesn't seem realistic. And certainly, the person could very well have something completely unrelated to his sexual encounters. But on an exam, I've already narrowed down my choices to the STIs.

Certain diseases are overrepresented. How many doctors have actually seen a pheochromocytoma or diphtheria or prostitute's pupils (tertiary syphilis)? Yet those are exam-perfect diseases; I've seen all three multiple times.

Real life differential diagnosis involves a list of possible causes for the patient's symptoms, some more likely than others. A test question involves one right answer and four wrong answers. As a result, test questions rely heavily on buzz-words and key phrases. Lyme disease never presents without a target rash. Chagas disease only happens to immigrants from South America. A malar rash in someone on a new drug regimen? Has to be procainamide or hydralazine. An old man can't pee? It's BPH.

Is this medicine?

It'll apparently get you a license, and a good residency too. I know I probably come across as harsh and/or bitter, but I don't think there are any alternatives. And it's actually not bad; pedagogically, we have to start with black-and-white facts (all heart attacks present with crushing left-sided chest pain radiating to neck and back) before we can deal with the gray of real life.

Here's something else. Consider this: an old man with uncontrolled hypertension and polycystic kidney disease comes in with the "worst headache of his life." A CT is normal. What's the next step? It's a lumbar puncture. This is mandatory. Everyone with a suspected subarachnoid hemorrhage and normal CT gets an LP. On a standardized exam though, there could be test-takers who pass who got this question wrong. On an exam, they can pass if they do well on the other questions. In real life, this question cannot be missed.

I don't care if my doctor knows whether herpesvirus is a DNA or an RNA virus (it's DNA), but I do care if my doctor knows I get an LP to rule out subarachnoid hemorrhage (or at least, my lawyers will care after I'm dead). Sorry, I think I overplayed the drama with that one.

There's really no way around it; these are simply the shortcomings of an exam format that tries to assess whether people are ready to make critical decisions about patients' health and lives. These are the intrinsic problems to the USMLE. I'll write (rant) about the extrinsic problems in the next post.