When preparing for Step 1 and Step 2 CK, most people use UWorld and the UWorld Self-Assessments (UWSAs). But how similar are UWorld and the UWSAs to Step 1 and Step 2 CK? And how effective are they in helping you prepare?
You will often hear people after their tests say something like, “my test was NOTHING like UWorld!” Others will swear that there is a remarkable similarity. What is the truth?
This article will discuss:
- The rules for making Step 1 and Step 2 CK questions,
- How Step 1 and Step 2 CK questions are different,
- How well UWorld and UWSAs match the Step 1 and Step 2 CK formats,
- The concepts of “signal vs. noise” that can help you analyze any question, and
- Much more
How Are USMLE Questions Written?
When we prepare for the USMLEs, we want our preparation to mirror the real thing as much as possible. This helps us solve the problem of “transfer.” Transfer means that once we learn a concept in one context (e.g., the classroom), we can use it in another (e.g., the real world). It is useless if we can spout off a random fact we crammed if it doesn’t help us on actual exams.
As such, you may be wondering how the NBME writes the USMLE questions. The short answer is that USMLE questions are nothing like most med school exams. Med schools tend to focus on rote memorization due to the horrible system of incentives that makes good teaching nearly impossible. Med schools use busy, overworked professors who don’t receive a salary for teaching. The result? They write the most expedient questions – the ones they can scrape from their lecture slides.
The USMLEs Are Nothing Like the Memorization-Heavy Med School Exams
These fact-heavy questions miseducate medical students. Most students end up thinking that memorization is all they need. Instead, the USMLEs blindside them by focusing on the application of concepts.
This focus on concepts is particularly true for Step 1. One Step 1 question-writer I spoke with claimed that half of the pathology questions on Step 1 are applications of general principles. This is in stark contrast with the strategy many med students follow of cramming facts and hoping to spot memorized buzzwords.
Signal vs. Noise
A critical concept to understand for the USMLEs is “signal vs. noise.” “Signal” is basically anything that purposefully leads you to the correct diagnosis/answer. For example:
A 56-year-old man is brought to the emergency department with crushing chest pain radiating to his left shoulder. He has a history of type 2 diabetes, hypertension, and hyperlipidemia. He has a 30-pack-year smoking history. An EKG demonstrates ST-elevations in leads II, III, and aVF.
So this guy has an MI. How do we know? Well, everything in the vignette points to that diagnosis – his coronary risk factors, his symptoms, his EKG, etc. In other words, this vignette has only “signal,” and no “noise”.
“Noise” Makes It Harder to Analyze a Question
What would noise be, then? Noise would be anything that does NOT lead you to the correct diagnosis/answer. Let’s take the same example, and mix in some “noise”:
A 56-year-old man is brought to the emergency department with chest discomfort for the past 3 hours. He recently returned from a nonstop flight from Seoul to Los Angeles. His past medical history includes diabetes, Marfan syndrome, and mitral valve prolapse. He drinks 2-3 beers on the weekends, and doesn’t smoke or use illicit drugs. Chest x-ray is unremarkable. An EKG demonstrates ST-elevations in leads II, III, and aVF.
What did you notice about the second vignette relative to the first? There is a lot more “noise” in the second one. In fact, there were things added in there to confuse you – these give you enough information to raise the possibility for an alternative diagnosis.
In addition, the “signal” was weaker. Gone is “crushing chest pain radiating to his left shoulder.” Now it’s simply, “chest discomfort.”
So, how do these vignettes compare with what you might see in Step 1, Step 2 CK, UWorld, or the UWSAs?
Step 1 = Almost All Signal and No Noise
The first vignette is similar to many Step 1 questions in that it is almost all signal, an no noise. Almost everything in Step 1 questions is there for a reason.
Step 2 CK = Lots More Noise, Weaker Signal
Step 2 CK, in contrast, is much more like the second vignette. The nonstop flight? Meant to make you wonder if the chest pain might be a DVT/PE. Marfan syndrome? Increases the risk of an aortic dissection. Mitral valve prolapse? A common sequela of Marfan, and can cause chest discomfort in its own right.
What Do UWorld/UWSAs Do Well?
UWorld rose to fame because its questions closely modeled Step 1 questions. They did an excellent job of writing questions that made you think and apply concepts – just like Step 1 requires.
Many of the questions still do a good job of forcing you to think. In the other QBanks I’ve reviewed (Kaplan, Amboss, USMLERx, etc.) many of the questions focus on knowing details. While this is important, I have been less impressed with the non-UWorld questions’ ability to force you to apply concepts. This focus on minutiae is particularly pronounced with USMLERx in my experience.
UWorld, in contrast, has more concept-type questions mixed in, which is its greatest strength.
Additionally, UWorld Step 2 CK does an excellent job of adding “noise” to the questions. Shelf- and Step 2 CK-takers need to contend with the increased “noise” and weaker “signal” in the questions. Just like in real clinical scenarios, not all the data you receive is relevant.
UWorld mirrors this increase in “noise” better than other QBanks I’ve seen.
Is UWorld Harder Than Step 1?
Many students feel the USMLE/NBME questions are more “vague” (more on this below). However, UWorld does seem to be harder than Step 1 in one respect. One thing to note is that UWorld Step 1 seems to have more “noise” than is found in most Step 1 questions.
In other words, while the clinical findings in a standard Step 1 question should be straightforward, UWorld does seem to include more unnecessary/unhelpful “noise” in their Step 1 Qbank. While it is likely to help students transition to Shelf exams and Step 2 CK, know that for the most part the NBME isn’t trying to trick you. (At least on Step 1!).
Can UWorld Maintain Its Quality While Increasing the Number of Questions?
While UWorld has done a great job of adding more conceptual-type questions, the question quality seems to have dipped in recent years. Note, this is my own personal opinion in reviewing questions together with students. I’ve noticed that since they’ve expanded the number of questions in the QBank, the quality seems to have dropped.
The biggest way I notice the quality dropping is that they seem to have added many more questions that focus on minutiae. Particularly for Step 1, the hardest questions are often NOT the ones that focus on factual recall. Instead, it is the concept application-type questions that haven’t kept up in terms of number.
For Step 2 CK, there are fewer of these concept application-type questions. Instead, Step 2 CK is much more about question interpretation – being able to analyze/interpret clinical scenarios. UWorld continues to use a lot of “noise” effectively to mirror the Step 2 CK format.
Is the Wording on UWorld Different From the USMLEs?
One noticeable way that UWorld and other QBanks differ from the USMLEs is the wording. Students often note that they have a harder time understanding what NBME and USMLE questions are asking. This difficulty understanding the wording is especially true when transitioning from doing UWorld.
If you find the NBMEs to be harder to understand, strongly consider doing more of them to practice. There are currently six Step 1 NBMEs. While there are only three Step 2 CK NBMEs currently, there are plenty of subject-specific NBMEs for the shelf subjects. For example, there are four shelf NBMEs for EACH of neurology, internal medicine, surgery, pediatrics, Ob-Gyn, psychiatry, and many others. Give yourself enough practice so that you can remain flexible in your ability to answer these questions.
How Well Do the UWSAs Predict Final USMLE Scores?
A final question is how well UWSAs predict final USMLE scores. While there is no official tally, historically UWSAs have tended to overpredict USMLE scores. (In other words, UWSAs have a reputation of predicting scores that are higher than the final score). It is difficult to verify these claims. However, in my experience, I have noticed that often the UWSAs did overpredict scores. This was particularly true for those students who had done a lot of UWorld questions. (Not surprising since presumably the authors’ foci and style are very similar).
So what should you do if you are worried about the UWSAs predictive power? Our advice would be to simply take an NBME to corroborate the findings. Relying on any single indicator of your score is iffy.
Concluding Thoughts
Test writers shape Step 1 and Step 2 CK questions to test our “transfer” ability. Can we actually USE the equation we memorized? Do we know how to interpret the direct vs. indirect bilirubin in the context of a nauseous patient with fever?
QBanks – and UWorld in particular – are excellent ways to address the problem of “transfer” in medicine. In knowing the similarities, strengths, and weaknesses of UWorld, you can form a more effective USMLE preparation plan.
How similar do you find UWorld with the NBMEs, Step 1, and Step 2 CK? In what ways do you think they are different? Let us know in the comments!
