Flashcards for Impressive Step 1 ScoresBUY NOW
Flashcards for Impressive Step 1 Scores

blog

NBME Self Assessments: Ultimate Guide for the USMLEs and Shelf Exams

Have you mastered microbiology yet??

Sign up to receive a FREE micro starter deck of more than 130+ Anki flashcards that I used to get >85% on microbiology on USMLE World. You'll be begging to get microbiology questions on your exam!

Subscribe
by Alec in Plan

The NBME Self-Assessment Services (informally known as the NBME Practice Exams) are the cornerstone of evaluating your progress for your USMLE preparations.  In short, they are the exams that are the most predictive of what your true USMLE score will be, if you were to take your exam at a particular time.  In my work as a USMLE tutor, they are critical to both establish a baseline, as well as to track your progress over time/assess the effectiveness of our studying.

Here I address the most common questions people have for the NBME Practice Exams, as well as the surprising time you should take your first NBME.

What are the NBME Practice Exams?

The NBME Self-Assessment Services provide the official practice exams for the USMLE Step 1, Step 2 CK, Step 3, and Shelf exams.  They are scaled internally, and correlated against students’ actual scores, and as such they are the most accurate predictors known of what your USMLE/Shelf exam scores will be.

They are created by the National Board of Medical Examiners (which is how they get their name), which is the same organization that develops and manages the USMLE.  To read more about the National Board of Medical Examiners, click here.

In other words, the NBME Self-Assessments are THE practice exams for the USMLEs and Shelf Exams.

How much do they cost?

All NBME Self-Assessments cost $60 (see below for more).  Clinical Mastery Series exams cost $20 each.

How do I sign up for the NBME Practice Exams?

Go to the NBME Self-Assessment Services website here: https://nsas.nbme.org/home

What do I need to sign up?

Essentially, anyone can register for the NBME Self-Assessment Services practice exams. You do NOT need to have registered for any of the USMLEs, do NOT need to be ECFMG-certified (for IMGs), or have any verifiable form of credentialing.

Which NBME Practice Exams to take for Step 1? Step 2? Shelf exams?

The nomenclature is confusing for these practice exams.  For some odd reason, the names for the practice exams don’t match up with the exams they are supposed to represent.  Below are what the exams are called on the NBME SAS website, and what exams they correlate with:

Comprehensive Basic Science = USMLE Step 1

Comprehensive Clinical Science = USMLE Step 2 CK

Comprehensive Clinical Medicine = USMLE Step 3

Clinical Mastery Series = Shelf Exams

What kind of feedback will you get from your exam?  How useful is it?

As of writing this article, each of the NBMEs (minus the CMS exams) will show you the correct answer, however you will NOT see any explanation of why your answer or the others were incorrect.

In my experience as a USMLE and Shelf tutor for a number of years, every single student I have worked with has gotten a substantial number of questions (think 20-50%) wrong not because they don’t know the information, but rather because they make what I call “silly mistakes” – they misinterpret a lab finding, don’t read carefully, or generally don’t fully apply everything that they know.  When you read through the answers afterwards, it is often apparent that many of the wrong answers have nothing to do with your depth/breadth of knowledge.

This is true whether the student ultimately was aiming to pass or if they scored 260+ – you’re leaving tons of points on the table for reasons beyond a lack of knowledge. You can certainly become aware of this from doing UWorld or Kaplan questions, but it becomes even more clear when you take an NBME exam, especially since you get an estimate of your score afterwards.

Standard- vs. Self-Paced?

There are two “Timing Modes” for taking the NBMEs.  You can either choose “Standard-Paced” or “Self-Paced.”  Generally speaking, the “Self-Paced” exams allow you to take up to 4x as much time per section.

Generally, I recommend doing Standard-Paced, even if you’ve been practicing doing untimed questions.  My rationale is that your exam will be at a standard-pacing, and since your goal is to know what your score would be if you took your test now, the most accurate way to do that is if you take it in close to exam settings.

NBME Practice Exams Standard- vs. Self-Paced

Source: “Administration Period.”  NBME Website: https://nsas.nbme.org/nsasweb/doc/administration_periods.htm

How predictive are they of the final score?

For Step 1: There is no study that I have seen looking at the predictive value of NBMEs vs. students’ final scores.  That said, at least for Step 1, the NBMEs are quite accurate; it is uncommon to see a student score more than +/- 5 points differently than their last 2 NBMEs and rare to see a difference of +/- 10 points.  As a corollary to this, if you are shooting for a 240+ and are only scoring 200+ on your NBMEs, and are hoping against hope that you will magically improve your score on your real exam, you would be the exception and not the rule.

For Step 2: I have seen much weaker predictive power for the NBMEs for Step 2 CK.  Whereas it is rare to see someone score +/- 10 points on Step 1 relative to their previous practice exams, it is routine to see this for Step 2.  I’m not sure why this is – perhaps many fewer students take NBMEs for Step 2, which hurts the predictive power, or because so much of the exam depends on your interpretation abilities that having a “good” or “bad” test day could have a much larger impact than Step 1.  Regardless, proceed with caution, and give yourself an extra cushion if you are at risk of not passing.

For Shelf Exams: Their predictive value is closer to the NBMEs for Step 2 than for Step 1.  It is not uncommon to see students score +/- 10 from their CMS to their final Shelf score.  This is likely due to small sample sizes of questions, both for the CMS and the actual Shelf Exams themselves.  The CMS/Shelf Exams have about 1/4 the number of questions as the standard NBME/USMLE – a stray guess/miss will have a much larger impact on the scores here.

For Step 3: There is no score given for the lone NBME.  This is likely because a portion of your grade is based on Clinical Case Simulations, which are NOT a part of the NBME.

Why do some students score substantially less than predicted than their NBMEs?

It is rare that someone’s score is substantially below their predicted score, especially for Step 1 (see above).  You wouldn’t necessarily know that from looking through message boards, though; it seems like every other post is someone wailing about how their score was so much different than was predicted, and how the test was NOTHING like any of the UW/NBME questions, etc.

Hysterics aside, there are several reasons why students may score substantially below their predicted NBME scores.  First, if students used boot-legged copies (see below), or repeated the same NBME exams, their predictive power is invalidated in my experience.  Particularly when a student repeats an NBME, even more than a year after they took it the first time, I have seen significant (10-15+ points) overprediction of one’s score on previously-taken NBMEs relative to students’ final scores.  Recalling questions, even vaguely, can change how you take the test, and can inflate your score.  Second, in large enough bodies of people in a normal distribution, you would expect a small percentage, by random chance, to score substantially differently than their predicted scores.  Finally, I have known students who were experiencing (and not addressing) higher levels of underlying anxiety who panicked on their exam, changed their entire approach (i.e. guessed blindly), and who scored substantially below their predicted (up to a 70 point drop in one case).

Should I take my first NBME before my “first pass” through material?

Yes, unambiguously.  One of the biggest mistakes I see students make is that they wait to take their first NBME.  It’s understandable: they feel like it wouldn’t be “accurate,” or that they would be very discouraged because they already “know” it will be a low score.

“What’s the point?” they ask.

The point: not only do you need to establish a baseline, but you ALSO must know how effective your studying is.  Let’s say that you didn’t plan to take an NBME until 1 month before your exam, and the score on it was 220.  Is that good?  YOU HAVE NO IDEA.

If you’d taken an NBME 2 months before that, and had scored a 180, and had improved by 40 points in 2 months to a 220, that’d be good progress!  But what if you’d scored a 220 on your NBME 2 months ago, and were still scoring 220 now?  That same score would be, in fact, representative of something quite different – poor studying.

Let’s be clear: you should NOT feel completely prepared by the time you take your first NBME.  That’s not the point!

UWSA vs NBME for predictive value – does UWSA overestimate one’s score?

There is a fair amount of debate amongst students about whether the UWorld Self-Assessments overestimate one’s score.  Generally, my experience with those has been mixed (mine were an underestimate, but then again they only go up to 265; for some of my students it has been an underestimate, while a couple have seen an overestimate), although generally I have seen a trend towards overestimation of the UWSAs by at least 10 points for a large number of students.

My general advice is to take the UWSA scores with a grain of salt, and to not put too much stock in it.  If you want to do an 8-block test to simulate a USMLE Step 1 type environment, do one UWSA first, followed by an NBME, and use the NBME as an estimate of your score.  You may not even want to review 8 whole blocks, so you could even do the NBME as “Standard” to save $10, and review the UWSA!

When/how often should I take my NBME exams?

I took an NBME exam every 2-4 weeks, which worked well for me and is what I recommend for my students.  The general goal is to see if you’re making progress, and if so, how quickly.  I don’t recommend much more frequently than one NBME every other week, since as discussed previously, your progress usually doesn’t vary much week to week – it takes 2 or more weeks to really see if you’re making significant progress.  Plus, from a learning perspective, if you’re going to review them on your own, you would probably learn more from just doing 4 blocks of USMLE World questions, and studying from the explanations which are designed to teach you.

Which NBME Practice Exam should I take?  Which are most like the real exam?

USMLE Step 1 has evolved over the years.  It used to be much more knowledge-based – either you knew the fact, or you didn’t, but the tests now are much more clinical vignette based.

NBME 7 was the last exam that was still like the “old” test (more memorization/fact-heavy) and has since been retired.  Everything from NBME 12 onward is much more like the “real” exam – longer question stems with more complicated clinical vignettes.  They are all scaled internally, and correlated with what students’ true exams scores were, so even if one test is “harder” or another is “easier” this will be reflected in the curve, as students will either have more or less answers correct.

As such, if you are planning to take all of the NBME practice exams over the course of your studying, I recommend starting with the lowest-numbered exam, and to take the later NBMEs closer to your exam (e.g. starting with NBME 13 first, then doing 14, etc.).  Not only will that save the NBMEs that are likely more similar to the exam for closer to the exam, but also for Step 1, if your study period encompasses March, your exam will be retired and you will have another one available (See “MED SCHOOL HACK” below).

If I take a Clinical Mastery Series (CMS) exam, should I search message boards for all the answers?

This is a personal decision, but generally I don’t recommend going crazy and trying to find all of the answers to questions from your CMS Shelf practice exam.

“But my friend told me that s/he had a question that was EXACTLY like one that she saw on her CMS exam – doesn’t that mean I should memorize all the questions and answers from that practice test?”

I don’t doubt that there are some questions that overlap – I actually found a question that was word-for-word from the “Free 150.” Does that mean you should memorize those questions too? The problem is that in ANY collection of 150-200 questions, you are likely to have at least one that will overlap significantly with a question on your test, whether it is 5 blocks of Kaplan questions, so the fact that we’ve all had overlap with these tests doesn’t mean you should go out and memorize the answers to all the NBME practice tests.

Time is a scarce resource, and spending the (often) hours necessary to review all of the questions from the CMS by searching through forums is time that I simply used to do more Kaplan and USMLE World questions.

If you DO decide that you MUST know all of the CMS Shelf exams, I would suggest either a) setting a time-limit for yourself of how long you will spend per question/total, so as to not go overboard, and/or b) go over the test with someone you trust to know the answers.

I delayed my Step 1 date by a significant amount of time (6+ months) – when should I re-start the NBMEs?

Life happens, and plans change – some students are forced to delay their dedicated studying/Step 1 exam by significant amounts of time.  If you are in this position, when should you re-start taking your NBMEs?

You may have already used a significant number of them, so this is an important question since you want to use your remaining practice exams most effectively.  As discussed before, you do NOT want to repeat old NBMEs, even if it has been over a year since you took them last, since the repeated NBMEs will be of questionable predictive value.

You should re-start your NBMEs when you are closer to your dedicated studying – something around 2-4 weeks prior.  If you delayed by a significant period of time, it likely means you are not able to focus effectively on your studies, so tracking your progress via NBMEs will be of limited use, and must be balanced with not running out of NBMEs.  As for the previous NBMEs you took, they will serve as your previous baseline – measure your progress in your new dedicated studying relative to them.  Also, ask yourself what you will be doing differently the next time, and how to give yourself sufficient time/preparation to expect a different result the second time.

NBME 19 (or the most recent): should I take this right before my exam?

As of this writing, NBME 19 is the most recent NBME released.  As such, there is a bit less known about it.  It’s up to you, but if it were me, I would probably do NBME 18 before my test, and do NBME 19 as the second-to-last exam.

When are the NBME Practice Exams released?

Typically, one new USMLE Step 1 NBME is released every year in early March.  USMLE Step 2 NBMEs seem to be released less frequently and so there isn’t a clear pattern.  There is still only a single USMLE Step 3 NBME.

What if I’ve already taken all of my NBMEs?  Can other things be used as a substitute?

If you’re in the unfortunate scenario of having already used all of the NBMEs, you have a few options.  First, you can use the UWSAs, which again, are likely an overestimate of your score, but better than nothing.  Second, as long as you took the NBMEs at roughly the same time, and at least 6 (ideally more) months have passed since you took them, you can use the NBMEs as a rough estimate of improvement.  In other words, if you repeated NBMEs 17 and 18, and their scores initially were 160 and 170, but on repeat 6 months later they are 200 and 220, I wouldn’t trust the actual score of 220.  However, I would take the improvement of 20 points as a good sign, and as a rough estimate of your level of improvement.  Third, you can try the various online score converters/calculators that will take your UW percentages, Kaplan diagnostic test score, or “Free 150” scores and convert them to an actual score, although this in my experience is the least accurate method, and would be a last resort, especially if you have repeated UW or your other QBanks and the percentage you are using is the one from the second time through.

BONUS TIP

“Offline” bootlegged NBME Practice Exams: accurate predictor or waste of time?

As mentioned before, each exam is scaled differently, so a 90% on one will convert to a different score than 90% on another.  I know of no such set of score converters for each NBME exam, and as such, anything you are likely to find online that is not the official NBME SAS score will likely be a poor estimate of your true score.

I’m all for saving money, but as someone once told me, “The USMLEs aren’t the place to be cutting corners.”  This advice applies to the NBME Self-Assessment Services.

How long will it take for me to raise my NBME exam scores by 20-40 points?

This is a question I get a lot.  Here are 4 students’ scores (including mine) graphed over time who used the Yousmle Method, to give you an idea of how much time it has taken others that I have worked with.

NBME UWSA Score Graph USMLE Step 1

My NBMEs Graphed Over Time

You can read more about these students’ experiences here.

 

Have you mastered microbiology yet??

Sign up to receive a FREE micro starter deck of more than 130+ Anki flashcards that I used to get >85% on microbiology on USMLE World. You'll be begging to get microbiology questions on your exam!

Subscribe