NBME Self Assessments: Ultimate Guide for the USMLEs and Shelf Exams

by Alec Palmerton, MD in Plan

8 comments

The NBME Self-Assessment Services are the cornerstone of evaluating your USMLE progress. Also known as the NBME Practice Exams, they predict your Step 1 and Step 2 CK scores. Versions also exist for Shelf exams, which can be used to predict your Shelf scores.

In my work as a USMLE tutor, they are critical to:

• Establish a baseline,
• Assess for content and/or skills gaps,
• Track your progress over time, and
• Assess the effectiveness of our studying

In this article, you will learn:

• The different kinds of NBME practice exams, and when to use them;
• How much NBMEs may over-/underpredict your final score;
• Basics like where to sign up, how much they’ll cost, and how many exams there are;
• The surprising time you should take your first NBME;
• Whether you should take your test self- or standard-paced;
• Which NBMEs don’t give you an actual predictive score
• Why some students score lower than their predicted score (and what to do about it);
• Much more

Updated: December 1, 2023 to reflect Step 1 being pass-fail, many new CBSSA and CCSSAs, and much more

Table of Contents

Don’t Take Your USMLE Without Taking an NBME Self Assessment

The National Board of Medical Examiners (NBME) writes the USMLE and Shelf exams. To help students prepare, the NBME also releases practice exams. These NBME practice exams are the official practice exams for the USMLEs and Shelf Exams. (The “NBME Self-Assessments” are often referred to as simply “NBMEs.” You will often see them referred to by their number, as in “NBME 25,” or “NBME 31”).

Currently, there are:

• 7 NBME Self Assessments for Step 1 (CBSSA 25-31)
• 6 NBME Self Assessments for Step 2 CK (CCSSA 9-14)
• 1 NBME Self-Assessment for Step 3 (CCMSA 5)
• 29 NBME Subject Exams for Shelf exams

So, what’s the big deal with the NBME Self-Assessments? The NBME tracks students’ scores and measures them against their actual scores. As such, they are the most accurate predictors of your USMLE/Shelf exam scores.

Your USMLE scores are a critical part of your residency application. As such, you shouldn’t take your Step 1 or Step 2 CK without taking at least one NBME Self-Assessment. (And ideally multiple).

The Basics: Cost, Where to Sign Up

• All NBME Self-Assessments cost $60 (see below for more).
• Subject (Shelf) exams cost $20 each.

To sign up:

• Go to the NBME Self-Assessment Services website here.

Anyone Can Sign Up

Essentially, anyone can register for the NBME Self-Assessment Services practice exams.

You do NOT need to have:

• Registered for any of the USMLEs,
• ECFMG certification (for IMGs), or
• Have any verifiable form of credentialing.

NBME Self-Assessment Length

The Self-Assessments are shorter than the corresponding exam (but each NBME block is longer). Each USMLE has 40-question blocks that are 1 hour each; each NBME block is typically longer.

Below are the names for each NBME and what USMLEs they correlate with:

How Predictive Are NBMEs for Step 1 and Step 2?

So, how predictive are the NBMEs for Step 1 and Step 2? Officially, these are the predictive value of the NBMEs for:

• Step 1: 2/3 of students will score +/- 13 points of their NBME from within 1 week
• Step 2 CK: 2/3 of students will score +/- 15 points of their NBME from within 1 week
• Step 3: No predictive score given

These are the official figures. (Note, however, that the NBME no longer publishes these data regularly). However, in my experience, I’ve found that:

Step 1 NBMEs Are Highly Predictive

As discussed above, officially, 2/3 of students’ Step 1 scores should be +/- 13 points of an NBME taken within a week. Here is data from the National Board of Medical Examiners:

Actual Step 1 Scores Minus Score Predicted by NBME Within 1 Week of Exam (Source: NBME)

What do you notice? The highest frequency group is the “0-9” point group. This means the largest group of students saw a 0-9 point improvement from their most recent NBME. It also implies that the Step 1 NBMEs (slightly) underestimate students’ final scores.

In my experience, it is uncommon to see students score more than +/- 5 points differently than their last 2 NBMEs. The difference of +/- 10 points is even rarer.

What does all this mean? Taking your test is unwise if your NBMEs aren’t close to your desired Step 1 score. This is doubly important for those who are at risk of failing. Most students who fail their USMLE have had fair warning. However, when they explain their decision to take the test, they often say they were:

• Hoping to have a good day, or
• They felt the need to meet a particular deadline (usually self-imposed)

Step 1 NBMEs are very accurate. You should expect your final Step 1 score to be close to your most recent NBMEs.

(To read Are You Ready to Take Your USMLE or Need More Time?, click here).

Step 2 CK NBMEs May (Substantially) Underpredict Scores

The Step 2 CK NBME Assessments are less predictive than the Step 1 Assessments. The NBME’s own data support this. 2/3 of students will score within 13 points of their Step 1 NBME but within 15 points of their Step 2 CK NBME.

Dig deeper, however, and the story gets even murkier. Here is the histogram provided from the NBME:

Actual Step 2 CK Scores Minus Score Predicted by NBME Within 1 Week of Exam (Source: NBME)

Take a moment to compare it to the Step 1 histogram above. What do you notice? There is a MUCH greater spread in scores for Step 2 CK relative to the NBME than for Step 1. In other words, Step 2 CK NBMEs may vastly underpredict actual Step 2 CK scores.

Side-by-Side Analysis Shows Degree of Step 2 CK NBME Underpredicton of Final Score

Here are the data presented side-by-side:

Actual Step 1 and Step 2 CK Scores Minus Score Predicted by NBME Within 1 Week of Exam

Viewed side-by-side, it appears Step 2 CK NBMEs likely underpredict your final score. (And probably by a lot). Here are some interesting statistics, according to the NBME’s own data:

The likelihood that your final score is equal/greater than last NBME within 1 week:

• Step 1: 68%
• Step 2 CK: 77%

Likelihood that final score is ≥ 10 than NBME within 1 week:

• Step 1: 31%
• Step 2 CK: 50%

Likelihood that final score is ≥ 20 than NBME within 1 week:

• Step 1: 7%
• Step 2 CK: 23%

Those statistics are pretty remarkable, particularly for Step 2 CK. It implies ~1 in 4 students will score at least 20 points higher on their actual test than predicted by the Step 2 CK NBME. (By comparison, fewer Step 1 students will accomplish this feat).

~1/3 Chance Actual Step 1 Score Will Be Lower. ~1/4 Chance Step 2 CK Score Will Be Lower.

We can do the same analysis for the likelihood that your score will be lower than predicted.

The likelihood that your final score is ≤ 1 lower than last NBME within 1 week:

• Step 1: 32%
• Step 2 CK: 23%

Likelihood that final score is ≤ 11 lower than last NBME within 1 week:

• Step 1: 9%
• Step 2 CK: 8%

Likelihood that final score is ≤ 21 lower than last NBME within 1 week:

• Step 1: 3%
• Step 2 CK: 3%

Here, we can see that the likelihood that there is < 1/3 chance your NBME will overpredict your Step 1 score. (It’s even lower for Step 2 CK).

That’s not to say you won’t have a negative surprise with Step 2 CK. However, based on the NBME’s data, self-assessments likely underpredict Step 2 CK scores.

This squares with my experience. It is uncommon to see someone score +/- 10 points on Step 1 relative to their previous practice exams. However, swings of +/- 10 points are routine for Step 2.

Although imperfect, NBME Self-Assessments are as close to a crystal ball as you’ll find in predicting your USMLE score.

Step 2 CK Score Volatility: Due to Subject Matter?

Why is there such a difference in the predictive power of the Step 1 vs. Step 2 CK NBMEs? Remember, there are fewer NBMEs for Step 2 CK (only 3, as opposed to 6 for Step 1). This may hurt their predictive power.

I suspect, however, that the difference in predictive power is due to the exams themselves. Much of Step 1 involves mastery of content. Having a good/bad day, won’t affect my overall knowledge. Thus, my Step 1 score is likely going to be similar to my prior NBMEs.

(To read How Are USMLE Questions Written? 9 Open Secrets for Impressive Boards Scores, click here).

However, doing well on Step 2 CK involves much more question interpretation. Knowing what each sentence means in context – in a timed setting – is a huge challenge. If you don’t sleep the night before, your ability to interpret the questions will take a hit. This likely causes both the NBMEs and the final scores much more volatile.

(To read The Secret to Scoring 250/260+ You Can Learn Right Now: Question Interpretation, click here).

Shelf Exam Predictive Value Questionable (But Better Than Nothing)

The NBME doesn’t offer data on the predictive power for the Shelf Self-Assessments. There is no official explanation given for why, although we can speculate. The cynical view is that the Shelf NBMEs likely aren’t very predictive. (This has been my experience).

It is not uncommon to see students score +/- 10 from their Subject Exam to their final Shelf score. Why such large swings?

Like with Step 2 CK, Shelf exams depend much more heavily on question interpretation (QI). As we discussed above, QI ability fluctuates day-to-day much more than content knowledge. Thus, you would expect performance on Shelf exams to be much more volatile.

The bigger issue, however, is likely due to sample size. NBME Self-Assessments for USMLEs have ~200 questions each. The Shelf-specific assessments have only 50 questions.

Even the Shelf exams themselves are shorter than the USMLEs. Each Shelf exam is 110 questions, whereas Step 1, which has the fewest questions, has 280 items. Like with all small sample sizes, there will be much more variability.

The Lone Step 3 NBME Doesn’t Give a Predicted Score

There is no score given for the lone NBME. Unlike Step 1 and Step 2 CK, Step 3 has an additional portion: the Clinical Case Simulations. The Step 3 Self Assessment doesn’t have a CCS portion, and so the NBME likely isn’t able to offer a predicted score.

Why Some Students’ Scores Drop Dramatically On Their Real Test

It is rare to score far below your predicted score (see above). You wouldn’t necessarily know that from looking through message boards, though.

It seems like every other post is someone wailing about:

• How their score was so much different than predicted, or
• How the test was NOTHING like any of the UW/NBME questions, etc.

Hysterics aside, there are several reasons why students’ scores may underperform their NBMEs.

Address Anxiety Issues and Don’t Repeat NBMEs

First, remember you should never take an NBME more than once. Repeating the same NBME reduces their predictive power. (Even if it’s been up to a year). I have seen significant (10-15+ points) score overprediction on previously-taken NBMEs. Recalling questions, even vaguely, can change how you take the test, and can inflate your score.

Second, boot-legged copies of NBMEs are unlikely to give accurate scores. Every NBME has a different scale for how much each question is worth. Even if you knew how many items you got wrong, it wouldn’t lead to an accurate score.

Third, statistically, a small number will experience large swings in scores. This is true for normal distributions, although as we saw above, this is rare.

Finally, I’ve known a few students who panicked on their exam. They changed their entire approach (i.e., guessed blindly). Their scores plummeted below their predicted (up to a 70 point drop in one case).

Do NBME Self Assessments Show the Answer and Explanation?

NBME Self Assessments show the answer AND explanation for the Step 1, Step 2, and Shelf practice exams.

How Should You Review Your NBME Self-Assessment?

The NBME used to not give answers OR explanations to questions. They give answers and explanations now. However, their explanations are often lacking when compared to commercial question banks, like UWorld.

As such, use NBME Self Assessment review time to focus on gaps, rather than trying to learn from the questions themselves. In particular, focus on how they are writing questions, as the style is hard to imitate for commercial QBanks.

(To read NBME Self Assessments: Guide to Reviewing Your USMLE Practice Test, click here).

Look for How Many Questions You Miss Because of Interpretation

That said, there is one thing you should do when reviewing an NBME:

Be sure to assess how many questions you missed on topics where your knowledge is already good.

Up to 50% of students’ wrong answers are for “silly” reasons. In other words, they:

• Misinterpret a lab/physical exam finding (or just ignore it),
• Don’t read carefully,
• Misunderstand what the question is asking, or
• Generally, don’t fully apply everything that they know.

Don’t make the common mistake of assuming every wrong question is due to a lack of knowledge. Many of the wrong answers have nothing to do with your depth/breadth of knowledge.

These mistakes of question interpretation are most dangerous if you’re aiming for 240-260+. There is only so much you can increase your knowledge. Much of scoring 240+ deals involves limiting your unforced errors.

Even if you’re trying to just pass Step 1, many wrong questions are not due to a lack of knowledge. You can become aware of this by doing the UWorld or Kaplan QBanks. However, it becomes even more evident when you take an NBME exam. This is especially true since you get an estimate of your score afterward.

(To read The Secret to Scoring 250/260+ You Can Learn Right Now: Question Interpretation, click here).

Standard- vs. Self-Paced?

There are two “Timing Modes” for taking the NBMEs. You can either choose “Standard-Paced” or “Self-Paced.” Generally speaking, the “Self-Paced” exams allow you to take up to 4x as much time per section.

Generally, I recommend doing Standard-Paced, even if you’ve been practicing doing untimed questions. Your exam will be at a standard-pacing. With the Self-Assessments, your goal is to predict your final score. The best way to do is to have your practice approximate the real thing as much as possible.

Source: “Self-Assessments – Common Questions.”  NBME Website: https://www.nbme.org/common-questions/examinees-self-assessments).

When Should I Take My First NBME?

One of the biggest mistakes I see students make is that they wait to take their first NBME. Specifically, they wait until they’ve done a “First Pass” though First Aid +/- UWorld. (In other words, they want to review everything in First Aid and UWorld first before doing their first NBME. I don’t recommend this).

It’s understandable. Students often want to wait to take their first NBME because:

• They’re worried it won’t be “accurate” because they haven’t reviewed enough
• A low score would damage their confidence or
• They already “know” it will be a low score

“What’s the point?” they ask.

Let me be blunt: you should take your first NBME as early as possible. Here are the reasons to take an NBME practice exam at the beginning:

• You need to establish a baseline, but also
• You must know how effective your studying is

Without a Baseline NBME, You Have No Idea How Well You’re Studying

Let’s say that you didn’t plan to take an NBME until 1 month before your exam, and the score on it was 220. Is that good? You have no idea.

Let’s say you’d taken an NBME 2 months before that, and had scored a 180. You would have improved by 40 points in 2 months to a 220: that’d be good progress!

But what if you’d scored a 220 on your NBME 2 months ago, and were still scoring 220 now? In this latter scenario, your score wouldn’t have improved at all. This would indicate an inadequate approach that you should address.

Let’s be clear: you should NOT feel wholly prepared by the time you take your first NBME. That’s not the point! Instead, your first NBME establishes a baseline and gauges your studying effectiveness.

First-Years: Consider Taking an NBME (Long) Before Dedicated Studying

If you’re in your first year of medical school, it’s a good idea to takean NBME early on. This is true even though is pass-fail. Some students even take an NBME in the summer between their first and second year. Doing this can help you understand what studying for the USMLEs involves.

You might think this is strange advice. By summer, you probably haven’t studied all the material yet. So, a practice test won’t predict your final score very well. Then why do it?

The USMLEs are different from most medical school tests. They focus on how well you can apply important concepts, not memorizing minutiae. However, many medical school tests do focus on small details. This can make it hard for students to know how to study for the USMLEs. It’s sad that studying lots of facts might help you do well on medical school tests, but it won’t help much with the USMLEs or being a doctor.

So, where does an early NBME fit in? It’s hard to see how different medical school tests are from the USMLEs until you take a practice test. But once you’ve taken your first NBME, you’ll see the differences. After that, you’ll better understand how to get ready for Step 1.

If you’ve memorized PowerPoint slides for two years, you can’t suddenly start understanding concepts deeply. Taking an NBME early can help you start this shift sooner. This will be good for your scores on the Steps, and it will also be good for your future patients.

Do UWSAs Overestimate One’s Score?

People often argue about whether the UWorld Self-Assessments (UWSAs) overestimate your score. The UWSAs are UWorld’s version of the NBME Self-Assessments.

Much of the time, UWSAs seem to overestimate students’ scores. (You can see students’ actual UWSAs and final scores here). More students have found that their scores were too high than too low.

Even though UWorld has presumably changed the way they calculate these scores, this overprediction has seemed to persist. Why? My theory is that because most people take UWSAs after doing a lot of UWorld questions, they’re used to the style and content of UWorld. This helps boost their score on the UWSA questions written by the same authors. However, when they take an actual USMLE, it can lead to underperformance.

Here’s my advice: don’t take the UWSA scores too seriously. Instead, you can use them to practice for an 8-block test, which is a good idea. To practice for an 8-9 hour exam, you can do one UWSA first, then an NBME. Between the two, I’d use the NBME to guess your score, unless you’ve already used up your NBMEs.

(To read 1-2 Weeks Before Your USMLE? Read This, click here).

How Often Should I Take NBME Self-Assessments?

We’ve already discussed why you should take an NBME early to establish a baseline. But you’re probably wondering: how often should you do an NBME Self-Assessment? The short answer: it depends on your exam and timeline.

Remember, your goal with NBMEs is to track your progress. You do NOT want to run out of them. There is nothing worse than taking your test without knowing what you might score.

Here are my recommendations for how often to take NBME practice tests.

Step 1: Every 2-4 Weeks, Depending on Timeline

I recommend students take an NBME every 2-4 weeks. Why? The general goal is to see if you’re making progress, and if so, how quickly.

Peoples’ scores don’t go up week by week. It takes 2+ weeks to see significant progress.

As such, I don’t recommend taking an NBME more frequently than every other week. If your timeline is more than 6 months, I will wait 1-2 months in-between practice exams. Remember, it’s terrible (and avoidable) to run out of NBME Self-Assessments. Space out your practice tests accordingly.

(To read Want to Add 20-60+ Points to Step 1? Here’s How Long It Takes (16 Experiences), click here).

Step 2 CK: Full Self-Assessments Every 1-2 Months, with Subject Exams Every 1-2 Weeks

Remember, the accuracy of the Step 2 CK Self-Assessments is suspect. This is doubly true for the NBME Subject Exams. As such, don’t read into any single score. Since the scores are less accurate, your goal should be to track your progress.

You can use the NBME Subject Exams (e.g., Medicine, Surgery, or Family Medicine) every week. While the individual scores don’t tell you much, the trend and overall scores are useful. I’ve tutored many Step 2 CK students who scored 260+. Their Shelf NBMEs are usually in the high 80s/low 90s on the Subject Exams.

What Order Should I Take the NBMEs?

So, which NBME Self-Assessment should you take? Which gives the best score prediction? And which exam best simulates the real exam?

The short answer is that no clear evidence exists that one NBME is better than the others. Let me explain.

It Doesn’t Matter Which Exam is “Hardest” Because Each Exam Has Its Own Scale

Each exam has its own scale. This means that questions are worth different amounts of points for various assessments. The upshot? Some tests will be “harder” than others. However, each exam’s scale should take that into consideration.

Sometimes an exam gets a reputation (sometimes deserved) for being a better predictor. Among the old set of Step 1 NBMEs, NBME 18 was thought of as the most “predictive.” However, in March 2019, all but one of the Step 1 Self-Assessments were replaced. They did the same thing in March 2021.

Currently, no evidence suggests one NBME is better than another. This leads me to my next point: you should take the lowest-numbered NBME first.

Take the Lowest Numbered Exam First (It Will Be Retired First)

I recommend starting with the lowest-numbered exam first. (E.g., start with NBME 25, then move to NBME 26, etc.). Why? Typically, the NBME replaces Self-Assessments beginning with the oldest first. If you’ve taken an NBME that will be replaced, you get a “bonus” NBME to do. If not, you’ve replaced one unseen NBME with another one. (See more below).

Can I Repeat My NBMEs?

If you can help it, you don’t want to do the same NBMEs again. Even a year later, I’ve seen NBMEs overpredict by 10-20 points. So, plan to take an NBME 2 weeks before your final test. Spread out any other exams you haven’t done yet.

The old NBMEs you took will be your new starting point. You can see how much you’re improving in your further dedicated study time compared to them.

When Does the NBME Release New Self Assessments?

Typically the Step 1 NBMEs are updated in March. Most recently, five CBSSAs (Step 1 NBMEs) were replaced in March 2021.

• There were no Step 1 Self-Assessments released in 2018
• 5 of the 6 Self Assessments were replaced in March 2019
• 5 of the 6 Self Assessments were replaced in March 24, 2021

The NBME seems to release Step 2 CK Self Assessments less frequently, although they released a few more once Step 1 became pass-fail. There isn’t a clear pattern.

There is still only a single USMLE Step 3 NBME.

Taken All the NBMEs Already? Here Are Your Options.

If you’re in the unfortunate scenario of already using all of the NBMEs, you have a few options.

1. UWSAs Are Your Second Best Option Behind the NBME Self Assessments

First, you can use the UWSAs. Again, these likely overestimate your score, but they’re better than nothing.

2. Repeat NBME Practice Exams with Caution

You can try to do the NBME Self Assessments again. I only suggest this if it’s been at least 6 (or even better, 12) months since you last took the exam. It’s best if you never looked over the NBME after taking it.

If you do the Step 1 CBSSAs again, I would suggest subtracting 10% from your score as a safe guess if you took it in the past six months, especially if you looked over the questions afterward. If you do the Step 2 CCSSAs again, I would suggest subtracting 20 points from your score as a safe guess if you took it in the past six months, especially if you looked over the questions afterward.

Is taking away 10% and 20 points from the CBSSA and CCSSA too much? Maybe. But, with Step 1 being pass or fail, the goal is to take it and pass for sure. And since Step 2 is even more important, it’s always better to be safe than to be sorry.

3. Beware of Online Score Converters/Calculators, Especially If You’ve Done Multiple Passes

Third, you can try various online score converters/calculators. These take multiple variables (e.g., UWorld scores) and predict your actual score.

In my experience, it is the least accurate method. It would be my last resort, especially if you have repeated UW or your other QBanks.

It’s fine to repeat QBanks. And QBank scores might predict your final score. However, having already studied the questions will likely limit their predictive power.

“Offline” Bootlegged NBME Practice Exams: Accurate Predictor or Waste of Time?

NBME Self-Assessments can be expensive. It costs $60 for a practice test and $20 for the Subject exams. There are ways to change scores for offline NBMEs to match each exam’s scoring scale. But should you use the unofficial NBME copies that you can find online?

I suggest using the official NBME Self Assessments, not the “offline” NBMEs. (I don’t get any money from saying this). Why? Because an NBME is more than just a score. It also helps you find and focus on the subjects you need to work on.

For example, students who find the NBME style questions often struggle with “General Principle” questions. These questions test how well you can apply basic concepts. At first, they might seem like a bunch of random small details. But, when you can group the questions under the “General Principles” section, it’s much easier to see the pattern and practice those questions.

Concluding Thoughts: Don’t Take Your Test Until You’re Ready

The biggest mistake I see students make is they take their test and hope for a miracle.

Don’t Take Your Test If You Might Fail

One prominent example is students who fail their USMLE. While some students have scores tank on their exam, they’re in the minority. What’s more common is students feeling an urgency to take their exam. Typically the student feels external pressure to take their test, either because:

• Their school has set an arbitrary deadline (and they don’t ask for an extension)
• Friends/family keep asking them, “will you just take your test already?”
• They’ve already filed for an extension of their test permit. (And delaying further would cost extra money).

None of these reasons justify taking your test if you have a high risk of failing. Yes, failing a USMLE isn’t the end of the world. However, it will shut the door at many residency programs.

(To read Step 1 Fail: A Blessing in Disguise?, click here).

Ask Yourself: “Would I Be OK With My Score?”

Don’t take your test if you’re at high risk of failing. Just don’t.

If you’re above passing, but below your goal score, it’s a personal decision whether to take your test or not. In other words, determine if your score is within the range of your goals. You can find the average USMLE scores by specialty for US students and IMGs here.

If you’re not sure whether you should take your test, consider these articles:

• Are You Ready to Take Your USMLE or Need More Time?
• 1-2 Weeks Before Your USMLE? Read This

What do you think? Let us know in the comments!

Photo by Elijah Hiett.