250+ Mentality: What Basketball Teaches Us About USMLE Timing

by Alec Palmerton, MD in Uncategorized

2 comments

They say that if you have a hammer, everything looks like a nail. Medical students are guilty of this mentality. They get so used to approaching everything as if it’s a problem of knowledge. Not doing well on my med school exams? Need to cram more knowledge. Got a question wrong on UWorld? Learn more facts.

We are most guilty of this in our approach to the USMLEs. Have you ever sought out the most “comprehensive” source available? Or asked if something covered “everything I need to know for Step 1?” If so, you’re not alone. We’re all guilty of the knowledge-is-everything fallacy.

Don’t get me wrong – knowledge is essential. But focusing solely on information will hurt your score.

Scoring 250+ on the USMLE requires 3 things:

• Content mastery: excellent understanding of concepts
• Question interpretation: the ability to apply knowledge in a clinical context
• The right mentality

I’ve written extensively about the first and second points. Today, I want to discuss the mentality that is critical to scoring high on the USMLE. What’s more, this mentality can help you, regardless of your goal score.

In this article, you will learn:

• The mentality you need if you want a 250+ score
• What basketball can teach you about score outcomes
• Why easy and hard questions aren’t actually worth the same number of points
• How many points you’re missing because you can’t let hard questions go, and
• Much more

What Basketball Can Teach You About Improving Your Scores

To understand this mentality, I want to discuss basketball. Yes, basketball.

At first glance, there are few similarities between basketball and the USMLEs. Both involve “scores.” However, one is a fast-moving sport full of athletic/tall people. The other a high-stakes exam for medical students.

However, the mentality of basketball teams can help test-takers. Specifically, basketball can teach us better strategies for maximizing our scores.

Like 2-Point Shots, Not Every Question Is Worth the Same

One critical similarity between basketball and USMLEs is the scoring system. Made shots in basketball range in point value. Some are 1 point, some are 2-points, and some are 3-points. Within each category, each shot is worth the same amount.

But are all 2-point shots worth the same amount? In fact, no.

Let’s say I’m Shaquille O’Neal, a famous former NBA player. He’s renowned for being both:

• Dominant anywhere within 3 feet of the basket
• A below-average shooter

If Shaq dunks the ball, it’s worth two points. If he shoots a 2-pointer from just within the 3-point line, it is also worth two points.

However, in the context of the game, those two shots are not worth the same! Why? Because the likelihood of making a dunk is significantly higher than a long 2-pointer. In fact, we can calculate the NBA-wide expected value of each shot.

Expected value of each 2-point shot:

• Within 3 feet of basket = 1.3 points/shot
• Mid-range (farther from basket) = 0.8 points/shot

Yes, the shots are nominally worth the same amount. I don’t get any extra points for making a 2-point shot near the basket. However, the likelihood of making those shots is higher. Thus, their expected value is higher.

Likelihood of Getting USMLE Questions Right Determines Expected Value

USMLE questions have the same calculus. Each item is nominally worth the same. However, because the likelihood of getting them right differs, their values diverge.

How do we calculate a USMLE question’s expected value? Simple:

Point value of item x likelihood of getting it right

However, since each question is worth 1 point, we can say that:

Expected value = likelihood of getting it right

But how do we estimate the likelihood of getting it right? By looking at the average for other students’ getting that question right. In every QBank, each item will tell you:

• The correct answer
• How many chose the wrong answers
• The average number of people who selected the right answer

In basketball, more challenging shots have a lower % of made attempts. In the USMLEs, more difficult questions have a lower average correct.

For a 10-question block, the percentages might be:

• Question 1: 37% of students got it right
• Question 2: 82% of students got it right
• Question 3: 54%
• Question 4: 45%
• Question 5: 68%
• Question 6: 27%
• Question 7: 95%
• Question 8: 38%
• Question 9: 75%
• Question 10: 59%

In this block, the expected outcome would be:

0.37 + 0.82 + 0.54 + 0.45 + 0.68 + 0.27 + 0.95 + 0.38 + 0.75 + 0.59 = 5.8

Like in Basketball, Easier Questions are Worth More!!

This leads to several interesting conclusions. In basketball, 2 point shots all worth the same. But the “easier” ones are more valuable.

USMLE questions are the same. The items you have a higher likelihood of getting right are worth more. So what does that mean?

Like in basketball, more straightforward questions are worth more points! Especially if you struggle with timing, spend less time agonizing over harder questions. Let me show you an example.

Like in basketball, harder questions (shots) are worth less. Why? Because the probability of success is lower, their expected value is less.

Why Too Much Time on Hard Questions Hurts Your Score

NBA teams try to maximize the most valuable shot: shots at the rim. They work very hard to eliminate long 2-pointers. Their rationale is simple. The more high-value shots they take, the higher their expected score.

The same is true for USMLE questions. As illustrated before, more straightforward questions have a higher expected value. Conversely, harder questions have a lower expected value.

Yet so many of us perseverate on difficult questions. Some people spend as much as five or six minutes on a question! Even if they get it right, that will hurt their overall score, as I’ll argue below.

Let’s assume on a 10-question block, there are:

• 20% “Hard” questions (20% likelihood of getting it correct)
• 60% “Medium” questions (50% likelihood correct)
• 20% “Easy” questions (80% likelihood correct)

Each USMLE question should take, on average, about 90 seconds. But what if we spend extra time on the hard items? Let’s say we spend 300 seconds on one hard question and 120 seconds on the other. Spending 240 seconds additional, reduces the time/question of the remaining items from 90 to 60 seconds.

Time Spent on Questions Affects Likelihood of Getting it Right

However, because of the time limit, there is an opportunity cost. In other words, by spending more time on some questions, we take time away from others.

The more time spent on a question, the higher the likelihood of getting it right. The less time you spend on a problem, the lower your chances of getting it right. This should make intuitive sense.

But how much of a boost would spending more time have on a hard question? And how much would it decrease our odds of getting other questions correct? I can only speculate, however, conservative assumptions would be:

• Spending extra time on the hard items improves our odds by 10%
• Having less time for the remaining eight questions reduces the odds by 5%

By not moving past hard questions, your score may be 10+ points lower!

What are the results? By spending extra time on hard questions, we reduce our expected score by 4%! On an NBME, 4% fewer corrects would equal 8 items or about 10 or 11 points!

In other words, perseverating on hard questions may reduce your score by at least 10 points.

See What Your Average Score is When You Reduce Your Time/Question

Again, here are my assumptions:

• Spending extra time on a question gives us a 10% boost
• Less time on other questions makes it 5% less likely you’ll get the question right

Don’t agree with my assumptions? You can test this yourself! Give yourself only 60 seconds per question. Then look at your average for that block relative to different sections. You’ll likely find that your score is more than 5% less than if you gave yourself a full 90 seconds. (It perhaps goes without saying that 90 seconds is 50% more than 60 seconds).

Why is it So Hard to Let Go of Harder Questions?

We may know in theory that perseverating on harder questions hurts our scores. So why is it so hard to put into practice? Here are the two most common reasons:

1) Pride

The worst offenders are the ones we think we “should” get, but are stuck between two choices. We scan the question stem frantically looking for any possible clues. Meanwhile, time continues to pass.

If you just studied heart failure, you’ll feel guilty about not getting an MI question correct. “But I KNOW this!!” we think to ourselves.

Remember, even for topics we know, there will be questions we can’t answer. It doesn’t mean you’re a terrible person. It just means your normal.

2) Sunk Cost Fallacy

Have you ever thought, “I’ve already spent so much time on this question. I can’t move on – I MUST get this question right!” If so, you’ve fallen victim to the sunk cost fallacy.

In sunk cost, we make decisions based not on their expected utility. Instead, we spend more time/resources on the things we’ve already invested the most.

The problem? The questions we’ve spent so much time on already are the ones with the lowest expected return. By spending more time on them, we deprive the higher-value items of the time they need.

Concluding Thoughts

Like in basketball, the rational USMLE timing approach isn’t intuitive. We may know that the easy and hard questions are nominally worth the same amount of points. However, it is unintuitive to think that the expected value of simple items would be higher.

But what else can you do to improve your score? You want to make as many “hard” questions easier. In other words, you want to turn your long 2-pointer into layups/dunks.

Question interpretation – knowing what every sentence means – improves each question’s odds. Understanding the rationale behind every word skyrockets your chances of getting that question.

You can read more about question interpretation here.

If you’re ready to take the next step, sign up for the Online Course. Here, you’ll not master critical concepts, but apply them to making every question easier. Learning the meaning behind every vignette, you’ll maximize your score in less time.

To join the Online Course now, click here.

Photo by Chris Moore