The Perils of “Survivorship Bias”

From Katy Milkman, PhD, in The Scientific American:

My colleagues and I, we’ve been spending a lot of time looking at medical decision-making. Say you walk into an emergency room, and you might or might not be having a heart attack. If I test you, I learn whether I’m making a good decision or not. But if I say, “It’s unlikely, so I’ll just send her home,” it’s almost the opposite of survivorship bias. I never get to learn if I made a good decision. And this is supercommon, not just in medicine but in every profession.

Similarly, there was a work done that showed that people who had car accidents were also more likely to have cancer. It was kind of a puzzle until you think, “Wait, who do we measure cancer in?” We don’t measure cancer in everybody. We measure cancer in people who have been tested. And who do we test? We test people who are in hospitals. So someone goes to the hospital for a car accident, and then I do an MRI and find a tumor. And now that leads to car accidents appearing to elevate the level of tumors. So anything that gets you into hospitals raises your “cancer rate,” but that’s not your real cancer rate.

That’s one of my favorite examples, because it really illustrates how even with something like cancer, we’re not actually measuring it without selection bias, because we only measure it in a subset of the population.

How can people avoid falling prey to these kinds of biases?

Look at your life and where you get feedback and ask, “Is that feedback selected, or am I getting unvarnished feedback?”