Confirmation Bias (Part Two)
How to Overcome Your Own
Pre-Existing Beliefs

Recognize the problem. That doesn’t mean just recognizing that confirmation bias exists. It means recognizing that you are vulnerable to it just like everybody else.

Consciously seek out opposing views. Do that with regard to specific issues and controversies. But do it more generally as well. Regularly watch a newscast or subscribe to a magazine that sees everything from “the other side.” Find Internet aggregators that mostly collect sources from “the other side.”

Think of your opinions not as undeniable truths but as hypotheses that deserve to be tested. And then rethink what you know about how to test hypotheses. Even when we’re not emotionally committed and genuinely trying to get the right answer, we typically test our hypotheses by looking for evidence that we’re right. That biases what evidence we find. And it limits how much evidence we find. Collecting example after example of cases where we’re right doesn’t help us know where we might be wrong. We would learn more, usually, if we searched for disconfirming evidence instead of confirming evidence.

Interviewers who suspect a job candidate is introverted, for example, tend to ask questions that elicit introverted answers. They would be wiser to ask questions aimed at eliciting extroversion, if there’s any to be elicited. Not getting the extroverted answers you were probing for is stronger evidence of introversion than seeking and getting introverted answers. And if you do get the extroverted answers you were probing for, you learn that your hypothesis needs correcting.

Beware of overconfidence. The more confident you are about an opinion, the more strongly you will exercise your toolkit of confirmation bias defenses to protect yourself from evidence that you’re wrong. So work extra hard to expose yourself to opposing views on the issues where you’re surest of your own view.

Realize that “right” doesn’t mean 100% right. It’s a rare point of view that doesn’t have some solid facts and valid arguments to be cited on its behalf. So go find the solid facts and valid arguments on behalf of viewpoints you know are mostly wrong.

Always require yourself to be able to summarize the opposition’s strongest case. Whether you acknowledge that case publicly or keep it to yourself is a strategic communication decision. But if you can’t summarize it, you have fallen victim to confirmation bias. To test yourself, find an opponent to critique your summary.

On issues that are new to you, try to stay undecided longer, so your information-seeking is aimed more at figuring out what’s so and less at documenting why you’re right. It’s easier to overcome confirmation bias if you don’t have an opinion yet, not even a hypothesis.

But if you do have a hypothesis, don’t pretend you don’t. Instead, try to come up with two competing hypotheses. You’ll be less vulnerable to confirmation bias if you’re looking for information to help you assess which of your three hypotheses is correct than if you’re trying to confirm your only hypothesis. (According to some research, three is the magic number. Fewer than three leads to too much confirmation bias. More than three makes it too hard to organize what you’re learning.)

Even on issues that are extremely familiar, where you do have an opinion and not just a hypothesis, try to stay permanently tentative, open to new evidence. Act in black-and-white when you must, picking the side you stand for. But try to keep thinking in shades of gray.

When you run into evidence that seems to contradict your opinion, don’t make the mistake of assuming you must either reject the evidence or abandon your opinion. If you think those are your only choices, you’re all too likely to reject the evidence. The new piece of evidence might be an outlier, a tiny piece of truth that favors the other side even though most of the truth is on your side. It might be proof that you’re fundamentally wrong.

But the likeliest alternative is that it’s evidence that you’re partly wrong– that your prior opinion needs to be adjusted rather than abandoned or defended in toto. Keeping that third alternative in mind should make it easier for you to let contrary evidence in. In research studies of confirmation bias, the most successful participants used disconfirming information to modify incorrect aspects of their initial hypotheses.

Stay alert for surprises, especially for factoids that don’t make any sense to you. They may simply be anomalies than run contrary to the weight of the evidence. But they may be signals that you’ve been reading the evidence wrong, or partly wrong. Instead of assuming that they’re anomalies and shrugging them off, force yourself to take them seriously. Go looking for more examples of your anomaly, and start trying to figure out what it might signify. Much brilliant new thinking begins with the decision to take a weird, senseless factoid seriously.

Assess the quality of your information sources. Before the Internet, the main challenge of research was getting information. Now the main challenge of research is vetting information, deciding how much to trust it. Develop standards for deciding how much to rely on a source – standards that are independent of what viewpoint that source is supporting.

Look for signs of confirmation bias in your sources. Do they neglect or acknowledge the other side’s good arguments, good studies, and good facts? Do they sound overconfident or appropriately tentative?

Look for reasons why your sources would have wanted to reach the conclusions they reached. Mistrust sources with an economic or reputational interest in their conclusions. An example of the latter: a source who has long been identified with a particular viewpoint and would find it embarrassing to reach a conclusion that supports the other side.

Sources who look like they started out neutral are more trustworthy than sources who are on a side and reach the conclusion their side always reaches. But the most trustworthy sources are those who reach a conclusion they didn’t expect and didn’t want to reach, reluctantly conceding that the opposition is right about a particular point. (I’m not talking about converts; they’re often the most unreliable sources of all.)

Mistrust anecdotal evidence. If you’re genuinely neutral about a question, anecdotes can be useful preliminary sources of information, helping you figure out what might be true. But there are anecdotes to support virtually any position, and thanks to the Internet they’re easy to find. So once you’re on a side, your anecdote search becomes a pure exercise in confirmation bias. Use anecdotes to illustrate your opinion and perhaps even to enrich it. But don’t ever think they prove it. And if you’re an honest communicator, don’t pretend they prove it.

Mistrust quantitative evidence nearly as much as anecdotal evidence. Just as there are anecdotes to support virtually any position, the same is true of published studies.

Empirical research involves far more subjectivity than we like to imagine. Decisions about hypothesis formulation, research methods, data analysis, and data interpretation are all influenced by the researcher’s confirmation bias. And when all else fails, there’s always the file drawer; studies that reach conclusions the researcher wasn’t happy reaching are less likely to be submitted for publication. Readers’ judgments about other people’s research are also greatly affected by confirmation bias; we overlook methodological flaws in studies we agree with but quibble with studies we disagree with (if we read them at all).

When lots of studies by different researchers using different methodologies are all reaching pretty much the same conclusion, and only a few are reaching the opposite conclusion, then there’s a good chance it’s a sound conclusion. Not until then.

Even with quantitative evidence, but especially with opinions and interpretations, watch out for groupthink. To one extent or another, every field is a guild. Certain opinions are part of the occupational ideology of the field. Contrary opinions are unacceptable not just to espouse, but even to investigate. Contrary evidence is ignored until it becomes impossible to ignore, at which point a belated paradigm shift occurs.

So when nearly everybody in a field seems to be on the same side, their guild confirmation bias is likely to be part of the reason why. That doesn’t mean they’re necessarily wrong. It does mean they’re more often wrong than their near-consensus suggests. And even if they’re mostly right, it means they share the same blind spots to the ways in which they’re wrong.

Especially if you’re in the guild too or if you’re on the same side they’re on, pay close attention to what the apostates are saying – people on the fringes or beyond the fringes of the field. “Most experts believe…” turns out to be a weaker rationale for an opinion than we imagine. It’s too often circular: Most experts believe it because most experts believe it.

Pick your moments. Confirmation bias is an efficient tool we all use to save effort and emotional energy. Nobody can afford to start from Square One every time we consider any question, as if we had never considered it before. We’re entitled to the benefit of our prior knowledge and experience. We’re entitled to our attitudes and values. We’re even entitled to enjoy the pleasure of spending more time with people and websites we agree with than with disagreeable ones.

What counts is knowing that confirmation bias significantly distorts how we process information; knowing when it’s important to find a less biased way to check out whether we’re actually right or not; and knowing some ways to help overcome our confirmation bias when we have decided we should.