Our ability to focus attention on whatever is before us can be highly efficient, but that efficiency benefits us only when the object of our focus represents the full scope of the problem. If we watch a soccer game by focusing only on the side that possesses the ball, we have a chance of decoding that team’s strategy, but we will learn little about what the defensive side is (or is not) doing to counter it. This downside of focus creates one of the oldest and easiest ways for frauds, hucksters, and marketers to fool us into making bad choices. They don’t have to hide critical information from us—they only need to omit it, and count on us not to think about it ourselves.
For years, the two of us regularly saw an ad in our social media feeds featuring a photo of a middle-aged white guy in an ill-fitting pink shirt with the headline “5 Years from Now, You’ll Probably Wish You Grabbed These Stocks.” In smaller print, the caption explained, “He recommended Amazon in 1997 and Tesla in 2011, and he’s announcing his newest pick for the best stock to buy now.” Leaving aside the typical marketing nonsense (was he really announcing a new pick “right now” every single time that ad popped up?), the copy implies that this guy must really know what he’s talking about. After all, he was right about two of the biggest companies ever, so isn’t he likely to be right again?
A simple tool that we call the possibility grid can help us appreciate precisely what important information we do not have. The idea is to apply the statistical concept of a contingency table, common in behavioral science uses like signal detection theory or chi-square tests, to everyday questions—such as whose investment advice to take.
Imagine a two-by-two grid. The top row contains investments that Mr. Pink Shirt recommended, and the bottom row contains investments he did not recommend. The left column includes stocks that showed big gains, and the right column includes stocks that were losers. If we take Mr. Pink Shirt at his word, we would put Amazon and Tesla in the top-left box of the possibility grid. Those are stocks that Mr. Pink Shirt correctly predicted would do well, and we regret having missed out on them. But to more accurately grapple with whether we should trust Mr. Pink Shirt for future stock picks, we have to look at the rest of his grid.
No professional investment advisor can survive by recommending only one stock every 14 years. He must have picked others, but we have no idea whether those did well or poorly. It’s quite possible that the list includes duds like Zynga, Myspace, and Pets.com (“Because pets can’t drive!”). Stocks like those would fall in the upper-right cell: stocks he picked that were bombs. We wouldn’t regret missing out on those! We can also be pretty sure that he failed to pick some highly successful stocks, like Google, Facebook, and Mastercard. Had he picked those, he would have bragged about them as much as he did about Amazon and Tesla. A lot of companies have increased hugely in value since the late 1990s, so there must be a lot of stocks in that lower-left box. Finally, in the bottom-right box go all the other stocks: the ones he never picked and that didn’t do well.
The possibility grid for Mr. Pink Shirt’s claims
To avoid being deceived, we don’t need to know exactly how many stocks are in each box—just thinking about the possible contents of the full grid tells us there is no reason to believe that Mr. Pink Shirt, a guy who made two good picks in fourteen years, is worth paying attention to now.
The possibility grid is a universal tool to draw attention to what is absent. It alerts you to think about rates of success rather than stories of successes. Applied to scientific research, the possibility grid reminds us that we can’t evaluate the state of the literature by tallying up only the significant results—we also have to think about the studies that failed or never got published. And it tells us to be wary when someone claims that their intervention will improve your performance or your health if they don’t show that the gains they promise are more likely to occur with than without their product’s help.
Questioning Mr. Pink Shirt’s claims using the possibility grid
Once you master its logic, you will start to notice so many uses for the possibility grid that you will wonder how you got along without it for so long. A few more examples might help broaden your focus:
- Oprah Winfrey’s magazine O celebrated “great moments in intuition” with examples like that of Ray Kroc, who went with his gut and against the advice of his lawyers when he borrowed $2.7 million (in 1961 dollars) to buy out his partners in McDonald’s decades before it became the world’s largest restaurant chain. No mention is made of the businesspeople who followed their lawyers’ advice and succeeded, or any who ignored their lawyers and failed.
- News reports about Ahmad Khan Rahami, who planted several bombs in the New York City area in 2016, noted that he had traveled between the United States and Pakistan and other Islamic countries several times in the preceding twelve years, but they did not mention the millions of people who traveled with similar itineraries equally often but were not terrorists, or any terrorists (or alleged terrorists) who did not go back and forth regularly to Islamic countries.
- If we look for cases of people who died shortly after receiving a Covid-19 vaccine, we will find many—but we’ll miss the hundreds of millions who did not die, plus those who died on the same dates and hadn’t recently been vaccinated.
People who want to deceive us will go on endlessly about what’s in the top-left box while omitting the others. It’s entirely reasonable to draw conclusions from a small amount of evidence in that top-left box as long as there’s a plausible causal mechanism to explain why it’s virtually certain that the examples in it didn’t get there by chance. If someone lists people who died after being shot, it’s logical to infer that the bullets killed them because we know something about the physics and physiology of how guns can kill. Deceivers know that, of course—it’s why they often appeal to secret, complex, or untestable causal mechanisms (“quantum mechanics” in the social sciences, “professional experience” in fields that have no professional standards, and nebulous forces like “discipline” in business writing).
When someone hands us a reason for success, even an empty one, it becomes harder to think about what they’re not telling us—but even more important to do so.
Adapted from Chapter 1 of Nobody’s Fool: Why We Get Taken In and What We Can Do about It by Daniel Simons and Christopher Chabris. Copyright © 2023. Available from Basic Books, an imprint of Hachette Book Group, Inc.
When you buy a book using a link on this page, we receive a commission. Thank you for supporting Behavioral Scientist’s nonprofit mission.
