It Isn’t a Replication Crisis. It’s a Replication Opportunity

Let me briefly explain what business and behavioral science have in common.

They both do experiments.

Apart from that, everything is slightly different.

Business does not always perform experiments consciously. Sometimes, a business or a product is an experiment in itself. When the iPhone was introduced it represented a huge bet on a behavioral outcome—that people would value its strengths in usability and aesthetics over its functional weaknesses.

(We forget the weaknesses now, because we are all inured to them, but the iPhone battery life was so poor for the time that it needed to be charged overnight every night—something Nokia might have considered a deal breaker. We don’t see that as a failing today, as all smartphones share the same weakness, but at the time it was a brave bet).

In business, you don’t need to be “right.” You just need to be right enough.

For all its many flaws, business is also blessed with a clever funding mechanism. By and large, the more successful your hunch or theory—or lucky accident—the more money you are given to pursue it.

Not only are business experiments not always conscious but, at the most basic level, what businesses ask from experimentation is markedly different from what scientists seek. In science, the dream is to uncover a universal, timeless truth or law. In business, we don’t need to be right in general—we just need to make the best decision for the situation at hand.

Both of these approaches—the universal and the contextual—have their strengths and their weaknesses.

You can see something of this business–academic divide in the difference between William Sealy Gosset and R. A. Fisher and their approaches to statistical significance. Gosset, discoverer of the Student’s t-test, worked for Guinness, and he devised the statistical formula to help the brewer maintain the consistency of its hops, and thus its beer, while measuring only small samples of ingredients.

He would have nothing to do with Fisher’s later assertion that p < 0.05 was a threshold for statistical significance, believing it to be arbitrary. From the more pragmatic standpoint of the businessman, he believed that the level of proof you required for a decision depended on the consequences of the decision.

In business, you don’t need to be “right.” You just need to be right enough—or right with sufficient frequency to occupy a lucrative market space. Sometimes all you need to be is less wrong than your competitors.

Of course, like any experiments, business experiments often yield strange results. The business world is full of unexpected failures and inexplicable successes. Why, for instance, don’t people use moist toilet paper? Why are Americans so weirdly guilt ridden about taking vacations? And what is so popular about Red Bull—an expensive drink that really doesn’t taste very nice? (In research, people hated it.)

These weird findings make consumer capitalism a kind of Galapagos Islands for the academic behavioral scientist.

Yet where behavioral scientists have done a much better job than businesspeople is in categorizing those apparent anomalies and in attempting to find recurring patterns and commonalities, something the business mind is woefully disinclined to do. Just as plenty of dog breeders, horse breeders, and pigeon fanciers used Darwinian thinking in their specialized fields long before Darwin codified his theory (one dog breeder explained his method to Darwin as, “I breed many, and I hang many”), plenty of businesspeople have made use of behavioral science every day without making any attempt to transpose generalized findings into other fields.

Designing the check-in experience for a hotel chain is not like designing the 787: you probably want to give reason a back seat.

As Amos Tversky once said, he was “merely examining in a scientific way those things which were already known to advertisers and used-car salesmen.” Robert Cialdini embedded himself in sales forces to investigate the same thing. This work, in codifying human behavior, has been priceless.

In my own field, some of the greatest advertising end lines paradoxically contain an apparent negative. “Fresh Cream Cakes, Naughty but Nice.” “We’re Number Two So We Try Harder.” “Reassuringly Expensive.” No one in advertising had considered investigating the persuasive power of an admitted weakness until Cialdini.

If I have a criticism of the academic approach, it is only that some academics seem to shy away from evolutionary explanations—possibly because in some academic circles evolutionary psychology seems to be a kind of kryptonite, fatal for one’s career once touched. Also, in academia the dominant currency—reputation, rather than money—cannot be traded between disciplines, which limits the pursuit of interdisciplinary work.

But science also has another facet, which is appropriate in some settings but not in others. This is the French enlightenment “bit of science,” as distinct from the Scottish enlightenment bit: the idea of the supremacy of “reason.” Here I am a sceptic.

Reason works well when you have universal, context-free laws that apply at all times and in all places, where there is a single right answer to the relevant question, and where the opposite of a good idea is a bad idea. When people design a Boeing 787, you want them to use reason a great deal.

What we have learned from behavioral science is that human behavior—including human economic behavior—does not meet these criteria, despite valiant but misguided attempts of economists to pretend that it does. Designing the check-in experience for a hotel chain is not like designing the 787: you probably want to give reason a back seat.

In physics, the opposite of a good idea is a bad idea.

In marketing, and hence in business, the opposite of a good idea can be another good idea.

So, depending on context, expectation, culture, and mood, a good hotel check-in experience can be perfunctory, brief, and highly automated. Or it can be lavish, highly personalized, and solicitous in the extreme. There is no single right answer; there may be many right-ish answers.

I first noticed this when reading Cialdini’s principles of persuasion. The principles are in some ways slightly contradictory. You might use “scarcity,” or you might use “social proof” (ubiquity). This rather reflects an old saying in the ad industry—that there are “two ways to sell a product: you either tell people that very few people own something, so it must be good, or you explain that lots of people own that thing, so it must be good.”

In physics, the opposite of a good idea is a bad idea. In marketing, and hence in business, the opposite of a good idea can be another good idea.

I now notice this seeming contradiction more and more. I once proposed to a fast-food client, in defiance of all economic logic, that they should raise the price of a product that wasn’t selling as well as expected. I was lucky—demand subsequently went up. This seems ridiculous, until you consider the two different modes in which someone might dine at a fast-food chain. They might be looking for a bargain, or they might be looking for a treat. Something priced in the middle of the menu (as this item was) satisfies neither of those criteria.

(McDonald’s in the U.K. has a clever approach to this. They have five flavors of wrap, only one of which is discounted by £1 on any given day of the week. Someone chasing a bargain will sacrifice their chosen flavor for the “wrap of the day” and save a pound. Someone less price sensitive will pay £1 more for the flavor they want. That’s price discrimination, baby!)

Anthropologists have long been aware of this paradoxical element of human behavior. Pierre Bourdieu observed how the same thing, in a different context, can take on an opposite meaning. Returning a gift is an act of generosity in economic terms. In human terms, however, it is a massive insult. In economic terms it makes sense to pay your spouse for sex or to leave cash on the table after a dinner party. In anthropological terms it really, really doesn’t.

So it does not surprise me in the least that sometimes the strategy of choice reduction works, and sometimes it doesn’t. If I am in a busy supermarket and somewhat indifferent about whether to buy jam, too wide a selection may cause me to buy nothing at all. On the other hand, if I have driven 40 miles to visit a superstore called “World of Jam,” it is pretty unlikely that I will go home empty-handed having been paralyzed by choice.

I never expected the paradox of choice to apply everywhere. After all, that’s why it’s a paradox. What was useful to me was that I suggested that an airline test it on its website, and the change worked. It is a valuable finding not because it is universally true but because it is counterintuitive—a person reliant only on reason would assume that the greater the choice of destinations, the more likely someone is to book a flight. It was a triumph of behavioral science that we tested it at all.

Given these inherent, context-sensitive contradictions in human behavior, the replication crisis doesn’t surprise me at all. Nor does it bother me. Very few business “rules” hold under all conditions. Can you sell sunglasses for £200? In a drugstore, no; in an airport, yes.

In business there is no replication crisis, because there is no appetite for or expectation of replication. In fact the notion that small changes in context can change the rules of human perception and behavior presents a wonderful world of opportunity.

Depending on context, it is likely that the right answer to A can be B, the opposite of B, or somewhere in between. And it is in that area—of coming up with more than one right answer to the same problem—that business has the upper hand on science.