Why Randomness Doesn’t Feel Random

Unfortunately, when it comes to understanding random phenomena, our intuition often lets us down. Take a look at the image below. Before you read the caption, see if you can pick out the data set generated using truly uniform random numbers for the coordinates of the dots (i.e., for each point, independent of the others, the horizontal coordinate is equally likely to fall anywhere along the horizontal axis and the vertical coordinate is equally likely to fall anywhere along the vertical).

Three data sets, each with 132 points. One represents the position of the nests of Patagonian seabirds, another the position of ant colony nest sites and the third represents randomly generated coordinates. Can you guess which one is which?

The truly randomly distributed points in the figure are those in the left-most image. The middle image represents the position of ants’ nests that, although distributed with some randomness, demonstrate a tendency to avoid being too close together in order not to overexploit the same resources. The territorial Patagonian seabirds’ nesting sites, in the right-most image, exhibit an even more regular and well-spaced distribution, preferring not to be too near to their neighbors when rearing their young. The computer-generated points, distributed uniformly at random in the left-hand image, have no such qualms about their close proximity.

If you chose the wrong option, you are by no means alone. Most of us tend to think of randomness as being “well spaced.” The tight clustering of dots and the frequent wide gaps of the genuinely random distribution seem to contradict our inherent ideas of what randomness should look like.

It is precisely because of this cognitive bias that some of my recent research has focused on developing metrics that are able to tell whether a spatial pattern is random or not, completely taking human perception out of the equation. Researchers in my group use these tools to determine whether cells in a developing embryo are more spaced out than we might expect and to characterize and better understand the beautiful striped patterns of zebrafish.

Randomness can leave our human brains poorly placed to make sensible deductions, and, unfortunately for us, it is a part, to a greater or lesser degree, of many everyday situations we are faced with, from the arrival time of the next bus to the next song that is dealt to us by our music players when set to shuffle.

Most of us tend to think of randomness as being “well spaced” … genuinely random distributions seem to contradict our inherent ideas of what randomness should look like.

As a case in point, after noticing a disproportionate number of Steely Dan songs playing on his iPod shuffle, journalist Steven Levy questioned Steve Jobs directly about whether “shuffle” was truly random. Jobs assured him that it was and even got an engineer on the phone to confirm it. A follow-up article Levy wrote in Newsweek garnered a huge response from readers having similar experiences, questioning, for example, how two Bob Dylan songs shuffled to play one after the other (from among the thousands of songs in their collections) could possibly be random.

We ascribe meaning too readily to the clustering that randomness produces, and, consequently, we deduce that there is some generative force behind the pattern. We are hardwired to do this. The “evolutionary” argument holds that tens of thousands of years ago, if you were out hunting or gathering in the forest and you heard a rustle in the bushes, you’d be wise to play it safe and to run away as fast as you could. Maybe it was a predator out looking for their lunch and by running away you saved your skin. Probably, it was just the wind randomly whispering in the leaves and you ended up looking a little foolish—foolish, but alive and able to pass on your paranoid pattern-spotting genes to the next generation.

Nowadays, in the absence of the danger of predation, the main use to which our species puts this long-honed auditory skill is in alleging backmasking—the practice by which a reversed message is included in an audio recording. Despite the fact that it makes little sense, people have claimed to hear, “Here’s to my sweet Satan, the one whose little path would make me sad, whose power is Satan. He will give those with him 666. There was a little tool shed where he made us suffer, sad Satan,” when Led Zeppelin’s “Stairway to Heaven” is played backwards.

Despite denials by the band that they had deliberately encoded secret messages into their tracks, a 1982 session of the Consumer Protection Committee of the California State Assembly was played reversed segments of “Stairway to Heaven” and asked to vote on a bill (which ultimately didn’t pass) that would see mandatory warning labels placed on music containing “dangerous” backward messages.

Self-declared “neuroscientist” William Yarroll testified to the committee that teenagers need only hear backmasked songs three times before the subliminal messages were “stored as truth,” turning them into disciples of the antichrist. Although Led Zeppelin were deemed the principal offenders, Yarroll alleged that he had found messages in the reversed music of other bands, including Queen and the Beatles.

We ascribe meaning too readily to the clustering that randomness produces, and, consequently, we deduce that there is some generative force behind the pattern.

This auditory wishful thinking is just one example of the phenomenon known in the psychology literature as pareidolia, in which an observer interprets an ambiguous auditory or visual stimulus as something they are familiar with. This phenomenon, otherwise known as “patternicity,” allows people to spot shapes in the clouds and is the reason why people think they see a man in the moon. Pareidolia is itself an example of the more general phenomenon of apophenia, in which people mistakenly perceive connections between and ascribe meaning to unrelated events or objects. Apophenia’s misconstrued connections lead us to validate incorrect hypotheses and draw illogical conclusions. Consequently, the phenomenon lies at the root of many conspiracy theories—think, for example, of extraterrestrial seekers believing that any bright light in the sky is a UFO.

Apophenia sends us looking for the cause behind the effect when, in reality, there is none at all. When we hear two songs by the same artist back-to-back, we are too quick to cry foul in the belief that we have spotted a pattern, when in fact these sorts of clusters are an inherent feature of randomness. Eventually, the dissatisfaction caused by the clustering inherent to the iPod’s genuinely random shuffle algorithm led Steve Jobs to implement the new “Smart Shuffle” feature on the iPod, which meant that the next song played couldn’t be too similar to the previous song, better conforming to our misconceived ideas of what randomness looks like. As Jobs himself quipped, “We’re making it less random to make it feel more random.”

Excerpted from How to Expect the Unexpected: The Science of Making Predictions—and the Art of Knowing When Not To by Kit Yates Copyright © 2023. Available from Basic Books, an imprint of Hachette Book Group, Inc.

​When you buy a book using a link on this page, we receive a commission. Thank you for supporting Behavioral Scientist’s nonprofit mission.