When a Theory is Too Good to Be True: Fallacies in Perception Research

This article was originally published on The Psych Report before it became part of the Behavioral Scientist in 2017.

Over the past few decades, research in the fields of perception and psychophysics has seemingly demonstrated that our vision is inherently tied to the current psychological, emotional, or physical state of our body. Wearing a heavy backpack makes hills appear steeper (Bhalla & Proffitt, 1999); holding a baton makes objects appear closer (Witt, Proffitt, & Epstein, 2005); holding your arms out to the side makes doorways appear narrower (Stefanucci & Geuss, 2009). Findings like these suggest that the image we see is the product of our brain coordinating information about our visual environment with information about our bodily state. Subtle changes in, say, our body’s position, produce noticeable changes in how we perceive our environment, or so the theory goes.

Several scientists have argued that having distorted vision allows you to better adapt to your environment. Walking up a hill with a heavy backpack requires burning more calories, so seeing the hill as steeper allows your body to anticipate the extra burden. Holding a baton makes a nearby object easier to reach. Holding your arms out makes some doorways impassable.

There are many results like these, and when added together they build a bold, intriguing theory about how we see and interact with our visual environment. However, a growing body of research has raised substantial doubts about this theory, as well as the experimental validity of the evidence supporting it. An article in July’s Perspectives on Psychological Science, called How “Paternalistic” Is Spatial Perception? Why Wearing a Heavy Backpack Doesn’t—and Couldn’t—Make Hills Look Steeper (Firestone, 2013), offers a new perspective in this debate and argues why a theory like this, as intriguing as it may be, cannot actually be true.

Paternalistic Vision

Chaz Firestone refers to this theory as Paternalistic Vision, in order to illustrate the guiding role that your brain supposedly takes on during perception. Recall the claim that wearing a heavy backpack makes hills look, actually look, steeper; this result suggests that your brain is “paternalistic” in that it makes your visual system see the hill as steeper than it really is, because of the added energy it would take your body to walk up that hill (Bhalla & Proffitt, 1999). The word “paternalistic” in this theory is reminiscent of an overbearing father, constantly trying to influence your decisions based on his Father-knows-best view of the world.

However, after reviewing much of the Paternalistic Vision literature, Firestone points out that many of the effects are illogical, inconsistent, and arbitrarily-sized. He concludes:

“The deepest shortcoming of the paternalistic vision hypothesis is not that the weight of the empirical evidence is against it… Rather, it never could have been the right account of the relevant results in the first place. It is just the wrong kind of theory for the job.”

Even if the experimental results are taken at face value, this theory cannot be true. Here’s why.

First, the effect sizes found in various experiments do not follow consistent patterns. Sometimes the effect sizes are very dramatic, and other times, the effect is inconsequential. For example, people who drank an artificially-sweetened drink reported hills to be 50% steeper in grade than those who drank the same drink, only sweetened with sugar (Schnall, Zadra, & Proffitt, 2010)—a rather large effect. Participants who held a 15-inch baton that extended their reach by about 50% judged an object in front of them to be about 5% closer than it actually was (Witt, Proffitt, & Epstein, 2009)—a rather modest effect. People holding their arms out wide (about 40-inches apart) judge doorways to be about 3% smaller than people who held their arms by their side (Stefanucci & Geuss, 2009)—a very small effect, considering the fact that the width of the doorway would only look about an inch smaller.

Not only are these effect sizes inconsistent, but they are also measured in units that are incomparable and arbitrary. Proponents of the theory often mention that we see the world in perceptual units that are based on the capabilities of our body (e.g. Linkenauger, Witt, & Proffitt, 2011). My phone is 2 arm-lengths away from me; the pole is 14 steps away from me; that car is 10 calories away from me. In this sense, having paternalistic vision is useful because it produces distortions that provide you with information tailored to your body’s needs, in order for you to make the most efficient decisions possible (Proffitt, 2006). However, this claim falls apart when you are forced to make a decision between performing two different actions with two conflicting perceptual units. Firestone points out that these perceptual units are incommensurable; that is, there is no common way to compare the distortions across different actions. Imagine that a cockroach is three arm-lengths away from you (arm-lengths are units of reach). It also happens to be two steps away from you (steps are units of walking). Which action do you choose if you want to squash the bug? Having paternalistic vision should help you select which action to perform by informing you which action is best suited for your body’s current state. Nevertheless, when choosing between doing two different actions, there is no common denominator—no conversion scale—that allows your body to compare between the two possible actions.

One possible objection to this argument is that energy expenditure, measured in calories burned, could be a way to compare different actions. If your body predicts the amount of calories that will be used up for each action, it would be beneficial to see an updated visual environment that reflects possible energy expenditures; for example, a steep hill would look even steeper when wearing a backpack, because it would take more energy to walk up it. However, Firestone points out that this kind of argument would fail to account for findings that do not involve energy use whatsoever. One experiment showed that thinking about a close friend right before guessing the steepness of a hill made people give lower estimates of the hill’s slant than people who weren’t told to think about their friend (Schnall, Harber, Stefanucci, & Proffitt, 2008, Experiment 2). Having real or imagined social support does not change the amount of calories you’ll have to burn in order to walk up a hill. Thus, even an objective measurement like calories burned would fail to unite the various Paternalistic Vision findings.

Perhaps the most striking argument against Paternalistic Vision is that the theory does not have an explanation for why some of these substantial “perceptual” effects are not perceptually noticeable. You can try—and fail—to notice these changes yourself; next time you’re in front of a doorway, hold your arms out to your side. Bring them back in. Out, in, out, in. Is the width of that doorway shrinking before your eyes? Put on your backpack, stand at the bottom of a hill, and look to the top of the hill. Take the backpack off. Put it on. Notice any Inception-esque landscape warping? Proponents of the theory acknowledge—and cannot resolve—the obvious fact that we don’t notice our visual environment contorting before our eyes (Proffitt, 2006).

But given the number of published findings that support Paternalistic Vision, the question still remains: how are so many researchers able to find significant effects in their experiments?

Experimental Methods

One possibility is that some of the evidence and support for the Paternalistic Vision hypothesis can be explained away as a result of demand characteristics in the experiment; this is when participants slightly change their usual behavior because they—either consciously or unconsciously—know the hypothesis of the experimenters.

From an experimental standpoint, there are a few ways to question whether experimental demand is present, and they generally follow a similar methodology. First, examine the findings that are consistent with the Paternalistic Vision hypothesis. Then, replicate the findings to ensure that the “effect” exists. After making minor, strategic tweaks to the experimental design, rerun the experiment. If the effect disappears (or appears when it clearly shouldn’t), then the new results cast a shadow of doubt across the original research, and questions arise about possible demand characteristics in the original experiment.

For example, Frank Durgin and colleagues demonstrated how demand characteristics could account for the finding that wearing a heavy backpack makes hills look steeper (Bhalla & Proffitt, 1999; Durgin, Baird, Greenburg, Russell, Shaughnessy, & Waymouth, 2009). First, the effect was replicated. Having people put on a heavy backpack does make them tend to give higher estimates of slant. Then, another experimental condition was added in which a cover story was provided for why they were wearing a backpack (wires and other experimental equipment were in the backpack, and the participants were told that the backpack was taking measurements during the experiment). The people who were told the cover story did not give overly exaggerated estimates of slant, but the ones without these new instructions did. In this case, wearing a backpack produced a non-perceptual effect on their estimates of the slope of hills. The backpack didn’t make participants see the hill differently, it was part of an experimental demand that biased them to tell the experimenter that the slope was higher.

Interestingly, even without knowing the specific confounding factor that is driving the result, we can still learn about effects that should not have logically been found.

Firestone and Scholl demonstrated this fact by reexamining an experiment that showed when participants think about an immoral event from their past, they tend to judge the room they are in to be a darker shade than if they reflected on a moral event (Banerjee et al., 2012; Firestone & Scholl, in press, Experiment 4 and 5). If true, this claim would be a sensational finding that bridges emotion and visual perception. First, they replicated Banerjee’s finding on Amazon’s crowdsourcing website, Mechanical Turk, and showed that people actually do rate the room they are in to be “darker” (on a scale of 1 to 7) after writing a description about an unethical event in their lives. Then, after changing the rating of lightness and darkness from a 1-7 scale to a gradient of seven different shades of grey, Firestone and Scholl reran the experiment. Again, participants who wrote about unethical deeds rated the room to be darker by choosing darker patches of grey. This replication raises several questions: if participants who wrote about an unethical event saw the room as darker, why don’t the patches of grey also look darker (therefore canceling out the effect)? If this change in mood after reflecting on your past is a perceptual phenomenon, why is it that only the room looks darker? Is there something unique about rooms that intimately tie them to our visual perception? After a while, the questions raised from this finding become frivolous and detract from the larger point that there must be a non-perceptual explanation for this finding. Reflecting on an unethical event does not make the world look darker—sunglasses do this, not your mood.

The compelling methodology described above is based on a logical fallacy known as the El Greco Fallacy. El Greco was a 16th century Spanish artist who became known for the noticeably long-faced characters in his paintings. In the 20th century, an ophthalmologist claimed that the reason for this painting style was because El Greco must have suffered from an astigmatism—a vision distortion that makes the environment appear stretched out. But if El Greco’s vision was distorted because of an astigmatism, would the faces he painted on his canvas actually reflect that distortion?

Portrait of a Man, El Greco (1604)

No. Logically this cannot be true. Imagine yourself wearing “astigmatism goggles” that make everything you see look vertically elongated; if you painted a face onto a canvas, that face would also look distorted to you. But if you took the goggles off, you would realize that you had just painted a normal face. If the world El Greco saw was unnaturally elongated, and he reproduced images on the canvas as he saw them, then the faces he painted would be similarly scaled and appear normal for non-visually-impaired viewers.

It is a fallacy to assume that El Greco had an astigmatism based on the elongated faces on his canvases, just as it is incorrect to assume that when people think of unethical deeds their perception of color changes.  As Firestone writes, “Distortions must ‘cancel out’ when the means of reproduction would be distorted in just the same way as the stimulus being reproduced,” (Firestone & Scholl, in press). If you really saw the world as distorted, you would see the whole world as distorted. By claiming that a certain action forces your brain to, say, make faces appear longer, rooms appear darker, or—as we will soon see—doorways appear narrower, many researchers have fallen victim to the El Greco Fallacy, which indicates that some results in favor of the Paternalistic Vision account are likely due to a confounding, non-perceptual factor.

Firestone and Scholl demonstrated this when they reexamined the finding that participants who held a relatively long pole across their body saw  the width of a doorway (referred to as an aperture) to be smaller (Stefanucci & Geuss, 2009, Experiments 2 and 3). Again, if true, this is certainly worthwhile to remember whenever we try to walk through doors while holding a poster tube. Firestone and Scholl began by replicating this finding (Firestone & Scholl, in press, Experiment 1). In the experiment, participants were told to hold a pole parallel to the ground and imagine passing through the aperture. They then had to turn and tell an experimenter to adjust a measuring tape so that the widths of the aperture and the measuring tape were the same. Here, participants who were holding a pole consistently set the width of the measuring tape to be narrower than did participants who were not holding a pole.

Next, to test the El Greco Fallacy, the experimenters modified this setup slightly. Instead of matching the width of the aperture to a measuring tape, participants in Experiment 2 matched the width of the aperture to a second adjustable aperture. After all, if holding a pole across your body makes apertures look smaller, then that effect should apply to all apertures. The distance between the two widths should be the same, much like in the previous experiment where changes in darkness should have applied to all surfaces. However, this is not what happened, and the original effect was replicated; participants set the adjustable aperture to a smaller width than the one in front of them—a perfect illustration of the El Greco Fallacy, and an indication that demand characteristics may have caused the effect.

To explore these results, participants in Experiment 3 were told that the reason they were holding the pole was to help them with balance, much like a tightrope walker. The experimenters also said that they were testing out different poles for their study and pointed to an array of different sized poles in the corner of the room. With this new cover story, the effect of holding the pole disappeared. And thus, what started out as a spectacular claim about our visual perception was shown to be simply the result of demand characteristics in the experiment.

The Allure of the Sensational

While some findings in psychology are truly sensational, it remains important to question the validity of published results. A theory that can withstand logical and experimental critiques can provide powerful insight into the human brain, but for Paternalistic Vision, this does not appear to be the case.

Some of the results supporting Paternalistic Vision can be explained away by understanding the implicit demands that come with participating in any psychology experiment. Other results can be refuted by replicating them in situations where they should not exist. But even if some of the findings supporting Paternalistic Vision can withstand the experimental scrutiny mounting against them, a compelling case has been made that the theory is flawed to the point that it logically is not—and can not be—true.