Is Perception a ‘Controlled Hallucination’?

EDITOR’S NOTE: This is the first of a three-part series on the Human Mind Conference, an “interdisciplinary event” held in Cambridge, England, in June 2017, “bringing together a wide range of experts from across the humanities and the cognitive sciences to discuss key aspects of mental life and experience.”

Is human perception a controlled hallucination? That was the claim advanced in a pair of talks at the Human Mind Conference in Cambridge, England in June, one by Anil Seth, a neuroscientist at the University of Sussex, the other by Andy Clark, a philosopher at the University of Edinburgh. They were not advancing the radical thesis, made by some overeager neuro-philosophers, that all experience is an illusion. Rather, Seth and Clark made the case that there is no bright dividing line between hallucination and ordinary perception.

Our common view is that, in normal perception, the mind neutrally receives information from the eyes, ears, and other sensory organs, whereas in a hallucination, the mind injects its own ideas into the perceptual process. In a hallucination, what we expect or believe becomes what we literally see. What Seth and Clark aim to show is that this model of hallucination — in which the mind uses its existing beliefs to tinker with our senses — actually describes how perception normally works. Are they right?

The Naïve Model of Perception
There is a common-sense view that informed early efforts to build a cognitive model of perception. This is sometimes called the “sense-data” theory of perception, but for the sake of generality, let’s call it the “naïve” model of perception.

Consider how the naïve model would explain vision. Visual perception arises out of a neatly hierarchical, bottom-to-top process, similar to the way digital cameras linked to image-processing software recognize text, faces, or lanes in a roadway. The naïve story goes something like this:

• First, light enters the eyes and impacts the rod and cone cells of the retina.
• Next, these cells create an electrical signal that encodes the image, and send this image along the optic nerve to the neural cortex.
• The neural cortex detects patterns in the electrical signal and reconstructs visual elements.
• These perceptual parts are delivered to the mind as a whole perceptual image.
• Finally, the conscious mind discerns interpretations of the image.

A crucial assumption of the naïve model is that there exists a “raw” stage of perception, an un-interpreted image that has not been affected by our beliefs about the world. Most versions of the naïve model hold that this stage of perception is not merely something that happens in our neurons, but is also something to which we can have direct experiential access.

As the philosopher Alasdair MacIntyre has pointed out, it follows from the naïve model — or perhaps is a founding assumption of the naïve model — that every person in every culture initially sees the same thing when looking at the night sky: a black plane interrupted by occasional bright points. It is only at a secondary stage of interpretation that people begin to disagree about whether this is actually an image of stars and planets, or of pinpricks in a sphere exposing illumination behind, or of marauding gods.

So the naïve model separates interpretation and perceptual mistakes out of the normal process. Seeing the night sky as the inner surface of a sphere is not a false perception, but rather only a mistaken interpretation of an underlying, correct perception, one that is the same for anyone who looks at a night sky. And perceiving something that isn’t there at all — hallucinating — must arise out of some other set of cognitive mechanisms, which interfere with the normal perceptual process.

Problems with the Naïve Model
There are many shortcomings to the naïve model. Notice that it implies a causal ordering of perception. So, for example, you shouldn’t perceive an apple unless the light waves corresponding to an apple hit your eyeballs.

Yet a variety of evidence belies this expectation. In his talk, Andy Clark pointed to an experiment in which pure audio static was played for research subjects. But the subjects were told — falsely — that the audio might contain a distorted fragment of Bing Crosby singing “White Christmas.” And, indeed, a substantial portion of the subjects reported faintly hearing the jingle. Clark describes this as like seeing faces in clouds: “We can project the patterns we expect onto the world.”

The naïve model of perception faces a challenge if hallucination can be shown to be an everyday occurrence in sane people, rather than an extraordinary offshoot of cognitive impairment. And something like this conclusion seems to be implied by a wealth of recent research on “cognitive biases” that distort the normal perceptual process. This ever increasingly asterisked account of perception — a normal cognitive process along with thousands of separate, interfering processes — risks becoming explanatorily unwieldy.

One of the themes Clark and Seth stressed in their talks was the need for neuroscience to make progress in accounting for phenomenology, the philosophical term for studying the nature of first-person experience. And in a believable phenomenology, it’s not obvious that anything like “normal” or “raw” perception actually exists.

Consider something as simple as the perception of everyday objects, such as a remote control (during his talk, Seth held one up for the audience to illustrate the point). On the naïve view, the remote first appears to us as simply an arbitrary region in a field of shapes and colors, which we then quickly pick out and label as a remote. Yet even the bare shape that we perceive does not seem to be the same thing as the raw signal that hits our eyes. This is because we initially perceive the remote control as an object, a whole thing with a continuous surface. Perceiving the remote control as an object means that we perceive it to have a back — yet the back is actually obscured from our view.

Even to see an object as a “bare” or “raw” thing is to see it as an object. And this means that our perception of the remote — even before we consciously think of the concept “remote control” — is already doing interpretive work. That is to say, our perception of an object is already a reconstruction beyond the information delivered to us by our retinas.

Predictive Processing
One way to think about the problems with the naïve model is that it seems to be the reverse of real phenomenology — lived experience. In normal perception, the remote control comes to us already an object. But this means that normal perception is already an act of interpretation, of building up our world from the way we believe it to be.

Hence Clark and Seth urge us to reverse the naïve model and offer evidence for what they call “predictive processing.” As Seth put it in his talk:

If we think of the brain trying to figure out what’s out there in the world … then it’s faced with a problem — that it receives just ambiguous and noisy sensory signals, which are only indirectly related to things in the world, whatever they may be. So perception has to be a process of best-guessing or inference.

“The idea,” said Clark, “is that you’ve got an inner organization that has the resources somehow to construct through itself a kind of version of the incoming sensory information from the top down.” Thus in perception, we are “trying to construct the sensory information at the same time as the sensory information is coming in.”

How does this construction occur? The brain recognizes what is in a visual scene “by finding the best way to build it out of what you [already] know about how stuff operates out there in the world.” As Seth put it, perception arises out of a process “in which the brain combines sensory data with prior beliefs or expectations about the causes of that sensory data.”

To return to the earlier example, the mind is able to perceive a remote control as an object — not just as an arbitrary region in a flat image — because it already knows something about the wholeness, continuousness, and separation of objects. Watching the remote persist through rotations and motion in space, a continuous process, underscores our perception of a whole, persisting object.

Perception from the Inside Out
In his talk, Seth offered the arresting conclusion that perception happens as much from the “inside out” as the “outside in.” That is, perception is not a bottom-up process of electrical signals hitting the eye and being computed into progressively higher levels of organization. Rather, it is a process conducted by a mind that has already constructed its world, and is attempting to discover how the senses confirm or defy its expectations.

This process results in a great efficiency relative to the naïve model. Instead of neutrally analyzing sensory data, the brain checks the data for differences from its expectations. Clark and Seth refer to these differences as “prediction error,” and argue that the real purpose of the sensory organs is not to continuously reconstruct the external world from scratch. You “swap the forward flow of sensory information,” said Clark — the way perception is understood on the naïve model — “for the forward flow of prediction error. So you’re bringing a richly structured model to bear on the world, and all the brain needs to then process is deviations from that.”

Predictive processing also allows the brain to sort through the vast amount of incoming information, focusing only on what it considers most relevant. As Seth pointed out: “The more attention you pay to a sensory signal, the more influence it will have in updating the predictions that are trying to explain it away.”

This account of the active role of the interpreting mind is what leads to the slogan “perception is controlled hallucination.” Where the cognitive mechanisms of hallucination were once thought to be distortions, predictive processing theorizes that they are how perception always works. “Your brain tries to guess what’s out there,” said Clark, and “to the extent that the guess matches or…explains away the evolving sensory data, you get to perceive the world.”

Evidence for Predictive Processing
Clark and Seth offer a wealth of evidence to support the predictive processing model.

Some of this evidence comes from neurological studies. One such study, described by Seth, found signals propagating in opposite directions across the visual cortex, carried on different frequency bands. According to Seth, this finding is consistent with counter-flowing predictions and prediction errors, a continuous interchange between top-down prediction and bottom-up signal deviation. In a related study, researchers found that interrupting the top-down signal flow blocked research subjects from consciously perceiving motion, even though the bottom-up flow — the signals from the optic nerve — were left intact.

But the most compelling evidence offered by Seth and Clark came in the form of simple phenomenological experiments, a few of which they recreated during their talks. In one vivid example, Clark played a short audio file of a woman speaking, which had been deliberately garbled through sine-wave distortion to make it difficult to understand. Take a listen for yourself:

Now listen to the original, clean version of the same audio:

Now listen again to the first, garbled version:

If the experiment works as intended, the garbled audio should be much easier to understand on the second listening. Yet it’s exactly the same audio as before — the same external input, the same bottom-up signal jaunting across your cochlear nerve.

“I think what’s going on there,” said Clark, is that “you’ve obviously got a pretty good model of this language-y stuff already, so it doesn’t take much to sort of fire up a very good model of what that sentence was.” The clean audio gives you a new top-down model of the garbled audio, so that when you hear it again, your perception of it is richer. Clark even suggested that it is possible to become a sort of “native speaker” of this type of distortion, learning to understand these files without first listening to the clean versions.

According to Clark, this example aligns well with ordinary phenomenology, with our experience of experience. “Think about hearing a familiar song played on a poor radio receiver or in the shower…. You’ll hear that song a lot better than you’ll hear an unfamiliar song played on the same poor receiver.” On the naïve, bottom-up model, one would have to explain this phenomenon as the interpolation of some other cognitive mechanism such as a memory effect. But the garbled-audio experiment suggests that this effect is part of the normal functioning of perception. It’s not that something special happens for songs that you’re familiar with, but that you have “better priors” in this case. You are an “expert perceiver” for this particular song, much as expert craftsmen easily perceive imperfections in wood that non-experts would strain to notice.

Of course, to speak of this process as a “hallucination” is somewhat misleading. Unlike the “White Christmas” example, the signal in the garbled audio is actually there. The perception cannot be described as some sort of illusion or secondary effect. As Clark pointed out: “It’s not that your familiarity is making the fuzzy percept[ion] seem clearer. It’s not that memory later does a little bit of filling in.” Rather, your perception itself has genuinely been altered. The song “really does sound clearer.”

Imagination and Understanding
Predictive processing suggests several fundamental shifts, not only in our understanding of perception but in our overall picture of how the mind fits into the physical world.

Instead of understanding perception as a piece of biological software that first encounters the world as a realm of abstract geometry, predictive processing approaches perception as one of the many regulatory processes by which an organism sustains its own conditions for life. “Conscious selfhood is…very closely tied to being alive,” Seth observed. It “emerges because of and not in spite of the fact that we are flesh-and-blood organisms that need to keep ourselves going over time.”

Once again, the perception of ordinary objects provides an instructive example. Where the naïve model would have understood object perception as an attempt to reconstruct geometrical shapes, predictive processing is primarily interested in regulating some variable that is relevant to an organism’s survival, protecting that variable “against perturbations that might be incompatible with life.” The perception of objects as separate, whole things “is about the brain making predictions about the sensory consequences of actions…. I perceive this as an object because…my brain somehow knows what would happen if I were to make various actions with respect to it. It’s in that sense that I perceive that it has a back, even if I don’t directly perceive the back.” Because we are oriented towards an object as active beings who can manipulate it, and the object is oriented towards us as something relevant to our condition, the object breaks out of the background perceptual field as a separate thing. Other, less relevant features of our perceptual field might require training to recognize.

Thus the boundaries between perception, understanding, and imagination begin to blur. Each of these are different aspects of a single ongoing process, a single set of cognitive mechanisms. In this sense, “perception and understanding come at the same time,” as Clark put it. And both perception and understanding are inherently acts of imagination, of projecting our beliefs out onto the world and seeing what kind of friction the world sends back.

Clark pointed to an anecdote from ordinary experience: when a curtain moves in your peripheral vision and you perceive it as your dog, before remembering that your dog recently died. But, Clark said, “maybe that’s not a sort of grief-based malfunction, it’s just the prediction machine really doing what it should do: you’ve got strong priors and expectations, you’ve got a bit of sensory evidence, you start to construct the world as it would be, and then you get the prediction error saying no, sorry, it’s not like that.”

The terms “controlled hallucination,” and related ones like “inferred fantasy” and “virtual reality,” are useful rhetorical devices for illustrating what is distinctive about the theory of predictive processing. But Clark is resistant to adopting these terms in full. For none of them quite describes what the theory says about the nature of our perceptual contact with the world. Perception, on Clark’s view, “doesn’t cut us off from the world…instead, it opens us up to the world as it matters to a creature like us.”