Consciousness Is a Self-Model
Three different theories sort of agree that experience is like a virtual reality
Philosophers of consciousness love to speak about seeing the book (usually red) in front of them. For the best illustration of the mundane miracle of conscious perception, I go for trees and have for a long time. Me putting on my first pair of glasses: “Mom, I can see the leaves on the trees!”
By the merest glance at a tree I can immediately—no real delay to speak of— see a detailed pattern of tens of thousands of leaves. The slightest movement of my eyes, and the visual tapestry unrolls myriads more details, woven in colors, shapes, and textures, stacked by distance from me, and yet all part of one world and all right now, no waiting. It’s my private world, interleaved with memories, meanings, and motivations particular to me.
During the roaring ascent of science last century many said that consciousness is too private to be understood scientifically. But that changed, and we’ve had several decades of psychologists, neuroscientists, philosophers, and mystics trying to come to grips with consciousness.
There are now several handfuls of theories about consciousness that can be taken seriously, but most are not known beyond the specialist audience. You could get the impression that it’s all still a big mystery, that it’s The “Hard Problem” of science. And it’s true that practitioners sometimes ask whether we have to change our approach to knowledge, or even our very understanding of the universe, in order to crack consciousness.
However, there’s one time-tested approach that should be primary, but rarely gets addressed. To wit: if you want to understand something, you need to describe it first. In other words, we should start with what consciousness is and then we can figure out its how and why. The classic example is when early observations of nature, called “natural history”, eventually led to the powerful theories of current biology.
For my money, the best modern description of consciousness came from a philosopher: that the properties of conscious experience are like an “Ego Tunnel,” generated by a self-inclusive model of reality. This complex idea is compatible with a general theory of brain function called Predictive Processing, and a specific theory of consciousness, Attention Schema Theory. Taken together they say that our conscious experience derives from a process of the brain’s modeling or simulation of you and your surroundings.
Self-Modeling
The philosopher Thomas Metzinger was 19 and studying the mind-body problem at Goethe University when he began having Out-Of-Body experiences. OBE’s have long been used to argue for a dualist distinction between mind and body. That was not a reasonable idea to him. His world view shaken, he was influenced by a conversation with psychologist Susan Blackmore, who proposed that in his OBE he was merely moving around in his world model, a sort of mental map.
The idea of a mental world model is, of course, a theory, not a description. But Metzinger began a long study of what consciousness is. His approach was like a naturalist’s: what are the properties and behavior of everyday consciousness? What does it take to clearly describe it? What are its parts?
About 20 years later he organized what he had learned in a lengthy monograph, Being No One: the self-model theory of subjectivity. The essence is that we perceive a world around us because we make a mental model of the world. But we do this by including a model of our self in that world.
He wants future research to focus on explaining how the brain produces a set of features of (“constraints on”) consciousness, which is generated, he hypothesized, as a “phenomenal self-model” or PSM.
His 700-page review of nearly everything known about consciousness was not likely to become widely known to the public. Metzinger saw that the knock-on effects of understanding consciousness could be important to culture and everyday life. So, he used one of his favorite metaphors in writing a book for the general public.
Reading his book, The Ego Tunnel, was like being an alien anthropologist who is making the discovery of a lifetime. It was a shock to learn that there was so much that could be said, and so clearly, about consciousness.
The ego tunnel is Metzinger’s metaphor for conscious experience. It’s like a tunnel because: (1) our world model is limited in scope and thus there are limits, like tunnel walls, to what we experience consciously, (2) your ego — the self-model that seems to be (but is not) you the person — is metaphorically inside the tunnel as a simulated point of view, and (3) consciousness seems to move through time like you would through a tunnel.
Starting with his list of properties of consciousness, Metzinger proposed a first-level case for the How. It sort of has to be a mental model, made by the brain.
What it Means for the Brain to Model Something
I can’t keep blabbing about mental models without defining what such a thing might be. And knowing the purpose of the model will lead us to one kind of Why: for our survival as individual organisms.
A model is a simplified representation of something more complex. Some models can be allowed to dynamically change, and we call that a simulation. Our mental world model has a lot of simplifying to do. Our actual physical reality is a seething, quantum ocean of fields and vibrating, twinkling particles. In order to survive and thrive, we have to extract from that a world of solid objects, smells, terrain, temperatures, textures, threats, and treats. We have only a few error-prone senses to gather raw data for this world-modeling process.
Metzinger likened the ego tunnel to a virtual reality (VR). If you want to create a VR for someone you must program a simulation model of their imagined environment. At each successive moment, the simulation must predict how that environment will appear to them in the next moment, and the sim must include the person’s behavior in the VR, such as how they are “moving” and where they are “looking.”
A popular theory called Predictive Processing says that the brain does exactly those things in real life.
“We are not cognitive couch potatoes idly awaiting the next ‘input’, so much as proactive predictavores” — Andy Clark, Surfing Uncertainty
Predictive processing argues that the brain predicts its inputs from the outside world, and corrects those predictions when they are wrong.
Remember my awe at how quickly the mind perceives changes in a massively detailed world? Predictive processing explains that this happens because your overall world model is a hierarchy of predictive models at different levels of detail, all running in parallel.
Think of each model as generating questions or hypotheses. These are compared to input from a lower model. If the input does not fit the hypothesis, then the higher model corrects it, and then feeds the correction up to the model above it.
The lowest level deals with perception. Our senses react to raw stimulation from the quantum world, while higher levels of processing ask questions whose answers constitute parts of our world model.
Your visual sense says there’s an apparently green patch in view. The levels above that might ask: is this really the same green as other patches nearby that are in the shade? Does the shape hold together, or is it a reflection off the water? Did moving my eyes or body cause a change in apparent shape? Are my glasses on? Are my eyes watering?
Above these levels, we start modeling objects. Is that green thing a leaf? Yes, it is, or no, it’s a leafhopper. We then model more complex scenes from multiple objects. Am I in a garden? Is a wind blowing?
The Self-Model Emerges
At the highest levels questions and answers are more complex. They can relate to feelings (”Am I very pleased to see so much green life around me?”), concepts (”Does the new leaf color mean that the season is changing?”), memories (“Did that plant really grow so much since last week?”) and plans (”what can I do to help that plant survive the winter?”).
According to Metzinger, our highest-level model is of an overall Self with a sense of agency (control over action) and ownership over a body. The Self appears as an entity with a remembered history and an imagined future.
Part of the self-model allows us to imagine that other people have self-models so that we can try to predict their behavior. The ability to do this is called “mentalizing”, or having a “theory of mind”. It’s a common concept in the social, psychological, and mind sciences. Our self-model and our theory of others’ minds influence each other. This is especially important during infancy and childhood, when the self-model is being formed. We get so good at this, so quickly, that most kids make entirely mental models of imaginary friends.
What Simulation Theories Explain
Sim theories explain multiple features of consciousness, among them some that Metzinger found to be either vital, or at least common, as part of the What of consciousness.
One is unity. The essence of being conscious is that we experience a coherent world, inside and outside of our bodies, that is a unified whole. This experience is generated by our highest-level model, which binds together our internal thoughts and external experiences.
“The brain constantly creates the experience that I am present in a world outside my brain.” — Metzinger, The Ego Tunnel
Another feature is the present moment. We are only conscious of the immediate present. We deal with passing time by bringing the past into the Now as memories. We may also imagine the future, but this happens within the Now. The Now is purely a construction, and a vital one. By being “in a present” we experience being present ourselves in the phenomenal world created by our world model. We need this because, overwhelmingly, our survival and thriving depend on our responses to what’s happening around us at a given time. Plans and preparation are also necessary, but they too must be enacted at an actual time, some future Now.
A third feature is called transparency. We are aware of our conscious world, but normally not aware of the fact that we simulated it or how we did so. The transparency sometimes breaks, such as during illusions, during drug-altered states, or in lucid dreaming, where we have limited awareness of controlling our reality.
For our own well-being, transparency is automatic and hard to bypass. Metzinger says that we rely on the transparently present world as being “World Zero.” It’s the world we can take for granted as being real, so we don’t get confused by our memories and imagining.
Transparency: so we don’t get lost in visions of juicy steak and wind up on a tiger’s plate.
A fourth feature is mentalizing itself. We could not get along without it to understand others. It started evolving when our primate ancestors began living in more complex social systems.
A fifth “feature” is the extent of limitations and gaps in conscious content. Our experience of consensus reality is often in error. This follows from the nature of a model, which is an abstraction, a simplified approximation of something more detailed.
Attention Schema Theory and Intentionality
Neuropsychologist Michael Graziano took the modeling concept one step further. He said that, yes, we have brain models of many things such as body parts, sensations, and feelings. We also have a higher-level model of the self to account for our coherence in a physical body and our continuation over time. This idea somewhat resembles Metzinger’s Phenomenal Self Model.
But, he adds, we have to budget what “signals” in our world need more processing. We do this by controlling our attention. And to control something you need a model of it — that’s a basic cybernetic principle. So we model our attention.
The model of your attention by definition relates your self-model to the models of things that you think and perceive. For Graziano, it’s your attention schema (the model of your attention) that makes you able to say that you are conscious. The attention schema connects your self to the contents of your consciousness, and that is why you experience things. This idea resembles the philosophical idea of intentionality.
Philosophers such as Franz Brentano and Daniel Dennett assert that consciousness has the property of intentionality: that every mental thing exists as an object. That is, your mental relationship to a piece of conscious content must be as subject to object, as in: you see it, hate it, seek it, know it, remember it, think it, or are trying to grab it.
“a cognitive system … needs an internal model of itself as currently directed at an intentional object. … Such a flexible, continuously changing model .. Allows a system to consciously experience itself as being not only a part of the world, but being fully immersed in it …” — Thomas Metzinger, All about the Ego Tunnel
Intentionality is so basic to our consciousness that Metzinger said we must have a specific model for it: the Phenomenal Model of the Intentionality Relationship (PMIR). It gives consciousness a first-person perspective. He notes that consciousness can exist without this perspective, as in “… spiritual and religious experiences of a certain kind or fully depersonalized states during severe psychiatric disorders.” In these states phenomenal experience is “… not tied to an individual, consciously experienced first-person perspective.”
The content of the PMIR is the set of intentionality relationships in which you are engaged at any one time. Metzinger gives examples of PMIR content, such as “I am someone who is visually attending to the leaves on that tree”, “I am someone who is grasping the meaning of what I am currently reading”, or “I am someone who is intending to get another beer”.
We can relate this PMIR idea to Graziano’s attention model. Remember that the attention schema relates the self (model) to (models of) other conscious contents. This seems to me like a general version of the specific relationships that occur in Metzinger’s PMIR. To unify the two theories we might say that the attention schema is a prototype of, and supports, the intentionality relationships in the PMIR.
Metzinger, however, asserts the opposite: that Graziano’s attention schema “really is one special case of” the PMIR. Graziano and Metzinger carve up the problem of consciousness in different ways. It’s speculative to suggest whether they could reconcile their theories. However, it seems safe to say that, generally speaking, simulation theories do account for a sixth feature of consciousness: intentionality.
Both Graziano and Metzinger believe that their theories provide targets for empirical psychological and neuroscience research. Both say that computers might someday be able to do the modeling that is necessary and even sufficient for consciousness. From a computer science perspective, computers are basically tools for modeling/simulating, and they are extremely suitable and powerful for the purpose.
I have even speculated about how a computer might be “made conscious” by self and world modeling.
Note that my treatment of consciousness theories herein is highly condensed and lacks a lot of their nuances. There are also other strong theories of consciousness that are currently more popular, and all their proponents argue fiercely. Graziano, for example, says that his is the only explanatory theory, and the other theories are merely descriptive.
The Simulating Species
We don’t yet know much about what kinds of brain activity would correspond to a mental simulation. On the other hand, we have evolved increasingly sophisticated ways to simulate things with processes outside of the brain. We went from cave drawings to marks on clay tablets, to verbal and then written language, to poetry, to reasoning, to maps, to mathematical models, to computational simulations, and lastly, to virtual reality systems. Does our increased facility with models now deceive us into thinking of our brains as modelers and our selves as models? Or, does it bring us closer to figuring out the ancient mystery of consciousness — as a self-simulation?
Postscript: In case you think I have thoughtlessly ignored non-physical points of view on consciousness, see my prequel to the current post at:
See also:
Are We Already Living in Virtual Reality? Joshua Rothman, The New Yorker, 2018.
Self Models, Thomas Metzinger et al., Scholarpedia
Toward a Mature Science of Consciousness, Wanja Wiese, Frontiers in Psychology, 2018.
Why should any body have a self? Jacob Hohwy and John Michael. The subject’s matter: Self-consciousness and the body, 2017.
Vanilla PP for Philosophers: a Primer on Predictive Processing. Wanja Wiese & Thomas Metzinger. In Philosophy and Predictive Processing, 2017
The problem with all of these "theories of consciousness" is that none of them even begin to address the actual "Hard Problem". None of them have even the slightest thing to with actually *explaining* why/how there's "something it's like" to be us.
They're interesting theories about cognition, about how the brain processes and makes use of information, but the only way they relate to conscious experience is by *presupposing* a conscious, experiencing subject.
There's absolutely no need for actual subjective experience in any of these models. They could just as well be descriptions of a non-conscious AI's operating system or something. This is true of literally every so-called "theory of consciousness".
IMHO.