How to Escape the Matrix: Part 1

Let’s do a little thought experiment together. Imagine that last night while you slept, while you were in the deepest unconscious stages of sleep, you were drugged and kidnapped. Your body was moved to a secret location, where unknown individuals conducted complex brain surgery to connect your sensory systems directly to a neural interface and virtual reality system. When you wake, you find yourself immersed in a virtual world much like the environment envisioned in the Wachowski Brothers film The Matrix.

Our thought experiment is a restatement of the philosophical conundrum known as the ”Brain-in-a-Vat”. In the BIAV scenario, a brain is removed from its body and maintained in a vat wherein it is provided sensory inputs are generated by a computing system of some sort. Both The Matrix and the BIAV are extensions of technologies that actually exists (at least to some extent) and which are known commonly as Virtual Reality or VR.

Brian in a Vat experiences walking
From http://en.wikipedia.org/wiki/Brain_in_a_vat

Within the existing state of the art, VR systems employ a variety of sensing systems to measure the body of the participant and then use conventional computers and audio/visual displays to produce an illusion of physical reality. For example, the typical virtual reality configuration includes a low latency tracker to measure the position and orientation of the user’s head and also a head mounted display with two separate video channels providing an illusory three dimensional world to the user’s eyes.

A virtual reality system includes a Virtual Reality Generator, or VRG for short, which is a special purpose computational system designed especially for the purpose of producing a virtual environment for presentation to one or more users. When connected to the sensors and displays the VRG performs all needed computations to produce the virtual reality to be experienced.

The VRG is implemented as software program running on modern von Neumann style digital computers using multi-core processors, conventional operating systems, etc. and these systems are in turn interconnected by conventional computer networks, i.e. The Internet. Through this software, the user or “participant” of the virtual reality system is embedded within a sensing-simulation-display (SSD) loop.

The SSD loop gives rise to the virtual reality experience, which the user experiences via various displays as a place or space. As the user moves or acts in the real world, these movements are transformed into corresponding acts in the virtual world. For example, if I lift my hand, my computer-generated avatar should lift its hand in the identical manner and into the same gesture as my real hand.

Virtual reality displays include the already mentioned head mounted stereoscopic display or HMD, but also include various projection technologies such as the CAVE, 3D sound display methods, tactile displays and force feedback systems, and more. The process of generating these displays is called rendering and the process of measuring the user’s body position and actions is known as sensing. Interestingly both the process of mapping the user’s body coordinates into the coordinate system of virtual reality and the rendering displays involve similar types of mathematical transformations.

The world simulation itself is a software program or system that performs additional computation in addition to these needed transformations. The purpose of these computations is to model some aspect of the real world or generate the virtual place. For example, the simulation software could represent the physics of an automobile race and include representation of the objects related to this simulation. The simulation software models the states of all objects within the world and computes any interactions between objects and the dynamics of these objects and systems over time.

If the construction of the virtual environment is successful, the user will experience a feeling of presence in the virtual world, an experience best understood as the sensation of “being there” or being somewhere at least which of course begs the question of where one is exactly when immersed in virtual reality. A related notion is the idea of immersion, which is the sensation of being “inside” of a place or space. Finally, the user may experience suspension of disbelief and accept the virtual environment as “real” and will respond as if the situations encountered were really happening. For example, a person might cower to avoid being hit with a virtual weapon in the virtual environment.

The VRG is a computer; in essence not that different from the computer I am using right now to write this essay. The first true virtual reality generators were digital computers and developers use standard workstations and personal computers with added graphics hardware to accelerate production of images and offload certain mathematical operations.

Back to our thought experiment, in order to escape The Matrix, first we need to know it exists. We assume here that you do not know with certainty that reality has been replaced with a simulation. After you are secretly kidnapped and interfaced to The Matrix, your experience of the simulated reality seems perfect. Except something is wrong and what it is, is hard to describe exactly. Your experience of the reality is somehow “wrong” and you might be diagnosed (in the virtual world) with something like the Capgras delusion in which the victim believes friends and relations are imposters. Is there any way for you to prove the existence of The Matrix and the reality of your situation as a BIAV?

Here for the purposes of our thought experiment we assume a conventional VRG with an advanced neural sensory interface. That is, we are going to forget about all the difficulties of the rendering and display part of the problem engendered in current VR technologies, and assume a “perfect” neural interface exists such that the induced sensory impressions cannot be directly distinguished from real sensory impressions. At least at first glance, The Matrix seems real enough. It looks, sounds, and even tastes just like it should.

However real it appears to be to the subjects in world, the world experienced via The Matrix can be seen to be an illusion if we know how it is generated and can see the system from outside. The situation is further understood by considering the mappings between various elements of the virtual reality system and the real universe in which it exists as shown in the figure below.

We will assume as shown in the figure that the VRG exists inside of a universe or reality we call “U” in which the VRG is constructed. This is the “real world” we are experiencing right now. The participant or user of the virtual reality system also exists in this universe U but they experience the virtual world “V” an alternative universe or reality generated by the VRG. Their experience occurs via an interface labeled “I”. The VRG is a conventional but very powerful computer of some sort, but fundamentally it is a Turing Machine and all world V that it can generate are Turing computable.

We will consider that the embedding universe, U, has a set of physics that govern its existence and dynamics. The subject of our experiment knows the laws of physics of universe U and we ask whether they can conduct any experiment in the virtual world V that shows it is not equivalent to U. The answer is yes, in fact many such experiments are possible.

For example, consider measuring the speed of light c. We know that in the universe U, our reality, the speed of light is 299,792,458 m/s. This is form light in a vacuum and the observed speed of light varies depending on the medium through which the light is passing and the wavelength of the light.

Note that all signals in our VRG exist in the universe U. Therefore none of them may exceed the speed of light in U. Further, all signals within the VRG must propagate slower than c since the device is made of some material (silicon, copper, fiber optics, etc.) If we try to measure the speed of light in V therefore, we always get a value less than c even in a virtual vacuum. Further, if we are careful, we will notice some other aberrations in our measurements that signal the presence of the simulation.

For example, even with our advanced neural interface, signals must propagate from the interface hardware present in the brain, to the VRG, and then back again. The so-called round trip time is irreducible and cannot be eliminated from the environment. This effect also impacts the ability of the VRG to model the physics of embedding universe.

Consider a two person multi-user virtual world with a shared VRG sometimes known as the world server. If the first user sends a signal, say by turning on a virtual flashlight, this signal must propagate to the second user through a communications network and computer and this propagation will always be slower than the speed of light c in the embedding universe. Further, if we now consider a many user world, the speed of a light signal measured will vary depending on the distances between users in the universe U and not only their distances in the virtual world V. This aberration should be detectable by a carefully constructed virtual experiment in V.

Notice however that there is nothing inherent in the virtual world that limits the speed of objects at all. That is, we can build a virtual environment that allows virtual objects to have velocities greater than c however even with these worlds, no signal can propagate as fast as the speed of light c. Note that this means that the physics of the virtual world V must be different than the physics of the embedding universe U. This is only one example, and a careful analysis of the physics of V shows that there are a number of ways in which the physics of V must differ from that of U.

It is worth noting here that the first virtual environments barely modeled physics at all, for example objects did not have mass, and movements were computed by simple integration of accelerations and velocities. Relativistic effects such as time dilation and length dilation which are observed in reality are not modeled at all in current virtual reality environments, and notably even the rendering process does not generally take these effects into account. Note that these types of differences in the physics of V can be detected, and our subject can use these to determine his predicament.

In his seminal paper, Nick Bostrom proposed The Simulation Hypothesis, which suggests that our reality may itself be some sort of simulation created by entities unknown. In other words, Bostrom is proposing that our reality is itself a sort of simulation or “matrix” in which we find ourselves embedded. We might sensibly ask ourselves, “Is there some way to detect this situation if it is true?”

The previous discussion outlines some possible physical experiments we might conduct to detect simulation artifacts and related effects. These effects do not appear in our reality as it is, that is, physical experiments conducted by scientists would produce different results if our universe were this type of virtual reality world. We can say with some certainty that reality is not a simulation or at least it is not any sort of simulation that humans currently know how to build. And certainly not the sort of system envisioned here. Reality as it exists cannot be a program in a network of modern digital computers.

While performing physical experiments in the virtual world can illuminate its virtuality, it does not obviously lead to a means of escape. Our subject can learn of his situation, but it is less than obvious how to use this knowledge to engineer any type of escape “hatch”. Specifically, since his motor neurons have been disconnected from his body his actions such as walking, running, talking, etc. only take place in the virtual world V and not in U. In the embedding universe, all that happens is some nerve impulses and computation in the VRG; there is no person actually running or walking etc. and the subjects body (if he still has one) does not move. If there is no means for an action in V to have an effect in U, then we can see that our subject can only learn of his situation but he cannot ever hope to escape back into U.

Today’s dynamic physical simulation systems have quite limited capability and can only simulate the interactions between small numbers of objects. More importantly, the more interactions and objects simulated, the higher the computational costs in the VRG. In fact naively constructed dynamic physical simulations scale badly, and developing methods to allow for more scalable systems and larger more complex simulations is an active and open area of research.

Note that all computations in the VRG are physical events in the universe U, and further all computations in the VRG require energy inputs and produce heat as a side effect and by product. By increasing the number of computations in the VRG, a person who is trapped in the virtual world V may still have an effect in U. This effect is limited to an indirect increase in the consumption of energy and production of heat however. But nevertheless, this leads us to suggest one potential means of escape, overloading the VRG and crashing it. For example, our subject might force the VRG to compute the dynamics of a very large set of interacting objects such as millions of marbles or balls bouncing in a constrained area. Although beyond the scope of this article, it is also known that certain types of quantum simulations scale badly on Von Neumann multiprocessors.

The computational costs of simulating the dynamics rises exponentially, and so does the energy consumption and heat output of the VRG. If enough objects can be introduced into the virtual world, the simulation might crash or the VRG could overheat and cease functioning. However, it seems likely that the architects of the world would anticipate this possibility and prevent this sort of attack. Existing virtual worlds such as Second Life and the Sims Online have suffered proliferating object attacks in practice and already include some safeguards against them. It is easy to imagine the hypothetical architects of our more advanced system protecting against this possibility.

Connecting the outputs of a specialized software program running in the VRG to the neural interface produces the simulated world V. But since this is nothing more than a computer program, it may contain bugs or fail to handle various error conditions. These types of flaws are exploited in attacking existing computer systems such as web servers. We might consider for example the use of buffer overflow attacks. It is plausible that the virtual world software will have flaws of this sort that might be exploited by someone with some knowledge of how the VRG software is constructed.

Finally, many programs of this sort such as existing video games include a “super user” mode that is most commonly part of a debugging console or test environment used during development. Many of the cheat codes used by gamers, access these sorts of controls or tools although they are usually intended to be disabled (or at least hidden) from the players. It might be possible for us to access a similar debug facility from within the virtual world if it exists.

There is however one major problem with this entire line of reasoning. While it might be possible to perform physical experiments to detect the presence of a simulation, and further while there might exist possible modes of “escape”, this all hinges on the subject remembering the laws of physics of the embedding universe U. and believing the results of their experiments. If I don’t recall the speed of light in U and I have no preconceptions about how the physics should work, I can’t detect these minor anomalies. Perhaps more correctly, I would detect them but not consider them noteworthy or indicative of a simulation.

Both memory and belief have neural correlates and any technology sufficiently advanced to present a complete sensory interface at the neural level would also be capable of interfacing with these other subsystems in the brain. An interface to memory could for example erase the subject’s memory of the physics in U or even of the embedding universe’s existence. This would eliminate the subject’s ability to compare the observed physics of the virtual world with their prior knowledge. They would instead remember only the physics of the virtual world V.

Further, such a system could directly interface with the mechanisms of belief and thought. In practice it might be possible to edit thoughts before they entered conscious awareness. The subject would made to believe The Matrix was real by direct neural stimulation, and any thoughts to the contrary would be jammed or erased before the subject could became aware of them.

In fact we don’t even need a neural interface to accomplish this, as there are pharmacological approaches that achieve similar results. One substance that has recently received some attention in the popular press is scopolamine, which in sufficient dosages can eliminate the subject’s ability to exercise free will. In combination with presentation of a virtual environment, the effect can be total belief in the virtual world as real. The subject can simply be told to suspend disbelief and they will do so.

There is no obvious means of escape from this situation. Any evidence that the world is a simulation will be erased from your memory. Any belief you hold that the world isn’t real, will be erased or altered. Any actions you take that might overload or break the VRG system will be detected, blocked, or reversed. How can you possibly escape?

In the film, Neo is presented with the choice of two pills: the red pill and the blue pill. If he takes the blue pill, he will “wake up in [his] bed and believe whatever [he] want[s] to believe.” Alternatively, if he takes the red pill, he will “stay in Wonderland” and learn “how deep the rabbit hole goes”. The operation of the red seems to show how the subject how The Matrix operates on the subjects’ mechanisms of belief. Taking the blue pill renormalizes the subject erasing any conflicting memories or beliefs that might lead them to dis-believe in the illusory universe V (in a very similar manner to that already described above).

Look around. Are there any things in our environment that operate as the imaginary red and blue pills? Do there exist memetic systems, mechanisms, or even chemical substances that reinforce prior beliefs and lead people to simply believe what they are told? Can we invent a system or method that frees us from this sort of trap?

Timothy Leary and Robert Anton Wilson discuss the notion of our individual reality tunnels, and how these are generated in the mind. Charles Tart expanded the idea to the social sphere with the notion of a “consensus trance” from which we must “wake up”. Various methods from Eastern martial arts traditions, various meditation practices, and Western hermetic science and esotericism, psychedelics and related technologies and techniques all come to mind here. But there seems to be a problem.

Isn’t our knowledge of the possibility of a red pill itself contradictory with the idea that our universe is a belief manipulating Matrix or similar system? How can we even imagine the red pill if we are already trapped in The Matrix? Why would the architects ever let us even imagine freedom? The problem is perhaps related to the conundrum of our imagining non-algorithmic processes in what appears otherwise to be a purely algorithmic computational universe.

Setting this apparent paradox aside for a moment, we might also wonder as above if there is some method or information that we might employ to make our own version of the blue pill? What does the blue pill actually do? Is it a thing or a process or ??? Is there a tool or method that we can use to come to know the truth?

(to be continued)

References

The Matrix as Metaphysics, David Chalmers
The Simulation Hypothesis, Nick Bostrom
http://en.wikipedia.org/wiki/Capgras_delusion
http://en.wikipedia.org/wiki/Dynamical_simulation
Limitations of physics engines: http://www.youtube.com/watch?v=GMQr8xfgTrY
MonteCarlo scaling in quantum simulations http://arxiv.org/pdf/cond-mat/0408370.pdf
Pentland’s article: http://dl.acm.org/citation.cfm?id=91444
Scaling MMOs: http://www.cs.cornell.edu/people/~wmwhite/papers/2009-ICDE-Virtual-Worlds.pdf
Buffer overflows: http://en.wikipedia.org/wiki/Buffer_overflow
Neural correlates of belief: http://theconnectome.wordpress.com/2011/10/28/brain-scans-lucid-dreams/
http://www.biopsychiatry.com/scopolamine/borrachero.html
http://en.wikipedia.org/wiki/Red_pill_and_blue_pill

Peter Rothman is an independent researcher, applied mathematician and computer scientist. He was one of the earliest innovators and developers of commercial virtual reality applications and did award winning research in the field of dismounted infantry simulation for the U.S. Army. Mr. Rothman designed and developed several early simulation environments, visualization and decision aids for the Air Force and other government organizations in the 1980s and 1990s. Mr. Rothman was one of the earliest developers of data visualization and virtual reality applications for finance, and he pioneered the use of virtual reality for visualization of command and control systems as well as in simulation and modeling of complex systems in the U.S. Air Force.