Ultimate VR

In his book Reality+ philosopher David Chalmers restates a common view about the future of VR.

My guess is that within a century we will have virtual realities that are indistinguishable from the nonvirtual world. Perhaps we’ll plug into machines through a brain-computer interface, bypassing our eyes and ears and other sense organs. The machines will contain an extremely detailed simulation of a physical reality, simulating laws of physics to track how every object within that reality behaves (xiii).

Chalmers entertains the possibility that we may now be living in a simulation — echoing the so-called simulation hypothesis, a provocative thought experiment that certain philosophers like to recruit to test ideas about reality, embodiment, evolution, mind, intelligence and consciousness.

Under this scenario, the ultimate VR achievement will be “an extremely detailed simulation” of a shared geometrically ordered physical universe, complete with rules, laws and interactions in which objects and elements grow, compete, decay, reproduce and persist.

As the simulator is hypothetical, it can be projected so far into the future that we don’t need to worry about the impossibility of such a scenario or the amount of computational power needed to maintain a persistent and highly detailed artificial universe, let alone multiple universes, or simulations within simulations of universes.

Why VR?

In the 1990s I published a book (Technoromanticism) in which I attempted to explain why the idea of virtual worlds appeals to so many people — or at least I tried to position the concept of VR within a legacy of utopian and dystopian thinking in which people invent a future and project themselves into it — and out again.

The idea that you might enter and exit an alien virtual version of the world you inhabit has something in common with deep-seated psychological and cultural memes about rights-of-passage as explored by anthropologists Victor Turner and Claude Levi-Strauss as well as Sigmund Freud and his acolytes about the allure of returning to the security of the womb — the Oedipus complex.

The fantasy of ultimate VR is not just an example of technical extrapolation, but speaks to certain anxieties, hopes, dreads, and expectations, wrapped up in a series of culturally charged imaginings that draw on the legacy of Plato’s cave, the Garden of Eden, the descent of Heaven to Earth, varieties of religious experience, the philosophies of idealism and Romanticism.

Perceptual VR

I think that VR as a vehicle of philosophical reflection is overrated. I am also surprised at the barren account of VR on which it is based, relying as it does on detail in 3D modelling, an infinite regress along a vector of ever increasing geometrical, textural, sensory and interactive detail — albeit with direct conduits to the brain.

Here’s an alternative rendering of the technical VR challenge on which to base a more perceptually-based simulation hypothesis.

1. Attention-based models. The world is not ever present to us in geometrical detail, but according to our actions and needs. We direct our attention according to the task at hand, that in turn affects our perceptions. The technical VR challenge should be to detect what the VR visitor is attending to and expecting, and to provide the perceptual stimuli appropriate to that task. If we take on board some of the ideas advanced by Anil Seth about the role of imagination (hallucination) in perception, then the human cognitive apparatus may even fill in the blanks without the need for endless detail in a digital model.
2. Persistence of experience. Then there’s the question of persistence. If you come back to the same virtual place then you expect things to be as you left them. Rather than devoting digital memory to maintaining a vast universe of 3D information, the simulation could be based on mechanisms to keep track of the sensory experiences of the individual visitor.
3. Shared experience. But it is a shared universe. So the simulator has to keep a record of the experience of all visitors, especially where the experiences overlap and interact. Second Life, Minecraft, Decentraland and massively multiplayer online role-playing games address the challenge of a persistent and shared world by maintaining large databases of geometrical, textured, interactive objects and spaces. All players/visitors access the same database. But the idea is not scalable; there’s the problem of storing and updating data and the computational costs of bringing details into view — e.g. loading “levels” into local memory as needed in gaming. A shared perceptually-based VR experience poses severe and interesting technical challenges that require solutions other than mega-scale databases of spatial coordinates and programs.
4. Diversity of experience. Not everyone’s experience of the world is the same. The perceptions of the world do not have to align so closely. I once undertook a course in basic plant identification. What had hitherto appeared to me as world of undifferentiated swathes of greenery were suddenly transformed into a diverse array of plant types and species. People with other areas of expertise and experience see the world in ways that evade my own perceptions. Seeing differently enlivens teams and social interaction. The simulation hypothesis ought to accommodate different ways of seeing, and provide facility for learning, agreeing, disagreeing and negotiating what constitutes the shared perception in view.

If the simulation hypothesis is to work, as a plausible speculation, then it needs to present technological innovations that accord more closely with the operations of shared human perception as we understand them. As yet I know of no VR platform (e.g. Unity 3D) that supports this kind of perceptual VR.

I still think that any ultra VR that supports the simulation hypothesis is impossible. Perhaps at least this proposal for perceptual VR or its variants helps clarify further the nature of those impossible challenges.

References

• Chalmers, David. Reality+: Virtual Worlds and the Problems of Philosophy. London: Penguin, Alen Lane; Kindle Edition, 2020. 
• Coyne, Richard. Technoromanticism: Digital Narrative, Holism, and the Romance of the Real. Cambridge, Mass.: MIT Press, 1999. 
• Seth, Anil. Being You: A New Science of Consciousness. London: Faber, 2021. 

Note

• Featured image shows a screen display from Obduction, the immersive puzzle game by Cyan.