When talking about VR, we usually hear the word immersion or immersive content. Did you ever stop to ponder on the meaning of the term “immersion”? Is it just connected to this technology? And to what extent is this VR experience Immersive? What are the limits, if they exist?
Probably not! So just stick with us through this quick explanation of what exactly it is that makes a VR Experience immersive, and what are the breakthroughs lately that made this experience even more “immersive”
To begin with, let us get something straight, the word immersion according to Nikolayi Engelmann in his book “Virtual Reality Gaming” means:
“Immersion (the technical term for immersing oneself in an artificial world) describes the effect caused by a situation, environment or graphic representation which makes the user’s consciousness recede into the background so that the virtual environment is perceived as reality.”
Nikolayi Engelmann
There are two types of immersion: physical and mental. The physical one tends to stimulate the mental one. I am not going to bore you with tons of definitions and a lot of technical jargon but let me just explain to you briefly what each is.
Mental immersion happens when the user dives right into a certain story line. For example, when you submerge yourself in a book or the state you reach when you are very hooked to the plot of a movie and not aware of your surroundings any more.
Physical immersion is the cherry on the top; all the physical things that our diverse senses perceive increase and strengthen mental immersion. Meaning the moment we have the VR headset on, everything we perceive while turning our heads will always be in that virtual reality environment.
We are “shielded” from the outside world.
What could increase this immersion you ask? It is pretty intense as it is.
I am quite sure that most of you have been to a 4D film theatre, those who have not please do! Pirate movies are extremely fun with 4D but let’s stay on the subject here. 4D film is a marketing term for an entertainment presentation system combining a 3D film with physical effects that occur in the theatre in synchronization with the film. Effects simulated in a 4D film may include rain, wind, temperature changes, strobe lights, and vibration. The feeling of immersion here is very real and let me tell you that that feeling can be achieved especially well through virtual reality nowadays and it’s much easier than with a normal TV series or a film.
As I said, the moment we put the headset on, we are in a different realm, our bodies and brains are more prone to trust what we see. We all too often let ourselves be deceived in our everyday lives by what we see. Our brain can quickly adapt to Virtual Reality and accepts the “unreal” world. But what about our other senses? the body sends signals that it is actually standing or sitting still.
Some of these “aspects” have been solved and made the immersion experience so much more intense. Here is a list of all these breakthroughs that we once thought were fantasy.
Realistic Sound
Being able to tie a sound to its location is an important cue for users navigating the virtual world. A lack of spatial sound or a mismatch in the perceived source of the sound will draw the user’s attention away from the virtual world. For that reason, most or all sounds should be spatialized
That’s why audio has become an important area where VR leaders are focused on innovation. The latest advances in VR audio technology – including sophisticated “spatial audio” solutions – are making improved sound “localization” possible in virtual environments.
Before we get into details, you should have a brief idea on how our brains process sounds.
While processing visual input comes naturally to us, there is actually no difference between seeing in real life and in the virtual world, processing sound in VR is a whole other story. The way our brain computes sound, is a bit more complex: it processes the sound location based on cues for things like direction and distance. We recognize lateral direction – aka a sound’s position to our right or left for those of you who slept during bio and physics lessons – using factors such as:
Interaural time difference (ITD), the difference between the times sounds reach the two ears
Interaural level difference (ILD), the difference in sound pressure level reaching the two ears
And that’s just right to left. Our brains also need to recognize the location of sounds in front or back of us, or above or below us.
To do that “localization,” we clue into the ways that sounds originating from different directions interact with the geometry of our bodies differently: The unique shapes and sizes of our heads, necks, torsos, and outer ears cause changes in sound filters and reflections that our brains use to infer direction and elevation. These localization cues are just a few of the factors our brains consider when determining sound position. There is also the distance localization, the time delay between the event and the sound reaching our ears, the mixture of source of sounds, etc. All of these considerations (among others) play into the development of VR audio technologies, which use specialized recording systems and algorithms to mimic natural audio.
“Binaural audio is critical to an immersive experience within the context of VR. We consider audio to be 50 percent of the immersive experience.”
–Adam Somers, Engineering Manager of JauntVR
Headsets nowadays take into consideration all of these aspects and more, there are even special VR headphones, e.g. 3D Sound Labs sells headphones with motion sensing technology customized for each user. Dysonics also sells a motion sensor that can be attached to existing headphones to make audio more location-smart. The Oculus Quest for instance has built-in speakers that support spatial audio, and they work well. The time when we used speakers in each corner of the room is long behind us.
The spatial audio algorithms will continue to improve… but only as fast as VR adoption increases: Improving the algorithms will demand more user data, which tech companies will only get if more and more people use VR.
Freedom of mobility
For a VR device to create fully immersive experiences, it must blend real-world elements into the virtual one, and enable natural interactions with the digital content.
“A realistic virtual reality experience requires freedom of movement. Being connected to an external computer with cables handicaps a user’s natural movement. The future of virtual reality must “cut the VR cord” allowing the user to operate freely, immersed within the experience.”
These words seemed so impossible to bring to real life years ago but look where we are at now, there are currently nowhere near as many untethered VR headsets available in the burgeoning virtual reality market, compared to PC VR (tethered VR) and smartphone VR.
These self-contained headsets (made popular by releases such as the Oculus Go and Oculus Quest) are uncabled, i.e. users don’t have to limit themselves to a restricted space. In general, they offer a more powerful VR experience than smartphone VR but less powerful than tethered VR. This is about to change with 5G and the various resources put on developing additional HMD’s for standalone VR.
Intuitive Controls and Interaction
It should be easy to do things like select, grasp, manipulate, carry, throw, and place objects or other environmental features. The controllers are getting better and better, more intuitive and more natural but holding a device like this in your hand significantly dampens your immersion and lowers your VR experience quality. A more natural and intuitive way to interact with virtual and augmented reality would be to directly use your hands and fingers as the input device – like in the real world. And we are soon getting to the point where we will not need the controllers anymore. The level of interaction is improving so much and with the introduction of hand/finger tracking with full degree of freedom, the controllers’ days will soon be behind us. As I am writing this article, Apple is working on fingertip and face mapping tools for VR and AR. Who knows what tomorrow will hold? I mean we are even talking about VR Synthetic ‘Skin’ and how it is bringing a Sense of Touch to Virtual Reality
It is still in the prototype phase, but even the prototype surprised so many people and made the experience more intense.
Virtual reality’s capabilities have come so far that a remarkably convincing sense of immersion is now possible, offering worthwhile immersive experiences to users.The future of virtual reality will go beyond computer-generated content by bringing real-world objects into the digital environment in real time. We are already seeing early prototypes of these integrated examples, such as through the ways Virtual reality is being used in the medical field, for instance. Intel’s work with Surgical Theater, which let surgeons rehearse difficult procedures with their own surgical tools within a virtual rendering of the operation environment. This allows for not only better medical training, but also for the patient to be brought along on the journey.
This journey will certainly take time and a lot of hard work, but we are excited to be a part of it and excited to see STAGE evolve. The next step we took was with the Oculus Quest so you can now enjoy using STAGE while