Mixed Reality Technology Advances “Mirror Worlds’
Virtual Reality is a disruptor. VR is disrupting human behavior and how we behave as a species. In the past 20 years there have been two other big disruptors: the internet and the smartphone. Virtual reality will revolutionize our world in more profound ways than the Internet. Let’s look at how the current state of 360 video will become part of mixed reality.
We don’t know right now how VR and AR will eventually look. That’s the fun part of being a VR Producer today. I guarantee you that it won’t be a brick of plastic strapped to your face. Today we see AR/VR as a title for many articles, talks, and as a descriptions for conferences. I predict the need to distinguish between VR and AR will dissolve soon. Mixed Reality (MR) will become the common term we use to communicate where and how we had an experience.
Wearing stylish glasses with high definition screens isn’t too far off. The decision to shut out physical reality or overlay imagery and data will become ambiguous. Hopefully AR will return to it’s roots as the acronym for Actual Reality!
Mixed Reality + Physical Reality = Voxels
Augmented reality is more complex technology than virtual reality, but it’s catching up. Our mixed reality glasses we wear in the future will have cameras on them. Cameras that scan and analyze the physical space in front of you to create voxels.
A voxel represents a value on a regular grid in three-dimensional space. Similar to the two dimensional pixels in a bitmap images (JPEG/GIF) we use today. Voxel data will beam up to satellites and back down to your experience of physical reality with images, animation, and text data precisely overlapped on your vision of physical reality. Learn more here : Pixels and voxels the long answer.
360ยบ Video Is Immature Technology for AR
Right now we have two forms of 360 video for virtual reality. Monoscopic (2D) and stereoscopic (3D). The point of view (POV) is from one position. Camera movement can give a nice illusion of movement, but it’s not giving the viewer agency to move around freely.
360 degree video serves it’s purpose for both mono and stereo 360 VR. The immersive quality of being in live action is here now, but we want more. We want room scale freedom of movement in a 360 VR scene. Let’s look at the future of 360 imagery and how it will be part of augmented reality.
360 Photogametry
The next phase of photo real 360 imagery is photogrammetry. Photogrammetry is the science of making measurements from photographs. The output of photogrammetry is a 3D model of some real-world object or scene. With a 3D model made from Actual Reality (AR) we can have agency in the 360 VR experience. Adding props, products, and videos on walls, will make it more interactive.
360 Light Field Imagery
The wholly grail of photo realistic mixed reality 360 imagery is light field technology. Light field camera sensors capture information about the light intensity emanating from a scene, and the direction of the light rays in space. A more accurate spacial perception with the ability to change focus is the result.
Why is light field the wholly grail of 360 VR image capture? Our brain-eye system uses inputs of optical depth, light depth/shape, and focus depth commingled together to produce our visual perception of reality. Light field capture will give us the closest photo real representation in the future mixed reality media productions.
Suspended belief without cognitive overload is the key to successful 360 VR production today. Ultimately we want the empathy of film with the agency of games and the experience of life.
Convergence of the next phases of photogametry and light field technology, with 3D spacial audio, will enhance the goal of suspended belief. Not subjecting your audience to cognitive overload will stay in the realm of a good VR Producers.
In five years we’ll could have light field 360 cameras and more mixed reality experiences available. The 360 VR we have now is very compelling and how companies that are using VR to reach their customers are seen as more innovative.