Transporting people into a world of the imagination is nothing new, as anyone who has been to a planetarium – or read a book – can testify. But the power of modern AV is making the experience more lifelike than ever, thanks to advances in augmented reality, virtual reality and immersive video.
“Augmented Reality (AR) is probably best known through Pokemon Go!,” says Tim Holmes, director of research and development at Acuity Intelligence. “But more generally it overlays information or graphics on a live scene – so think Arnold Schwarzenegger’s point-of-view shots in Terminator. The information or graphics are projected on to clear lenses or, in the case of apps like Pokemon Go!, the image from a camera is augmented with an additional layer of graphics before being presented on a screen.”
AR applications are wide-ranging, from bringing museum exhibits to life and guiding visitors round a strange city to adding the graphics to Match of the Day.
According to Johan Besnainou, director, France and Spain for Antycip Simulation VR is typically used in b2b for conception, production, design, R&D, simulation and training. In the consumer industry, VR is mainly used for gaming.
Immersion in a new world
While AR is about seeing the real world in a new way, virtual reality (VR) immerses the viewer in a whole new world that is entirely computer-generated. This is usually achieved by wearing wraparound 3D goggles or entering an enclosed space where the virtual world is projected on to every surface.
“With AR you can check out how a new sofa might look in your living room, but with VR you could see how it would look in a house that hasn’t even been built yet,” says Hannu Anttila, vice-president of marketing and business development at VividWorks.
This is just the tip of an iceberg of possible applications that seem to touch almost every sector, from inspiring schoolchildren and training surgeons to designing cars and demonstrating jet engines.
“One of the key elements of VR is the ability for the user to explore the virtual world at will rather than be directed through it,” says Holmes. “Immersive Video (IV, also called 360° video) on the other hand is a more filmic, or linear version of VR that’s been shot and edited together to tell a story, meaning that the same things happen at the same time each time you watch it.
“The big difference from regular film is that IV completely wraps around you so you can look wherever you want, whenever you want. So you might choose to look at different things on a repeat viewing and have a completely different experience.”
“IV’s immersive experience can be further enhanced with stereoscopic depth of field and binaural sound that moves with the viewer’s field of vision,” says Richard Wormwell, head of 360 production at dock10. “IV is mostly being used for experience-based events such as music, sports and journalism.”
AR headsets are similar to glasses, with the AR components projected on to the lenses in much the same way as heads-up satnav in a car is projected on to the windscreen. “Very few functioning AR headsets are available today, although several are expected to come to market in the next year or so,” says Holmes.
Perhaps the most recognised is the Microsoft HoloLens that sells to developers for around £2,700. So most AR today is implemented on mobile phones, using the camera to stream a realworld video with the augmented components shown on top.
“AR is a great method of engagement as it can all take place from people’s own devices with them simply downloading an app,” says Dave Elliott, enterprise business development manager at Holovis. “Hardware investment can be kept to a minimum and content refreshed as often as possible with a simple app update, which could become available as people earn more points to gamify almost any activity but keep return rates high.”
Wormwell explains how AR combines its virtual overlay with the real world: “In a broadcasting studio, an optical tracking system provides tracking data giving precise XYZ co-ordinates of studio camera locations to powerful software. 3D CGIs (computer generated imagery) built using specialist software are then placed in the studio as objects that can be tracked round or interacted with.
“Mobile AR uses GPS to identify a geographical location that’s been tagged with a CGI object, and the device contains software, sensors, compass and digital projectors to display the CGI over a real world object.”
For VR and IV there are two types of headset. The more expensive (£300-£600) units like the Oculus Rift, HTC Vive and Sony PlaystationVR have an integrated screen, but today require a wired connection to a pretty powerful computer or gaming console. The cheaper units (£5-£100) such as the Samsung Gear VR, Google Daydream or Google Cardboard use a mobile phone as the screen and computer and so aren’t tethered to a PC.
High-end VR is starting to become more affordable. “During 2017 PC costs have begun to fall, and Vive and Rift will be joined by a range of cheaper headsets designed by Microsoft but manufactured by companies such as Dell, HP and Acer,” says Michael Boreham, senior analyst at Futuresource who adds: “The Holy Grail is untethered VR, and HTC is working towards this with the launch of a Wi-Fi add-on to allow untethered movement without impacting the VR experience. All-in-one headsets are another alternative. Intel is developing its Project Alloy headset which uses built-in Windows hardware, and several all-in-one headsets are available for mobile VR which use a built-in Android device instead of a separate mobile phone.”
VR may allow users to physically walk around within their virtual world. “Sensors are positioned in the room to track the headset’s location, and controllers can be used to interact with the environment by moving objects around and picking them up, or by clicking buttons or flicking switches,” says Alastair Reece, head of DBpixelhouse.
“There are lots of off-the-shelf VR environments and games, or you can have a bespoke one created. Your aims and objectives will help determine the most suitable software, as bespoke environments need to be built on the chosen platform and aren’t cross-compatible.”
IV content may also be computer-generated, but most is recorded from live action, in either 2D or 3D.
“A rig of cameras or an omnidirectional camera captures the live video from every direction around a fixed spot,” explains Wormwell. “Specialist video editing software stitches the videos and audio together seamlessly, analysing common images and sounds within the multiple camera feeds and linking them together.”
The main drawback of headset-based VR and IV (apart from the inconvenience of having a big lump of plastic clamped to your head) is that it seals you off from everyone around you. The alternative is a virtual environment that can be shared by a group, such as a CAVE – a space where all walls feature 3D volumetric projection and users wear headtracked glasses so the world moves to their true perspective. “This maintains the group experience, and in the case of collaborative design allows teams of people to examine the same detail together,” says Elliott.
The same can apply to IV. “We achieve immersion by designing and building domes, cylinders and custom installations on to which the 360 degree video is projected, so you can be totally immersed accompanied by the rest of the people in the room,” says Jake Rowland, head of business development at Igloo Vision.
But before we get too carried away with the amazing nature of this fast emerging tech, switching to a full digital approach brings financial and human challenges, warns Antycip Simulation’s Besnainou.
“In order to best benefit from VR, there is a need to educate and train people. A number of specialist schools already exist worldwide, but there is still a blatant shortage of professionally trained people who are capable of using VR applications. This is both true for the public and private sector,” he says.
“This situation will improve within five years as more and more schools and departments within schools are created, but we may need to wait 15 years to reach a tipping point where the industry will get enough human resources. Training and learning is a linear process whereas VR technology is subject to an exponential curve.”
Case study: Attol Architecture
Andrew Lucas Studios designed this virtual reality representation of a property renovation in London’s Belsize Park by Attol Architecture. The physical space was recreated using technical drawings, plans and photographs, enabling viewers to explore the property inside and out, and interactively change furniture layouts. The virtual house can be viewed using a Google Daydream or Samsung Gear VR headset, or via a touch screen on a mobile phone or tablet.
Case study: Cedar Point
US theme park chain Cedar Fair has worked with Holovis to gamify its Cedar Point park in Ohio by using augmented reality to integrate virtual storytelling content into the physical park. Visitors download a free phone app, join a virtual ‘team’ and compete to earn points and accomplish various challenges hidden throughout the park. CGI content appears to interact with physical fixtures of the park, and even queuing is enlivened with special themed content. Benefits for the park range from increasing dwell time in restaurants and shops to generating more coverage on social media.
Case study: virtual shopping
Virtual reality is replacing real-world research in costly bricks-and-mortar test stores. Acuity Intelligence’s AcuityVR tool helps brands and retailers to explore new concept store designs and test how products will perform in store using VR with integrated eye-tracking technology.
VR is also being used by IKEA and Amazon for kitchen design and furniture retail, and could enable brands and retailers to set up unique and even personalised shops in virtual malls, says Acuity.
Case study: Near miss simulator
Holovis developed its virtual reality Near Miss Simulator to help construction firms combat complacency when working at height. Users are immersed into the environment with a VR headset, spatial audio, a motion platform and special effects, and given a series of interactive tasks to complete by a virtual coach. At any moment the coach can trigger a slip and virtually drop the user to the ground. It’s scarily lifelike, say those who’ve experienced it.