Technology innovation agency, Inition, has produced a virtual reality experience for Dubai-based system integrator, DigiComm that takes visitors to the UAE’s iconic Burj Khalifa on a high-flying spy mission. When wearing the VR headset users transform into an undercover operative attempting to prevent an impending cyber-attack.
A multi-sensory exeprience
The Mission 828 experience sees the user scale 828 metres of the Burj Khalifa – the world’s tallest structure – to intercept an overhead satellite, retrieve secret devices and base jump from the building’s summit. By making use of multi-sensory technology and effects such as physical cues, 360-degree spatial sound and simulated wind, the VR creation produces an immersive experience whilst showing users some of Dubai’s most spectacular views from a bird’s eye perspective.
The experience was built using the Unity game engine and intended to be viewed using the HTC Vive headset. High-end CG elements were overlaid with real world footage to ensure the experience was as realistic as possible and 360-degree scenes were captured at height using drones.
To film the static Dubai footage Inition was given permission to climb up to the top of the Burj Khalifa’s spire; a privilege only a few people have enjoyed. Footage was shot in multiple directions in super high-res and stitched together as a full 360-degree 16K frame using Autopanao Giga software.
“We shot over 30 motion plates which we distorted and composited back over the top of the equirectangular plate to bring the world alive,” says Lee Spooner, lead artist at Inition. “This final 8K 360 video was used as a background environment for the full VR experience.”
On top of the world
For the base jump portion of the experience the team needed to create an experience that would make people feel like they were jumping off the tallest building in the world. Content was captured using a Matrix 600 drone filming in 6K 360/3D and fitted with an INSTA 360 Pro camera which was flown up from the ground to the top of the spire.
To obtain sign off from Burj Khalifa Security, UAE Aviation Authority and UAE Military Inition needed to fit GPS monitoring and recording kit to the drone which later proved useful in modelling the sequence.
“On the day of filming, despite the drone pilot having a clear line of sight of the drone, the piloting signal strength was subject to intermittent interference from the mass of the building and the alignment of satellites required to safely drive the drone,” says Spooner. “The point of the shot was to fly the drone as close to the side of the building and as safely possible, which prevented it from flying higher than just over 300 metres.”
Therefore, footage was only captured from halfway down the building to the ground. The team still had to find another 500m of content from the top. First Inition sourced a 3D model of the Burj Khalifa and added textures and shaders shot on the day to match the drone footage visuals.
“We added the static 360-degree footage as a spherical backplate and animated a camera jumping off the spire. The motion was camera matched to the drone footage so they ended up in the same place. This was rendered in 360 at 8K using V-Ray software and 3ds Max,” says Spooner.
Stabilising the sequence
The footage captured by the drone was shaky due to strong winds, but stabilisation was achieved using tools within After Effects. Inition also created a believable transition between the rendered sequence and the drone footage by using the moment the realtime CG parachute was deployed to suddenly jerk the user’s viewpoint up to give the sensation of a sudden change of speed and direction. The whole sequence was colour matched and 360 motion blur was applied to the first half to add a sense of speed and to make the full sequence seem like one consistent realistic piece of footage.
As the drone could only gain altitude at a certain speed the parachute drop footage was 10 minutes long when 30 seconds was required. Speeding up the footage made the traffic and surrounding roads seem to be travelling at over 200mph. “For normal static shots this would be a simple paint out job,” says Spooner, “but we had the complication of it being a moving shot dropping down over 300m in height and the even more technical challenge of it being equirectangular footage, with the distortion spherical footage applies.”
Inition tried to isolate the footage using standard tracking workflows, but the most successful solution was using 360 video product Mocha VR’s new planar tracking tools to isolate the roads and remove the moving traffic using clean plates.
A VR learning curve
An increasing number of tourist attractions are adding value through complementary immersive experiences such as Mission 828. Inition recently created a similar vertigo-inducing VR experience for The Shard, giving them the opportunity to experiment with convincing users’ brains they’re at immense height without making them feel nauseous.
“This learning was invaluable when it came to creating the project’s all-important realism, especially with multi-sensory inputs,” says Inition’s ceo Adrian Leu.
“But already being well-versed in creating VR height experiences didn’t mean there weren’t other challenges to overcome, such as flying drones to capture 360/3D images at altitude and combining 360-degree footage with CGI in a seamless way.