Creative tech helps crack the code

Museum-goers are improving their espionage knowledge at SPYSCAPE, a new museum and attraction in Midtown Manhattan that places an emphasis on immersive user experiences and sophisticated exhibits. Zoe Mutter examined the AV elements behind the code-breaking spectacle.

Immersive storytelling techniques and customised experiences are being used to explore the world of spycraft at an interactive experience in the heart of New York. Through a multitude of creative and stimulating exhibition spaces, SPYSCAPE is providing detailed and entertaining insight into espionage and intelligence operations. The project’s development began in London in 2014, with the team in New York coming on board in 2016.

Extensive research was carried out to aid the attraction’s construction, including valuable input from experts in the worlds of museums, architecture, experiences, spying, technology, hacking and psychology who provided assistance with core concepts, structures, ideas for visitor flow and experience.

Architects Adjaye Associates’ involvement in the 60,000-square-foot project saw them create a design that draws from the architectural language of significant spy organisations whilst integrating bespoke features such as smoked glazed doors and interactive videowalls with fibre cement elements and dark grey acoustic panelling. Varied lighting strategies, transparencies between floors and screens and perforations helped establish spaces that shift users’ vantage points.

Contemporary digital arts organisation onedotzero also partnered with the SPYSCAPE team at the project’s inception, consulting and working across areas such as tone of voice, branding, architectural interiors, signage, experiences and technology. The consultancy role included providing creative feedback and suggesting technological solutions. Shane Walter, creative director at onedotzero, acted as an executive creative director with Sophie Walter, working across experiential, production and budgeting.

Custom-made laser tunnels test reactions alongside a range of psychometric tests. Players must press the illuminated buttons while avoiding breaking the beams of 13 lasers that criss-cross to form a light maze

Exhibits with impact
Housed within bespoke pavilions, SPYSCAPE’s range of exhibition spaces focus on seven spy themes. The exhibition arrangement plays with users’ perceptions, using partially obscured circulation spaces that open up into immersive, fully interactive multi-media environments. Each pavilion, including a weathered steel drum with bespoke curved panelling, features a distinctive design and material palette crafted around its unique content.

Technology used in the spaces falls into two categories – linear content and interactive experiences. The technical and creative team behind the immersive exhibits used linear content to embed moving image and audio into the space in a way that suited the aesthetic of the museum, it was visually consistent throughout and provided a broad creative canvas to tell stories.

Sound design is as important as the museum’s visual ingredients, says Shelby Prichard, chief of staff at SPYSCAPE: “Working with sound artist, Nick Ryan, we created rich soundscapes that help immerse visitors in our stories and tie the spaces between the galleries together.”

An example of this is the Surveillance gallery which takes visitors on a 12-minute audio journey into the world of CCTV and surveillance. Another significant experience occurs at the beginning of the visitor’s journey, as they enter the Briefing exhibit. Playing over three walls of the space in surround sound, the three-minute film immerses visitors in the world of espionage and introduces them to the experience to come. The film was created in collaboration with Territory, a creative studio known for its work on films such as Bladerunner 2049, The Martian and the recent Spielberg blockbuster film, Ready Player One.

Technology used in the spaces falls into two categories – linear content and interactive experiences

Encouraging interaction
To achieve true visitor engagement the SPYSCAPE team ensured there was the opportunity to interact in a variety of ways, as well as introducing more traditional and familiar elements such as touch screens and tablets.

“We have voice analysis, handwriting recognition, pulse readers, 3D sound design, facial recognition, RFID and lasers,” says Prichard. “When a visitor buys a ticket, they are given an RFID wristband which allows us to identify them in the space and tailor certain aspects of the experience for them, such as their name and preferred language.”

The visitor journey begins with an experience called The Briefing. Guests then encounter the first of 12 question stations and then the Encryption Challenge. An iiyama 65in, 4K touchscreen (PROLite TF6537UHSC-B1AG) traditional multi-touch table features in the first gallery, used to teach the basics of writing encrypted messages. From there visitors experience deception booths which teach them how to spot lies. The booths use sophisticated facial recognition technology and software that can recognise facial movements and decipher personality to test these skills.

Elsewhere, the eight-metre wide Surveillance Gallery room features 360-degree video projection of surveillance feeds on its circular internal wall. This game-format exhibit uses a combination of eight Epson EB-L1105U, eight ELPLU03 and a 7thSense Design Proton Infinity media server. Epson projectors were chosen for their solid performance, and 7thSense servers for their versatility and control in allowing mixed source media content presentation. In this space voice-controlled software is also used to put visitors’ powers of observation to the test.

In the next stage of the spy-themed adventure, custom-made laser tunnels test visitors’ reactions. The Special Operations laser interactive consists of 198 game buttons that can be illuminated during the game. Players must press the illuminated buttons to increase their score whilst avoiding breaking the beams of 13 lasers that criss-cross the tunnel to form a light maze. Whilst Prichard could not reveal details of all of the technology behind the tunnels, some of the products used include EDAM and LabJack controllers, DMX devices and bespoke game applications. Additionally, a custom written iPad management application allows staff to monitor and control the tunnels in realtime. Visitors can also take a range of psychometric tests, delivered on column-mounted iPads

The spy experience culminates in a mirrored room where visitors scan their personal RFID bands to reveal their scores and spy roles based on how they have interacted in the space. This appears on LG 55UH5C 55in, 4K signage displays set behind mirrors. Other visual kit playing a part in the exhibits includes rack-mounted PCs and Extron DTP HDMI 4K 330 Tx/Rx.

“As with the content, we chose technology to use in the interactives that would best serve the story they are telling, but also with an eye on the future. We see them as creative canvases, endlessly updatable with new content, games and experiences,” says Prichard.

LED lighting is ubiquitous throughout the exhibit, with granular control of just about every fixture allowing control of intensity, colour temperature and hue

Amplifying architecture and emotions
Developing lighting schemes for the museum’s spaces was placed in the capable hands of Lighting Workshop’s Steven Espinoza, who worked closely with architects Adjaye Associates to develop the design and experience.

“First and foremost, we looked to support the architecture and amplify the multitude of emotions that it evokes,” says Espinoza. “The variety of architectural volumes, how they are organised and how they are illuminated was carefully planned. Some spaces create intimacy with artefacts, while others can become almost overwhelming in scale. In many instances, technology, including large-scale animated lighting gestures, introduces moments of awe that produce meaningful waypoints and bring balance to the experience. It is all very carefully orchestrated.”

Espinoza also recognised the museum as being inherently fluid and something that would evolve over time. As exhibitions come and go, the spaces need to be able to adapt to change so it was important to provide a lighting system that was flexible. Phillips iPlayer, Phillips Light System Manager and Paradigm ETC control systems were therefore provided to maximise adaptability and define how each space feels.

For Espinoza, SPYSCAPE is all about the narrative and the experience: “In most areas we try to make the lighting dissolve into the architecture. In a few rare instances we make lighting moves that are unmistakable. The vault – an LED canopy on the second floor consisting of 3,500+ LED tubes, each individually programmable – is one of these instances. Here, we partnered with Arktura and Philips Color Kinetics to create extraordinary results.”

LED lighting is ubiquitous in all of Lighting Workshop’s projects due mostly to energy efficiency and how the marketplace has reacted to the technology. Espinoza required granular control of the lighting of just about every fixture throughout SPYSCAPE. LED offered the perfect solution, allowing him to control intensity and, in many cases, colour temperature and hue.

Storytelling through sound
Audio was a key component of SPYSCAPE’s experience design from the start. Creative director, Ross Philips, invited Nick Ryan to act as audio director, having recognised the vital role that a considered approach to audio could play in such a unique and ambitious project.

Ryan developed a strategy for audio across all exhibits and crafted an audio journey. “The audio in the museum acts on an aesthetic and functional level,” he explains. “Aesthetically, audio is designed to work in unison with that of the architecture, motion graphics, lighting, objects, narratives, and overall visitor journey. Bespoke music and tonal sound design plays an important aesthetic role in the museum too, with soundscapes, setting a distinctive cinematic mood.”

Music was composed in a way that would not dictate specific interpretations, but instead gives the visitor the ingredients for their own subjective emotional response to stories, objects, the space and its overall message.

Audio design was also deliberately and necessarily functional. Content was finely balanced, equalised and tuned to the acoustics of each space to assist visitors in absorbing the many and varied pieces of visual information. Gallery sound systems and content were also designed to immerse visitors in spatial audio to enhance their sensory experience, delineate one gallery from another and contextualise the physical artefacts and screen-based exhibits more broadly in textures related to the particular subject matter.

Some spaces create intimacy with artefacts…

“Audio is used as the primary information channel at various points in the museum, delivering facts, stories, signals and instructions through voice and sound effect design,” says Ryan. “Lastly, audio is a key component of the UX design of all interactive and experiential elements in the museum, such as special ops and deception challenges, where it provides vital navigation and interaction feedback to the visitor. All of the audio, whether serving a function or feeling, is designed to fit together as a ‘score’ to SPYSCAPE.”

Early in the creative process audio director, sound designer and composer Ryan appointed Davey Williamson as technical sound designer and Adam Armitage as sound effects editor. Williamson’s experience working on large-scale installations as a technical sound designer, audio systems architect and mixing engineer was invaluable. He had collaborated with Ryan in 2016 on A Living River, one of the worlds’ largest sound installations at Gatwick Airport.

“We worked in parallel when creating content for SPYSCAPE to develop creative concepts and solutions for playback systems that could deliver audio in an innovative way,” says Ryan. “Developing concepts from initial ideas through to CAD models, Davey could then represent our recommendations for hardware and configuration, such as placement of speakers and design of bespoke immersive systems, to the architects Adjaye Associates and other parties, to ensure the building was acoustically compatible and ready to accommodate specific units precisely where they were required.” Audio content was created for over 30 bespoke short films and projections, showcased throughout the galleries, all of which required original sound design.

Many of the films feature a combination of archive footage – much of which is mute – accompanying detailed motion graphics. “I recognised we needed a sound effects editor who was used to reconstructing sound for archive material, such as an Enigma machine or a military tank, with the highest level of accuracy, as well as having the skills to create intricate bespoke sound effects for the motion graphics,” says Ryan. “I also wanted the sound design to be approached with filmic techniques in its attention to detail and impact.”

Ryan knew the area to explore was feature film sound effects design and more specifically the work of Adam Armitage, a sound effects editor with extensive experience of feature film and TV drama. Armitage is also familiar with the subject of spying, having worked as sound effects editor on The Night Manager. He worked through SPYSCAPE’s films, track laying the effects and delivering them to Ryan for additional sound design, scoring, mixing and mastering. They also collaborated with sound designer, composer and audio visual artist, Tim Cowie, who created the sound and music and film for the Briefing space with Territory Studio.

A spatial audio experience
Two spatial sound systems feature in the museum which the audio team considers to be world firsts and refers to as the “audio hero” environments.
“In the Briefing – which takes the visitor up to entry level – the lift moves so smoothly you can’t tell it’s in motion. An immersive film, projected on to three of the walls, introduces SPYSCAPE and through the use of incredible visual effects and animation, takes the visitor on a dramatic journey up four floors, into orbit above the earth and down again into Manhattan,” says Ryan.

The audio team felt the introductory segment demanded a special sound system where audio could be positioned and panned accurately against the film’s detailed visual elements, such as passing satellites and simulated elevator movements. They also wanted the audio to be powerful and vital to this was the ability to pan sound in all three dimensions, with control over height, as well as having considerable headroom and bass extension. Williamson designed a 19.1 channel system using L-Acoustics speakers and amplifiers, with the entire system concealed behind a mini perforated projection screen.

“Davey and I mixed Tim Cowie’s composition and sound design for this film in the space itself using a combination of Logic, Protools, L-Acoustics L-ISA processor and Pyramix. By using the L-ISA processor and panner we could achieve highly accurate panning of up to 64 individual audio elements within the 3D space at any one time. The spatial resolution, huge dynamic range and extended frequency response of the system enabled us to create an incredibly immersive and unique audio experience. No ordinary elevator music!”

The other much larger audio ‘hero’ is found in the Surveillance Gallery where the goal was to create a sound system which would immerse the visitor in “sound textures of surveillance.”

Other spaces can be almost overwhelming in scale

“From crackles and whirrs of server farms through to the tapping of Morse code, we wanted these sounds to be all around, for every source to be different and of the highest spatial resolution possible. We also wanted this system to provide the museum with a unique capability for sound: a system with incredible spatial resolution, scale and fidelity,” says Ryan.

This was achieved by designing a 38.1 channel system using 38 L-Acoustics 5XT speakers, two sb18i subs and 10 LA4X amplifiers. The speakers were configured as three rings of 10 at floor level, ear level and ceiling height plus eight on the ceiling. The pair of sb18i subs were flown in the centre of the ceiling. As it would have been impossible to mix the content in a conventional surround mixing stage the content was first mixed on a half-sized test system they built in London and then in the gallery itself in New York.

In addition to audio from the exhibits, each gallery also features an array of at least five ceiling speakers which play back a spatial soundscape intended to differentiate galleries and contextualise exhibits within the museum. Bose DS100SE speakers were chosen for projections and screens and Bose DS100F speakers for ceiling soundscapes due to their fidelity and impressive low-end extension. The soundscapes were mixed in situ – pushing carts with a mobile rig from gallery to gallery – using a combination of Apple Logic Pro, Ableton Live, Figure 53’s Qlab and Merging Pyramix.

In the same vein as the project’s visual components, the audio was always designed with the visitor in mind. The audio team decided on the placement of every audio source in the museum with respect to the listener, an exhibit, the room or other sources of sound. Ryan says: “We conceived every gallery as an immersive system made up of mono or stereo sources – from screens and other audio-based exhibits – which together create a distributed canvas.

Have your say

or a new account to join the discussion.