Immersive Media Wall

In 2014 I was contracted by local company Formula-D to help produce a Virtual Gulf Stream for the new high-tech Phillip & Patricia Frost Museum of Science in Miami.

The interactive wall spanned 90 feet with 4K projection and 12 channel dynamic audio. Myself and Andries Odendaal worked together and installed the project in December of 2016. Delays were mostly part on the construction of the museum which made things difficult. Most of the footage describes the installation and calibration which happened in December. The opening happened in May of 2017.

Andries focused on the 3D renders and realtime schooling (flocking) behaviour while I focused more on the video, audio and sensor integration. The entire project was built in OpenFrameworks - just another testament to this framework. The duration of the project meant we went through multiple upgrades of the framework without much hassle.

The content is a mix of realtime reactive 3D elements like particles, tuna, sardines and other fish and video produced specifically for this interactive wall like the whale.

For the audio I built a bespoke OpenFrameworks app which initially used FMOD which is a popular gaming audio library. I struggled to scale it up even though there shouldn't have been any issues to go beyond 8 channels. Our setup required 10 channel directional, with 1 ambient and 1 sub woofer. Due to time constraints at the time I tested and decided to move to Ableton Live which worked out well. The the OF app would communicate with Ableton via my OSC Controller and distribute the audio across the directional 10 channels. Videos would be spawned when a person is close to the wall and the video audio would have to play where the person is standing next to the wall, i.e. directional. We also had a full 10 channel pan which happened when the whale would move from one side to the end of the wall. The panning audio meant the whale audio would follow the visuals on screen through the 10 channels.

This was a challenging project with lots of curve balls but a great one looking back. The video I posted is more of a behind the scenes and can't do the real thing justice. So if you find yourself in Miami go and support the museum as there are a bunch of other cool high tech projects implemented and well worth seeing.

Phillip & Patricia Frost Museum of Science