Magic Leap may well be creating the future of the digital music immersion experience, with the first example of this being a collaboration with Icelandic experimental rock band Sigur Ros. Marc Hogan, from Pitchfork, recently got a chance to try out a musical experience on the near-final version of Magic Leap's hardware, and did so alongside the band that helped create it. The in-engine screenshots and the story of the experience that he brought back with him tell a tale not only of a Mixed Reality company and an experimental band creating something amazing together, but of the potential future of musical immersion and engagement through technology.
The evolution of that art started out with the humble phonograph, letting folks listen to music in the comfort of their own homes. From there, music videos brought concept art and video concerts into people's homes. After that, some artists began embracing the digital landscape by creating entire albums that were hosted inside apps for smart devices, or creating music videos meant to be consumed in virtual reality, or even handing over a bit of control to users through musical video game collaborations. Magic Leap and Sigur Ros have created something that mixes and transcends all of those. The app, called Tonandi, lets users interact with the music by interacting with visual representations of it in the form of tiny sprites that share a name with the app itself. While you cannot directly control Tonandi, they will interact with you and with your environment, providing a fully immersive, real-time experience that's meant to inject the visual splendor and raw magic of a Sigur Ros concert into your personal world. You can summon different sounds by interacting with your environment, and with the tiny Tonandi, which means that each experience with the app will be different. This, of course, is only one possible manifestation of mixed reality music.
What Sigur Ros and Magic Leap created together could be tweaked and used with just about any band's music, or a new type of experience entirely could be created. To name just a few easy examples, having a band play live in your living room would be a simple feat for a Magic Leap kit, as would putting you in the shoes of one of the band members, or even overlaying a live concert environment onto wherever you happen to be. Naturally, individual bands could kick this up a notch and add some of their signature style. A Deff Leppard mixed reality app could turn your kitchen light fixtures into pyrotechnics, or a U2 concert in your backyard could utilize 3D sound and lighting effects to turn the moon into a stage light and a speaker. Satirical death metal band Dethklok could inflict its signature malice upon a live audience that includes you, or you could find yourself inside one of J-Rock sensation Acid Black Cherry's notoriously surreal music videos, surrounded by puppets, angels, nurses, or the entirety of early 20th century France, just to cherry pick a few examples from the band's repertoire.
Interactive elements could easily affect the creation of music, as well. Imagine using a mixing and mastering suite that had switches and knobs attached to your walls, then hearing the effects you just finished adjusting in real time simply by picking up your guitar or singing a few notes. Realistic turntables could appear on your kitchen counter, allowing you to remix, scratch, and drop your own music. Something akin to Rock Band and Guitar Hero is also easy to envision, with mixed reality. Making music by moving your limbs and objects, like a sound-based version of Google's Tilt Brush, is another of the many possibilities inherent in the intersection of music and mixed reality.
Virtual reality offers a mostly passive experience that requires special controllers to interact. Augmented reality does mostly the same thing, but projects that experience out onto the real world. Mixed reality, where augmented reality considers a user's environment and can be interacted with using one's bare hands, is what allows for experiences like Tonandi, and all of the potential examples listed above. Magic Leap is a pioneer in this new medium, and will almost undoubtedly be one of the biggest names in the medium's early years, if not the biggest. That means that it will be the one to watch during the early development of the mixed reality music scene.