Music Video • Virtual Production • Volumetric Capture
MIYAVI
「Need for Speed」
Japanese rock-star Miyavi wanted to make a music video during the spring 2020 lockdown, with the goal of creating a unique visual experience for his fans.
We were presented with a few challenges - Miyavi couldn’t travel, we only had a small team to work with (all working from home) and a limited budget compared to similar productions.
To solve these, we turned to virtual production and 3D game engines to make the music video. This allowed us to work remote with a skeleton team.
The Story
In a lifeless virtual world of ruins, Miyavi finds himself with video-game inspired superpowers and the need to break free of his lockdown-induced inanition. The world became a symbol of our own mood at the time after months of isolation. We consider the ruins an alternate version of this planet that Miyavi inhabits alone.
We wanted to embrace our love of gaming by merging tool and story. The engine allowed us to create mystical surreal worlds with little effort, and were inspired by our favorite visual moments in games to create them.
Technologies
We made use of two core technologies: full-body volumetric capture and real-time rendering via game engine Unity. These tools allowed us to construct the world of the music video from scratch. Volumetric capture gave us a full body 360 degree view of Miyavi (similar to a 3D model), which gave us full freedom to move the camera anywhere we wanted to.
We chose Unity (over Unreal) in this case due to it’s more advanced treatment of volumetric data and simpler integration into visual effects graph for complex mesh effects.
The entire shoot was remote, with director and creative team in Los Angeles while Miyavi was in the studio in Tokyo with his producer and stylist.
Anatomy of a shot
Miyavi in a volumetric capture studio. Dozens of RGB camera record simultaneously to reconstruct a 3D model for each frame of the animation.
The resulting file (similar to alembic sequence) is brought into Unity and a set is built around it. The textures are setup as a streaming image sequence.
The resulting final scene with lighting, VFX and camera motion.
Virtual Cinematography
Lighting was done completely in Unity. The real-time engine allowed us to see the output of our work instantly, and allowed for limitless creativity for our cinematographer.
We made extensive use of Unity's timeline feature to animate and sequence the video. Everything from effects controls, lighting changes to camera motion was manipulated. We could see the shots we made as soon as we created them, making iteration fast and creative.
The World.
A tour around one of the sets in Need For Speed
Remote color sessions used Zoom and a shared Unity Teams project
To bring an isolated world of ruins to life, we imagine our environments like theatre sets. Lighting engulfs the visual focal point of the scene - this not only helped us create the illusion of music video set, but also limited the scope of set we had to build given our short timeline. Our world is filled with half-broken objects, weathered grounds and the remnants of a past civilization.
Set design, materials, lighting and color was all done in-engine with very little work done in other software.
We also made use of Quixel Megascans to quickly prototype and build sets. We ended up with 14 set prototypes over the course of 2 weeks, even though only a handful made it into the final video. Photogrammetry was often used to fill in distant set pieces and make up for bulk in the entire set. Many of the materials are custom blended materials.
All color grading was done by our colorist directly in Unity in three remote video sessions. What you see in the video is 99% of what we saw in the Unity editor. The only element added in post was fine 16mm film grain via Nuke.
Superpowers
Virtual Miyavi wouldn’t be complete without his own unique superpowers. When Miyavi performs his guitar is an experience in itself, he’s been called “samurai guitarist” because of his impressive solo choreography. To capture the raw feeling of power and energy at a live Miyavi concert, we used his guitar as the focal point of our visual effects.
Raw BMW
BMW was kind enough to provide additional support for our video. In return we used one of their latest cars in the video. We received a 7.3 million vertices CAD file of the car complete with mechanical details and meshes for every single seam. The mesh went through considerable optimization to run in real-time in Unity. Car paint and PBR materials were achieved with a mix of custom materials and Unity’s car paint shader to add accuracy down to minute scratches and reflections.
