Once, you might see strange sensors or generated graphics as a kind of novelty, used for their foreign, futuristic quality. Now, a generation immersed in such tech exploits these tools because it’s second nature.
That’s the message of the creators behind a fantastic trans-media beatbox performance starring Japan’s Ryo Fujimoto. And while you’ve seen each clever gimmick on its own, here they try tickingall the boxes at once. Musical gloves dripping in wires with flex sensors – check. Muscle sensors – check. Glowing-blue heart-rate sensor behind the ear – check. LEAP motion sensor detecting gestures of the gloves on top of what the gloves are doing – check.
Keep watching the video, because after the usual behind-the-scenes business, the performance starts – and it’s stunning.
“We’re a generation raised on video games … immersed in technology,” confides Eddie Lee, the wizard at game studio Funktronic Labs who provided the visuals and their interactions. “It’s only natural that we express ourselves with technology.” And so Lee’s visuals are flattened and retro with reason: they’re cultural touchstones, a visual language that feels native.
Ryo Fujimoto is Humanelectro, the cyborg beatboxer/musician.
Takuma Nakata of TN2 provided concept and direction; Tusbasa Nakata of TB2 the hardware design.
More from the site (see link for Japanese text):
Electrical signals radiate from the body of Ryo Fujimoto via the various sensors that are connected to his face, arms, and hands. These sensors track heart rate, muscle movement, and finger positions in real-time. The data is then converted directly into audio and visuals, thereby allowing the expressive unification of man and machine, and the birth of “Σ (SIGMA)”.
In July 2013, Ryo Fujimoto published a live LeapMotion performance onto Youtube, which caught the attention of Eddie Lee. Eddie contacted Ryo, discovered that they shared an interest in audiovisual technology, and decided to collaborate. Takuma Nakata joined the project to direct the performance and to build custom hardware that Ryo Fujimoto would use during the live event.
This video above is a recording of the live event that took place in Tokyo, Japan. And it is the first leap of project “Σ”.
The group next dreams of adding robotics and still more tech. We’ll be watching.