WORK

In September 2023, at the Ars Electronica festival of media arts in Linz, Austria, Masatane Muto, a DJ living with the intractable disease amyotrophic lateral sclerosis (ALS), showcased a dance performance using an avatar.
 
Mr. Muto, located in Tokyo, wore electromyography sensors that captured his biometric data from minute muscle activities. This data was used to control an avatar in the metaverse, allowing him to perform at the venue in Linz from afar.
 
Project Humanity is a collaborative work between Dentsu Lab Tokyo, NTT, and WITH ALS.

We were initially given three missions.
The first was to convert the biometric information obtained from the electromyography sensors worn by Mr. Muto into control data, enabling the free manipulation of the avatar.
The second was to remotely control the avatar in a music performance, synchronizing it with the music, where even slight delays or misalignments can be critical.
The third mission was to integrate these elements and realize a groove-filled performance remotely.
 
Additionally, in order to accomplish these missions, we needed to meet various conditions. These included creating natural movements for the avatar using limited control data from both sides of his neck, both wrists, and both ankles. Among other tasks, we also had to devise a system for transmitting electromyography data from Mr. Muto in Tokyo to the venue in Linz and create a system for synchronizing the avatar's movements with the music's rhythm.
 
Just like the previous "ALL PLAYERS WELCOME (link)" project in 2022, we approached this performance from both the perspective of a performer and that of a tech director who assesses technical feasibility. We engaged in ideation sessions for the most suitable performance equipment, overall system design, and technical direction, to solve the challenges at hand.
 
Moreover, in arranging the equipment that formed the basis of the performance, we served as a communication hub, handling interactions in different languages, dealing with the stringent constraints of a venue where multiple presentations were happening simultaneously, and managing the restrictions of bringing equipment overseas. Despite the different constraints and environments compared to domestic settings, we contributed to ensuring the smooth progress of the performance.