POWER OF INNOVATION

NTT began developing Kirari! in 2015. Immersive telecommunication technology aims to “transmit whole” and recreate sporting event spaces and live concert spaces in real time via networks in distant locations. Through a combination of three cutting-edge technologies unique to NTT, it is now possible to transmit highly realistic spatial information from distant locations.

These three technologies are: “Real-time Extraction of Objects with Arbitrary Background”, which enables only images of the moving athletes to be extracted in real time, even in situations where chromatic key filming utilizing a monochrome blue or green background is not employed; “Sound Video Synthesis Technology for a New Video Viewing Experience”, through which an entire sports arena is filmed separately using multiple 4K cameras and these images connected in real time to create high-definition wide images; and “Advanced MMT”, a high-realism media synchronization technology capable of synchronizing various imaging and sound technologies. With “Advanced MMT” it is possible to transmit and synchronize not only video images and sound but also lighting and other special information, creating an even more realistic sense of “being there” in accordance with the size of the broadcasting studio and broadcasting facilities.

A world in which it is possible to experience sporting or entertainment events taking place at a distant location in their entirety—including the venue atmosphere—anywhere, anywhere: NTT is accelerating R&D with the aim of achieving just such a world.

Kirari! technology has also had a powerful impact on society in trials and at the social experiment stage. In November 2017, a collaborative project was conducted with popular techno-pop group PERFUME as the first phase of “FUTURE-EXPERIMENT.JP”, a project proposed by DOCOMO. Performances by the three PERFUME members, each on a different stage in three different cities—New York, London, and Tokyo—were synchronized in real time and broadcast live around the world.

By combining Kirari! technology with DOCOMO’s high-capacity, low-latency 5G technology and synchronizing the sounds and images of the three performers, it was possible to broadcast images live on the Internet that made it appear as if the three PERFUME members were all singing and dancing together on the same stage. Thanks to the perfectly synchronized performances of the PERFUME members and “Advanced MMT” image/sound synchronization technology centered on 5G circuits and Kirari! technology, the project was able to demonstrate just how far audio and visual delays can be eliminated with this technology, indicating the tremendous potential for being able to stream live broadcast of contents providing completely new video experiences (available from 2020 onwards) via the Internet to anywhere in the world.

For the second phase of the “FUTURE-EXPERIMENT”, an exhibition match between Rio 2016 Paralympic Wheelchair Fencing Gold Medalist Beatrice Maria Vio and Beijing 2008 Olympic Fencing Silver Medalist Yuki Ota was broadcast using the “Multi-view Live Viewing” sports broadcasting system which enables viewers to select the images they wish to view while freely switching between multi-view images filmed using nine cameras.

Such an experiment regarding audiences’ watching of matches filmed simultaneously using multi-view cameras is extremely rare in not only the world of fencing, but in the entire world of sports, and the project embodied new possibilities for watching fencing, a sport in which matches can be decided in a moment’s movement, making it difficult for general spectators to determine what was the winning blow even when watching with the naked eye.
Watching a fencing match utilizing real-time switching between images filmed with nine cameras was made possible by “Advanced MMT” image/sound synchronization technology centered on “high-speed” and Kirari! technology. For sports involving fast movement, the challenge is how clearly images can be seen even when transmitted at a high speed with an elevated frame rate. By continuing to take on the challenge of making sports-viewing even more enjoyable, NTT is presenting new sports-viewing styles with an eye to the year 2020.

At the “NTT R&D Forum 2018”—an event where the latest results of NTT’s R&D are presented—held in February, a demonstration of Kirari! technology was conducted. Onto an oblong screen measuring 16 m × 5 m that had been erected inside the venue were projected 12K-wide high-resolution images of figure-skater Daisuke Takahashi that had been filmed using multiple 4K cameras and then connected. A new form of image-viewing providing bird’s eye views of the ice rink overall was presented using sound image synthesis technology.

Furthermore, in addition to CG effects, various effects were also added to enable audiences to better enjoy the sense of “being there” in the skating rink, with bands of light wrapping around the skater as he spun and the entire skating rink suddenly transforming into a dance hall. Viewers were enveloped in excitement as the venue was filled with a sense of power and presence; it was as if they were watching Daisuke Takahashi performing right there, before their very eyes.

Enabling the extraction and real-time transmission and reconstruction of spaces in distant locations, NTT technology shows potential for future entertainment accompanying a “sense of realism” that has never before been experienced. It is now possible to provide viewers with completely new viewing experiences, as if a sporting event is taking place before their very eyes, in a form far exceeding any “sense of realism” that they may have previously experienced through video images.

In the year 2020, it is anticipated that this technology will be used in the viewing of sporting and entertainment events, but also a wide range of other fields. It is possible for athletes at a sports arena or performers at a concert venue to become one with their audiences and share emotions in real time, despite the event being held in a location far away from the audience. In order to realize such as world, NTT will continue to present innovative ways to utilize cutting-edge technology.