R&D TECHNOLOGIES

Cutting-edge NTT communication technology that will change the world

R&D TECHNOLOGIESThe future of watching sports transcends physical location and spaceThe spectacle of maneuvering the wind and waves at sea unfolding right before your eyes

Rina Niijima
You Have to Read the Tide and Harness the Wind. I Want to Communicate How Deep Sailing Can Get.

Sailing is a sport in which participants head toward markers set on the water's surface, racing against the clock. They must figure out the course in a limited time, understand the wind that drives their vessels, and meet the challenge with a strategy to sail around the markers as quickly as possible. The difficulty is that because the race takes place off the coast, it is difficult to watch it from land with the naked eye.

Is there a new sports spectator experience that can provide us a more exciting and immersive feeling while watching those competitions?

Watching sailing has traditionally been something to do from a seawall with binoculars. NTT's advanced communication technology offers a new way of watching sailing at sea right in front of your eyes, delivering never-before-experienced sensations and excitement.

Press Release : Technical cooperation with Kirari! Ultra-realistic Communication Technology for the "TOKYO 2020 5G PROJECT" to Recreate New Spectator Experiences – Live Transmission of 12K Ultra-wide Composite Images of an Entire Sailing Event to Remote Locations –

The Tokyo Organising Committee, in cooperation with Intel Corporation, Nippon Telegraph and Telephone Corporation, and NTT DoCoMo, Inc., is implementing this project which provides a new sports spectator experiences within the venue utilizing 5G technology at the Tokyo 2020 Games.
NTT, which is engaging in research and development to realize the remote world, support Tokyo 2020 Organising Committee of the Olympic and Paralympic Games with its communication services.

AFTER REPORT

Immersion and a sense of unity as if you're at the venueDelivering a new beyond-real viewing experience

Immersion and a sense of unity as if you're at the venue Delivering a new beyond-real viewing experience

Sailing saw hot competition at Enoshima Yacht Harbor.
In the Tokyo 2020 Games women’s 470 class sailing, Japan’s Ai Yoshida and Miho Yoshioka finished seventh, continuing their strong performance following the Rio de Janeiro 2016 Olympics.

Watching sailing has traditionally been something to do from afar with binoculars and on a small monitor. But NTT has achieved real-time synthesis of a super-wide video with a horizontal resolution of 12K using a 5G network and the “super-immersive communication technology Kirari!,” transmitting images to a gigantic 55-m-wide screen just above the ocean’s surface in front of spectators in Enoshima.

The 55m-wide screen with a resolution of 12K, showing the race at the Tokyo 2020 Games sailing venue on the ocean’s surface in front of the audience area

The 55m-wide screen with a resolution of 12K, showing the race at the Tokyo 2020 Games sailing venue on the ocean’s surface in front of the audience area

The project began with the development team wanting to project the race as a gigantic image covering the audience’s entire field of view, for a dramatic experience that feels as if the race is going on right before their eyes.
NTT's development team visited the venue before the event started, evaluating the project from various angles, doing countless tests and evaluations. There were two key technological points that enabled the sense of immersion, as if the race that was taking place far away on the sea was unfolding right in front of the audience.

1:「5G」

The race was filmed by a drone with three 4K cameras and a boat, also with four 4K cameras. The high-volume video data shot off the coast was transmitted via high-speed communication to a screen with a horizontal resolution of 12K (with a resolution about 3 times higher than high-quality TVs for general home use) set just above the water surface in front of the audience. The "5G" communication technology was vital for the servers receiving the videos to instantly synthesize them, transmitting to the 55-m-wide screen with no time lag.

A boat with 4K cameras on the bow, shooting the hot contest at sea

A boat with 4K cameras on the bow, shooting the hot contest at sea

A drone with 4K cameras flew over the ocean to transmit high-volume data at high speeds, using 5G

A drone with 4K cameras flew over the ocean to transmit high-volume data at high speeds, using 5G

2: "Super immersive communication technology Kirari!"

The biggest hurdle in connecting the video images, taken by multiple 4K cameras in real time and projecting them onto the wide screen, was how to treat the gap between the videos. To deliver vivid and dramatic videos of the heated race at sea, the videos were seamlessly synthesized in real time and delivered live.

Videos taken at sea were synthesized to deliver the heat and excitement to the audience

Videos taken at sea were synthesized to deliver the heat and excitement to the audience

A

A "virtual stand" was set up to deliver the cheering of families and friends in remote locations to the athletes

Although the event went ahead with no audience, a "virtual stand" was created for athletes to receive the cheering of families and friends in remote locations on a super-wide screen. The super-wide video of the race was also transmitted to the distant Tokyo Big Sight (the Main Press Centre) onto a 13-m-wide hyper-resolution LED screen, giving the press corps from around the world the sense of immersion and unity as if they were at the venue. This offered them a glimpse how they might watch sports in the future.

The dramatic video images captured by the 4K cameras were transmitted in real time to the press corps at Tokyo Big Sight

The dramatic video images captured by the 4K cameras were transmitted in real time to the press corps at Tokyo Big Sight

The experience highlighted the thrill of watching sports at the venue and the ease of watching it remotely. Accommodating both these needs, NTT will apply its communication technology to bring new value to the future of watching sports that is “safe and effortless, with a sense of immersion.”

READ ARCHIVE(UPDATE 07/02/2021)
Exploring an innovative viewing experience that surpasses the real thing keyboard_arrow_down

Exploring an innovative viewing experience that surpasses the real thing

Sailing is a sport in which sailors let their vessels glide across the water's surface using wind to propel the sails, competing in terms of speed and technique. To enable viewers to feel the race's excitement at sea up close, NTT has achieved real-time synthesis of a super wide video with a horizontal resolution of 12K using a 5G network and super immersive communication technology Kirari!, projecting the image onto a gigantic screen 50 m wide set on the ocean's surface. Using the wide marine screen, NTT will turn sailing into a near-real experience that feels as if viewers are watching the race from a cruise ship anchored near the race area instead of having to watch from seawalls with binoculars like before.

Technology that fills the entire human field of vision Doing it all to deliver the sensations and excitement of being at sea

Video of the sailing race will be broadcast not only in Enoshima Yacht Harbor, Kanagawa, the venue for the Olympic Games Tokyo 2020, but also transmitted to the distant Tokyo Big Sight (the Main Press Centre) thus offering media staffs a chance to share a sense of immersion and unity in Enoshima. NTT's leading communication technologies offer an arena for viewers and sports fans to share in the excitement as a new way of enjoying watching sports.

READ ARCHIVE(UPDATE 07/02/2021)
Communication technology that fills the entire human field of vision keyboard_arrow_down
TECHNOLOGIES

Communication technology that fills the entire human field of visionDoing it all to deliver the sensations and excitement of being at sea

As a sport that takes place at sea far away from land, sailing has always had the reputation that it is a sport whose dynamism is hard to convey to spectators. Despite this challenge, NTT combined its multiple advanced communication technologies in order to deliver the sensations and excitement of being at sea to as many people as possible. By connecting and synthesizing videos from multiple 4k cameras in real time, high-definition and wide-angle videos that could not be produced by a single camera are generated, and then separate video and sound data are synchronized and transmitted over a 5G network, thereby delivering a fully immersive, powerful video on the ocean itself, filling the audience's full field of vision.

Technology that fills the entire human field of visionTechnology that fills the entire human field of vision Doing it all to deliver the sensations and excitement of being at sea

INTERVIEW

You Have to Read the Tide and Harness the Wind. I Want to Communicate How Deep Sailing Can Get.

Rina Niijima
Rina Niijima

Event: Windsurfing, class: iQ FOiL
Having started windsurfing in the first grade of elementary school, Ms. Niijima competed in multiple international championships from junior high school. In the Second Youth Olympic Games (2014/Nanjing), she won seventh place even though she was the youngest participant at age 14. From high school, she switched to the official Olympic category of class RS:X, winning sixth place at the 2017 Youth World Sailing Championship/Youth Sailing ISAF World Championship. She now competes as an adult.

Sailing is a sport in which athletes must read the tide and harness the constantly changing wind to reach the goal. NTT's ultra-realistic communication technology Kirari! transmits in real time the heat of the race to the audience watching from the stands. The offshore race will be displayed in real time on a gigantic wide marine monitor floating in Enoshima Yacht Harbor. We interviewed Rina Niijima, the young windsurfer on the Japanese national team, and asked her to share her take on the technology's future and potential.

*This interview was conducted in June 2021.

Nothing feels better than interpreting the wind correctly and winning
-I suppose that because of COVID-19, you have had fewer opportunities to compete over the last year and could not train as you wanted.

Niijima For me, my college shifted to online classes, so I had more time to train. You can only practice windsurfing when there is enough wind in the daytime, but because the college went online, I could make use of the time I would otherwise have been commuting to campus. I was also able to attend class remotely in Miyakojima, the windy place where we trained, so actually I could focus on windsurfing better than usual.

-You managed to balance your academic work and training.

Niijima Yes, but until last November, competitions were cancelled, so we had little opportunity to demonstrate the good results of our training. We could not train overseas, either, which was frustrating because even if I performed well in Japan, I had no way of comparing my results to my overseas rivals.

-Since you were born and grew up near the sea in Kamakura, I am sure you have always been familiar with marine sports. When did you actually start windsurfing?

Niijima I started riding a board when I was 4, but of course I have no recollection of that (laughs). I joined the junior club in the first grade of elementary school mostly to see my friends at sea-this was an extension of playing on the beach. Winning the national junior championships in second grade and then losing to a rival the following year prompted me to take on windsurfing more seriously.

-Your father runs a windsurfing store and he was a former professional windsurfer. Has he been a tough mentor since you were a child?

Niijima Not so much when I was small. However, he was tough on me when I was in junior high school. I wanted to study hard to enter high school, but my parents kept telling me to stop going to a cram school and to train instead (laughs). I was a huge rebel then, quitting windsurfing for six months and studying. But once I managed to pass the school entrance exam and entered high school, I switched my focus. Until that time, I had a feeling that my father was forcing me to train. I then switched from junior equipment to the RS:X board, which is an Olympic category, and started to focus on the sport by my own will, setting my eyes on the Tokyo 2020 Games.

-In sailing, you have to race to the goal while skillfully understanding where the wind and tide are going. What is the sport's appeal for you, and which points determine victory?

Niijima Put simply, the appeal is that it is not simple. You cannot win unless various elements fit together correctly. No matter how much you work on it, you keep facing new tasks. But that's the fun of it, the most enjoyable part for us athletes. The determining factor is your ability to interpret the wind. If you cannot do that correctly, you might take a detour to get to a point you must clear, which will of course consume your energy unnecessarily. That's why athletes pay attention not just to the waves around them but the mountains and clouds in order to accurately figure out how the following wind will come. There is so much to think about, which makes it a complex sport, but because of the hard work, it feels fantastic when you are able to guess right and win.

-What do you think is the most important factor to win in a windsurfing race?

Niijima Weight is one key concept. After the Tokyo 2020 Games, the official windsurfing equipment will switch from RS:X to iQFOiL, toward the next 2024 Paris Games. I have been training with the new equipment, but with iQFOiL, you are at an advantage if you are heavier. You can get speed more effortlessly than with the old board, with weaker wind, but to maintain that posture, you must be steadily grounded on the board. So, I have worked hard to put on extra weight and gained 8 kg over the past year. It is a little sad, even though it is for the race (laughs).

Everything from the dynamism of the race to individual windsurfers' strategies will be felt through the screen

-At the Tokyo 2020 Games, for the first time NTT's communication technology Kirari! will change the way people watch sailing. The heated race off the coast will be displayed on the gigantic screen right in front of the audience, making them feel as if they are watching the race at sea.

Niijima Until now, people have had to use binoculars to watch the race far from the shore, or check where the athletes were online, which uses the information from the GPS devices worn by the athletes. In windsurfing, you normally do not have any audience except in some major international championships. So, it is amazing that the audience will be able to enjoy the dramatic race right there in real time. If you watch the entire race on the wide-vision monitor, you will come to understand the unique way of appreciating windsurfing, like how different athletes deploy different strategies to determine their courses. In the future, if you could install such a screen off the coast and cheer on the athletes, the athletes will feel encouraged like never before, because the race at sea is a completely solitary battle.

Read more of the interviewkeyboard_arrow_down
-Lastly, please tell us your goals.

Niijima I am sure the audience will, even if they have never seen windsurfing before, immediately experience the intensity and speediness by which athletes nearly crash into each other. If you learn a little about the principle of the wind that propels the sail, I am sure you will enjoy watching sailing at greater depths and find it more riveting. My goal is to compete in the Paris 2024 Games in the iQFOiL class. I do not know what happens beyond that, but I want to stay active until the Los Angeles 2028 Games. My dream for the future is just to get married and be a mom (laughs). In the end, I would like to take up my dad's store and train children who are looking to windsurf to compete on the global stage.

A step toward a new experience of watching sports that creates a new style of watching

With the new 5G-based way to enjoy watching sports even from afar with a sense of immersion and a feeling of being at the venue, this project is a big step forward toward realizing the remote world of the future envisioned by NTT.

OTHER R&D TECHNOLOGIES