What kind of sounds are you surrounded by as you read this article? There are many sounds around us, such as conversations with people around us, TV sounds, favorite music, and ambulance sirens ringing outside the house. These sounds are information that plays an important role in communication, knowledge acquisition, and avoidance of danger. But not all the sounds that come into each of us are just as important. Some sounds may be noisy or noisy. Then, what kind of sound do you really want to hear or know?

This time, we will focus on “Ontenna,” a device that converts sound information into a device that transmits vibration and light. Through Ontenna, we will tell you about the online event that everyone thought about “listening to the sound” that you do not usually stop and think about.

“Ontenna” is a device that converts the loudness of sound into vibration and light intensity. Used by attaching to hair or collar of clothes (Image courtesy of Fujitsu Limited)

The title of the event is “Technology to overcome the difference of” listening to sound “”. We invited three members from the xDiversity project, which will move into the “Research Area” * of Miraikan, and listened to the opinions of the viewers while thinking about it. The xDiversiy project is researching various technologies that can overcome the physical and ability differences of each of us. Among them, the three people who are working on research on Ontenna are Tatsuya Honda (Fujitsu Ltd.) who is the developer of Ontenna, Yusuke Kanno (Associate Professor, The University of Tokyo) who studies AI technology, and AI. This is Yuri Nakao (Fujitsu Laboratories Ltd.) who studies how to use such technologies.

From the upper left of the photo, Sakiko Tanaka, Tatsuya Honda, Yusuke Kanno, Yuri Nakao

Ontenna where anyone can feel the sound

First of all, I was asked to introduce what kind of efforts have been made at Ontenna so far. It seems that the reason for the development of Ontenna was that Mr. Honda met a hearing-impaired person when he was in college. Ontenna was developed with people with disabilities because they wanted to convey sound to people who are deaf. Currently, not only are they distributed free of charge to Deaf schools nationwide, but there are also events that people who can and cannot hear using Ontenna can enjoy together. This time, I was introduced to an example of tap dancing. By wearing Ontenna, you can feel the rhythm of tap dance with your body by vibration. In addition, you can enjoy the feeling of unity as everyone on the spot glows at the same time in time with the rhythm.

Enjoy tap dancing at Ontenna

Possibility to expand by combining AI

Combining such Ontenna with AI opens up various possibilities. By learning a computer using machine learning, which is one of the AI ​​technologies, it is possible to create an Ontenna that identifies and responds only to specific sounds that you want to distinguish, such as your own name. However, each party needs a wide variety of things. The sounds you want to hear are also different. It is difficult to create an Ontenna that meets all of these needs, and there is a limit to how researchers and engineers can imagine what the parties need.

“The principle is that the person with a disability should decide what he or she wants to be and what he or she wants to do. However, if you are not familiar with machine learning, what can you do with machine learning? I can’t imagine if it can be done, “says Mr. Kanno, who believes that it is ideal for the general public, who is not a researcher, to master machine learning. Aiming for that, he and Mr. Honda and Mr. Nakao held a workshop in which people who could hear and people who could not hear were paired and created a machine learning model to identify the sounds they wanted to distinguish. Mr. Nakao says that these workshops are not only for the purpose of improving the functions of Ontenna, but also for giving them an opportunity to think about how to interact with AI. When AI technology has evolved to support every aspect of our lives, we will become “people who depend on AI support” and “people who are aware of the limits of AI and devise their own ways of using it.” There is a possibility of polarization. In response to this problem, he said, “Telling that anyone can move and design AI by themselves will close the gap between the two. In that sense, I think these activities are important.” It was.

What can you do with Ontenna? What do you want to do

So what do you want to do with Ontenna and in combination with Ontenna and machine learning? Mr. Sugano, who likes new music that he has never heard, is thinking, “If you can’t hear, how can you feel the new music?” “By learning various kinds of music through machine learning, I think we can create a model that detects rock-like, classical-like, and hip-hop-like music. But even with such a model, we can express new music well. I don’t think I can do it. I’d be happy if there was something that could convey the unique characteristics of new music. ” There seems to be no answer yet, but how can I get it? Thinking about it may be an opportunity to discover new aspects of music that you didn’t realize just by listening to it. Regarding the approach to music using Ontenna, viewers also commented that “converting music into other expressions leads to the fundamental question of what music is, which is interesting.” ..

We received many other comments during the event. For example, “Ontenna is for people who are deaf, but I wish I could have something fun like Ontenna for people who are blind (from my 1st son).” That’s a nice opinion! On the other hand, Mr. Honda said, “For example, by combining image recognition and Ontenna, we will recognize the movement of waving” bye bye “and make Ontenna vibrate. You don’t know if you shake it, but if you do this, you may notice that you waved your hand regardless of whether you can see it or not. “

Also, using image recognition technology may be a good way to make people with hearing impairments aware when they are called by voice from a place that is not in sight. “I can do my best to detect what is being called only by sound, but I think it would be fairly easy to achieve if a camera that can see 360 ​​degrees is installed,” says Mr. Sugano. It is easier to make it possible to judge “called” by image recognition by machine learning the characteristics of the person who is calling than to judge by the called voice (information that you do not understand because you can not hear it). .. It’s an important point of view that not only is it possible to provide support based on sound information just because you can’t hear the sound, but there are times when another approach is more efficient. And by developing technology for people who cannot hear or see, new possibilities for people who can see and hear, such as being able to feel the movements of people behind who cannot be seen by anyone. One of the interesting things about technology is that it may expand the sex.

Stay tuned for upcoming events! !!

During the event, we talked about various other contents. I can’t tell you everything on my blog, so if you are interested, please check the YouTube archive!

Miraikan will continue to hold events focusing on the efforts of Ontenna and other xDiversity projects. An important starting point for xDiversity research is the “difference” of each of us. We are looking forward to the participation of everyone who has various “differences” such as “differences” in opinions and “differences” in ideas as well as physical “differences”!

* What is the “research area” of Miraikan?
The “Research Area” next to the exhibition area is home to 12 external project teams that promote cutting-edge science and technology research. Various project teams such as chemistry, life science, robotics, informatics, cognitive science, and psychology are engaged in research every day. But this is not just a place for researchers. It is also a place where visitors can participate in cutting-edge research. Researchers are waiting for you to work together and create the future.


Written by: Sakiko Tanaka (National Museum of Emerging Science and Innovation, Science Communicator)
Students and teachers are all free and have strong characters. Having lived such a fulfilling high school life, I learned the joy of meeting various people. After that, he went on to the science course that he was interested in and completed graduate school. While working at a company, I faced a wall where it was difficult for me to know scientific information. I am working at Miraikan because I want to meet various people and share the fun of science, and to create a society where everyone can live comfortably.