Sony is known for products familiar to general consumers, such as the smartphone Xperia and the pet robot aibo. The company is researching and developing not only such products but also various new technologies within the group, and one part of them is shown at an event called “Sony Technology Day”.

On December 7, “Sony Technology Day 2021” was held online to showcase the new technologies that the Sony Group is working on. Here, we will focus on technologies that are familiar to us general consumers, such as the VR head-mounted display (HMD), which is equipped with two 4K resolution organic EL (OLED) micro-displays and realizes high image quality of 8K with both eyes. I look back.

In addition to online explanations, we will also provide a place for the press to experience the two technologies of “OLED micro display + low latency HMD system” and “manipulator” that reproduces delicate human hands. Was done. Please also read the report article (written by Atsushi Yamamoto) covering them.

8K “real world” with both eyes.High image quality + low latency VR HMD system

The new VR HMD system introduced at this event is equipped with two OLED micro-displays that achieve 4K resolution, achieving high resolution of 4K for one eye and 8K for both eyes. You can project a high-definition 3D space.

  • Prototype of “OLED micro display + low latency HMD system” unveiled at Sony Technology Day

The built-in OLED micro display is close to a square and has a size that is not much different from a 10-yen coin. Although it is very small, when I actually looked at the HMD prototype at the experience exhibition, the image quality was surprisingly high.

  • OLED micro display built into the prototype HMD

In conventional HMDs, the “screen door effect” that makes the pixel boundaries look like a mesh and reduces the sense of detail has often occurred, but this prototype can hardly be felt. did. You can enjoy clear and detailed images such as the showroom drawn in 3DCG and the live-action images of women and dogs, each with a three-dimensional effect.

It was a short experience because it was a prototype, but it seems that we can fully expect the black expression of the dark part unique to OLED.

  • Pixel image of OLED microdisplay.Although it is small, it has a high image quality and the “screen door effect” is suppressed compared to the conventional (left).

In particular, I was amazed at the high expressiveness and reality of the 3D model drawn in CG. In the demo, I was able to see the 3D model of Sony’s mirrorless camera “α9” in a showroom-like space, but the texture expression was extremely high from the leather-like grained grip to the details of each dial and ring. It was. Involuntarily (even though it was an empty space), I tried to reach out and touch it, and laughed with the commentator who was giving the demonstration lecture.

  • The 3D model (left) of the mirrorless camera “α9” seen on the HMD (right end) was too real and I wanted to touch it.

The built-in OLED micro-display was developed by utilizing the microfabrication technology cultivated in the development and manufacture of CMOS image sensors to realize multi-pixel and miniaturization, while also utilizing the device and circuit technology cultivated in display development. Says.

In addition, by combining the information of multiple sensors, the amount of delay is reduced in the entire system and the processing time is shortened. The texture of the material and facial expressions of the person can be expressed in high definition in real time according to the movement of the viewer’s head.

  • Sony’s neckband speaker (bottom center) was also used in this HMD demo.

Speaking of Sony’s HMD, in the past there was the “HMZ series” for personal use equipped with organic EL, and now the “PlayStation VR” for PS4 / PS5 is on sale. In addition, although details have not been revealed, SIE has announced that it is developing a next-generation VR system for PS5.

In addition to industrial applications, the HMD system unveiled this time is “expected to be used in the entertainment field.” We will be paying attention to how this technology will be commercialized and appear in front of us.

Three technologies introduced in PS5 to improve the immersive feeling in the game

Sony Technology Day introduces the three technologies of “Tempest 3D Audio” used in “PlayStation 5” (PS5), “Haptic Feedback” and “Adaptive Trigger” of DualSense wireless controller.

Tempest 3D audio is designed from DSP (digital single processor) to realize accurate audio positioning, and it is said that sound can be heard from anywhere in 360 degrees with the image of being inside a sphere with innumerable speakers. It’s a technology. It is one of the main features of PS5, and you can experience it with compatible headphones such as genuine headsets, as well as TV speakers connected to PS5.

  • Image of listening to Tempest 3D audio with a headset

The haptic feedback provided in the DualSense wireless controller changes the vibration of the newly developed dual actuator according to the situation in the game, realizing a variety of tactile reactions and providing an immersive experience. In addition, the L2 / R2 button uses an adaptive trigger, and the small precision gear and high torque motor built into it create a powerful tactile sensation in real time in response to various actions in the game.

  • Actuator that produces the effect of haptic feedback (right) built into the DualSense wireless controller (left)

A compatible game is required to experience these technologies, but you can experience both with the action game “ASTRO’s PLAYROOM” pre-installed on PS5. PS5 continues to be difficult to obtain, but it is a recommended title that you should try once you get it.

The latest manipulator that can hold soft things without crushing

One of the highlights of the hands-on exhibition for the press was the “manipulator,” which reproduces the delicate movements of human hands and fingers. It is also called a robot arm and is a technology that has already been used in industrial applications, but the major feature of Sony’s newly developed manipulator is that it can carefully handle objects whose characteristics are unknown.

  • A humanoid robot equipped with a “manipulator” that reproduces the delicate movements of human hands and fingers

The robot’s minions on display were equipped with soft parts with a total of 16 (4 x 4) protrusions. This part, which is softer than the cat’s paws, is equipped with a pressure sensor, and it is possible to detect how much pressure is applied to each protrusion to deform it. From the change in the pressure distribution, it is a mechanism that detects the sign that the object being grasped slips in real time, adjusts the force it has appropriately, and keeps holding it without dropping it.

  • The hands of the robot on display are equipped with soft parts that detect the pressure distribution.

In addition, a distance sensor is provided on the part of the human palm to grasp the distance from the finger to the object. In combination with the above pressure sensor, you can hold an object in an appropriate position and posture.

In the demo, there are only sweets such as soft eclairs and cakes, rose flowers, vegetables, egg shells, etc. that should be taken care of by human hands, and you can freely choose and have them. I was able to make it. One of our strengths is that we can keep holding things so that they do not break the things we have, detect the force in the direction in which they are likely to rotate, and do not drop them.

  • A manipulator developed by Sony with a soft paprika.I keep holding it firmly without applying too much force and distorting it

In addition, he used the trick of pouring water (beads) into an empty cup with both hands, using the fine beads that fill the bottle as water. It is a situation that tends to fail unexpectedly, such as dropping a bottle (or a cup) with a human hand and throwing the contents, but in this manipulator demo, the operation was done without difficulty.

  • Demonstration of pouring water (beads) into an empty cup with both hands, using the fine beads that fill the bottle as water.

In this way, Sony’s newly developed manipulator can grasp objects as delicately as a human hand, so in situations where it was difficult to introduce robots in the past, such as human care support and product display work at stores. You can expect to be active.

When you think of Sony robots, you can think of things with a lot of entertainment elements such as the dog-shaped “aibo” and the “QRIO” that can walk on two legs, but this new technology is to realize “robot technology that enriches people’s lives”. It seems to be a new step towards.

Sony Technology Day introduced various other technologies. Let’s take a quick look at them with the official video.

Technology for analyzing and visualizing human movements such as sports

“Virtual production” that synthesizes live-action video and CG in real time

Achieving both high image quality and production efficiency, application of super-resolution technology to Ray Tracing

Stacked SPAD distance sensor that can be applied to in-vehicle LiDAR and drones

“Earth Mimamori Platform” that informs the “change” of the earth by sensing technology