Overview

SRI, a leading research institute, collaborated with Sony Electronics to integrate Sony’s Spatial Reality Display (SRD) — a glasses-free 3D display — with SRI’s XRGo robotic telemanipulation platform. This collaboration aimed to enhance operator experience and precision in remote robotic control through stereoscopic 3D visualization.

The Challenge

SRI sought to improve the fidelity and usability of its XRGo robotic control system. While VR headsets can be ideal for some implementations of XRGo, headsets also limit the user’s awareness of the surrounding environment and can impact interactions with nearby co-workers. SRI wanted a solution that could deliver real-time, glasses-free 3D visualization without compromising on performance or compatibility with existing workflows.

The Solution

To address SRI’s requirements, Sony worked with SRI to adapt XRGo’s robot telemanipulation software for streaming in 3D stereo video. The solution processes video from a 3D stereo camera located on a robotic arm and delivers it to the Sony SRD in real time.* This allows operators to experience an immersive, glasses-free visualization environment, significantly enhancing their situational awareness and control precision. Sony and SRI conducted extensive user testing, with iterative feedback loops to fine-tune depth cues, minimize latency, and ensure intuitive user interaction within the 3D interface.

The Outcome

This integration delivered a marked advancement in how operators interact with remote robotic systems. Equipped with real-time stereoscopic visualization through the Sony Spatial Reality Display, operators were able to perceive depth with remarkable clarity, which translated into more precise manipulation and control of the robotic arm. The enhanced visual feedback reduced errors and improved response time, enabling users to approach complex tasks with greater confidence and efficiency. Moreover, the system’s glasses-free design further contributed to operator comfort, making extended sessions more feasible without fatigue. This technological leap not only brought additional capabilities to the XRGo system but also set a new standard for immersive teleoperation within SRI, paving the way for future research and applications in various domains such as remote surgery, hazardous environment exploration, and advanced scientific visualization.

Executive Quote

“The Sony Spatial Reality Display monitor enables a new type of deeply immersive experience on our XRGo platform, advancing our mission to make robotic telemanipulation more efficient and effective for users across numerous industries.” Nicholas Marion, Robotics Engineer, SRI

*Computer required with a recommended Windows 10/11 PC with a CPU of Intel Core i7-9700K @3.60GHz or faster;16GB of memory or higher, and a 3D graphics card with a Passmark score of 18,000 or faster.