Insider Brief
- Trossen Robotics and Stereolabs are integrating calibrated stereo-camera systems into Trossen’s Workbench and Rivet robot-learning platforms to improve data collection for physical AI training.
- The platforms combine dual WidowX Pro arms, Nvidia Jetson AGX Orin computing and three Stereolabs cameras, including wrist-mounted ZED X Nano units capable of 60 fps and depth sensing from as close as 3 centimeters.
- The systems use GMSL2 connections for longer cable runs and synchronized camera feeds, with support for ROS 2, Nvidia Isaac Sim and Isaac Lab and public demonstrations planned in San Francisco in August.
Trossen Robotics and Stereolabs are collaborating on robot-learning platforms aimed at improving the quality of data used to train physical AI models.
The collaboration will put Stereolabs’ ZED X Mini and ZED X Nano stereo cameras into Trossen’s Workbench and Rivet systems, stationary and mobile platforms built around two robotic arms. Stereolabs is a wholly owned subsidiary of lidar maker Ouster.
Collecting large amounts of synchronized visual and sensor data while people remotely demonstrate tasks for robots to imitate has its limitations when it comes to training robots, Stereolabs noted. Research teams often assemble those systems themselves using separate robot arms, USB cameras and custom software, which can lead to calibration differences, motion blur and timing problems between cameras.
“Trossen has done what few others have: put the camera at the heart of a complete, calibrated data-collection system,” said Stereolabs president Cecile Schmollgruber said in the announcement. “The Workbench with Stereolabs ZED X Mini and ZED X Nano turns every demonstration into training-grade data, and that’s what will move Physical AI forward.”
The Specs
The Workbench and Rivet combine two WidowX Pro six-degree-of-freedom arms with onboard Nvidia Jetson AGX Orin computing and a three-camera Stereolabs vision system.
Key specifications include:
- Robot arms: Two WidowX Pro 6-DoF arms
- Reach: 700 millimeters to 1,000 millimeters
- Payload: 4 kilograms to 6 kilograms
- Repeatability: 1 millimeter
- Compute: Nvidia Jetson AGX Orin with 64 GB of memory
- Vision: One center-mounted ZED X Mini and two wrist-mounted ZED X Nano cameras
- Teleoperation: Supported over local networks and the internet
- Software: Native support for ROS 2, Nvidia Isaac Sim and Isaac Lab
The Cameras
The camera layout is designed to capture both the overall workspace and close-up views needed for manipulation, according to Stereolabs. A ZED X Mini mounted in the center provides a wider view of the scene along with depth information, while a ZED X Nano on each robot wrist captures the gripper and objects at close range.
The ZED X Nano uses two 2.3-megapixel, 1920-by-1200 global-shutter sensors that can capture video at up to 60 frames per second. Unlike rolling-shutter cameras, global shutters capture an entire frame at once, reducing distortion and blur when robot arms move quickly.
Stereolabs said the cameras can calculate depth from distances as close as 3 centimeters, which is intended to improve data collection during grasping and other fine-manipulation tasks.
GMSL2 Instead of USB
The system uses GMSL2 connections rather than conventional USB links. GMSL2 is a high-speed camera connection that supports longer cable runs and more reliable synchronization. Locking, interference-resistant cables connect the three cameras to the Jetson computer.
“The Physical AI community is migrating to GMSL2 because USB can’t handle the long cable runs from the end effector to compute that real robots demand,” Trossen CEO Matt Trossen said. “Stereolabs ZED X Nano gives us the signal stability, image quality, and throughput to take Physical AI from the lab into hardened industrial deployments. Teams should spend their time collecting demonstrations and training policies, not integrating and calibrating camera rigs.”
Trossen said the architecture allows video recording, encoding and AI inference to run simultaneously while keeping camera feeds synchronized. To maintain the quality of the views from wrist-mounted cameras is a vibration-resistant onboard IMU.
Trossen plans to demonstrate the systems during its Physical AI Residency in San Francisco from Aug. 12 through Aug. 28 and at the Actuate conference Aug. 18-19.
