Robbyant Releases Open-Source AI Model to Improve Robots’ Sense of Touch
By Greg Bock · · 3 min read
Insider Brief
- Chinese embodied AI company Robbyant, part of Ant Group, has released LingBot-Touch, an open-source AI model designed to improve robots’ sense of touch and ability to handle objects.
- Trained on nearly 5,000 hours of tactile data, the model supports three sensor types and was evaluated in 26 real-world tests involving two robotic hands and four AI action models.
- Robbyant reported average improvements of 14 to 26 percentage points in task success rates, including a bottle-cap unscrewing test where success rose from 56.25% without tactile feedback to 90% using LingBot-Touch.
Chinese embodied AI company Robbyant has released an open-source AI model designed to give robots a better sense of touch and improve their ability to handle objects.
Called LingBot-Touch, the model converts signals from different tactile sensors into a common format that robotic control systems can use. According to the company, which is part of the Ant Group, trained it on nearly 5,000 hours of real-world tactile data and evaluated it in 26 tests involving two robotic hands and four AI action models.
In one test, a Sharpa Wave robotic hand controlled by the T-Rex model successfully unscrewed a bottle cap 56.25% of the time without tactile feedback. Feeding raw sensor signals directly into the action model increased the success rate to 62.5%, while LingBot-Touch raised it to 90%, according to Robbyant.
Combining Different Types of Touch Sensors
Robotic hands use different sensing technologies to detect contact and pressure, but their signals can vary depending on the sensors and how they are arranged. As a result, a model trained to interpret one type of sensor may not work with another, requiring developers to collect additional data and retrain their systems.
Robbyant claims LingBot-Touch addresses the problem by processing signals from different sensors through separate encoding pathways and mapping contact points into a shared fingertip coordinate system. The model then interprets contact location, force distribution and how those measurements change over time.
The resulting information is then converted into a common representation that different robotic control models can use.
Technical details of LingBot-Touch include:
- Sensor types: Magnetic tactile, visual tactile and piezoresistive sensing.
- Robotic hands tested: Sharpa Wave and XHand.
- Action models evaluated: LingBot-VLA, π0.5, Diffusion Policy and T-Rex.
- Lightweight version: LingBot-Touch-Tiny contains approximately 5.3 million parameters.
- Model overhead: The lightweight version adds approximately 0.13% to 0.16% to the total model size when used with LingBot-VLA, π0.5 or T-Rex.
All 26 real-world manipulation tests used LingBot-Touch-Tiny, according to the company.
Testing Across Different Hardware
Robbyant also tested whether LingBot-Touch could interpret signals from a robotic hand whose sensor had not been included in its initial training data.
The XHand robot uses a magnetic tactile sensor that was absent from the model’s pre-training dataset and its previous experience with magnetic sensing came entirely from data collected using gloves worn by humans. For the experiment, the team kept LingBot-Touch’s parameters unchanged and trained only the action model using demonstrations performed with XHand.
The company said the approach allows tactile data collected from human-worn gloves and remotely operated robotic hands to be used together during training.
Robbyant also offers a larger version of LingBot-Touch, which showed additional potential in perception tests. The company reported that the model continued to benefit from more pre-training data as the dataset approached 5,000 hours.
Although LingBot-Touch has been integrated with Robbyant’s LingBot-VLA system, it is designed to work with other robotic control models and different sensor configurations, the company pointed out.
The company has released the technical report and code, with the model available through Hugging Face and GitHub.
