Insider Brief
- Foxglove added agentic AI capabilities to its robotics data platform to help developers search robot data, investigate failures, compare runs and prepare datasets for model development.
- The platform includes an Agent Sidebar for built-in analysis and an MCP server that lets external AI agents interact with Foxglove through Model Context Protocol.
- Foxglove also introduced Nvidia Cosmos-powered Semantic Search for finding behaviors in unlabeled robot data, along with Comparison Mode and Remote Access for analyzing model changes and debugging deployed robots.
Foxglove has added new agentic AI capabilities to its robotics data platform to help developers find failures, compare robot runs and turn large volumes of sensor data into training and evaluation datasets more quickly.
Autonomous machines can generate large amounts of synchronized video and sensor data, but engineers often review only a small portion of it, according to Foxglove. Finding the particular events that caused a failure or exposed a weakness in a model can require substantial manual work and the company said its new tools let engineers describe in plain language the behavior, failure or dataset they want to investigate rather than manually searching through video, telemetry, lidar, logs and other robot data.
“Robots are increasingly powered by AI, but the workflows used to develop and improve them remain largely manual,” co-founder and CEO Adrian Macneil said in the announcement, which was made at Actuate 26. “Foxglove brings intelligence into the development loop, allowing engineers to describe the behavior they need to investigate or the dataset they want to create, and let agents do the work.”
The platform’s capabilities are available through two interfaces. The Agent Sidebar is built into the Foxglove application and can search data, create visualizations, investigate failures, compare runs and organize results into datasets. The Foxglove MCP server allows outside AI agents to interact with the platform through Model Context Protocol.
Semantic Search
The company also introduced Semantic Search, developed with Nvidia and powered by the Nvidia Cosmos video-text embedding model. Foxglove said developers can enter a description of an action or event and retrieve matching segments from unlabeled robotics data.
Those results remain linked to the other information recorded at the same time, including telemetry, logs and sensor signals. That allows engineers to move from finding an event in video to examining what the rest of the robot was doing when it occurred.
Foxglove said Semantic Search works with its bring-your-own-storage architecture, allowing customers to index data stored in their own cloud environments without first creating another copy of the raw logs.
Comparison Mode
Comparison Mode is designed in part to help developers identify changes between successive model and software releases. Remote Access addresses a different part of the process by allowing engineers to examine deployed robots without traveling to the site, according to Foxglove.
Underlying the system is time-synchronized data from cameras, lidar, radar, telemetry and logs. Foxglove said keeping those sources tied to the same robot run gives its AI tools the context needed to investigate events rather than relying on isolated images or text descriptions.
