Insider Brief
- Researchers at Osaka Metropolitan University have developed a method for generating synthetic training data for agricultural AI systems using realistic virtual tomato farms, potentially reducing one of the major bottlenecks in developing autonomous harvesting robots.
- The study, published in Smart Agricultural Technology, used images collected by agricultural robots together with Unreal Engine 5 and 3D Gaussian Splatting to create virtual tomato farms that automatically generate training images and labels for AI models.
- The researchers found that AI systems trained on the synthetic datasets were able to detect tomatoes in real-world images, suggesting the approach could reduce the time and cost of developing agricultural robots and be adapted for other crops.
Researchers at Osaka Metropolitan University have developed a method for generating synthetic training data for agricultural AI systems using realistic virtual tomato farms, a technique that could reduce one of the major challenges in developing autonomous harvesting robots.
The study, published in Smart Agricultural Technology, focuses on using virtual environments to train AI models that identify and assess the ripeness of tomatoes, according to the university. The approach could help reduce the need for labor-intensive data collection and manual image labeling, which remain significant challenges in agricultural robotics.
Reseachers pointed out that modern harvesting robots rely on computer vision systems to locate fruit and determine whether it is ready to be picked. Training those systems typically requires large numbers of real-world images in which every tomato must be manually identified and labeled. That the process, researchers noted, becomes even more difficult because lighting conditions, plant structures and growing environments vary widely between farms and seasons.
Virtual Tomatos
To tackle the problem, the research team, led by Takuya Fujinaga of OMU’s Graduate School of Engineering, created a virtual agricultural environment capable of automatically generating realistic images of tomato plants along with corresponding training labels.
The virtual farms were reconstructed using images collected by agricultural robots and real tomatos, then researchers used Unreal Engine 5 and 3D Gaussian Splatting to create detailed digital representations of plants, fruit and surrounding conditions. The resulting simulations were designed to replicate real-world situations, such as changing lighting, overlapping leaves, shadows and partially obscured tomatoes.
Using positional data within the virtual environment, the system automatically generated annotations identifying tomato locations and ripeness levels, researchder said. The framework also exported labels in YOLO format, a widely used standard for training object detection systems.
The Findings
The researchers then used the synthetic datasets to train AI models and found that the resulting systems were able to detect tomatoes in real-world images, demonstrating that virtual environments can serve as a practical source of training data for agricultural AI.
“By comparing differences in the shape of 3D tomato models, lighting conditions, and the amount of data, we identified conditions that affect AI accuracy,” Fujinaga said. “Understanding how lighting, tomato shape, and dataset size affect detection performance are important discoveries for improving the model in the future.”
While the work focused on tomatoes, the researchers said the findings could apply to a broader range of crops and harvesting applications. The approach could potentially reduce the time and cost associated with building AI systems for agricultural robots while making it easier to adapt those systems to different growing environments.
Featured image: The virtual environment (left) is created using real-world tomato farm images (right) manually obtained from camera data collected by agricultural robots. (Credit: Osaka Metropolitan University)
