Perception Tools for Collaborative and Autonomous Pruning for Bi-Manipulator Robot in Table Grape Vineyards

Abstract

Robots have found extensive applications in various domains, including social and industrial settings. However, the agricultural sector is still in the early stages of incorporating robotics, fac- ing unique challenges and specialized requirements. Recognizing the potential, initiatives like the CANOPIES project by the European Union aim to tackle precision viticulture challenges and contribute to the development of robotics in agriculture. The project’s ultimate goal is to enable agricultural personnel and robots to collaborate on a variety of agronomic activities, such as harvesting and pruning of table grape vineyards. This challenge exhibits a multiplicity of complex areas including control, human-robot interaction, computer vision, and mechanics. To address these challenges, the project’s objectives are being pursued through the develop- ment and implementation of a collaborative bi-manipulator robot, designed and optimized by multiple European universities. This Master’s Thesis addresses the problem of perception in the pruning procedure, aiming to contribute to the CANOPIES project. Through collaboration with experienced agronomists over the course of approximately one year, innovative instruments integrating classical and reliable computer vision methodologies have been developed. These tools enable the robot to success- fully complete the pruning task and cover essential aspects of robotics in table grape vineyards, including autonomous vineyard software, vine segmentation, trellis wire reconstruction, and pruning point identification. The developed solution has undergone rigorous development and testing using data from Aprilia’s field (Italy) and a lab mock-up setup. After conducting field evaluations, the vineyard soft- ware system has shown promising outcomes. The vine segmentation demonstrates satisfactory performance, while the F1-scores for trellis wire and bud detection reach 88.3% and 66.8% re- spectively. Consequently, the solution effectively fulfills the requirements set by the CANOPIES project