Title:
|
Fruit detection and 3D location using instance segmentation neural networks and structure-from-motion photogrammetry
|
Author:
|
Gené Mola, Jordi; Sanz Cortiella, Ricardo; Rosell Polo, Joan Ramon; Morros Rubió, Josep Ramon; Ruiz Hidalgo, Javier; Vilaplana Besler, Verónica; Gregorio, Eduard
|
Other authors:
|
Universitat Politècnica de Catalunya. Departament de Teoria del Senyal i Comunicacions; Universitat Politècnica de Catalunya. GPI - Grup de Processament d'Imatge i Vídeo |
Abstract:
|
The development of remote fruit detection systems able to identify and 3D locate fruits provides opportunities to improve the efficiency of agriculture management. Most of the current fruit detection systems are based on 2D image analysis. Although the use of 3D sensors is emerging, precise 3D fruit location is still a pending issue. This work presents a new methodology for fruit detection and 3D location consisting of: (1) 2D fruit detection and segmentation using Mask R-CNN instance segmentation neural network; (2) 3D point cloud generation of detected apples using structure-from-motion (SfM) photogrammetry; (3) projection of 2D image detections onto 3D space; (4) false positives removal using a trained support vector machine. This methodology was tested on 11 Fuji apple trees containing a total of 1455 apples. Results showed that, by combining instance segmentation with SfM the system performance increased from an F1-score of 0.816 (2D fruit detection) to 0.881 (3D fruit detection and location) with respect to the total amount of fruits. The main advantages of this methodology are the reduced number of false positives and the higher detection rate, while the main disadvantage is the high processing time required for SfM, which makes it presently unsuitable for real-time work. From these results, it can be concluded that the combination of instance segmentation and SfM provides high performance fruit detection with high 3D data precision. The dataset has been made publicly available and an interactive visualization of fruit detection results is accessible at http://www.grap.udl.cat/documents/photogrammetry_fruit_detection.html. |
Abstract:
|
Peer Reviewed |
Subject(s):
|
-Àrees temàtiques de la UPC::Enginyeria agroalimentària -Àrees temàtiques de la UPC::Enginyeria de la telecomunicació::Radiocomunicació i exploració electromagnètica::Teledetecció -Remote sensing -Fruit -- Breeding -Structure-from-motion -Fruit detection -Fruit location -Mask R-CNN -Terrestrial remote sensing -Teledetecció -- Aplicacions agrícoles -Fructicultura |
Rights:
|
|
Document type:
|
Article - Submitted version Article |
Share:
|
|