A VISION SYSTEM FOR DATE HARVESTING ROBOT
Hamdi Altaheri · Zenodo (CERN European Organization for Nuclear Research) · 2019
In date cultivation, manual harvesting is the dominant method used, which is inefficient in terms of both time and the economy. Advanced agricultural automation such as robotic harvesting can significantly increase quality and yield as well as reduce production costs and delay. One of the most important aspects of harvesting robots is their ability to interpret and analyze visual data. Accurate vision system to detect, classify, and analyze fruits in real time is critical for the harvesting robot to be cost-effective and efficient. However, practical success in this area remains limited due to the difficulties caused by unstructured and unconstrained agricultural environments. Furthermore, research on machine vision for date fruits in the pre-harvesting and the harvesting stages is scarce. Hence, this research aims to develop intelligent systems for date fruit harvesting robotic in an orchard environment, including date fruit detection and segmentation, variety classification, maturity analysis, and automated harvesting decision system. We propose an efficient deep learning based machine vision framework for date fruit harvesting robots that consists of three classification systems used to classify date fruit images in real time according to their variety, maturity, and harvesting decision. Deep convolutional neural networks (CNNs) are utilized with transfer learning and fine-tuning techniques. In the case of date fruit segmentation, we propose a robust method to detect date fruits and segment them from the background (trunk, stalks, ground, sky, etc.). The method utilizes a superpixel clustering and local binary patterns with support vector machines. This research also creates a comprehensive dataset for date fruits that can be used for multiple tasks including automated harvesting, visual yield estimation, and classification tasks. The dataset has been fully labeled, coded, and released with their associated files to the research community in the IEEE DataPort repository [1] (http://dx.doi.org/10.21227/x46j-sk98). The resources of this research, including source codes, datasets, models, and test videos, are available for the benefits of researchers on the website https://daterobotic.hamdialtaheri.com/.