CN-116958032-B - Fruit target harvesting decision-making method, system and device for air-ground cooperation

CN116958032BCN 116958032 BCN116958032 BCN 116958032BCN-116958032-B

Abstract

The invention provides a fruit target harvesting decision-making method, system and device with air-ground cooperation, and relates to the technical field of agricultural informatization, wherein the method comprises the steps of determining a target detection path based on first image information acquired in the air of a target orchard; the method comprises the steps of controlling a ground unmanned vehicle carrying with a vision sensor, collecting second image information in a target orchard based on a target detection path, determining yield distribution conditions and maturity conditions of fruit targets in the target orchard based on the second image information and the first image information, and determining a harvesting strategy of the fruit targets in the target orchard based on the yield distribution conditions and maturity conditions of the fruit targets and the topography information of the target orchard. According to the invention, the harvesting strategy of the fruit targets in the target orchard is determined based on the yield distribution condition and the maturity condition of the fruit targets and the topographic information of the target orchard, so that the accurate and efficient automatic harvesting of the fruit targets with different yield distribution and different maturity can be realized.

Inventors

ZHANG LINHUAN
ZHANG RUIRUI
CHEN LIPING
Yi tongchuan
WU MINGQI
GUO ZHIQIANG

Assignees

北京市农林科学院智能装备技术研究中心

Dates

Publication Date: 20260512
Application Date: 20230314

Claims (6)

1. A fruit target harvesting decision-making method by air-ground cooperation is characterized by comprising the following steps: determining a target detection path based on first image information acquired above a target orchard; Controlling a ground unmanned vehicle carrying a vision sensor, and acquiring second image information in the target orchard based on the target detection path; Determining the yield distribution condition and the maturity condition of fruit targets in the target orchard based on the second image information and the first image information; determining a harvesting strategy of the fruit targets in the target orchard based on the yield distribution condition and the maturity condition of the fruit targets and the topographic information of the target orchard, wherein the topographic information of the target orchard is determined based on the first image information; The determining a harvesting strategy of the fruit targets in the target orchard based on the yield distribution condition and the maturity condition of the fruit targets and the topographic information of the target orchard comprises the following steps: Determining a harvesting batch of the fruit targets in each preset area in the target orchard, the number of harvesting vehicles required to be dispatched for harvesting the fruit targets in each preset area, and a working path for harvesting the fruit targets by each harvesting vehicle based on the yield distribution condition and the maturity condition of the fruit targets and the topographic information of the target orchard, wherein the working path comprises the following steps: Harvesting fruits in a first batch in a region with the maturity in the target orchard being greater than a first preset value, harvesting fruits in a second batch in a region with the maturity in the target orchard being greater than a second preset value but less than the first preset value, and harvesting fruits in a third batch in a region with the maturity in the target orchard being less than the second preset value; the determining a target detection path based on first image information acquired above a target orchard includes: determining, based on the first image information, position information of each fruit tree in the target orchard and terrain information of the target orchard, and determining the target detection path based on the position information of each fruit tree and the terrain information of the target orchard, including: Acquiring crop row edge information of fruit trees in the target orchard by utilizing an edge detection algorithm based on the first image information, filtering data of fruit tree crowns based on a preset threshold value, performing plane fitting on the filtered data, calculating gradients between adjacent crop rows, identifying obstacles between the crop rows, and planning a target detection path for a ground unmanned vehicle by combining the position information of each fruit tree, the crop row edge information and the information of the obstacles; The first image information comprises fruit target distribution information of a canopy above a fruit tree, the second image information comprises fruit target distribution information of a canopy below the fruit tree, the yield distribution condition and the maturity condition of the fruit targets are obtained by fusing the second image information and the first image information through an image stitching technology, a complete three-dimensional map is generated, and then the three-dimensional map is used for determining the terrain information.
2. The air-ground cooperative fruit target harvesting decision method of claim 1, further comprising: Based on a harvesting strategy of the fruit targets in the target orchard, dispatching a harvesting vehicle to perform harvesting operation in the target orchard.
3. An empty-land collaborative fruit target harvesting decision-making system, comprising: unmanned aerial vehicle with vision sensor, ground unmanned aerial vehicle with vision sensor and ground station management platform; the unmanned aerial vehicle is used for acquiring first image information in the air of a target orchard based on a shooting instruction sent by the ground station management platform and sending the first image information to the ground station management platform; The ground unmanned aerial vehicle is used for acquiring second image information in the target orchard based on a target detection path determined by the ground station management platform and sending the second image information to the ground station management platform; The ground station management platform is used for determining the target detection path based on the first image information, sending the target detection path to the ground unmanned vehicle, determining the yield distribution condition and the maturity condition of fruit targets in the target orchard based on the second image information and the first image information, determining the harvesting strategy of the fruit targets in the target orchard based on the yield distribution condition and the maturity condition of the fruit targets and the topographic information of the target orchard, wherein the topographic information of the target orchard is determined based on the first image information; The determining a harvesting strategy of the fruit targets in the target orchard based on the yield distribution condition and the maturity condition of the fruit targets and the topographic information of the target orchard comprises the following steps: Determining a harvesting batch of the fruit targets in each preset area in the target orchard, the number of harvesting vehicles required to be dispatched for harvesting the fruit targets in each preset area, and a working path for harvesting the fruit targets by each harvesting vehicle based on the yield distribution condition and the maturity condition of the fruit targets and the topographic information of the target orchard, wherein the working path comprises the following steps: Harvesting fruits in a first batch in a region with the maturity in the target orchard being greater than a first preset value, harvesting fruits in a second batch in a region with the maturity in the target orchard being greater than a second preset value but less than the first preset value, and harvesting fruits in a third batch in a region with the maturity in the target orchard being less than the second preset value; the determining a target detection path based on first image information acquired above a target orchard includes: determining, based on the first image information, position information of each fruit tree in the target orchard and terrain information of the target orchard, and determining the target detection path based on the position information of each fruit tree and the terrain information of the target orchard, including: Acquiring crop row edge information of fruit trees in the target orchard by utilizing an edge detection algorithm based on the first image information, filtering data of fruit tree crowns based on a preset threshold value, performing plane fitting on the filtered data, calculating gradients between adjacent crop rows, identifying obstacles between the crop rows, and planning a target detection path for a ground unmanned vehicle by combining the position information of each fruit tree, the crop row edge information and the information of the obstacles; The first image information comprises fruit target distribution information of a canopy above a fruit tree, the second image information comprises fruit target distribution information of a canopy below the fruit tree, the yield distribution condition and the maturity condition of the fruit targets are obtained by fusing the second image information and the first image information through an image stitching technology, a complete three-dimensional map is generated, and then the three-dimensional map is used for determining the terrain information.
4. The air-to-ground cooperative fruit target harvesting decision system of claim 3, further comprising a harvesting vehicle controller and a harvesting vehicle; the harvesting vehicle controller is used for dispatching the harvesting vehicle to perform harvesting operation in the target orchard based on the harvesting strategy of the fruit targets in the target orchard determined by the ground station management platform.
5. An empty-land cooperative fruit target harvesting decision-making device, characterized by comprising: The first determining module is used for determining a target detection path based on first image information acquired above a target orchard; The image acquisition module is used for controlling the ground unmanned vehicle with the vision sensor, and acquiring second image information in the target orchard based on the target detection path; The second determining module is used for determining the yield distribution condition and the maturity condition of the fruit targets in the target orchard based on the second image information and the first image information; the third determining module is used for determining a harvesting strategy of the fruit targets in the target orchard based on the yield distribution condition and the maturity condition of the fruit targets and the topographic information of the target orchard, wherein the topographic information of the target orchard is determined based on the first image information; The determining a harvesting strategy of the fruit targets in the target orchard based on the yield distribution condition and the maturity condition of the fruit targets and the topographic information of the target orchard comprises the following steps: Determining a harvesting batch of the fruit targets in each preset area in the target orchard, the number of harvesting vehicles required to be dispatched for harvesting the fruit targets in each preset area, and a working path for harvesting the fruit targets by each harvesting vehicle based on the yield distribution condition and the maturity condition of the fruit targets and the topographic information of the target orchard, wherein the working path comprises the following steps: Harvesting fruits in a first batch in a region with the maturity in the target orchard being greater than a first preset value, harvesting fruits in a second batch in a region with the maturity in the target orchard being greater than a second preset value but less than the first preset value, and harvesting fruits in a third batch in a region with the maturity in the target orchard being less than the second preset value; the determining a target detection path based on first image information acquired above a target orchard includes: determining, based on the first image information, position information of each fruit tree in the target orchard and terrain information of the target orchard, and determining the target detection path based on the position information of each fruit tree and the terrain information of the target orchard, including: Acquiring crop row edge information of fruit trees in the target orchard by utilizing an edge detection algorithm based on the first image information, filtering data of fruit tree crowns based on a preset threshold value, performing plane fitting on the filtered data, calculating gradients between adjacent crop rows, identifying obstacles between the crop rows, and planning a target detection path for a ground unmanned vehicle by combining the position information of each fruit tree, the crop row edge information and the information of the obstacles; The first image information comprises fruit target distribution information of a canopy above a fruit tree, the second image information comprises fruit target distribution information of a canopy below the fruit tree, the yield distribution condition and the maturity condition of the fruit targets are obtained by fusing the second image information and the first image information through an image stitching technology, a complete three-dimensional map is generated, and then the three-dimensional map is used for determining the terrain information.
6. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the air-ground collaborative fruit target harvest decision method of any one of claims 1-2 when the program is executed.

Description

Fruit target harvesting decision-making method, system and device for air-ground cooperation Technical Field The invention relates to the technical field of agricultural informatization, in particular to a fruit target harvesting decision method, system and device for air-ground cooperation. Background In recent years, various fruit harvesting machines, such as a handheld fruit harvesting machine, a tree-like wrapped fruit harvesting machine and the like, are appeared on the market, and the fruit harvesting machines can improve the fruit harvesting efficiency and alleviate the problem of labor shortage. But the existing fruit harvesting machine cannot achieve accurate harvesting of fruit targets with different maturity, is low in harvesting quality, cannot achieve differentiated harvesting task allocation of different yield areas in an orchard, and is limited in harvesting efficiency improvement. Disclosure of Invention Aiming at the problems in the prior art, the invention provides a fruit target harvesting decision method, a fruit target harvesting decision system and a fruit target harvesting decision device for air-ground cooperation. In a first aspect, the invention provides a fruit target harvesting decision method for air-ground cooperation, comprising the following steps: determining a target detection path based on first image information acquired above a target orchard; Controlling a ground unmanned vehicle carrying a vision sensor, and acquiring second image information in the target orchard based on the target detection path; Determining the yield distribution condition and the maturity condition of fruit targets in the target orchard based on the second image information and the first image information; And determining a harvesting strategy of the fruit targets in the target orchard based on the yield distribution condition and the maturity condition of the fruit targets and the topographic information of the target orchard, wherein the topographic information of the target orchard is determined based on the first image information. Optionally, according to the fruit target harvesting decision method for air-ground coordination provided by the invention, the determining the harvesting strategy of the fruit target in the target orchard based on the yield distribution condition and the maturity condition of the fruit target and the topographic information of the target orchard includes: Determining a harvesting batch of the fruit targets in each preset area in the target orchard, the number of harvesting vehicles required to be dispatched for harvesting the fruit targets in each preset area and the operation path of harvesting the fruit targets by each harvesting vehicle based on the yield distribution condition and the maturity condition of the fruit targets and the topographic information of the target orchard. Optionally, according to the fruit target harvesting decision method for air-ground coordination provided by the invention, the determining the target detection path based on the first image information acquired above the target orchard includes: Determining the position information of each fruit tree in the target orchard and the terrain information of the target orchard based on the first image information; and determining the target detection path based on the position information of each fruit tree and the topographic information of the target orchard. Optionally, the method for fruit target harvesting decision-making by air-ground cooperation provided by the invention further comprises the following steps: Based on a harvesting strategy of the fruit targets in the target orchard, dispatching a harvesting vehicle to perform harvesting operation in the target orchard. In a second aspect, the present invention further provides a fruit target harvesting decision system for air-ground cooperation, including: unmanned aerial vehicle with vision sensor, ground unmanned aerial vehicle with vision sensor and ground station management platform; the unmanned aerial vehicle is used for acquiring first image information in the air of a target orchard based on a shooting instruction sent by the ground station management platform and sending the first image information to the ground station management platform; The ground unmanned aerial vehicle is used for acquiring second image information in the target orchard based on a target detection path determined by the ground station management platform and sending the second image information to the ground station management platform; the ground station management platform is used for determining the target detection path based on the first image information, sending the target detection path to the ground unmanned vehicle, determining the yield distribution condition and the maturity condition of the fruit targets in the target orchard based on the second image information and the first image information, determining the harvesting strategy of the fru