Search

CN-122004045-A - Closed-loop servo fruit stem shearing and picking system and closed-loop picking method thereof

CN122004045ACN 122004045 ACN122004045 ACN 122004045ACN-122004045-A

Abstract

The invention discloses a closed-loop servo fruit stem shearing and picking system and a closed-loop picking method thereof, and belongs to the technical field of agricultural intelligent equipment and agricultural robots. The method comprises the steps of step 1, target perception and candidate generation, step 2, constructing a shear point generation model based on the candidate generation in step 1, step 3, performing closed loop alignment by adopting visual servo based on the shear points formed in step 2, step 4, performing cutting based on the alignment in step 3, step 5, judging the cutting state in step 4, performing secondary repair cutting, rollback realignment or giving up and recording if judging is not completed, and step 6, fusing the environment and the maturity and scheduling updating. The invention is used for solving the problems of unstable positioning, uncontrollable open loop, non-intelligent decision, non-traceable operation and the like of the existing water accumulation.

Inventors

  • YAO TONG
  • GAO ZIHANG
  • CHEN ZIHENG
  • GU WENHAO

Assignees

  • 哈尔滨工业大学（威海）

Dates

Publication Date
20260512
Application Date
20260209

Claims (10)

  1. 1. The closed-loop servo fruit stem shearing and picking control system is characterized by comprising a visual identification module, an environment monitoring module and a core control system; The visual recognition module is used for acquiring related information of a target fruit and a fruit stem and outputting a shearing point and uncertainty/confidence; the environment monitoring module is used for collecting parameters such as temperature and humidity and the like to form a spatial distribution characteristic; The core control system is used for executing the scheduling strategies of shear point generation, visual servo closed loop alignment, shear state criterion and error correction and fusion of the maturity and environment.
  2. 2. A closed loop picking method of a closed loop servo fruit stem shearing picking system, wherein the closed loop picking method uses a closed loop servo fruit stem shearing picking system as defined in claim 1, the closed loop picking method comprising the steps of: Step1, target perception and candidate generation; step 2, constructing a shear point generation model based on the candidate generation in the step 1; Step 3, based on the shearing points formed in the step 2, adopting visual servo closed loop alignment; Step 4, executing shearing based on the alignment in the step 3; Step 5, judging the shearing state in the step 4, and if judging that the shearing state is not finished, executing secondary shearing, returning to realign or giving up and recording; and 6, fusing the environment and the maturity, and scheduling and updating.
  3. 3. The method according to claim 2, wherein the step 1 is specifically to collect the image and depth information to obtain the candidate fruit target set and the confidence thereof.
  4. 4. The method according to claim 2, wherein the step 2 is specifically that, for the ith target, a three-dimensional point of the root of the fruit stem is obtained Three-dimensional point with center of fruit body Fruit stem direction unit vector The method comprises the following steps: Wherein the shear point Defined as the point offset in the direction of the fruit stem: Wherein the offset is According to fruit size And (3) adaptive determination: Wherein, the 、 、 And Is a preset or calibrated parameter.
  5. 5. The method according to claim 3, wherein the step 3 is to project the shearing point to the image coordinates , ) The projection coordinate of the center of the guiding opening of the scissors is% , ) Defining an image error: The control system generates the omnidirectional speed and the lifting speed of the chassis according to the error: When meeting the requirements And visual confidence Triggering shearing during the time, ensuring alignment stability and ensuring alignment stability.
  6. 6. The method according to claim 1, wherein the step 5 is specifically collecting steering engine current Construction of window energy index : When a contact peak occurs and falls back to a threshold within a time window, the following is satisfied: And continue And if the criterion is not met within T_ { max }, triggering an error correction strategy.
  7. 7. The closed loop picking method of claim 6, wherein the trigger error correction strategy is if at maximum shear time If the internal condition is not met, triggering error correction action, namely executing micro-return + para-position + secondary compensation, or if the confidence of the target is reduced, discarding the target and recording failure reasons, and switching to the next target.
  8. 8. The method of closed loop picking as claimed in claim 7 wherein step 6 is specifically visual maturity probability Environment priori Fusion to obtain comprehensive score : Constructing a scheduling objective function based on scoring and mobile cost : By maximising The picking sequence and the path are obtained, and efficiency improvement and risk reduction are realized.
  9. 9. A closed loop servo stem shearing and picking control system using the system of claim 1, wherein the closed loop servo stem shearing and picking control system is applied in facility agriculture, plant factories and greenhouse cultivation.
  10. 10. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a computer program which, when executed by a processor, implements the method according to any of claims 2-8.

Description

Closed-loop servo fruit stem shearing and picking system and closed-loop picking method thereof Technical Field The invention belongs to the technical field of agricultural intelligent equipment and agricultural robots, and particularly relates to a closed-loop servo fruit stem shearing and picking system and a closed-loop picking method thereof. Background Along with the large-scale development of facility agriculture, plant factories and greenhouse cultivation, the requirements of fruit and vegetable harvesting links on the operation efficiency, stability and traceability are continuously improved. Traditional fruit picking mainly depends on manual completion, and has the problems of high labor intensity, obvious influence of personnel proficiency, tension of seasonal labor, poor operation consistency, high management cost and the like. In this context, automatic picking equipment based on mobile platforms, machine vision and end effectors is becoming a direction of research and industry interest. Existing indoor agricultural picking equipment generally adopts a camera to acquire images, identifies fruit positions through target detection or traditional image processing, and drives a mobile platform and a mechanical arm/clamp/cutter to finish picking. However, in the real operation scene of indoor agriculture, the spatial distribution and imaging conditions of fruits have remarkable complexity that the fruits are always in a state of branch and leaf shielding, mutual overlapping or half shielding, backlight, reflection, local shadow, illumination change with time and the like exist in a greenhouse, and in addition, the fruit skin is vulnerable, the intensity difference of fruit stalks is large, and the flexible swing of plant branches is obvious. These factors make the "detect-para-pick" procedure subject to insufficient stability in engineering applications. Most of the schemes in the prior art output only two-dimensional information such as the center of the fruit detection frame, the size of the bounding box, etc., and estimate the target position based on this and perform approaching and cutting. Under the conditions of leaf shielding, fruit overlapping and illumination change, the detection frame is easy to shake, drift or false detection, so that the target position is unstable along with the change of the frame. More importantly, the fruit detection frame cannot directly describe the spatial posture and the shearable position of the fruit stalks, if the fruit frame coordinates are used for open-loop alignment, the shearing points are easily deviated from the fruit stalks, the fruit peels are damaged or the branches and leaves are erroneously sheared, and the picking success rate and consistency are difficult to ensure. In the prior art, an open loop strategy is mostly adopted, namely, after a target is identified, a moving and shearing action is executed, and online judgment and feedback control on whether to align, shear and error shear are lacked. Under the conditions of thickness change of fruit stalks, interference of branches and leaves, shaking of plants and the like, anomalies such as continuous shearing, empty shearing, clamping shearing, pulling of plants and the like often occur, and after the anomalies occur, manual intervention is often needed for resetting or repositioning, so that operation is interrupted, and the engineering requirements of indoor agriculture on unattended and continuous automatic picking are difficult to meet. Therefore, the lack of a closed loop mechanism of "para-cut-decision-error correction" is an important reason for restricting the practical use of the existing scheme. Indoor agriculture has obvious space microenvironment difference, and the temperature and humidity, ventilation and illumination conditions of different areas in the same greenhouse/channel can be different, so that the fruit ripening speed and the ripening consistency are affected. The existing picking scheme generally decides picking objects and sequences only according to single visual recognition results, and lacks decision mechanisms for fusing environmental information such as temperature and humidity with maturity evaluation, so that picking path planning is unreasonable, repeated round trip is caused, picking amount per unit time is low, meanwhile, process records related to fruit quality are difficult to form, and follow-up quality tracing, refined planting management and data closed-loop optimization are not facilitated. In summary, the existing indoor agricultural fruit picking technology still faces the problems of unstable positioning, uncontrollable open loop, intelligent decision making, non-traceable operation and the like under a complex environment. In order to improve the success rate, stability and continuous unattended capability of picking, a system capable of realizing accurate generation of shearing points, closed-loop alignment of visual servo, judgment of shearing state