Search

CN-116012773-B - Shearing method, shearing device, robot and storage medium

CN116012773BCN 116012773 BCN116012773 BCN 116012773BCN-116012773-B

Abstract

The application relates to a shearing method, a shearing device, a robot and a storage medium. The method comprises the steps of obtaining a first image of the first depth camera, which is shot by the workbench towards the plant, determining whether a branch to be sheared exists or not according to the first image, determining the average depth of the plant when the branch to be sheared exists, controlling the mechanical arm to move, adjusting the observation height of the second depth camera, obtaining a second image of the second depth camera, which is shot by the second depth camera towards the plant, determining the branch to be sheared of the plant according to the second image, and controlling the end effector to shear the branch to be sheared. The application can cut efficiently and with high precision.

Inventors

  • Guo Zhanfan
  • YE XIYANG

Assignees

  • 上海非夕机器人科技有限公司
  • 非夕科技有限公司

Dates

Publication Date
20260505
Application Date
20221201

Claims (13)

  1. 1. A cutting method, characterized by being applied to a robot comprising a table for placing plants, a first depth camera disposed on top of the table, a robotic workstation, a robotic arm disposed on the robotic workstation, and a second depth camera and an end effector disposed on the robotic arm, the method comprising: Acquiring a first image of the plant photographed by the first depth camera towards the workbench, wherein the first depth camera overlooks the plant from the top; Determining whether a branch to be sheared exists or not according to the first image; Determining the average depth of the plant when it is determined that there are shoots to be sheared; Controlling the mechanical arm to move according to the average depth, so that the second depth camera is driven by the mechanical arm to adjust the observation height, and the observation height of the second depth camera is matched with the average depth, wherein the observation height of the second camera = the height difference between the first depth camera and the robot platform base-the average depth + constant, the constant is determined according to the type of plants, and the constant is used for adjusting the observation height to shoot the complete images of the plants; Acquiring a second image shot by the second depth camera towards the plant, wherein the second depth camera observes the plant from the side; Determining branches of the plant to be sheared according to the second image; and controlling the end effector to cut the branches to be cut.
  2. 2. The method of claim 1, wherein the determining whether there is a branch to be cut from the first image comprises: Identifying the first image according to a first neural network model, and determining whether a branch to be sheared exists or not; the first neural network model is obtained by training a first image set, and any image in the first image set is provided with labeled branches to be sheared.
  3. 3. The method of claim 1, wherein after the determining of the branches of the plant to be sheared from the second image, the method further comprises: determining a first position and a first posture of the end effector according to the positions of the branches to be sheared; And controlling the mechanical arm to move according to the first position and the first gesture, so that the end effector can detect the branches to be sheared in the shearing area of the end effector.
  4. 4. A method according to claim 3, wherein controlling the movement of the robotic arm according to the first position and the first pose to enable the end effector to detect the branch to be sheared within a shearing area of the end effector comprises: Controlling the mechanical arm to move, so that the end effector moves to the first position under the drive of the mechanical arm, and the posture of the end effector is adjusted to be the first posture; When the distance sensor of the end effector detects that the branch to be sheared exists within a first preset distance, the branch to be sheared can be detected in the shearing area of the end effector, and the distance sensor is arranged on the end effector and used for detecting the distance between an object in the shearing area of the end effector and the distance sensor.
  5. 5. The method of claim 4, wherein controlling the movement of the robotic arm based on the first position and the first pose to enable the end effector to detect the branch to be sheared within a shearing area of the end effector comprises: Controlling the mechanical arm to move, so that the end effector moves to the first position under the drive of the mechanical arm, and the posture of the end effector is adjusted to be the first posture; When the distance sensor of the end effector does not detect the branches to be sheared, the mechanical arm is controlled to move, so that the end effector is driven by the mechanical arm to adjust the position and/or the gesture of the end effector according to a preset adjustment strategy; And when the distance sensor of the end effector detects that the branch to be sheared exists within a first preset distance, determining that the end effector can detect the branch to be sheared in the shearing area of the end effector.
  6. 6. The method of claim 5, wherein the method further comprises: After the end effector is driven by the mechanical arm to adjust the position and/or the posture of the end effector according to a preset adjusting strategy, when the distance sensor of the end effector does not detect the branches to be sheared, triggering the workbench to rotate, and driving the plant to rotate to a second position by the workbench; determining a third position and a second posture of the end effector at the second position according to the position of the branch to be sheared; and controlling the mechanical arm to move according to the third position and the second gesture, so that the end effector can detect the branches to be sheared in the shearing area of the end effector.
  7. 7. The method of any one of claims 3 to 6, wherein after said controlling the robotic arm to move in accordance with the first position and the first pose such that the end effector detects the branch to be sheared within a shearing region of the end effector, and before said controlling the end effector to shear the branch to be sheared, the method further comprises: controlling the end effector to move along the branches to be sheared towards the trunk of the plant; When the end effector moves along the direction of the branches to be sheared towards the trunk of the plant, if the force applied to the end effector along the extending direction of the branches to be sheared does not exceed a first threshold value, the end effector is controlled to move a second preset distance along the direction of the branches to be sheared towards the trunk of the plant; When the end effector moves along the branch to be sheared towards the trunk of the plant, if the end effector receives a force exceeding a first threshold value along the extending direction of the branch to be sheared, the end effector is controlled to move a third preset distance along the branch to be sheared towards the direction away from the trunk of the plant; Controlling the end effector to swing reciprocally along a direction perpendicular to the branches to be sheared; and controlling the end effector to clamp the branch to be sheared.
  8. 8. The method of claim 7, wherein after said controlling said end effector to grip said branch to be sheared and before said controlling said end effector to shear said branch to be sheared, said method further comprises: The end effector is controlled to move a fourth predetermined distance away from the trunk of the plant.
  9. 9. The method of any one of claims 1 to 6, wherein after said controlling the end effector to shear the limb to be sheared, the method further comprises: controlling the end effector to move away from the trunk of the plant; And controlling the end effector to shear the branches to be sheared when the end effector is subjected to a force towards the trunk exceeding a second threshold.
  10. 10. A cutting device, characterized in that it is applied to a robot comprising a table for placing plants, a first depth camera arranged on top of the table, a robot workstation, a robotic arm arranged on the robot workstation, and a second depth camera and an end effector arranged on the robotic arm, the cutting comprising: The first acquisition module is used for acquiring a first image of the plant, which is obtained by shooting the plant by the first depth camera towards the workbench, wherein the first depth camera overlooks the plant from the top; the first determining module is used for determining whether the branches to be sheared exist according to the first image; a second determining module for determining an average depth of the plant when it is determined that there are branches to be sheared; The first control module is used for controlling the mechanical arm to move according to the average depth, so that the second depth camera is driven by the mechanical arm to adjust the observation height, and the observation height of the second depth camera is matched with the average depth; wherein the observed height of the second camera = the difference in height of the first depth camera and the robotic stage base-average depth + constant, the constant being determined according to the type of plant, for adjusting the observed height to take a complete image of the plant; the second acquisition module is used for acquiring a second image shot by the second depth camera towards the plant, wherein the plant is observed by the second depth camera from the side face; A third determining module, configured to determine a branch to be cut of the plant according to the second image; and the shearing module is used for controlling the end effector to shear the branches to be sheared.
  11. 11. A robot comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, carries out the steps of the method of any one of claims 1 to 9.
  12. 12. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 9.
  13. 13. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any one of claims 1 to 9.

Description

Shearing method, shearing device, robot and storage medium Technical Field The present application relates to the field of robotics, and in particular, to a shearing method, apparatus, robot, and storage medium. Background The branches of the plants are required to be picked when growing to a certain length, the branches to be picked are identified in a manual mode at present, and then the identified branches to be picked are manually sheared. The shearing method is low in efficiency, high in labor intensity and inaccurate. Therefore, how to improve the shearing efficiency and the shearing precision is a problem to be solved at present. Disclosure of Invention In view of the foregoing, it is desirable to provide a high-efficiency, high-precision shearing method, apparatus, robot, storage medium, and computer program product. In a first aspect, the application provides a shearing method applied to a shearing robot, the shearing robot comprising a workbench for placing plants, a first depth camera arranged on top of the workbench, a robot workstation, a mechanical arm arranged on the robot workstation, and a second depth camera and an end effector arranged on the mechanical arm, the method comprising: acquiring a first image of the plant photographed by the first depth camera towards the workbench; Determining whether a branch to be sheared exists or not according to the first image; Determining the average depth of the plant when it is determined that there are shoots to be sheared; Controlling the mechanical arm to move according to the average depth, so that the second depth camera is driven by the mechanical arm to adjust the observation height, and the observation height of the second depth camera is matched with the average depth; acquiring a second image shot by the second depth camera towards the plant; Determining branches of the plant to be sheared according to the second image; and controlling the end effector to cut the branches to be cut. In one embodiment, the determining whether the branch to be cut exists according to the first image includes: The first neural network model is obtained by training a first image set, any image in the first image set is provided with labeled branches to be sheared, and it is understood that labeling can be performed by a user or by a machine, and the labeling is feasible and is not limited herein. In one embodiment, after the determining the branches of the plant to be sheared according to the second image, the method further comprises: Determining a first position and a first posture of the end effector according to the position of the branch to be sheared; And controlling the mechanical arm to move according to the first position and the first gesture, so that the end effector can detect the branches to be sheared in the shearing area of the end effector. Wherein the first position is in the vicinity of the branch to be sheared, and the branch to be sheared is located within the shearing area of the end effector when the end effector is moved to the first position and the posture of the end effector is adjusted to the first posture. In one embodiment, the controlling the movement of the mechanical arm according to the first position and the first gesture, so that the end effector can detect the branch to be sheared in the shearing area of the end effector includes: When the distance sensor of the end effector detects that the branch to be sheared exists within a first preset distance, the end effector is determined to be capable of detecting the branch to be sheared in a shearing area of the end effector, and the distance sensor is arranged on the end effector and used for detecting the distance between an object in the shearing area of the end effector and the distance sensor. In one embodiment, the controlling the movement of the mechanical arm according to the first position and the first gesture, so that the end effector can detect the branch to be sheared in the end effector shearing area includes: Controlling the mechanical arm to move, so that the end effector moves to the first position under the drive of the mechanical arm, and the posture of the end effector is adjusted to be the first posture; when the distance sensor of the end effector does not detect the branches to be sheared, the mechanical arm is controlled to move, so that the end effector is driven by the mechanical arm to adjust the position and/or the posture of the end effector according to a preset adjustment strategy, and when the distance sensor of the end effector detects that the branches to be sheared exist within a first preset distance, the end effector is determined to detect the branches to be sheared in a shearing area of the end effector. In one embodiment, the method further comprises: After the end effector is driven by the mechanical arm to adjust the position and/or the posture of the end effector according to a preset adjusting strategy, when the distance sensor of the en