CN-121973207-A - Unordered grabbing method and unordered grabbing system for robot based on 3D vision

Abstract

The invention discloses a robot unordered grabbing method and system based on 3D vision, which relate to the field of vision grabbing and comprise the following steps: and configuring a 3D vision acquisition module to perform continuous acquisition of the incoming material region, performing definition of a robot base coordinate system, a camera coordinate system and an incoming material station coordinate system, and setting tray positioning consistency constraint. The method comprises the steps of forming deterministic mapping of a clamp and a strategy under multi-category incoming materials, improving grabbing success rate, reducing deformation and falling risk, selecting and improving collision safety in a disordered environment by minimum safety clearance constraint and path cost, maintaining operation stability by action time accounting and beat conformity judgment on a beat side, realizing traceability by sorting accuracy targets and a task log writing mechanism, and finally reducing shutdown waiting and secondary faults in a static incoming material and structure separation risk scene by alarm and separation treatment rules.

Inventors

• HOU ENZHEN
• ZHOU LINGFEI
• YIN CHAO
• LIANG SHANGDA
• LI XINGYU
• WANG CHENG
• GU FANGZHU
• LIU XIAOGENG
• WANG WEIMING
• LIU JIANQIU
• XIE RUIXIN
• LI JINXING
• LI HENGSHAN

Assignees

• 中国能源建设集团广东省电力设计研究院有限公司

Dates

Publication Date

20260505

Application Date

20260202

Claims (8)

1. 1. The disordered grabbing method of the robot based on the 3D vision is characterized by comprising the following steps of: S100, configuring a 3D vision acquisition module to perform continuous acquisition of an incoming material region, performing definition of a robot base coordinate system, a camera coordinate system and an incoming material station coordinate system, and setting tray positioning consistency constraint; S200, judging the category of the target object, outputting a target category label and a candidate grabbing area, setting a clamp type selection strategy, leading out a combined grabbing strategy entry, executing clamp switching of a clamp quick-change device, and setting clamp control signal access parameters; s300, setting a hardware parameter set, performing capture pose set generation and capture pose scoring value calculation, performing small hardware flat plane segment screening and magnetic attraction confirmation, performing non-flat outline clamping boundary frame generation and clamping in-place confirmation, performing attraction action and claw holding bottom constraint verification, and performing height detection probe triggering and claw clamping height establishment signal verification; s400, configuring an obstacle model, setting an obstacle set, generating a grabbing path section, setting unordered environment collision constraint and beat accounting parameters, executing grabbing action, and finishing judgment of an adsorption confirmation signal and a clamping in-place confirmation signal; S500, reading a target class label, outputting a placement strategy based on the target class label, executing beat control, setting beat conformity judgment, setting a sorting accuracy target, and writing a task log for tracing; S600, setting a static incoming material triggering and grabbing state judging mechanism, executing non-grabbing continuous alarming and offline manual processing temporary storage position closed loop, and configuring structure separation risk fitting special alarming and separation treatment rules.
2. 2. The method for unordered grabbing by a robot based on 3D vision according to claim 1, wherein the step S100 specifically comprises: S100.1, configuring a 3D vision acquisition module to perform continuous acquisition of an incoming material region, and performing definition of a robot base coordinate system, a camera coordinate system and an incoming material station coordinate system; The 3D vision acquisition module is provided with Mei Kaman De camera 3D cameras, the Mei Kaman De camera 3D cameras face the in-frame material feeding area, the charging basket material feeding area and the tray material feeding area to perform continuous acquisition, and the point cloud data set and the depth map data set are continuously acquired and output; The 3D vision acquisition module sets continuous acquisition frequency to be not lower than 10Hz, the 3D vision acquisition module generates acquisition sequence numbers for each frame of point cloud data set and each frame of depth map data set, the 3D vision acquisition module generates acquisition time stamps for each frame of point cloud data set and each frame of depth map data set, the resolution of the acquisition time stamps is set to be 1ms, the acquisition sequence numbers and the acquisition time stamps are written into point cloud data set entries and are written into depth map data set entries, and the point cloud data set entries and the depth map data set entries form corresponding binding relations; The robot execution module establishes a robot base coordinate system and cures the origin position and the axial direction of the robot base coordinate system, the Mei Kaman D camera 3D camera mounting bracket establishes a camera coordinate system and cures the origin position and the axial direction of the camera coordinate system, and the feeding station establishes a feeding station coordinate system and cures the origin position and the axial direction of the feeding station coordinate system; The robot execution module executes camera robot external parameter calibration, the camera robot external parameter calibration outputs a coordinate transformation matrix, and the coordinate transformation matrix is used for mapping a point cloud data set under a camera coordinate system to a robot base coordinate system and generating a target point cloud data set; s100.2, setting tray positioning consistency constraint; The AGV places the tray into the bracket, the repeated positioning precision of the AGV is set to be +/-10 mm, the repeated positioning angle deviation of the AGV is set to be +/-1 DEG, the bracket is provided with a flange and a guide deviation correcting wheel, the flange and the guide deviation correcting wheel execute limit fitting correction on the outer edge of the tray, and the position consistency of the tray after being placed is ensured; And setting a tray in-place judging logic for the incoming material station, wherein the tray in-place judging logic comprises flange attaching state judgment and guide deviation correcting wheel attaching state judgment, the pose error threshold value of the tray in-place judging logic is set to be translational + -10mm and yaw angle + -1 DEG, when the tray in-place judging logic is established, the incoming material station locks the relative relation between an incoming material station coordinate system and a robot base coordinate system, and a point cloud data set corresponding to an incoming material area of the tray is mapped to generate a target point cloud data set.
3. 3. The method for unordered grabbing of a robot based on 3D vision according to claim 1, wherein S200 specifically comprises: S200.1, judging the category of a target object, outputting a target category label and a candidate grabbing area, setting a fixture model selection strategy, and leading out a combined grabbing strategy entry; the robot executing module reads the target point cloud data set, the robot executing module executes target point cloud segmentation operation in an effective work area of the material feeding station, the target point cloud segmentation operation outputs a target point cloud segmentation cluster set, and each target point cloud segmentation cluster in the target point cloud segmentation cluster set corresponds to one candidate target object point cloud cluster; The robot execution module calculates three-dimensional external cuboid size parameters for the candidate target object point cloud clusters, wherein the three-dimensional external cuboid size parameters comprise length, width and height, the robot execution module calculates in-plane point numbers and total point cloud point numbers for the candidate target object point cloud clusters, and executes target class label judgment based on the ratio of the three-dimensional external cuboid size parameters to the in-plane point numbers, and the target class labels are defined as cartons, woven bags, sacks, turnover boxes and single electric power fittings; The robot execution module sets the carton size threshold to be 600 multiplied by 500mm in maximum size and 240 multiplied by 220 multiplied by 150mm in minimum size, the robot execution module sets the carton shape deformation threshold to be 5mm, and when the carton size threshold and the carton shape deformation threshold are met, the target type label is judged to be the carton; The robot execution module sets the reference sizes of the woven bags and the gunny bags to 600 multiplied by 400 multiplied by 300mm, the robot execution module sets the deviation threshold value of the external sizes of the woven bags and the gunny bags to +/-30 mm, and when the deviation threshold value of the external sizes of the woven bags and the gunny bags is met, the target type label is judged to be the woven bags and the gunny bags; The robot execution module sets a turnover box size threshold value to 600 multiplied by 400mm, when the turnover box size threshold value is met, the target type label is judged to be a turnover box, and when the paper box size threshold value, the woven bag and gunny bag reference size and the turnover box size threshold value are not met, the target type label is judged to be a single electric power fitting; The robot execution module generates a candidate grabbing area based on the target category label, wherein the candidate grabbing area consists of a geometric center point of a candidate target object point cloud cluster, a surface normal vector set and a grabbing approaching direction, and the candidate grabbing area is written into a candidate grabbing area record for clamp selection and clamp switching; the robot executing module establishes a clamp type selection strategy, the clamp type selection strategy establishes a corresponding relation between a target type label and a clamp name, the clamp name comprises a carton sucking disc clamp, a gunny bag special clamp, a turnover box special clamp and a special hardware single clamp, when the target type label is a carton, the robot executing module selects the carton sucking disc clamp, and the carton sucking disc clamp is matched with a carton grabbing weight threshold of 50kg and a carton size threshold; When the target type label is a single electric power fitting, a special hardware fitting single clamp is selected by the robot execution module, the special hardware fitting single clamp enables a combined grabbing strategy of an 80 magnetic chuck and a pneumatic parallel clamping jaw, the 80 magnetic chuck corresponds to a flat plane adsorption working condition of the single electric power fitting, the pneumatic parallel clamping jaw corresponds to a non-flat shape clamping working condition of the single electric power fitting, and the robot execution module is based on a clamp matching grading value Screening the execution pose of the combined grabbing strategy entry in the candidate grabbing area record; S200.2, executing clamp switching of a clamp quick-change device, and setting clamp control signal access parameters; the robot execution module is provided with a clamp quick-change device, the clamp quick-change device comprises a quick-change primary-secondary disc, the quick-change primary-secondary disc supports a load of 350kg, the robot execution module sets clamp control signal access parameters, the clamp control signal access parameters comprise input and output point position capacity, servo control group number capacity and gas path channel capacity, the input and output point position capacity is set to 219 points, the servo control group number capacity is set to 4 groups, and the gas path channel capacity is set to 40 paths; The robot execution module triggers the clamp quick-change device to execute a clamp switching process based on the target type label, and outputs a clamp ready state signal after the clamp switching process is completed.
4. 4. The method for unordered grabbing by a robot based on 3D vision according to claim 1, wherein the step S300 specifically comprises: S300.1, setting a hardware parameter set, performing capture pose set generation and capture pose scoring value calculation, performing small hardware flat plane segment screening and magnetic attraction confirmation, and performing non-flat outline clamping boundary frame generation and clamping in-place confirmation; When the clamp ready state signal is established, the robot execution module reads candidate grabbing area records of which the target class label is a single electric power clamp, and establishes a clamp parameter set which comprises a clamp size vector, a clamp weight parameter and a clamp material parameter, wherein the upper limit of the clamp weight parameter is set to be 50kg, and the clamp material parameter is used for judging the magnetic force adsorption condition of the 80 magnetic chuck; The robot execution module generates a grabbing pose set in the candidate grabbing area record, wherein the grabbing pose set comprises grabbing position vectors and grabbing gesture matrixes, and grabbing approach vectors of the grabbing gesture matrixes are taken from a surface normal vector set in the candidate grabbing area record; the robot execution module calculates a grabbing pose scoring value for the grabbing pose set, and writes the grabbing pose scoring value into grabbing pose set items; the robot execution module sets a small hardware fitting set, wherein the small hardware fitting set comprises QP7, QP10, W-7B, WS-10, U-7, Z-7 and PH-10, when the target class label is a single electric power hardware fitting and the hardware fitting parameter set is matched with the small hardware fitting set, the robot execution module starts an 80 magnetic chuck adsorption working condition; when the target type tag is a single electric power fitting and the magnetic attraction confirming signal does not meet 120ms, enabling a robot executing module to start a pneumatic parallel clamping jaw clamping working condition, and generating a clamping boundary frame in a candidate target object point cloud cluster by the robot executing module, wherein the clamping boundary frame comprises a clamping center position vector and a clamping opening direction vector; The robot execution module sets a clamping jaw opening amount target value which is formed by clamping the boundary width of the boundary frame and adding a safety margin, wherein the safety margin is set to be 6mm, the robot execution module sets a clamping force target value to be 60N to 110N, and the clamping force target value takes a high value along with the increase of hardware weight parameters; s300.2, executing adsorption action and holding claw bottom constraint verification, and executing height detection probe triggering and claw clamping height establishment signal verification; When the target type label is a carton, a robot executing module reads the clamp name of the carton sucking disc clamp, the robot executing module sets the maximum size of the carton to 600 multiplied by 500mm and the minimum size of the carton to 240 multiplied by 220 multiplied by 150mm, the robot executing module sets the shape deformation threshold of the carton to 5mm, and when the shape deformation of the carton meets 5mm and the size of the carton meets the maximum size and the minimum size of the carton, a lifting mechanism of the carton sucking disc clamp executes descending action to complete adsorption action, and after the lifting mechanism executes ascending action, a holding claw of the carton sucking disc clamp executes closing action to form bottom constraint; When the target type labels are woven bags and gunny bags, the robot execution module reads the clamp names of the gunny bag special clamp, the robot execution module sets the deviation threshold of the outline dimensions of the woven bags and the gunny bags to be +/-30 mm and the grabbing weight threshold of the woven bags and the gunny bags to be 50kg, and when the deviation of the outline dimensions of the woven bags and the gunny bags meets +/-30 mm and the grabbing weight meets 50kg, the gunny bag special clamp executes the clamping action to complete the sucking action, and the gunny bag special clamp maintains the clamping state and synchronously executes the bottom-covering constraint closing action; When the target type label is a turnover box, the robot execution module reads the clamp name of the special turnover box clamp, the robot execution module sets the size threshold of the turnover box to be 600 multiplied by 400mm, and when the size threshold of the turnover box is met, the robot execution module drives the special turnover box clamp to move to the position above the candidate grabbing area of the turnover box.
5. 5. The method for unordered grabbing of a robot based on 3D vision according to claim 1, wherein the step S400 specifically comprises: S400.1, configuring an obstacle model, setting an obstacle set, generating a grabbing path segment, and setting unordered environment collision constraint and beat accounting parameters; After the clamp ready state signal is established, the robot execution module reads the target point cloud data set and the target class label, establishes an obstacle model under a robot base coordinate system, and the obstacle model adopts voxel to occupy and express, and the voxel resolution is set to be 5mm; The robot execution module constructs an obstacle set, wherein the obstacle set comprises a material frame boundary, a material basket boundary, a bracket flange and an identified object which does not grasp, the material frame boundary, the material basket boundary and the bracket flange are written into the obstacle set by material station structure parameters, and the identified object which does not grasp is written into the obstacle set by a candidate object point cloud cluster which does not enter a grasp confirmation state in a target point cloud partition cluster set; The robot execution module reads the effective grabbing pose, and generates a grabbing path section by taking a grabbing approach vector corresponding to the effective grabbing pose as constraint, wherein the grabbing path section comprises an approaching section, a lower exploring section, a clamping action section, a lifting section and an evacuating section, the continuity of the pose at the tail end of the approaching section and the initial pose of the lower exploring section is maintained, the continuity of the pose at the tail end of the lower exploring section and the initial pose of the clamping action section is maintained, the continuity of the pose at the initial pose of the lifting section and the pose at the tail end of the clamping action section is maintained, and the continuity of the pose at the tail end of the lifting section is maintained; The robot execution module sets unordered environment collision constraint, the collision constraint adopts a minimum safety clearance threshold value, the minimum safety clearance threshold value is set to be 10mm, the robot execution module discretely samples 100 path points in each grabbing path section, the robot execution module calculates the minimum distance between the robot body envelope corresponding to each path point and the safety barrier set, and the grabbing path section with the minimum distance smaller than 10mm is judged as an inexecutable grabbing path section and the grabbing path section recalculation is triggered; the robot execution module sets the robot linear speed to be 2m/s for beat accounting, the robot execution module sets the robot speed percentage to be 90%, and the robot execution module converts the robot linear speed according to the robot speed percentage and is used for grasping path section time accounting; Robot execution module is based on snatch path cost value Selecting a grabbing path segment with the minimum grabbing path cost value from the grabbing path segment candidate set as an executing grabbing path segment, and writing the executing grabbing path segment into a grabbing execution queue; s400.2, performing grabbing action, and finishing judgment of the adsorption confirmation signal and the clamping in-place confirmation signal; The robot execution module drives the clamp to execute the approaching section, the lower detecting section and the clamping action section according to the grabbing execution queue, when the target type label is a carton, the robot execution module drives the carton sucker clamp to execute the vacuum adsorption action, the carton sucker clamp is provided with a sponge sucker or an array uniformly distributed multilayer sucker, the carton sucker clamp is provided with a one-way valve structure for isolating local air leakage, the robot execution module reads the output of a vacuum pressure sensor and sets a vacuum adsorption confirmation threshold value as-55 kPa, and when the output of the vacuum pressure sensor meets-55 kPa and is stably kept for 120ms, the robot execution module judges that the vacuum adsorption confirmation is established and drives the lifting section and the evacuation section; when the vacuum adsorption confirmation is not met or the magnetic adsorption confirmation is not met or the clamping in-place confirmation is not met, the robot execution module terminates the lifting section and executes the withdrawal movement of the withdrawal section, and the robot execution module marks the record of the corresponding candidate grabbing area as a grabbing failure state and triggers the recalculation of the grabbing path section.
6. 6. The method for unordered grabbing by a robot based on 3D vision according to claim 1, wherein the step S500 specifically comprises: S500.1, reading a target class label, outputting a placement strategy based on the target class label, executing beat control, and setting beat conformity judgment; when the target class label is a carton, the placement strategy outputs the carton discharging station placement position, when the target class label is a woven bag and a gunny bag, the placement strategy outputs the woven bag and gunny bag discharging station placement position, and when the target class label is a turnover box, the placement strategy outputs the turnover box discharging station placement position; When the target type label is a single electric power fitting, the robot execution module reads the fitting parameter set and generates a fitting model label, the fitting model label is limited to QP7, QP10, W-7B, WS-10, U-7, Z-7 and PH-10, and the placement strategy binds the fitting model label with the specified classification temporary storage position and outputs the placement pose of the specified classification temporary storage position; The robot execution module drives the tail end pose of the evacuation section to transition to a placement approaching pose according to the placement pose, the placement approaching pose and the placement pose keep grabbing approaching vectors constrained in the same direction, and the placement approaching distance threshold is set to be 40mm; The robot execution module establishes an action time accounting parameter, wherein the action time accounting parameter comprises lifting mechanism adsorption time and grabbing path section execution time, the lifting mechanism adsorption time of the carton sucker clamp is set to 2.5s, the lifting mechanism adsorption time of the gunny bag special clamp is set to 2.5s, the carton sucker clamp corresponds to the whole action time to 14s, the gunny bag special clamp corresponds to the whole action time to 14s, the turnover box special clamp corresponds to the whole action time to 12s and meets the 5 pieces/min beat target; the robot execution module sets a beat upper limit time threshold, the beat upper limit time threshold takes 14s for the carton sucker clamp and the gunny bag special clamp, the beat upper limit time threshold takes 12s for the turnover box special clamp, and the robot execution module calculates actual action time for each grabbing execution queue and outputs beat coincidence degree; S500.2, setting a sorting accuracy target, and writing a task log for tracing; The robot execution module sets a sorting accuracy target as 99%, and generates a task log entry for each grabbing execution queue, wherein the task log entry comprises a target class label, a hardware model label, an effective grabbing pose, a clamp name, a clamp number and a grabbing pose scoring value Matching score value of clamp Grabbing path cost value The clamp serial numbers are respectively assigned and fixed to the carton sucker clamp, the gunny bag special clamp, the turnover box special clamp and the special hardware single clamp by the clamp quick-change device to be unique serial numbers; and the robot execution module writes task log entries into the task log after the placement completion state signal is established, the task log writing period is set to be not higher than 200ms, when the execution result is marked as successful, the robot execution module enters the next target point cloud data set processing flow, and when the execution result is marked as failed, the robot execution module marks the candidate grabbing area record as a grabbing failure state and triggers grabbing pose set update.
7. 7. The method for unordered grabbing of a robot based on 3D vision according to claim 1, wherein the step S600 specifically comprises: S600.1, setting a static incoming material triggering and grabbing state judging mechanism, and executing non-grabbing continuous alarm and off-line manual processing temporary storage position closed loop; After the 3D vision acquisition module outputs the point cloud data set, the robot execution module leads out an inter-frame point cloud difference value which is used for representing the integral change amplitude of the items of the two adjacent frame point cloud data sets; The robot execution module sets a static incoming material judging threshold value to be 1.0mm, the robot execution module sets a static incoming material continuous frame number threshold value to be 30 frames, and when the continuous 30 frames of the inter-frame point cloud difference value meets 1.0mm, the robot execution module judges that the static incoming material state is established; after the static incoming material state is established, the robot execution module executes the graspable state judgment on the target point cloud segmentation cluster, the graspable state judgment simultaneously meets the establishment judgment of the exposed surface, the establishment judgment of the collision constraint of the unordered environment and the establishment judgment of the entrance of the clamping jaw, wherein the robot execution module reads the surface normal vector set and the grasping approaching direction recorded in the candidate grasping area, the robot execution module sets the included angle between the surface normal vector and the grasping approaching vector to be 20 degrees, the robot execution module sets the point number of the exposed surface to be 0.35, and the robot execution module judges that the exposed surface is established when the point number of the exposed surface is 0.35; the robot execution module leads out a non-grabbing continuous frame number counter, the non-grabbing continuous frame number counter is accumulated frame by frame along with the non-grabbing state in a static feeding state, the robot execution module sets the non-grabbing continuous frame number threshold value as 50 frames, and when the continuous acquisition frequency is not lower than 10Hz, the 50 frames correspond to the duration time for 5s; When the non-grippable continuous frame number counter meets 50 frames, the robot execution module locks the task number and outputs an alarm prompt signal to the control cabinet, wherein the alarm prompt signal comprises an audible and visual alarm output and a human-computer interface prompt text output, and the human-computer interface prompt text output writes the task number and the temporary storage position treatment guide of the offline manual processing; The robot execution module leads out a temporary offline processing position, the temporary offline processing position solidifies a position vector in a robot base coordinate system, the temporary offline processing position is guided to write in a manual unwrapping action, a manual re-swinging action and a re-identification triggering condition, the robot execution module sets the re-identification triggering condition as a tray in-place judging logic is established, the inter-frame point cloud difference value is continuous for 10 frames and meets 1.0mm, and after the re-identification triggering condition is established, the robot execution module releases a task number locking state and clears an unclampable continuous frame number counter, and the robot execution module re-reads a target point cloud data set and refreshes a target point cloud segmentation cluster set and a candidate grabbing area record; S600.2, configuring special alarm and separate disposal rules of the structure separation risk fitting; The robot execution module establishes a structure separation risk fitting set, the structure separation risk fitting set is limited to NX-2 and NUT-2, when the target class label is judged to be a single electric power fitting and the fitting parameter set is matched with the structure separation risk fitting set, the robot execution module executes two-part unconnected state judgment on the candidate target object point cloud cluster, the robot execution module executes secondary segmentation on the candidate target object point cloud cluster and generates a first sub-cluster and a second sub-cluster, and the robot execution module calculates the minimum connection distance between the first sub-cluster and the second sub-cluster; When the two parts are not connected, the robot execution module outputs an alarm prompt signal to the control cabinet and locks the task number, the human-computer interface prompt text outputs and writes a separate storage treatment rule and a separate grabbing treatment rule, wherein the separate storage treatment rule places a first sub-cluster corresponding part and a second sub-cluster corresponding part into two fixed partition positions of a downlink manual processing temporary storage position, the two fixed partition positions solidify position vectors in a robot base coordinate system, the separate grabbing treatment rule respectively generates candidate grabbing area records for the first sub-cluster corresponding part and the second sub-cluster corresponding part and respectively writes the candidate grabbing area records into a grabbing execution queue, and an interval time threshold between the two grabbing executions is set to be 2s for reducing grabbing failure and dropping risks.
8. 8. A 3D vision-based disordered grabbing system for a robot, for executing the four-way shuttle-based three-dimensional warehouse scheduling method according to any one of claims 1-7, comprising the following modules: The modeling calibration module is used for configuring the 3D vision acquisition module to execute continuous acquisition of the incoming material region, executing definition of a robot base coordinate system, a camera coordinate system and an incoming material station coordinate system, and setting tray positioning consistency constraint; the classification and model selection module is used for executing target object type judgment, outputting a target type label and a candidate grabbing area, setting a clamp model selection strategy, leading out a combined grabbing strategy entry, executing clamp switching of a clamp quick-change device, and setting clamp control signal access parameters; The pose adapting module is used for setting a hardware fitting parameter set, executing capture pose set generation and capture pose scoring value calculation, executing small hardware fitting flat plane segment screening and magnetic force adsorption confirmation, executing non-flat outline clamping boundary frame generation and clamping in-place confirmation, executing adsorption action and claw holding bottom constraint verification, and executing height detection probe triggering and claw clamping height establishment signal verification; The collision planning execution module is used for configuring an obstacle model, setting an obstacle set, generating a grabbing path section, setting unordered environment collision constraint and beat accounting parameters, executing grabbing action, and completing judgment of an adsorption confirmation signal and a clamping in-place confirmation signal; The sorting beat module is used for reading the target class label, outputting a placement strategy based on the target class label, executing beat control, setting beat conformity judgment, setting sorting accuracy targets, and writing task logs for tracing; the static alarm module is used for setting a static incoming material triggering and grabbing state judging mechanism, executing non-grabbing continuous alarm and off-line manual processing temporary storage position closed loop, and configuring special alarm and separate disposal rules of the structural separation risk fitting.

Description

Unordered grabbing method and unordered grabbing system for robot based on 3D vision Technical Field The invention relates to the field of vision grabbing, in particular to a robot unordered grabbing method and system based on 3D vision. Background In the logistics operation of storage and stations, incoming objects are in unordered stacking and multi-form mixing characteristics, common targets comprise cartons, woven bags and gunny bags, turnover boxes and single electric power fittings, when a robot performs grabbing sorting in an incoming material area, a charging basket incoming material area and a tray incoming material area in a frame, point clouds and depth maps are usually formed by relying on 3D vision, and recognition results are mapped to a robot base coordinate system, so that continuous operation flows of target recognition, grabbing pose generation, path planning, grabbing confirmation and classified placement are realized. The existing disordered grabbing scheme has the defects that firstly, the unified mapping of a camera coordinate system and a robot base coordinate system lacks dual constraint on the consistency of external parameter calibration residual errors and tray positioning, pose drift easily occurs under an AGV feeding scene, so that a candidate grabbing area and an effective grabbing pose are unstable, secondly, a static incoming material and a hooking overlapping scene lack continuous alarm and manual handling closed loop after grabbing state judgment, separation handling rules are not formed in the unconnected state of the hardware, and grabbing failure and dropping risks are increased. Disclosure of Invention The invention aims to overcome the defects of the prior art, and provides a method and a system for unordered grabbing of a robot based on 3D vision, which aims to continuously acquire a point cloud data set and a depth map data set through 3D vision, bind acquisition timestamps, establish a robot base coordinate system, a camera coordinate system and a charging station coordinate system, lock a coordinate transformation matrix by using an external parameter calibration residual error threshold, introduce a tray in-place consistency constraint into an AGV charging scene to solidify relative relation, promote unified stability of cloud coordinates of a target point from the source, trigger clamp selection and quick change switching based on a target category label, generate a grabbing pose set, calculate grabbing pose scoring value, combine an obstacle model and unordered environment collision constraint to execute grabbing confirmation, execute grabbing state judgment, non-grabbing continuous alarm, offline manual processing temporary position closed loop, trigger special alarm and separate handling rules of structural separation risk hardware under static charging conditions. Therefore, the application provides a robot unordered grabbing method and a system based on 3D vision, comprising the following steps: And S100, configuring a 3D vision acquisition module to perform continuous acquisition of the incoming material region, performing definition of a robot base coordinate system, a camera coordinate system and an incoming material station coordinate system, and setting tray positioning consistency constraint. And step 200, performing target object type judgment, outputting a target type label and a candidate grabbing area, setting a clamp type selection strategy, leading out a combined grabbing strategy entry, performing clamp switching of a clamp quick-change device, and setting clamp control signal access parameters. Step S300, setting a hardware parameter set, performing capture pose set generation and capture pose scoring value calculation, performing small hardware flat plane segment screening and magnetic attraction confirmation, performing non-flat outline clamping boundary frame generation and clamping in-place confirmation, performing attraction action and claw holding bottom constraint verification, and performing height detection probe triggering and claw clamping height establishment signal verification. And S400, configuring an obstacle model, setting an obstacle set, generating a grabbing path section, setting unordered environment collision constraint and beat accounting parameters, executing grabbing action, and completing judgment of an adsorption confirmation signal and a clamping in-place confirmation signal. And S500, reading a target class label, outputting a placement strategy based on the target class label, executing beat control, setting beat conformity judgment, setting a sorting accuracy target, and writing a task log for tracing. Step S600, setting a static incoming material triggering and grabbing state judging mechanism, executing non-grabbing continuous alarming and offline manual processing temporary storage position closed loop, and configuring special alarming and separate disposal rules of the structural separation risk fitting. In so