Search

CN-121978126-A - 3D machine vision processing method and system for unstacking cartons

CN121978126ACN 121978126 ACN121978126 ACN 121978126ACN-121978126-A

Abstract

The invention provides a 3D machine vision processing method and system for unstacking cartons, wherein the method comprises the steps of collecting an original point cloud image of the cartons to be unstacked through a 3D machine vision system, preprocessing the original point cloud image, controlling an industrial robot to drive a sucker to execute unstacking action through a robot control system, acquiring material parameters of the cartons to be unstacked through a material characteristic library of the 3D machine vision system, extracting surface pollutant characteristics of the cartons to be unstacked through an image segmentation algorithm of the 3D machine vision system, calculating stacking deformation of the cartons to be unstacked through the 3D machine vision system, constructing a point cloud noise correction model based on the material parameters, the surface pollutant characteristics and the stacking deformation, and performing noise filtration on the preprocessed original point cloud image through the point cloud noise correction model.

Inventors

  • XIE HONGWEI
  • TANG CHUANGANG
  • LI HUANBIN

Assignees

  • 广州威华视控技术有限公司

Dates

Publication Date
20260505
Application Date
20260108

Claims (10)

  1. 1. A 3D machine vision processing method for unstacking cartons, comprising: acquiring an original point cloud image of a carton to be unstacked through a 3D machine vision system, preprocessing the original point cloud image, and controlling an industrial robot to drive a sucker to execute unstacking action through a robot control system; Characterized in that the method further comprises: Acquiring material parameters of the cartons to be unstacked through a material characteristic library of the 3D machine vision system, extracting surface pollutant characteristics of the cartons to be unstacked through an image segmentation algorithm of the 3D machine vision system, and calculating stacking deformation of the cartons to be unstacked through the 3D machine vision system; Constructing a point cloud noise correction model based on the material parameters, the surface pollutant characteristics and the stacking deformation quantity, and performing noise filtration on the preprocessed original point cloud image by using the point cloud noise correction model; Acquiring environmental temperature and humidity data of an unstacking working area through a temperature and humidity sensor, calculating pose offset of the industrial robot for grabbing the carton to be unstacked according to the output result of the point cloud noise correction model, and calculating a vacuum degree adjusting value of the sucker according to the pose offset and the material parameter; And acquiring personnel dynamic data in the unstacking working area through a personnel dynamic detection camera, constructing a safety stop threshold model by combining the pose offset, the vacuum degree adjustment value and the output result of the point cloud noise correction model, calculating the risk value of unstacking operation in real time, and triggering the industrial robot to execute emergency stop action or safety avoidance action when the risk value exceeds the output threshold of the safety stop threshold model.
  2. 2. A 3D machine vision processing method for unstacking cartons according to claim 1, wherein the material parameters include modulus of elasticity and surface coefficient of friction of the cartons to be unstacked, and the surface contaminant characteristics include oil stain coverage and water stain average thickness of the surfaces of the cartons to be unstacked; The step of acquiring the material parameters of the cartons to be unstacked through the material feature library of the 3D machine vision system comprises the steps of calculating the similarity between the actual laser reflectivity of the cartons to be unstacked and the laser reflectivity of each standard material in the material feature library, determining the corresponding standard material as the material type of the cartons to be unstacked when the similarity is larger than a preset similarity threshold, calling the standard elastic modulus and the standard surface friction coefficient corresponding to the standard material as initial material parameters, and correcting the initial material parameters based on the correlation model of the stacking deformation and the material parameters to obtain final material parameters; The step of extracting the surface pollutant characteristics of the carton to be unstacked through the image segmentation algorithm of the 3D machine vision system comprises the steps of dividing an oil stain area and a non-oil stain area on the surface of the carton to be unstacked through the image segmentation algorithm, calculating the proportion of the oil stain area to the total area of the surface of the carton to be unstacked as oil stain coverage rate, and calculating the average water stain thickness of the surface of the carton to be unstacked through the laser reflection intensity difference value of the 3D machine vision system.
  3. 3. A 3D machine vision processing method for unstacking cartons according to claim 1, characterized in that the acquisition frequency of the temperature and humidity sensor is kept synchronous with the point cloud image acquisition frame rate of the 3D machine vision system; The method comprises the steps of calculating the pose offset of the carton to be unstacked by an industrial robot according to the output result of the point cloud noise correction model, calculating the initial pose offset of the carton to be unstacked based on the output result of the point cloud noise correction model, calculating the offset value of the environment temperature and humidity data and preset standard temperature and humidity data, carrying out coupling operation on the offset value and the output result of the point cloud noise correction model to obtain an environment compensation component of the pose offset, and superposing the initial pose offset and the environment compensation component to obtain the final pose offset.
  4. 4. The 3D machine vision processing method for unstacking cartons according to claim 1, wherein the output result of the point cloud noise correction model is a point cloud noise correction coefficient K, and the point cloud noise correction coefficient K is calculated by the following formula: ; Wherein, the The stacking deformation of the cartons to be unstacked is calculated by the deviation between the 3D point cloud image and a standard model of the cartons to be unstacked, wherein the unit is; the coverage rate of the greasy dirt on the surface of the carton to be unstacked is in units of 0-100; The average thickness of the water stain on the surface of the carton to be unstacked is in mm; the critical thickness of the water stain is an experimental calibration value, and the unit is mm; The actual elastic modulus of the carton to be unstacked is expressed in MPa; the elastic modulus is a standard material elastic modulus, an experimental calibration value and the unit is MPa; 、 、 、 All are experimental fitting coefficients, and are obtained through multiple groups of different pollution-deformation scene experiment calibration.
  5. 5. A 3D machine vision processing method for unstacking cartons according to claim 1 wherein the pose offset is calculated by the formula: ; Wherein, the Correcting coefficients for point cloud noise; the actual contact area of the sucker and the carton to be unstacked is calculated through a 3D point cloud image, and the unit is cm <2 >; the minimum effective contact area of the sucker is an experimental calibration value, and the unit is cm < 2 >; The vacuum degree is a standard vacuum degree, is an experimental calibration value, and is expressed in MPa; The current vacuum degree of the sucker is expressed as MPa; the unit is the real-time ambient temperature; the standard environmental temperature is an experimental calibration value, and the unit is the temperature; the real-time environmental humidity is expressed as% RH; the humidity is the standard environmental humidity, the experimental calibration value is expressed as% RH; 、 、 all are experimental fitting coefficients, and are obtained through experimental calibration of a plurality of groups of scenes with different humiture and contact areas.
  6. 6. A 3D machine vision processing method for unstacking cartons according to claim 1 wherein the vacuum adjustment value is calculated by the following formula: ; Wherein, the Standard vacuum degree, unit is MPa; The product of the volume and the material density of the carton to be unstacked, calculated through the 3D point cloud image, is obtained for the actual weight of the carton to be unstacked, and the unit is kg; the standard grabbing weight is an experimental calibration value, and the unit is kg; The friction coefficient of the actual surface of the carton to be unstacked is dimensionless; The friction coefficient is a standard surface friction coefficient, is an experimental calibration value, and is dimensionless; The pose offset is in mm; correcting coefficients for point cloud noise; 、 、 all are experimental fitting coefficients, and are obtained through experimental calibration of a plurality of groups of scenes with different weight-friction coefficients.
  7. 7. A 3D machine vision processing method for unstacking cartons according to claim 1, characterized in that the output threshold of the safety stop threshold model is a safety stop trigger threshold Tstop calculated by the following formula: ; Wherein, the The speed of the personnel entering the unstacking work area is calculated by an image frame difference method acquired by a dynamic personnel detection camera, and the unit is m/s; The unit is an angle between the personnel intrusion direction and the movement direction of the industrial robot; the safe intrusion speed is an experimental calibration value, and the unit is m/s; the unit is kg of the actual weight of the carton to be unstacked; the standard grabbing weight is kg; The pose offset is in mm; The vacuum degree adjusting value is calculated by the formula of claim 6, and the unit is MPa; the maximum vacuum degree adjustment amount is an experimental calibration value, and the unit is MPa; the point cloud noise correction coefficient is dimensionless; 、 、 All are experimental fitting coefficients, and are obtained through the experimental calibration of a plurality of groups of different personnel dynamic-grabbing parameter scenes.
  8. 8. The 3D machine vision processing method for unstacking cartons according to claim 1, wherein the step of triggering the industrial robot to execute an emergency stop action or a safety avoidance action comprises a preset emergency threshold, when the risk value exceeds the safety stop trigger threshold and is smaller than the preset emergency threshold, triggering the industrial robot to execute the safety avoidance action, driving the industrial robot to move in a direction away from personnel intrusion by a robot motion controller, wherein the greater the risk value is obtained by calculating the association between a preset proportionality coefficient and the risk value, the greater the moving speed is, and when the risk value exceeds the preset emergency threshold, triggering the industrial robot to execute the emergency stop action, and sending a control signal to a pilot valve by the robot motion controller, wherein the pilot valve cuts off a passage between a vacuum pump and a sucker, so that the sucker releases cartons to be unstacked.
  9. 9. The 3D machine vision processing method for unstacking the cartons according to claim 2, wherein a material characteristic library of the 3D machine vision system comprises standard parameters of at least three carton materials and two soft package materials, the standard parameters comprise standard elastic modulus, standard surface friction coefficient and standard laser reflectivity, the preset similarity threshold is calibrated through a plurality of groups of material matching experiments, the value range is 85% -95%, the step of correcting initial material parameters based on a correlation model of stacking deformation and material parameters comprises the steps of establishing a mapping relation between stacking deformation and elastic modulus and surface friction coefficient through the correlation model, inputting the calculated stacking deformation into the correlation model, outputting an elastic modulus correction and a surface friction coefficient correction, and correcting the standard elastic modulus and the standard surface friction coefficient in the initial material parameters to obtain final elastic modulus and surface friction coefficient.
  10. 10. A 3D machine vision processing system for unstacking cartons, which is applied to the 3D machine vision processing method for unstacking cartons according to any one of claims 1 to 9 and is characterized by comprising a multi-DOE stereo camera, a temperature and humidity sensor, a personnel dynamic detection camera, an industrial robot, a robot motion controller, a vacuum system, a pilot valve, a vacuum pump, an air compressor and a processor; the multi-DOE stereo camera is electrically connected with the processor and used for collecting an original point cloud image of a carton to be unstacked and transmitting the original point cloud image to the processor, the temperature and humidity sensor is electrically connected with the processor and used for collecting environment temperature and humidity data of an unstacked working area and transmitting the environment temperature and humidity data to the processor, the personnel dynamic detection camera is electrically connected with the processor and used for collecting personnel dynamic data in the unstacked working area and transmitting the personnel dynamic data to the processor, the processor is electrically connected with the robot motion controller and used for receiving the original point cloud image, the environment temperature and humidity data and the personnel dynamic data, executing a preset algorithm to calculate a pose offset, a vacuum degree adjusting value and a risk value and transmitting a control instruction to the robot motion controller, the robot motion controller is electrically connected with the industrial robot and used for driving the industrial robot to move according to the control instruction, the robot motion controller is electrically connected with the pilot valve and used for controlling on-off of the pilot valve according to the control instruction, the pilot valve is respectively connected with the vacuum pump and the vacuum compressor and the vacuum chuck, the vacuum chuck and the vacuum chuck are communicated with the vacuum chuck and the vacuum chuck system, the processor, when executing the computer program, implements the steps of the 3D machine vision processing method for unstacking cartons of any one of claims 1 to 9.

Description

3D machine vision processing method and system for unstacking cartons Technical Field The invention relates to the technical field of 3D visual processing, in particular to a 3D machine visual processing method and system for unstacking cartons. Background In the fields of logistics storage, food processing and the like, the paper box unstacking operation is a key link for connecting the storage and the transportation of cargoes, and the 3D machine vision technology becomes a core sensing means of the industrial robot unstacking operation because the paper box space pose information can be rapidly acquired. In the prior art, a 3D machine vision system generally performs conventional preprocessing such as gaussian filtering and downsampling on an acquired original point cloud image to remove simple interference factors such as ambient light and equipment shake, then calculates the capturing pose of an industrial robot directly based on preprocessed point cloud data, and controls a sucker to execute unstacking action through fixed parameters. However, the prior art has the obvious technical defect that the influence of the material characteristics of the cartons to be unstacked, such as elastic modulus, surface friction coefficient and surface pollutants, such as oil stain coverage rate, water stain thickness and stacking deformation, on the 3D point cloud noise is not considered. In an actual unstacking scene, deformation caused by carton stacking extrusion can lead to distortion of the form of point cloud, surface oil stains can destroy laser reflection uniformity, water stains can easily form specular reflection interference, and the three are not independent effects, for example, the interference of the oil stains on laser reflection can be non-linear aggravated in a high deformation state, the coupling effect greatly increases the noise error of the point cloud, and the conventional pretreatment means cannot effectively filter the coupling noise. The problem directly causes the calculation precision of the follow-up grabbing pose to be obviously reduced, the grabbing position deviation of the industrial robot is easy to occur, even the carton is caused to fall off, meanwhile, the noise residue also causes the dynamic risk assessment of personnel to lack reliable data support, the triggering time of the safety control action is misaligned, and the stability and the safety of unstacking operation are seriously affected. Based on the above-mentioned problems, a technical scheme capable of solving the problem that the point cloud noise is not thoroughly filtered due to the coupling of materials, pollutants and stacking deformation is needed currently so as to improve the operation precision and the safety reliability of the 3D machine vision unstacking system. Disclosure of Invention The invention aims to solve the defects in the prior art, and provides a 3D machine vision processing method for unstacking cartons, which comprises the following steps: acquiring an original point cloud image of a carton to be unstacked through a 3D machine vision system, preprocessing the original point cloud image, and controlling an industrial robot to drive a sucker to execute unstacking action through a robot control system; The method further comprises the steps of: Acquiring material parameters of the cartons to be unstacked through a material characteristic library of the 3D machine vision system, extracting surface pollutant characteristics of the cartons to be unstacked through an image segmentation algorithm of the 3D machine vision system, and calculating stacking deformation of the cartons to be unstacked through the 3D machine vision system; Constructing a point cloud noise correction model based on the material parameters, the surface pollutant characteristics and the stacking deformation quantity, and performing noise filtration on the preprocessed original point cloud image by using the point cloud noise correction model; Acquiring environmental temperature and humidity data of an unstacking working area through a temperature and humidity sensor, calculating pose offset of the industrial robot for grabbing the carton to be unstacked according to the output result of the point cloud noise correction model, and calculating a vacuum degree adjusting value of the sucker according to the pose offset and the material parameter; And acquiring personnel dynamic data in the unstacking working area through a personnel dynamic detection camera, constructing a safety stop threshold model by combining the pose offset, the vacuum degree adjustment value and the output result of the point cloud noise correction model, calculating the risk value of unstacking operation in real time, and triggering the industrial robot to execute emergency stop action or safety avoidance action when the risk value exceeds the output threshold of the safety stop threshold model. Preferably, the material parameters comprise the elastic modulus an