CN-122008247-A - Automatic hand-eye calibration method

CN122008247ACN 122008247 ACN122008247 ACN 122008247ACN-122008247-A

Abstract

The invention relates to the field of robots and industrial automation, in particular to an automatic hand-eye calibration method, which comprises the following steps of placing a calibration plate in a working space of a robot, moving the tail end of the robot to the position points of a central area of the calibration plate, carrying out path planning on the position points and driving the tail end of the robot to move to each position point, controlling the camera system to shoot the image of the calibration plate, and recording the position of the robot at the moment.

Inventors

ZHAO QIAN
LU ZHUSHI
CHEN ZEXIONG
GUAN YISHENG
XIE XIAO
YU SHUANGFEI
QI YULIANG
CHEN HAOFAN
LI YUE
DAI SHUDAN
LI ZHONGGAO

Assignees

广东工业大学

Dates

Publication Date: 20260512
Application Date: 20260407

Claims (10)

1. An automatic hand-eye calibration method is characterized by comprising the following steps: S1, mounting a camera system to the tail end of a robot, and adjusting the position of the camera system to ensure that interference between the camera system and an end effector of the robot does not occur; S2, placing the calibration plate in a working space of the robot, ensuring that the calibration plate is positioned at a position where the robot can surround and observe, moving the tail end of the robot to the position right above the calibration plate, enabling the normal vector of a flange at the tail end of the robot to be perpendicular to the plane of the calibration plate, and adjusting the tail end position of the robot up and down to enable the calibration plate to be full of the visual field of a camera system; S3, generating a series of pose points enabling an optical axis of a camera system to point to a central area of the calibration plate, planning a path of the pose points, driving the tail end of the robot to move to each pose point, controlling the camera system to shoot an image of the calibration plate, and recording the pose of the robot at the moment; S4, calibrating internal parameters and external parameters of the camera system by using a plurality of groups of shot calibration plate images and completing image registration of the camera system; And S5, based on the recorded robot pose data and the corresponding calibration plate image, establishing a hand-eye calibration equation and solving the hand-eye calibration equation to obtain a pose conversion relation between the camera system and the tail end of the robot, and completing the calibration of the hand-eye system of the robot.
2. The automated hand-eye calibration method according to claim 1, wherein in step S1, when the camera system is mounted to the robot tip, the position of the camera system is adjusted so that the optical axis of the camera system is in positive agreement with the Z axis of the robot tip coordinate system, i.e., the TCP coordinate system; In step S2, the position of the tail end of the robot is adjusted to lead the normal vector of the flange of the tail end of the robot to point to the center of the calibration plate, the position of the tail end of the robot is adjusted up and down, after the calibration plate is full of the field of view of the camera system, the coordinate of the tail end of the robot under the base coordinate system at the moment is recorded as P t =(x t , y t , z t ), the coordinate of the center point of the calibration plate under the base coordinate system of the robot is recorded as P o =(x t , y t , z t -r 0 ), and the vertical distance from the tail end of the robot to the center point of the calibration plate is measured Establishing a spherical coordinate system by taking the center point of the calibration plate as the origin To Creating a virtual hemisphere around the calibration plate for the radius; In step S3, the method for generating the pose points enabling the optical axis of the camera system to point to the central area of the calibration plate comprises the steps of randomly sampling points on a virtual hemispherical surface, generating a series of spherical pose points pointing to the center of the calibration plate, planning paths of the spherical pose points, driving the tail end of a robot to move to each pose point, controlling the camera system to shoot the image of the calibration plate, and recording the pose of the robot at the moment.
3. The automated hand-eye calibration method according to claim 2, wherein in step S3, the process of generating a series of pose points for pointing the optical axis of the camera system toward the center area of the calibration plate comprises the steps of: S31 knowing the points in the spherical coordinate system With points in a Cartesian coordinate system The conversion relation of (2) is: According to the translation relation between the coordinate systems, any point on the virtual hemispherical surface The expression under the robot-based coordinate system is: At this time, the position of any point P on the spherical surface under the robot base coordinate system is obtained ; S32, under the calculation base coordinate system, a vector n of a spherical point pointing to the center point of the calibration plate: Vector As the Z-axis direction of the end effector pose when the robot moves: Y-axis component of vector n The method comprises the following steps: (3) Wherein, the Is the Z-axis unit direction vector of the robot base coordinate system, and finally the direction vector of the X-axis can be obtained by the right hand rule : Finally, the calibration of the optical axis pointing of the camera is obtained robot tip pose in plate center area ; S33 for points on the virtual sphere Randomly sampling, substituting the coordinates into the tail end gesture of the robot, and obtaining a series of gesture points for enabling the optical axis of the camera system to point to the central area of the calibration plate.
4. The automated hand-eye calibration method according to claim 3, wherein in step S33, the radius is equal to Points on a range of virtual spheres And (5) randomly sampling.
5. The automated hand-eye calibration method according to claim 3, wherein in step S3, the path planning process for the pose points is that the inverse kinematics solution of the robot is solved for the spherical pose points under the robot base coordinate system, and the continuous robot motion track is planned according to the inverse kinematics solution.
6. The automated hand-eye calibration method according to claim 2, wherein in step S3, after the calibration plate image is captured, whether the calibration plate is successfully identified is detected, the robot pose and the calibration plate pose are recorded if the identification is successful, and the capturing is repeated until a preset number of valid calibration plate images are acquired.
7. The automated hand-eye calibration method according to claim 2, wherein in step S3, the spherical points generated by random sampling are first subjected to height screening, a height coefficient c is set, and only the spherical points of Z-axis coordinate Z s ＞c·r 0 in the robot base coordinate system are selected for subsequent inverse kinematics solution, wherein the height coefficient c=0.6 to 0.9.
8. The automated hand-eye calibration method according to claim 1, wherein the camera system comprises an RGB industrial camera and a 3D camera, and in step S4, the internal parameters and external parameters of the RGB industrial camera and the 3D camera are calibrated by using the photographed plurality of groups of calibration plate images, and image registration is completed, and a rigid body transformation matrix between the RGB industrial camera and the 3D camera is obtained by solving, so as to complete calibration between the cameras.
9. The automated hand-eye calibration method of claim 8, wherein the image distortion correction of the RGB industrial camera and the point cloud correction of the 3D camera are accomplished simultaneously when calibrating the internal and external parameters of the RGB industrial camera and the 3D camera.
10. The automated hand-eye calibration method according to claim 9, wherein in step S4, the image registration is performed by extracting corner points of the calibration plate, registering color images output by the RGB industrial camera with depth images or point cloud data output by the 3D camera through the corner points, and solving the rigid body transformation matrix between the two cameras by acquiring external parameters of the RGB industrial camera 、 And any single purpose external parameter of 3D camera 、 And (3) calculating rigid body transformation matrixes of the RGB camera and the 3D camera through the following steps of: 。

Description

Automatic hand-eye calibration method Technical Field The invention relates to the fields of robots and industrial automation, in particular to an automatic hand-eye calibration method. Background The existing hand-eye calibration method is mainly to utilize a calibration plate to calibrate by solving an AX=XB high-dimensional nonlinear matrix equation, wherein A is the transformation relation of the tail end of the robot relative to a base coordinate system, B is the transformation relation of a camera relative to the coordinate system of the calibration plate, and X is the required pose of the camera relative to the tail end of the robot or the pose of the camera relative to a base of the robot. In order to solve the hand-eye relationship, the robot is required to collect a plurality of calibration images (usually 15-20 images) under different poses, the calibration images have high quality, and the characteristic points of the calibration images can cover the whole image range, so the calibration process takes a long time. Disclosure of Invention The invention aims to overcome the defect of long time consumption of hand-eye calibration in the prior art, and provides an automatic hand-eye calibration method which can improve hand-eye calibration efficiency and shorten calibration time. In order to solve the technical problems, the invention adopts the following technical scheme: an automatic hand-eye calibration method is provided, which comprises the following steps: S1, mounting a camera system to the tail end of a robot, and adjusting the position of the camera system to ensure that interference between the camera system and an end effector of the robot does not occur; S2, placing the calibration plate in a working space of the robot, ensuring that the calibration plate is positioned at a position where the robot can surround and observe, moving the tail end of the robot to the position right above the calibration plate, enabling the normal vector of a flange at the tail end of the robot to be perpendicular to the plane of the calibration plate, and adjusting the tail end position of the robot up and down to enable the calibration plate to be full of the visual field of a camera system; S3, generating a series of pose points enabling an optical axis of a camera system to point to a central area of the calibration plate, planning a path of the pose points, driving the tail end of the robot to move to each pose point, controlling the camera system to shoot an image of the calibration plate, and recording the pose of the robot at the moment; S4, calibrating internal parameters and external parameters of the camera system by using a plurality of groups of shot calibration plate images and completing image registration of the camera system; And S5, based on the recorded robot pose data and the corresponding calibration plate image, establishing a hand-eye calibration equation and solving the hand-eye calibration equation to obtain a pose conversion relation between the camera system and the tail end of the robot, and completing the calibration of the hand-eye system of the robot. According to the automatic hand-eye calibration method, the optical axis of the camera system always points to the central area of the calibration plate, so that the success rate of the calibration plate in the visual field range of the camera system is improved, the number of invalid images is reduced, the calibration efficiency is improved, and the calibration time is shortened. Further, in step S1, when the camera system is mounted to the robot end, the position of the camera system is adjusted so that the optical axis of the camera system is in positive agreement with the Z axis of the robot end coordinate system, i.e., the TCP coordinate system; in step S2, the position of the tail end of the robot is adjusted to lead the normal vector of the flange of the tail end of the robot to point to the center of the calibration plate, the position of the tail end of the robot is adjusted up and down, after the calibration plate is full of the field of view of the camera system, the coordinate of the tail end of the robot under the base coordinate system at the moment is recorded as P t=(xt,yt,zt), the coordinate of the center point of the calibration plate under the base coordinate system of the robot is recorded as P o=(xt,yt,zt-r0), and the vertical distance from the tail end of the robot to the center point of the calibration plate is measured Establishing a spherical coordinate system by taking the center point of the calibration plate as the originToCreating a virtual hemisphere around the calibration plate for the radius; In step S3, the method for generating the pose points enabling the optical axis of the camera system to point to the central area of the calibration plate comprises the steps of randomly sampling points on a virtual hemispherical surface, generating a series of spherical pose points pointing to the center of the