CN-116794637-B - Laser radar, camera and IMU combined calibration device and method

CN116794637BCN 116794637 BCN116794637 BCN 116794637BCN-116794637-B

Abstract

The invention discloses a laser radar, camera and IMU combined calibration device, which relates to the technical field of environmental perception, and comprises a frame, a driving mechanism, a top plate, a collecting device, an inclination angle adjusting mechanism for driving the top plate to tilt forwards and backwards and a swing angle adjusting mechanism for driving the collecting device to swing horizontally; the combined calibration method introduces the real swing angle of the swing arm obtained by the measurement of the angle measuring instrument, and corrects the accumulated error of the IMU by a proportional integral method. The inclination angle adjusting mechanism and the swing angle adjusting mechanism drive the top plate and the acquisition device to move in a linkage way under the drive of the same driving mechanism, so that the linkage of the forward tilting movement, the backward tilting movement and the horizontal swinging movement of the acquisition device is realized, the detailed adjustment and the continuity guarantee of shooting angles are facilitated, a large number of images at different angles can be conveniently and conveniently shot and acquired, a large number of images shot and acquired at different angles are used for improving the accuracy of a calculation result, and finally the high-precision calibration of the sensor combined assembly is realized.

Inventors

ZHANG BINGLI
PAN ZEHAO
WANG XINYU
JIANG JUNZHAO
CHENG JIN
ZHANG CHENGBIAO
ZHANG YANGYANG
WANG YIXIN
YANG CHENGLEI
WANG YANHUI

Assignees

合肥工业大学

Dates

Publication Date: 20260512
Application Date: 20230423

Claims (10)

1. The laser radar, camera and IMU combined calibration device is characterized by comprising an inclination angle adjusting mechanism (2) for driving the top plate (5) to tilt forwards and tilt backwards and a swing angle adjusting mechanism (3) for driving the top plate (6) to swing horizontally, wherein the inclination angle adjusting mechanism (2) and the swing angle adjusting mechanism (3) are driven by the same driving mechanism (1) to drive the top plate (5) and the collection device (6) to move in a linkage manner; The inclination angle adjusting mechanism (2) comprises a threaded driven shaft (22), a threaded sleeve (23) and a swinging rod (24), wherein the swinging rod (24) is arranged at the front part and the rear bottom of the top plate (5), the threaded sleeve (23) is positioned below the front part of the top plate (5), the top end of the threaded sleeve is rotatably connected with the top plate (5) through the swinging rod (24) positioned at the front bottom of the top plate (5), and the threaded driven shaft (22) is inserted into the threaded sleeve (23) through the threaded fit of the bottom end of the threaded sleeve (23); The translational output end and the rotational output end of the driving mechanism (1) respectively output translational and rotational motions, the translational output end is rotatably connected with the top plate (5) through a swinging rod (24) positioned at the bottom of the rear part of the top plate (5), a revolute pair supporting forward tilting and backward tilting motions of the top plate (5) is formed between the translational output end and the top plate, the translational output end drives the rear part of the top plate (5) to reciprocate in the vertical direction, and the rotational output end is fixedly connected with an inclination angle adjusting rotational input end arranged at the bottom end of the threaded driven shaft (22) and drives the threaded driven shaft (22) to rotate around the axis of the threaded driven shaft.
2. The laser radar, camera and IMU combined calibration device according to claim 1, wherein the swing angle adjusting mechanism (3) comprises a crank (32), a swing rod (33), a sliding sleeve (34), a swing arm (35) and an angle measuring instrument for measuring the swing angle of the swing arm (35); The swing rod (33) is of an L-shaped rod structure formed by a horizontal section and a vertical section, the front end of the horizontal section is rotatably installed and connected with the frame (4), a revolute pair supporting the horizontal section to horizontally swing is formed between the horizontal section and the frame, the tail end of the vertical section is rotatably installed and connected with the front end of the swing arm (35), and two revolute pairs supporting the vertical section to drive the front end of the swing arm (35) to horizontally swing and to tilt forwards and backwards are formed between the vertical section and the frame; The tail end of the swing arm (35) is fixedly arranged on the acquisition device (6), the crank (32) is connected with the rotation output end in a mounting way through a swing angle adjusting input end arranged at the front end of the crank, and the tail end of the crank is connected with a sliding sleeve (34) sleeved on the horizontal section in a sliding fit way in a rotating way; A revolute pair around a vertical rotating shaft is formed between the tail end of the crank (32) and the sliding sleeve (34), a sliding pair along the horizontal section is formed between the sliding sleeve (34) and the horizontal section, and the rotation output end drives the crank (32) to rotate around the vertical rotating shaft at the front end of the crank.
3. The combined laser radar, camera and IMU calibration device according to claim 2, wherein the driving mechanism (1) comprises a driving motor (11), a driving cam (12), a lifting rod (13), a first rotating groove (14), a second rotating groove (15), a driving shaft (16) and a driving gear (17); The driving motor (11) is fixedly arranged on the frame (4), the output end of the driving motor is fixedly connected with the driving cam (12), the driving motor (11) drives the driving cam (12) to rotate around a fixed shaft of a horizontal rotating shaft, the lifting rod (13) is positioned above the driving cam (12), the bottom end of the lifting rod is matched with the driving cam (12), a cam pair for driving the lifting rod (13) to reciprocate along the vertical direction is formed between the lifting rod and the driving cam, the top end of the lifting rod (13) is used as a translational output end of the driving mechanism (1), and the lifting rod is rotatably arranged and connected to the bottom of the rear part of the top plate (5) through the swinging rod (24); The driving shaft (16) is rotatably mounted on the frame (4), a driving gear (17) serving as a rotation output end is fixedly arranged on the driving shaft, the front end of the first rotating groove (14) is rotatably mounted and connected to the middle of the lifting rod (13), the tail end of the first rotating groove is rotatably mounted and connected with the front end of the second rotating groove (15), the tail end of the second rotating groove (15) is fixedly connected with the driving shaft (16), and a rotating pair around a horizontal rotating shaft is formed between the front end of the first rotating groove (14) and the middle of the lifting rod (13), between the tail end of the first rotating groove (14) and the front end of the second rotating groove (15) and between the driving shaft (16) and the frame (4).
4. The laser radar, camera and IMU combined calibration device according to claim 3, wherein the inclination angle adjusting rotation input end and the swing angle adjusting input end are respectively an inclination angle adjusting gear (21) and a swing angle adjusting gear (31), are bevel gears, are symmetrically arranged on the upper side and the lower side of the driving gear (17), and are in meshed connection with the driving gear (17) which is the bevel gear.
5. The laser radar, camera and IMU combined calibration device according to claim 3, wherein the top end of the lifting rod (13) is of a supporting rod (18) structure extending horizontally to two sides, and two ends of the supporting rod (18) are respectively connected with the top plate (5) in a rotating and installing mode through one swinging rod (24).
6. The laser radar, camera and IMU combined calibration device according to any one of claims 1-4, wherein the swinging rod (24) is a swinging rod.
7. The combined laser radar, camera and IMU calibration device according to any one of claims 2-4, wherein the acquisition device (6) comprises a camera (61), an IMU (62), a laser radar (63) and a mounting plate (64), wherein the camera (61), the IMU (62) and the laser radar (63) are mounted and fixed on the mounting plate (64), and the mounting plate (64) is rotatably mounted on the top of the top plate (5); The mounting plate (64) is rotatably mounted on the top of the top plate (5) and forms a revolute pair around a vertical rotating shaft, and the tail end of the swing arm (35) is fixedly mounted on the side surface of the mounting plate (64).
8. The laser radar, camera and IMU combined calibration device according to claim 1, wherein the driving mechanism (1) and the inclination angle adjusting mechanism (2) are provided with a box body (8), the box body (8) is provided with a yielding hole (81) adapting to the swinging motion of the swinging rod (33), and the bottom of the frame (4) is provided with a plurality of casters.
9. A laser radar, camera, IMU joint calibration method, using the calibration device of claim 3 to calibrate a multi-sensor joint assembly, comprising the steps of: The method comprises the steps of firstly, installing and fixing an IMU (62), a laser radar (63) and a camera (61) on an installation plate (64), and adjusting the position of a calibration plate (7) to enable the calibration plate (7) to be located at the central position of a picture acquired by the camera (61); Step two, the combined calibration device performs data acquisition work of one period; thirdly, acquiring a camera external parameter matrix serving as a camera (61) calibration result according to the following formula : (Formula one); Wherein, the In the form of the coordinates of the camera pixel coordinate system, In the form of coordinates in the world coordinate system of the camera, f is the focal length of the camera, , The Pixel conversion unit mm/Pixel for x, y axis, , For the offset of the center of the projection screen relative to the optical axis, R is a rotation matrix, and t is a translation vector; fourth, a camera laser radar rotation translation matrix serving as a joint calibration result of the camera (61) and the laser radar (63) is obtained according to the following formula II-IV ; (Formula II); Wherein, the In the form of coordinates in the world coordinate system of the lidar, Is a rotation matrix for converting a laser radar coordinate system into a camera coordinate system, The translation vector is converted from a laser radar coordinate system to a camera coordinate system; (formula III); =1 (formula four); Wherein, the And The origin of the laser radar coordinate system and the origin space normal of the camera coordinate system are respectively And Is a unit vector of (a); fifthly, obtaining a corrected swing angle at the moment k according to the following formula ; (Formula five); Wherein, the For the real swing angle of the swing arm (35) at the moment k obtained by the angle measuring instrument measurement, i is the moment before the moment k, For the sampling time interval of time k and time t, For the swing angle measured by the IMU at time k, Is a gain factor of a proportion of the gain, Is an integral gain coefficient; representing three-dimensional rotation of IMU using quaternions : + ; Wherein the method comprises the steps of Indicating the rotation of the plane where the X and Y axes intersect about the positive X axis direction toward the positive Y axis direction, Indicating the rotation of the plane where the Z axis and the X axis intersect about the positive Z axis direction toward the positive X axis direction, A rotation of a plane intersecting the Y axis and the Z axis around the Y positive direction towards the Z axis positive direction; To be used for Adding constraints to the quaternion of the IMU: ; three-dimensional rotation amount combined with camera (61) Obtaining quaternion by function maximization solution ; Obtaining the optimal quaternion by pressing the following formula six : (Formula six); Will best quaternion Is converted into a matrix form, and a rotation matrix between the camera (61) and the IMU (62) for obtaining optimal estimation : ; Seven steps are pressed to obtain a translation matrix between the camera (61) and the IMU (62) : (Formula seven); Wherein, the For the camera translation obtained by calibration plate feature point tracking, The inertial translation amount is obtained through IMU pre-integration; then obtaining a camera IMU rotation translation matrix as a joint calibration result of the camera (61) and the IMU (62) ; Sixthly, integrating to obtain a joint calibration result of the camera (61), the IMU (62) and the laser radar (63), wherein the joint calibration result comprises a camera external parameter matrix Rotation translation matrix of camera laser radar And camera IMU rotation translation matrix 。
10. The method for combined calibration of a lidar, a camera and an IMU according to claim 9, wherein the data acquisition in the second step comprises the following steps: the driving motor (11) drives the driving cam (12) to rotate for half a circle, and simultaneously, the following three movements are driven, and meanwhile, the camera (61), the IMU (62) and the laser radar (63) of the acquisition device (6) continuously acquire data: First movement: The driving cam (12) drives the lifting rod (13) to move up and then down, so that the rear side of the top plate (5) follows the lifting rod to move up and then down; second movement: The driving cam (12) drives the lifting rod (13) to move up and then down firstly, the driving shaft (16) and a driving gear (17) fixed on the driving shaft are driven to rotate by the first rotating groove (14) and the second rotating groove (15), then the inclination angle adjusting gear (21) meshed with the driving gear (17) drives the threaded driven shaft (22) to rotate, the rotation is converted into linear motion through the threaded sleeve (23), and the front side of the top plate (5) moves down and then up along with the threaded sleeve (23); the first movement and the second movement together form the movement of the top plate (5) and the collection device (6) of tilting back before; third movement: The driving cam (12) drives the lifting rod (13) to move up and then down, the driving shaft (16) and a driving gear (17) fixed on the driving shaft are driven to rotate by the first rotating groove (14) and the second rotating groove (15), then the crank (32) is driven to swing by a swing angle adjusting gear (31) meshed with the driving gear (17), so that the sliding sleeve (34) drives the swinging rod (33) to do one-time reciprocating sliding along the horizontal section of the swinging rod (33) and simultaneously drives the swinging rod (33) to do one-time horizontal reciprocating swinging, and further the acquisition device (6) is driven by the swing arm (35) to do one-time horizontal reciprocating swinging; then, the driving motor (11) continuously drives the driving cam (12) to rotate for half a circle, so that all mechanisms of the combined calibration device are reset.

Description

Laser radar, camera and IMU combined calibration device and method Technical Field The invention relates to the technical field of environmental awareness, in particular to a joint calibration device and method for calibrating a joint assembly of a sensor. Background With the rapid development of the technology level, the environment sensing technology is increasingly applied to various fields including intelligent automobiles, and meanwhile, the fields also put higher demands on the accuracy of the sensed content of the environment sensing technology. The perceived content accuracy of the context awareness technique is directly affected by the awareness scheme settings. The multi-sensor combined sensing is a sensing scheme which is widely applied at present and relatively mature in technology, and the sensing of target content is realized by means of the multi-sensor combined assembly formed by the plurality of sensors in a group arrangement mode. Through prior art search, for calibration of the sensor, there are the following known technical solutions: Prior art 1: The hand-eye calibration method proposed by partial scholars can be separated from the limitation of the calibration plate and adopts self-calibration. However, from the aspect of experimental results, the calibration method inevitably loses part of calibration accuracy, and the application field of multi-sensor combined sensing has higher requirements on the calibration accuracy. Thus, the calibration method is obviously not applicable to the calibration of the multi-sensor combined assembly. Prior art 2: the prior art provides a joint calibration method, a joint calibration device and electronic equipment thereof, wherein the application number is 2018110369542, the application date is 2018.09.06, and the publication date is 2020.03.13. The combined calibration method is used for calibrating a visual inertia combined device, wherein the visual inertia combined device comprises an image acquisition unit and an inertia measurement unit. The joint calibration method comprises the steps of obtaining a visual pose sequence of an image acquisition unit by processing an obtained image data sequence, obtaining a visual computing data sequence corresponding to an inertial time stamp of an inertial measurement unit by fitting the visual pose sequence into a continuous spline curve, obtaining a time delay initial value between the image acquisition unit and the inertial measurement unit by performing cross-correlation degree analysis on the obtained inertial data sequence and the visual computing data sequence, and obtaining a pose initial value between the image acquisition unit and the inertial measurement unit by analyzing a minimized error between the visual computing data and the inertial data sequence. The implementation of the calibration method in the prior art depends on a large amount of image data acquired by the image acquisition unit from different visual angles, but a method and a device for supporting automatic acquisition of the image data are not correspondingly disclosed, and the image acquisition in a large amount and at multiple angles is performed manually, so that the problems of long acquisition time, low efficiency and great acquisition workload inevitably exist. Meanwhile, in order to eliminate the distortion characteristic of the acquired picture existing in the image acquisition unit, on the basis of meeting the calibration requirement, the possibility of continuously increasing the acquisition quantity of the image data to meet the orthodontic requirement of the acquired picture exists, and the problems of long time consumption and extremely high low-level acquisition workload of the calibration method are certainly further aggravated. In addition, the above-mentioned prior art adopts IMU and camera to carry out the calibration of sensor joint subassembly, and though can satisfy the needs of environment perception in the ideal operating mode basically, but can't adapt to like the obstacle shelter from more, image acquisition environment is bad and inferior grade not ideal operating mode inadequately, easily produces the problem that information acquisition is incomplete under the non-ideal operating mode, and this problem is extremely easy to lead to serious problems such as driving safety in some application occasions such as intelligent automobile field. The above prior art also has the problem of not taking into account the accumulated rounding errors of the IMU calculation process, which can lead to an increasing IMU measurement error over time. Through the above search, it was found that the above technical solutions do not affect the novelty of the present invention, and that the above prior art combinations with each other do not destroy the inventive aspects of the present invention. Disclosure of Invention The invention provides a laser radar, camera and IMU combined calibration device and method for avoiding the def