Search

CN-121999059-A - RGBD (red, green and blue) radar integrated machine based on separation filter

CN121999059ACN 121999059 ACN121999059 ACN 121999059ACN-121999059-A

Abstract

The invention belongs to the field of RGBD cameras, and particularly relates to an RGBD radar integrated machine based on a separation filter. The infrared imaging device comprises a flash laser radar, an RGB camera, a lens matched with the RGB camera and a detachable infrared filter, wherein the detachable infrared filter is installed separately from the lens, the calibration fusion stage is detached and installed in the normal use stage, the flash laser radar emits infrared light spots and outputs ranging data, the RGB camera shoots the infrared light spots and RGB color images in stages, and the fusion is completed through position correspondence by adopting a direct superposition model without coordinate matrix transformation. The integrated machine simplifies the calibration flow, reduces the resolution dependence, improves the fusion accuracy and solves the problems of complex traditional scheme and poor effect.

Inventors

  • LI ZHIXUAN

Assignees

  • 苏州赛瞳智能科技有限公司

Dates

Publication Date
20260508
Application Date
20260129

Claims (10)

  1. 1. The RGBD (red, green and blue) radar integrated machine based on the separation filter comprises a flash laser radar, an RGB camera and a lens matched with the RGB camera, and is characterized by further comprising a detachable infrared filter, wherein the detachable infrared filter is detached from the lens, the detachable infrared filter is detached in a calibration and fusion stage, the detachable infrared filter is installed in a normal use stage, the flash laser radar is used for emitting infrared light spots and outputting ranging data, the RGB camera shoots the infrared light spots in the calibration and fusion stage to obtain pixel information, an RGB color image is shot in the normal use stage, the RGBD radar integrated machine adopts a direct superposition model, the correlation of point cloud and RGB information is completed through the position correspondence of the pixel information and the ranging data, and the direct superposition model does not need coordinate matrix transformation, and fusion is realized only through image identification and data matching.
  2. 2. The RGBD (red, green and blue) radar integrated machine based on the separation filter according to claim 1 is characterized in that the flash laser radar is of a point-to-point architecture, each infrared light spot corresponds to one detection channel, the irradiation position of the infrared light spot corresponds to the position of point cloud ranging data output by the detection channel one by one, the calibration and fusion stage is carried out, the RGB camera shoots the infrared light spots to obtain pixel information corresponding to each detection channel on the RGB camera, the detachable infrared filter filters infrared light emitted by the flash laser radar in the normal use stage, the RGB camera obtains RGB color images, and the RGBD radar integrated machine endows the RGB information of corresponding pixels in the RGB color images to point clouds corresponding to the detection channels.
  3. 3. The RGBD (red, green and blue) radar integrated machine based on the separation filter according to claim 1 is characterized in that the flash laser radar is of a dodging architecture, a single emitted infrared light spot corresponds to a plurality of ranging channels after being dodged and diffused, the RGB camera is calibrated and fused to obtain an image, the image is subjected to binarization processing to obtain a spot centroid, the flash laser radar sets a gray level image output mode, black and white image data is output in the gray level image output mode, binarization processing is performed on the black and white image data and the centroid is calculated, the two centroids are in one-to-one correspondence to determine a central ranging channel corresponding to each spot centroid, the RGB camera is mounted with the detachable infrared filter to obtain RGB color images in a normal use stage, and the RGBD radar integrated machine is subjected to dodging processing after being correlated with the RGB information according to the centroid correspondence.
  4. 4. The RGBD (red, green and blue) radar integrated machine based on the separation filter according to claim 1, wherein a filtering wave band of the detachable infrared filter is completely consistent with an infrared wave band emitted by the flash laser radar, the lens and the detachable infrared filter adopt a plug-in dismounting structure, the mounting position of the detachable infrared filter is the front end or the rear end of the RGB camera lens, the optical parameters of the lens are not changed after the mounting, the lens and the photosensitive elements of the RGB camera are not damaged in the dismounting process, the detachable infrared filter uses optical glass as a substrate, and an infrared cut-off film layer is plated on the surface of the substrate.
  5. 5. The RGBD (red, green and blue) radar integrated machine based on the separation filter according to claim 1 is characterized in that an environment in which an RGB (red, green and blue) camera shoots infrared spots is a dark environment, the illumination intensity of the environment is not more than 10lux, only infrared light emitted by a flash laser radar is reserved as a light source, the shooting operation steps sequentially comprise removing the detachable infrared filter, adjusting the distance between the RGBD radar integrated machine and a shooting target to be 0.5-5 m, starting the flash laser radar to emit the infrared spots, adjusting exposure parameters of the RGB camera to enable the infrared spots to meet a gray value standard deviation of more than 30 in an image, the exposure parameters comprise exposure time to be 1 ms-10 ms, sensitivity to be ISO 100-ISO 400, shooting 10 images, selecting the image with the largest gray value standard deviation of the light spot area as a calibration image, and storing image data for subsequent pixel extraction or centroid calculation after shooting is completed.
  6. 6. The RGBD (red, green and blue) radar integrated machine based on a separation filter according to claim 2 or 3, wherein the threshold 128 of the binarization process is a demarcation value of black and white pixels, the threshold is determined according to the difference between gray values of an infrared light spot and a background in an image shot by an RGB camera, the threshold 128 is adopted when the difference is greater than 50 gray levels, the range of the threshold is adjusted to 100-150 when the difference is less than or equal to 50 gray levels, the adjustment method is to count the average gray value of a light spot area and the average gray value of a background area in the image, the intermediate value of the average gray value and the average gray value of the background area is calculated as an adjusted threshold, and the adjusted threshold needs to meet that the white pixel of the light spot area accounts for 80-95% and the white pixel of the background area accounts for no more than 5%.
  7. 7. The RGBD (red, green, blue and white) radar integrated machine based on a separation filter according to claim 3, wherein the mapping logic of the light spot centroid and a ranging channel is that centroid coordinates of an RGB camera shooting image and centroid coordinates of a flash laser radar gray scale image are calculated, one-to-one matching is carried out by adopting a Euclidean distance minimum principle, for each centroid coordinate in the RGB camera image, the centroid coordinate with the minimum Euclidean distance is searched in a centroid coordinate set of the laser radar gray scale image and is used as a corresponding coordinate, the matching is judged to be successful when the minimum Euclidean distance is smaller than 5 pixels, the ranging channel corresponding to the centroid coordinate of the laser radar gray scale image after the matching is the ranging channel corresponding to the centroid coordinate of the RGB camera image, and a mapping table of the centroid coordinates and ranging channel numbers is established for subsequent association process call.
  8. 8. The RGBD (red, green, blue, white) radar gray map output mode according to claim 3, wherein the resolution of the flash laser radar gray map output mode is consistent with the resolution of an image shot by an RGB camera, the resolution is 320×240 pixels, 640×480 pixels or 1280×720 pixels, the gray value range of each pixel is 0 to 255 in an 8-bit gray map format, the data storage format is a BMP or PNG format, the gray value of each pixel in the gray map is positively correlated with the laser reflection intensity at a corresponding position, the higher the reflection intensity is, the higher the gray value is, and the frame rate of the gray map is consistent with the shooting frame rate of the RGB camera and is 15fps to 30fps.
  9. 9. The RGBD (red, green, blue and white) integrated machine based on the separation filter according to claim 1, wherein the quantitative evaluation indexes of the fusion effect comprise the coincidence degree and the color accuracy, the coincidence degree is the coincidence proportion of the spatial positions of the point cloud and the RGB image, the calculation method is that the pixel number of the coincidence region is the percentage of the total corresponding pixel number of the point cloud, the coincidence degree value is not lower than 90%, the color accuracy is evaluated by adopting the color difference of CIE1976Lab color space, the color difference value is not more than 5, the test data comprise the coincidence degree and the color accuracy value under different distances and different illumination conditions, the test data are tested for 10 times under each condition, the average value is taken as a final test result, and the test distance covers 0.5 meter, 1 meter, 2 meter, 3 meter and 5 meter, and the test illumination condition covers dark environment, weak environment light and indoor regular light.
  10. 10. The RGBD (red, green and blue) all-in-one machine based on the separation filter according to claim 1 is characterized in that calibration and compensation measures of camera parameters after the detachable infrared filter is detached are that when the detachable infrared filter is installed at the front end of a lens, the camera internal parameters are not changed in the detaching process, additional calibration is not needed, when the detachable infrared filter is installed at the rear end of the lens, color calibration is carried out by shooting a standard color card image after detachment, the calibration step is to shoot a standard 24-color card image, the RGB average value of each color card color block is calculated and compared with the RGB standard value of the standard color card, a color deviation matrix is obtained, deviation correction is carried out on the RGB images shot later through a color correction algorithm, and the color deviation of the camera before and after detachment is not more than 3 RGB channel values.

Description

RGBD (red, green and blue) radar integrated machine based on separation filter Technical Field The invention belongs to the field of RGBD cameras, and particularly relates to an RGBD radar integrated machine based on a separation filter. Background In the field of RGBD (Red, green, blue) radar and camera fusion calibration is a core technical problem. The existing fusion scheme adopts a black-and-white grid plate to perform pose calibration, and a transformation matrix for fusing the black-and-white grid plate and the existing fusion scheme is required to be searched through a complex flow. The scheme has the obvious defects of complicated data acquisition, calculation and output processes and high calibration difficulty, and has extremely high difficulty in identifying a black-white grid junction area for a laser radar with lower resolution, so that the fusion accuracy is poor, and the actual application requirements cannot be met. Based on the above problems, a technical solution for simplifying the calibration process, reducing the resolution dependence and improving the fusion accuracy is needed, and the core problems of complex and low efficiency and poor fusion effect of the traditional solution are solved. Disclosure of Invention The invention aims to solve the defects existing in the prior art and provides an RGBD (red, green and blue) vision all-in-one machine based on a separation filter, which comprises a flash laser radar, an RGB camera, a lens matched with the RGB camera, a detachable infrared filter and a lens, wherein the detachable infrared filter is detached from the lens and installed in a calibration fusion stage, the detachable infrared filter is installed in a normal use stage, the flash laser radar is used for emitting infrared spots and outputting ranging data, the RGB camera shoots the infrared spots in the calibration fusion stage to acquire pixel information and shoots RGB color images in the normal use stage, the RGBD vision all-in-one machine adopts a direct superposition model, the correlation of point cloud and RGB information is completed through the position correspondence of the pixel information and the ranging data, and the direct superposition model does not need coordinate matrix transformation and realizes fusion only through image identification and data matching. The method comprises the steps of obtaining a red, green and blue (RGB) color image by a red, green and blue (RGB) laser radar, obtaining a red, blue and blue (RGB) color image by the RGB laser radar, and enabling the RGB laser radar to obtain the red, green and blue (RGB) color image by the RGB laser radar. The flash laser radar is further preferably of a dodging architecture, a single infrared light spot is emitted and is subjected to dodging diffusion to correspond to a plurality of ranging channels, a calibration fusion stage is performed, an RGB camera shoots the infrared light spot to obtain an image, the image is subjected to binarization processing to calculate a light spot centroid, the flash laser radar sets a gray scale image output mode, black and white image data is output in the gray scale image output mode, binarization processing is performed on the black and white image data, the centroid is calculated, the two centroids are in one-to-one correspondence to determine a central ranging channel corresponding to each light spot centroid, a normal use stage is performed, the RGB camera is mounted to obtain RGB color images, and the RGBD radar integrated machine correlates RGB information with the ranging channels according to a centroid correspondence and then executes dodging processing. Further preferably, the filter wave band of the detachable infrared filter is completely consistent with the infrared wave band emitted by the flash laser radar, the lens and the detachable infrared filter adopt a pluggable disassembly and assembly structure, the installation position of the detachable infrared filter is the front end or the rear end of the RGB camera lens, the optical parameters of the lens are not changed after the detachable infrared filter is installed, the lens and the photosensitive elements of the RGB camera are not damaged in the disassembly process, the detachable infrared filter takes optical glass as a substrate, and an infrared cut-off film layer is plated on the surface of the substrate. Further preferably, the environment of the RGB camera for shooting the infrared light spots is a dark environment, the illumination intensity of the environment is not more than 10lux, only infrared light emitted by the flash laser radar is reserved as a light source, the shooting operation steps are that a detachable infrared filter is removed, the distance between the RGBD laser integrated machine and a shooting target is adjusted to be 0.5-5 meters, the flash laser radar is started to emit the infrared light spots, exposure parameters of the RGB camera are adjusted to enable the infrared light spo