CN-122020545-A - Data processing method and device and vehicle

CN122020545ACN 122020545 ACN122020545 ACN 122020545ACN-122020545-A

Abstract

The embodiment of the application provides a data processing method, a device and a vehicle, which comprise the steps of acquiring data acquired by a plurality of sensors in the vehicle in a current period to obtain multi-mode data, determining an external parameter residual quantity based on the multi-mode data and a historical external parameter matrix determined in a historical period, converting the historical external parameter matrix by using the external parameter residual quantity to obtain a current external parameter matrix, wherein the current external parameter matrix is used for representing the relative pose of the plurality of sensors when the plurality of sensors work in the current period, and fusing the plurality of multi-mode data by using the current external parameter matrix to obtain fused data, wherein the fused data is used for representing the position information of at least one obstacle in an area where the vehicle is located. The application solves the technical problem of low accuracy of calibrating the external parameters of the multiple sensors.

Inventors

WANG ZIJIA
Huo Hongming
LI PENGLONG

Assignees

奇瑞汽车股份有限公司

Dates

Publication Date: 20260512
Application Date: 20260130

Claims (10)

1. A method of processing data, comprising: acquiring data acquired by a plurality of sensors in a vehicle in a current period to obtain multi-mode data; determining an extrinsic residual quantity based on the multi-modal data and a historical extrinsic parameter matrix determined in a historical period, wherein the historical extrinsic parameter matrix is used for representing the relative pose of a plurality of sensors when working in the historical period, and the extrinsic residual quantity is used for representing the deviation degree between the historical extrinsic parameter matrix and a current extrinsic parameter matrix to be determined; Converting the historical external parameter matrix by using the external parameter residual quantity to obtain the current external parameter matrix, wherein the current external parameter matrix is used for representing the relative pose of a plurality of sensors when working in the current period; and fusing the multiple multi-mode data by using the current external parameter matrix to obtain fused data, wherein the fused data is used for representing the position information of at least one obstacle in the area where the vehicle is located.
2. The method of claim 1, wherein the multi-modal data includes point cloud data acquired by a radar sensor and image data acquired by an image sensor, wherein the determining the residual amount of the extrinsic parameters based on the multi-modal data and a historical extrinsic parameter matrix determined during a historical period comprises: Projecting the point cloud data into planar image data by using the historical external parameter matrix; the amount of outlier residuals is determined based on the planar image data, the image data, and the historical extrinsic parameter matrix.
3. The method of claim 2, wherein the determining the extrinsic residual amount based on the planar image data, the image data, and the historical extrinsic parameter matrix comprises: Performing convolution processing on the plane image data to obtain a key vector and a value vector, and performing convolution processing on the image data to obtain a query vector, wherein the key vector is used for representing spatial feature distribution of the plane image data, the value vector is used for representing pixel value information of the plane image data, and the query vector is used for representing context correlation of the image data; converting the query vector by using the key vector to obtain an attention association matrix, wherein the attention association matrix is used for representing the association strength between the spatial position points in the image data and the spatial position points in the plane image data; performing fusion processing on the attention association matrix and the value vector to obtain fusion characteristics; and determining the external parameter residual quantity based on the fusion characteristic and the historical external parameter matrix.
4. A method according to claim 3, wherein said determining said extrinsic residual amounts based on said fusion features and said historical extrinsic parameter matrix comprises: converting the historical external parameter matrix to obtain a lie algebra coordinate, wherein the lie algebra coordinate is used for representing the rigid body transformation state of the historical external parameter matrix; and determining the external parameter residual quantity based on the fusion characteristic and the lie algebraic coordinates.
5. The method of claim 4, wherein said determining said extrinsic residual quantity based on said fusion feature and said lie algebraic coordinates comprises: Coding the lie algebra coordinates to obtain an external parameter embedded vector; Expanding the external parameter embedded vector to obtain prior information characteristics; Splicing the multi-mode fusion feature and the prior information feature to obtain a spliced feature; Decoding the spliced features to obtain feature images; and converting the characteristic image to obtain the external parameter residual quantity.
6. The method according to any one of claims 2 to 5, wherein said converting the historical external parameter matrix using the residual amounts of external parameters to obtain the current external parameter matrix comprises: and carrying out summation processing on the external parameter residual quantity and the historical external parameter matrix to obtain the current external parameter matrix.
7. The method according to any one of claims 2 to 5, wherein the fusing the plurality of multi-modal data using the current external parameter matrix to obtain fused data includes: and projecting the point cloud data into the image data by using the current external parameter matrix to obtain the fusion data.
8. The method according to any one of claims 1 to 5, further comprising: Identifying the fusion data to obtain the position information of the obstacle; and determining a moving track of the vehicle in a future period based on the position information, wherein the future period is later than the current period.
9. An information processing apparatus, comprising: The acquisition unit is used for acquiring data acquired by a plurality of sensors in the vehicle in the current period to obtain multi-mode data; A determining unit configured to determine an extrinsic residual amount based on the multimodal data and a historical extrinsic parameter matrix determined in a historical period, wherein the historical extrinsic parameter matrix is used to characterize a relative pose of a plurality of sensors when the sensors operate in the historical period, and the extrinsic residual amount is used to characterize a degree of deviation between the historical extrinsic parameter matrix and a current extrinsic parameter matrix to be determined; The conversion unit is used for converting the historical external parameter matrix by utilizing the external parameter residual quantity to obtain the current external parameter matrix, and the current external parameter matrix is used for representing the relative pose of a plurality of sensors when working in the current period; And the fusion unit is used for fusing the plurality of multi-mode data by utilizing the current external parameter matrix to obtain fusion data, wherein the fusion data is used for representing the position information of at least one obstacle in the area where the vehicle is located.
10. A vehicle, characterized by comprising: A memory storing an executable program; A processor for running the program, wherein the program runs the method of any one of claims 1 to 8.

Description

Data processing method and device and vehicle Technical Field The embodiment of the application relates to the field of vehicles, in particular to a data processing method and device and a vehicle. Background At present, the calibration method for external parameters of the vehicle-mounted multi-sensor mainly comprises an off-line calibration method and an on-line calibration method based on a natural scene. The on-line calibration method based on the natural scene is easy to influence the calibration result in certain environments. Both the above methods have the technical problem of low accuracy of calibrating the external parameters of the multiple sensors. There is currently no good solution to the above problems. Disclosure of Invention The embodiment of the application provides a data processing method, a data processing device and a vehicle, which at least solve the technical problem of low accuracy of multi-sensor external parameter calibration. According to one aspect of the embodiment of the application, a data processing method is provided, and the method can comprise the steps of obtaining data acquired by a plurality of sensors in a vehicle in a current period to obtain multi-mode data, determining an extrinsic residual quantity based on the multi-mode data and a historical extrinsic parameter matrix determined in a historical period, wherein the historical extrinsic parameter matrix is used for representing relative pose of the plurality of sensors when the plurality of sensors work in the historical period, the extrinsic residual quantity is used for representing deviation degree between the historical extrinsic parameter matrix and the current extrinsic parameter matrix to be determined, converting the historical extrinsic parameter matrix by the extrinsic residual quantity to obtain the current extrinsic parameter matrix, wherein the current extrinsic parameter matrix is used for representing relative pose of the plurality of sensors when the plurality of sensors work in the current period, and fusing the plurality of multi-mode data by the current extrinsic parameter matrix to obtain fused data, wherein the fused data is used for representing position information of at least one obstacle in an area where the vehicle is located. Further, the multi-modal data includes point cloud data acquired by the radar sensor and image data acquired by the image sensor, and the determining of the extrinsic residual amount based on the multi-modal data and the historical extrinsic parameter matrix determined during the historical period includes projecting the point cloud data as planar image data using the historical extrinsic parameter matrix, and determining the extrinsic residual amount based on the planar image data, the image data, and the historical extrinsic parameter matrix. Further, based on the planar image data, the image data and the historical external parameter matrix, the external parameter residual quantity is determined, and the method comprises the steps of carrying out convolution processing on the planar image data to obtain key vectors and value vectors, carrying out convolution processing on the image data to obtain query vectors, wherein the key vectors are used for representing spatial feature distribution of the planar image data, the value vectors are used for representing pixel value information of the planar image data, the query vectors are used for representing context correlation of the image data, converting the query vectors by the key vectors to obtain an attention correlation matrix, wherein the attention correlation matrix is used for representing correlation strength between spatial position points in the image data and spatial position points in the planar image data, carrying out fusion processing on the attention correlation matrix and the value vectors to obtain fusion features, and determining the external parameter residual quantity based on the fusion features and the historical external parameter matrix. Further, determining the extrinsic residual quantity based on the fusion feature and the historical extrinsic parameter matrix comprises converting the historical extrinsic parameter matrix to obtain a lie algebraic coordinate, wherein the lie algebraic coordinate is used for representing the rigid body transformation state of the historical extrinsic parameter matrix, and determining the extrinsic residual quantity based on the fusion feature and the lie algebraic coordinate. Further, based on the fusion features and the lie algebra coordinates, determining the residual quantity of the external parameters comprises the steps of encoding the lie algebra coordinates to obtain an external parameter embedded vector, expanding the external parameter embedded vector to obtain priori information features, splicing the multi-mode fusion features and the priori information features to obtain spliced features, decoding the spliced features to obtain feature images, and