CN-116128744-B - Method for eliminating image distortion, electronic device, storage medium and vehicle

CN116128744BCN 116128744 BCN116128744 BCN 116128744BCN-116128744-B

Abstract

The application provides a method for eliminating image distortion, electronic equipment, storage media and a vehicle, which comprises the steps of identifying a plurality of preset points in a constructed calibration coordinate system, determining a first pose relation between a preset real camera coordinate system and the calibration coordinate system by using pixel coordinates of each of the plurality of preset points in a preset real camera image, determining a shooting height of the real camera image according to the first pose relation, identifying the plurality of preset points in a preset virtual camera coordinate system, determining a second pose relation between the real camera coordinate system and the virtual camera coordinate system by using pixel coordinates of each of the plurality of preset points, and constructing a projection transformation relation between the virtual camera and the real camera under the shooting height according to the second pose relation, and mapping the real camera image into the virtual camera coordinate system and displaying the real camera image by using the projection transformation relation.

Inventors

TAN JINGYANG
WANG GUANGFU
Ye Nianjin
CHEN XIANLING
ZHAO LONG
Jia Lanpeng

Assignees

长城汽车股份有限公司

Dates

Publication Date: 20260505
Application Date: 20221123

Claims (9)

1. A method of eliminating image distortion, comprising: A plurality of preset points are marked in the constructed calibration coordinate system, and a first pose relation between a preset real camera coordinate system and the calibration coordinate system is determined by utilizing pixel coordinates of each preset point in a preset real camera image; determining the shooting height of the real camera image according to the first pose relation; marking a plurality of preset points in a preset virtual camera coordinate system, and determining a second pose relationship between the real camera coordinate system and the virtual camera coordinate system by utilizing the pixel coordinates of each of the preset points; According to the second pose relation, a projection transformation relation between the virtual camera and the real camera at the shooting height is constructed, and the real camera image is mapped into the virtual camera coordinate system by utilizing the projection transformation relation and is presented; said constructing a projective transformation relationship between said virtual camera and said real camera at said photographing height, comprising: Constructing a second perspective projection relationship between the real camera coordinate system and the virtual camera coordinate system by using preset camera internal parameters; According to the second perspective projection relation, a plurality of pixel coordinates and a plurality of virtual camera coordinates are utilized to establish a plurality of second linear equations related to a second camera external parameter, a second rotation matrix and a second translation matrix are determined by utilizing the plurality of second linear equations, and the second rotation matrix and the second translation matrix are used as the second camera external parameter; determining a vertical vector between an origin of the real camera coordinate system and a preset plane; constructing a projection transformation matrix by using the vertical vector, the camera internal parameter, the shooting height and the second camera external parameter; the formula of the projective transformation matrix H is as follows: where H represents the projective transformation matrix, K represents the camera internal parameters, Representing a second rotation matrix, T 2 representing a second translation matrix, T representing a transpose of the matrix, N representing a vertical vector, d representing a shooting height; And determining the projective transformation relation according to the projective transformation matrix.
2. The method of claim 1, wherein prior to identifying the plurality of preset points in the constructed calibration coordinate system, comprising: Setting the plurality of preset points; constructing the calibration coordinate system by using any preset point as an origin; and determining the calibration coordinates of each of the plurality of preset points in the calibration coordinate system.
3. The method of claim 1, wherein determining a first pose relationship between a preset real camera coordinate system and the calibration coordinate system comprises: Constructing a first perspective projection relation between the real camera coordinate system and the calibration coordinate system by using preset camera internal parameters; According to the first perspective projection relation, a plurality of first linear equations about a first camera external parameter are established by utilizing a plurality of pixel coordinates and a plurality of calibration coordinates, and the first camera external parameter is determined by utilizing the plurality of first linear equations; and determining the first pose relation according to the first camera external parameters.
4. A method according to claim 3, wherein said determining a second pose relationship between said real camera coordinate system and said virtual camera coordinate system comprises: Constructing a second perspective projection relationship between the real camera coordinate system and the virtual camera coordinate system using the camera internal parameters; establishing a plurality of second linear equations about a second camera external parameter by using a plurality of pixel coordinates and a plurality of virtual camera coordinates according to the second perspective projection relation, and determining the second camera external parameter by using the plurality of second linear equations; determining the second pose relationship according to the second camera external parameters; The plurality of virtual camera coordinates are obtained by marking the plurality of preset points in the virtual camera coordinate system.
5. The method of claim 1, wherein the mapping and rendering the real camera image into the virtual camera coordinate system using the projective transformation relationship comprises: determining pixel coordinates of all pixels in the real camera image; The pixel coordinates of all pixels in the real camera coordinate system are converted into the virtual camera coordinate system according to the proportion of the projection transformation relation; and presents the image in accordance with the converted pixel coordinates.
6. The device for eliminating the image distortion is characterized by comprising a first pose relation determining module, a shooting height determining module, a second pose relation determining module and a projection transformation module; The first pose relation determining module is configured to determine a plurality of preset points in a constructed calibration coordinate system, and determine a first pose relation between a preset real camera coordinate system and the calibration coordinate system by using pixel coordinates of the preset points in a preset real camera image; the shooting height determining module is configured to determine the shooting height of the real camera image according to the first pose relation; the second pose relation determining module is configured to mark the preset points in a preset virtual camera coordinate system, and determine a second pose relation between the real camera coordinate system and the virtual camera coordinate system by using the pixel coordinates of each preset point; The projection transformation module is configured to construct a projection transformation relation between the virtual camera and the real camera at the shooting height according to the second pose relation, and map the real camera image into the virtual camera coordinate system by using the projection transformation relation and present the real camera image; the projective transformation module is further configured to: Constructing a second perspective projection relationship between the real camera coordinate system and the virtual camera coordinate system by using preset camera internal parameters; According to the second perspective projection relation, a plurality of pixel coordinates and a plurality of virtual camera coordinates are utilized to establish a plurality of second linear equations related to a second camera external parameter, a second rotation matrix and a second translation matrix are determined by utilizing the plurality of second linear equations, and the second rotation matrix and the second translation matrix are used as the second camera external parameter; determining a vertical vector between an origin of the real camera coordinate system and a preset plane; constructing a projection transformation matrix by using the vertical vector, the camera internal parameter, the shooting height and the second camera external parameter; the formula of the projective transformation matrix H is as follows: where H represents the projective transformation matrix, K represents the camera internal parameters, Representing a second rotation matrix, T 2 representing a second translation matrix, T representing a transpose of the matrix, N representing a vertical vector, d representing a shooting height; And determining the projective transformation relation according to the projective transformation matrix.
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable by the processor, wherein the processor implements the method of any one of claims 1 to 5 when executing the computer program.
8. A computer readable storage medium storing computer instructions for causing a computer to perform the method according to any one of claims 1 to 5.
9. A vehicle comprising the apparatus for eliminating image distortion according to claim 6 or the electronic device according to claim 7.

Description

Method for eliminating image distortion, electronic device, storage medium and vehicle Technical Field Embodiments of the present application relate to the field of image processing, and in particular, to a method for eliminating image distortion, an electronic device, a storage medium, and a vehicle. Background For cameras with wide angle functions, such as fish-eye cameras for vehicles, the captured images tend to have significant distortion, and on the other hand, the fish-eye camera may have a downward tilt angle in order to capture more information about the vehicle body. Based on the above, the resulting image will have significant perspective distortion after the distortion of the wide-angle camera itself is removed. Based on this, a solution capable of removing distortion of the wide-angle camera itself and perspective distortion is required. Disclosure of Invention In view of the above, the present application is directed to a method, an electronic device, a storage medium and a vehicle for eliminating image distortion, so as to eliminate distortion and perspective distortion generated by a wide-angle camera. Based on the above object, the present application provides a method for eliminating image distortion, comprising: A plurality of preset points are marked in the constructed calibration coordinate system, and a first pose relation between a preset real camera coordinate system and the calibration coordinate system is determined by utilizing pixel coordinates of each preset point in a preset real camera image; determining the shooting height of the real camera image according to the first pose relation; marking a plurality of preset points in a preset virtual camera coordinate system, and determining a second pose relationship between the real camera coordinate system and the virtual camera coordinate system by utilizing the pixel coordinates of each of the preset points; and constructing a projection transformation relation between the virtual camera and the real camera under the shooting height according to the second pose relation, and mapping and presenting the real camera image into the virtual camera coordinate system by utilizing the projection transformation relation. Further, before the calibration coordinate system is constructed to identify the plurality of preset points, the method comprises the following steps: Setting the plurality of preset points; constructing the calibration coordinate system by using any preset point as an origin; and determining the calibration coordinates of each of the plurality of preset points in the calibration coordinate system. Further, determining a first pose relationship between a preset real camera coordinate system and the calibration coordinate system includes: Constructing a first perspective projection relation between the real camera coordinate system and the calibration coordinate system by using preset camera internal parameters; According to the first perspective projection relation, a plurality of first linear equations about a first camera external parameter are established by utilizing a plurality of pixel coordinates and a plurality of calibration coordinates, and the first camera external parameter is determined by utilizing the plurality of first linear equations; and determining the first pose relation according to the first camera external parameters. Further, determining a second pose relationship between the real camera coordinate system and the virtual camera coordinate system includes: constructing a second perspective projection relationship between the real camera coordinate system and the virtual coordinate system by using the camera internal parameters; establishing a plurality of second linear equations about a second camera external parameter by using a plurality of pixel coordinates and a plurality of virtual camera coordinates according to the second perspective projection relation, and determining the second camera external parameter by using the plurality of second linear equations; determining the second pose relationship according to the second camera external parameters; The plurality of virtual camera coordinates are obtained by marking the plurality of preset points in the virtual camera coordinate system. Further, constructing a projective transformation relationship between the virtual camera and the real camera at the photographing height, including: determining a vertical vector between an origin of the real camera coordinate system and a preset plane; Constructing a projective transformation matrix by using the vertical vector, the camera internal parameter, the shooting height and the second camera external parameter; And determining the projective transformation relation according to the projective transformation matrix. Further, mapping and presenting the real camera image into the virtual camera coordinate system using the projective transformation relationship includes: determining pixel coordinates of a