CN-116798034-B - Image data processing method and device and electronic equipment

CN116798034BCN 116798034 BCN116798034 BCN 116798034BCN-116798034-B

Abstract

The embodiment of the invention discloses an image processing method, an image processing device and electronic equipment, which are used for determining a first image labeling result by labeling a first sensor image, determining corresponding two-dimensional labeling coordinates according to the first image labeling result, preprocessing a second sensor image according to the two-dimensional labeling coordinates to obtain a preprocessed image, labeling the preprocessed image and determining a second image labeling result of the second sensor image. Therefore, in the embodiment, the first sensor image labeling result is used as prior information, and the second sensor image is automatically labeled, so that the same target object in different sensor images is automatically paired, and the image labeling efficiency and the accuracy of the target detection result are improved.

Inventors

XIAO NING
JIANG BO
NIU JINGBO
CAI YEHE
TONG MUCHENXUAN

Assignees

北京航迹科技有限公司

Dates

Publication Date: 20260512
Application Date: 20220314

Claims (12)

1. An image processing method, the method comprising: Acquiring a first image labeling result of a first sensor image; Determining corresponding two-dimensional annotation coordinates according to the first image annotation result; preprocessing the second sensor image according to each two-dimensional annotation coordinate to obtain a corresponding preprocessed image; Labeling each preprocessed image to determine a second image labeling result of the second sensor image; The method for acquiring the first image annotation result of the first sensor image comprises the following steps of: Labeling the first sensor image, wherein the labeling information of the first sensor image comprises the vertex coordinates of a three-dimensional labeling frame of each target object in the image, and the labeling is performed through a pre-trained first image labeling model; Mapping the marked first sensor image into a corresponding two-dimensional image according to the parameter information of the first sensor and the second sensor, wherein the marking information of the two-dimensional image comprises the vertex coordinates of a two-dimensional marking frame corresponding to each target object in the image, each vertex coordinate of the two-dimensional marking frame is determined according to each vertex coordinate of the corresponding three-dimensional marking frame, and the parameter information of the first sensor and the second sensor comprises the relative position and azimuth relation of the first sensor and the second sensor and the internal and external parameters of the first sensor and the second sensor; Determining the first image annotation result according to the annotation information of the two-dimensional image; the determining the corresponding two-dimensional annotation coordinates according to the first image annotation result comprises: The maximum value of each coordinate value is respectively increased by a first preset value, the minimum value of each coordinate value is respectively reduced by a second preset value, and the coordinates determined by the maximum value of each coordinate value after being increased and the minimum value of each coordinate value after being reduced are determined as the two-dimensional labeling coordinates; preprocessing the second sensor image according to each two-dimensional annotation coordinate, wherein the obtaining of the corresponding preprocessed image comprises the following steps: shearing the second sensor image according to the two-dimensional labeling coordinates to obtain an initial image; the initial image is placed on a predetermined background image to obtain the pre-processed image.
2. The method of claim 1, wherein cropping the second sensor image according to the two-dimensional annotation coordinates to obtain an initial image comprises: Shearing the second sensor image according to the two-dimensional labeling coordinates to obtain a sheared image; and amplifying the sheared image to acquire the initial image.
3. The method of claim 2, wherein the enlarging the cropped image to obtain the initial image comprises: Determining an amplification factor according to the size parameter of the second sensor image and the size parameter of the sheared image; and amplifying the sheared image according to the amplification factor to acquire the initial image.
4. A method according to claim 3, wherein determining a magnification factor from the size parameter of the second sensor image and the size parameter of the cropped image comprises: Determining a length ratio of each side of the second sensor image to the cropped image; And determining the amplification factor according to each length ratio.
5. The method of any of claims 2-4, wherein a magnification factor of the cropped image is no greater than a predetermined value.
6. The method of claim 1, wherein labeling the first sensor image comprises: And inputting the first sensor image into a first image annotation model for processing so as to annotate the first sensor image.
7. The method of claim 1, wherein labeling each of the preprocessed images to determine a second image labeling result of the second sensor image comprises: And inputting each preprocessed image into a pre-trained second image annotation model to be annotated so as to determine the second image annotation result.
8. The method of claim 1, wherein the first sensor image is a radar sensor image and the second sensor image is a camera sensor image.
9. An image processing apparatus, characterized in that the apparatus comprises: The first labeling unit is configured to acquire a first image labeling result of the first sensor image; the coordinate determining unit is configured to determine corresponding two-dimensional annotation coordinates according to the first image annotation result; the preprocessing unit is configured to preprocess the second sensor image according to each two-dimensional annotation coordinate, and obtain a corresponding preprocessed image; a second labeling unit configured to label each of the preprocessed images to determine a second image labeling result of the second sensor image; The first sensor image is a three-dimensional image, the first labeling unit is further configured to label the first sensor image, wherein the labeling information of the first sensor image comprises the vertex coordinates of a three-dimensional labeling frame of each target object in the image, the labeling is performed through a pre-trained first image labeling model, the labeled first sensor image is mapped into a corresponding two-dimensional image according to the parameter information of the first sensor and the second sensor, the labeling information of the two-dimensional image comprises the vertex coordinates of a two-dimensional labeling frame corresponding to each target object in the image, the vertex coordinates of the two-dimensional labeling frame are determined according to the vertex coordinates of the corresponding three-dimensional labeling frame, and the parameter information of the first sensor and the second sensor comprises the relative position and azimuth relation of the first sensor and the second sensor and the internal and external parameters of the first sensor; The coordinate determining unit is further configured to determine a maximum value and a minimum value of each coordinate value corresponding to each vertex coordinate of the two-dimensional labeling frame, respectively increase the maximum value of each coordinate value by a first preset value, respectively decrease the minimum value of each coordinate value by a second preset value, so as to determine the coordinates determined by the maximum value of each coordinate value after the increase and the minimum value of each coordinate value after the decrease as the two-dimensional labeling coordinates; The preprocessing unit is further configured to cut the second sensor image according to the two-dimensional labeling coordinates to obtain an initial image, and place the initial image on a preset background image to obtain the preprocessed image.
10. An electronic device comprising a memory and a processor, wherein the memory is configured to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method of any of claims 1-8.
11. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a computer program which, when executed by a processor, implements the method according to any of claims 1-8.
12. A computer program product, characterized in that the computer program product, when run on a computer, causes the computer to perform the method according to any of claims 1-8.

Description

Image data processing method and device and electronic equipment Technical Field The present invention relates to the field of computer technologies, and in particular, to an image processing method, an image processing device, and an electronic device. Background The more input data is during the training of the deep learning model, the more accurate the output result is during the inference, so that the more labeling data is needed in the model training process. In complex and changeable application scenes (such as an automatic driving running process, etc.), the data volume obtained daily is extremely large, the data can be marked only by a manual mode at present, the time and the effort are consumed, the progress is slow, and the requirement of model updating iteration is difficult to meet. Disclosure of Invention In view of the above, the embodiments of the present invention provide an image processing method, an image processing device, and an electronic device, which implement automatic labeling of a second sensor image by using a labeling result of a first sensor image as prior information, thereby implementing automatic pairing of the same target object in different sensor images, and improving efficiency of image labeling and accuracy of a target detection result. In a first aspect, an embodiment of the present invention provides an image processing method, including: Acquiring a first image labeling result of a first sensor image; Determining corresponding two-dimensional annotation coordinates according to the first image annotation result; preprocessing the second sensor image according to each two-dimensional annotation coordinate to obtain a corresponding preprocessed image; labeling each preprocessed image to determine a second image labeling result of the second sensor image. Optionally, the first sensor image is a three-dimensional image; The obtaining a first image annotation result of the first sensor image comprises: labeling the first sensor image, wherein the labeling information of the first sensor image comprises the coordinates of each vertex of a three-dimensional labeling frame of each target object in the image; mapping the marked first sensor image into a corresponding two-dimensional image according to the parameter information of the first sensor and the second sensor, wherein the marking information of the two-dimensional image comprises the vertex coordinates of a two-dimensional marking frame corresponding to each target in the image, and the vertex coordinates of the two-dimensional marking frame are determined according to the vertex coordinates of the corresponding three-dimensional marking frame; And determining the first image annotation result according to the annotation information of the two-dimensional image. Optionally, determining the corresponding two-dimensional labeling coordinate according to the second image labeling result includes: and determining the coordinates determined by the maximum value of each coordinate axis and the coordinates determined by the minimum value of each coordinate axis corresponding to each vertex coordinate of the two-dimensional annotation frame as the two-dimensional annotation coordinates. Optionally, determining the corresponding two-dimensional labeling coordinate according to the second image labeling result includes: Determining the maximum value and the minimum value of corresponding coordinate axes in the vertex coordinates of the two-dimensional annotation frame; And respectively increasing the maximum value of each coordinate axis by a first preset value, and respectively shrinking the minimum value of each coordinate axis by a second preset value so as to determine the coordinates determined by the maximum value of each coordinate axis after the increase and the minimum value of each coordinate axis after the shrink as the two-dimensional labeling coordinates. Optionally, preprocessing the second sensor image according to each two-dimensional labeling coordinate, and obtaining a corresponding preprocessed image includes: shearing the second sensor image according to the two-dimensional labeling coordinates to obtain an initial image; the initial image is placed on a predetermined background image to obtain the pre-processed image. Optionally, clipping the second sensor image according to the two-dimensional labeling coordinates to obtain an initial image includes: Shearing the second sensor image according to the two-dimensional labeling coordinates to obtain a sheared image; and amplifying the sheared image to acquire the initial image. Optionally, amplifying the clipped image to obtain the initial image includes: Determining an amplification factor according to the size parameter of the second sensor image and the size parameter of the sheared image; and amplifying the sheared image according to the amplification factor to acquire the initial image. Optionally, determining the magnification factor according to the si