CN-115909238-B - Image processing method and device, electronic equipment and storage medium

CN115909238BCN 115909238 BCN115909238 BCN 115909238BCN-115909238-B

Abstract

The application discloses an image processing method, an image processing device, electronic equipment and a storage medium, wherein the image processing method can be applied to the field of intelligent driving and comprises the steps of obtaining a ground mosaic around a target vehicle; the method comprises the steps of carrying out semantic segmentation on a ground mosaic to obtain at least one piece of ground semantic information, inputting the ground mosaic and the at least one piece of ground semantic information into a pre-trained generator of a generating type countermeasure network to generate a noiseless inverse perspective transformation graph corresponding to the ground mosaic, and carrying out semantic segmentation on the noiseless inverse perspective transformation graph to obtain at least one item of target semantic information. The method reduces the influence of ground reflection noise in the picture on the subsequent segmentation model, improves the accuracy of the segmentation model for predicting the pixel class in the picture, and improves the accuracy of an automatic driving perception system in interpretation of the surrounding environment.

Inventors

LI CHENXI
YUAN JINWEI
ZHANG ZHENLIN

Assignees

中汽创智科技有限公司

Dates

Publication Date: 20260512
Application Date: 20221027

Claims (8)

1. An image processing method, the method comprising: Acquiring a ground splice map around a target vehicle; carrying out semantic segmentation on the ground mosaic to obtain at least one piece of ground semantic information; Inputting the ground mosaic and the at least one piece of ground semantic information to a pre-trained generator of a generated countermeasure network, and generating a noiseless inverse perspective transformation diagram corresponding to the ground mosaic; carrying out semantic segmentation on the noiseless inverse perspective transformation diagram to obtain at least one item of label semantic information; The method comprises the steps of inputting the ground mosaic and at least one piece of ground semantic information into a pre-trained generator of a generated countermeasure network, generating a noiseless inverse perspective transformation graph corresponding to the ground mosaic, distributing weights to the at least one piece of ground semantic information to obtain weights corresponding to each piece of ground semantic information, obtaining weighted ground semantic information corresponding to each piece of ground semantic information based on the at least one piece of ground semantic information and the weights corresponding to each piece of ground semantic information, inputting the ground mosaic and the weighted ground semantic information into the pre-trained generator of the generated countermeasure network, and generating the noiseless inverse perspective transformation graph; The training process of the generated type countermeasure network comprises the steps of obtaining a sample ground mosaic and a noiseless ground mosaic corresponding to the sample ground mosaic, conducting semantic segmentation on the sample ground mosaic to obtain at least one piece of sample ground semantic information, inputting the sample ground mosaic and the at least one piece of sample ground semantic information into a generator of an initial generated type countermeasure network to generate a predicted noiseless graph, inputting the predicted noiseless graph and the noiseless ground mosaic into a discriminator of the initial generated type countermeasure network, determining model loss based on a discrimination result of the discriminator, indicating a gap between the predicted noiseless graph and the noiseless ground mosaic, and adjusting network parameters of the initial generated type countermeasure network until a preset training end condition is met based on the model loss to obtain the pre-trained generated type countermeasure network.
2. The image processing method according to claim 1, wherein the inputting the sample ground mosaic and the at least one piece of sample ground semantic information to the generator of the initially generated countermeasure network generates a predicted noiseless map, comprising: Distributing weights to the at least one piece of sample ground semantic information to obtain weights corresponding to each piece of sample ground semantic information; obtaining weighted sample ground semantic information corresponding to each piece of sample ground semantic information based on the at least one piece of sample ground semantic information and the weight corresponding to each piece of sample ground semantic information; And inputting the sample ground mosaic and the weighted sample ground semantic information to a generator of the initial generation type countermeasure network to generate the prediction noiseless graph.
3. The image processing method according to claim 1, wherein the performing semantic segmentation on the sample ground mosaic to obtain at least one piece of sample ground semantic information includes: Inputting the sample ground mosaic to a pre-trained inverse perspective transformation image segmentation model for semantic segmentation processing to obtain at least one piece of sample ground semantic information.
4. The image processing method according to claim 1, wherein the performing semantic segmentation on the ground mosaic to obtain at least one piece of ground semantic information includes: Inputting the ground mosaic to the pre-trained inverse perspective transformation image segmentation model to generate the at least one piece of ground semantic information; The semantic segmentation is carried out on the noiseless inverse perspective transformation diagram to obtain at least one item of slogan semantic information, which comprises the following steps: inputting the noiseless inverse perspective transformation map to the pre-trained inverse perspective transformation image segmentation model to generate the at least one item of taggant information.
5. The image processing method according to any one of claims 1 to 4, characterized in that before acquiring the ground map around the target vehicle, the method further comprises: acquiring a plurality of fisheye pictures shot by a plurality of fisheye cameras of the whole body of the target vehicle; determining an internal parameter corresponding to each fisheye camera and an external parameter corresponding to each fisheye camera; Based on the internal parameter corresponding to each fish-eye camera and the external parameter corresponding to each fish-eye camera, converting the fish-eye picture corresponding to each fish-eye camera into a ground coordinate system to obtain a plurality of ground subgraphs; And splicing the plurality of ground subgraphs to obtain the ground splice graph.
6. An image processing apparatus, characterized in that the apparatus comprises: the first acquisition module is used for acquiring a ground splice map around the target vehicle; the first semantic segmentation module is used for carrying out semantic segmentation on the ground mosaic to obtain at least one piece of ground semantic information; The image generation module is used for inputting the ground mosaic and the at least one piece of ground semantic information into a pre-trained generator of a generated type countermeasure network to generate a noiseless inverse perspective transformation diagram corresponding to the ground mosaic; The second semantic segmentation module is used for carrying out semantic segmentation on the noiseless inverse perspective transformation diagram to obtain at least one item of label semantic information; The image generation module comprises a first weight distribution module, a first information synthesis module, a noise-free image generation module, a pre-training generation type antagonism network generator, a noise-free inverse perspective transformation module and a pre-training generation type antagonism network generator, wherein the first weight distribution module is used for distributing weights to the at least one piece of ground semantic information to obtain weights corresponding to each piece of ground semantic information; The device further comprises a model training module used for training the generated type countermeasure network, the model training module comprises a second acquisition module used for acquiring a sample ground splice graph and a noise-free ground splice graph corresponding to the sample ground splice graph, a sample segmentation module used for conducting semantic segmentation on the sample ground splice graph to obtain at least one piece of sample ground semantic information, a prediction module used for inputting the sample ground splice graph and the at least one piece of sample ground semantic information to a generator of the initial generated type countermeasure network to generate a predicted noise-free graph, a loss determination module used for inputting the predicted noise-free graph and the noise-free ground splice graph to a discriminator of the initial generated type countermeasure network and determining model loss based on a discrimination result of the discriminator, the discrimination result indicates a difference between the predicted noise-free graph and the noise-free ground splice graph, and a parameter adjustment module used for adjusting network parameters of the initial generated type countermeasure network until a preset training end condition is met based on the model loss to obtain the pre-trained generated type countermeasure network.
7. An electronic device, comprising a processor and a memory, wherein at least one instruction or at least one program is stored in the memory, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the image processing method according to any one of claims 1 to 5.
8. A computer-readable storage medium, wherein at least one instruction or at least one program is stored in the computer-readable storage medium, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the image processing method according to any one of claims 1 to 5.

Description

Image processing method and device, electronic equipment and storage medium Technical Field The present application relates to the field of autopilot, and in particular, to an image processing method, an image processing device, an electronic apparatus, and a storage medium. Background At present, an automatic driving sensing system needs to acquire scene information around an automobile through hardware such as a laser radar and a camera carried by the automobile, the scene information generally needs to be further processed through a preset model, interpretation of the sensing system to the environment around the automobile is completed, the driving system is assisted to react, and safe driving of the automobile is guaranteed. In the underground garage, as shown in fig. 1, reflection effects caused by the problems of light irradiation and ground materials often occur, and in pictures shot in the camera moment, as shown in fig. 2, the ground reflection has the same pixel value with traffic sign lines on the ground due to exposure, so that trouble is caused to a model, the model regards the ground reflection as the traffic landmark on the ground, an incorrect perception result is output, the planning and decision of a subsequent vehicle are influenced, and the safe running of the automobile is influenced. In the prior art, the reflective area is treated as a background, or the part with higher brightness is eliminated based on the color distribution of the picture, however, if the picture has a brighter color, the reflective area is also considered to be removed, so that semantic information originally contained in the picture is affected, and the definition of the picture is disturbed. Disclosure of Invention In order to solve the problems in the prior art, the embodiment of the application provides an image processing method, an image processing device, electronic equipment and a storage medium. The technical scheme is as follows: in one aspect, there is provided an image processing method, the method including: Acquiring a ground splice map around a target vehicle; carrying out semantic segmentation on the ground mosaic to obtain at least one piece of ground semantic information; Inputting the ground mosaic and the at least one piece of ground semantic information to a pre-trained generator of a generated countermeasure network, and generating a noiseless inverse perspective transformation diagram corresponding to the ground mosaic; And carrying out semantic segmentation on the noiseless inverse perspective transformation diagram to obtain at least one item of label semantic information. In another aspect, there is provided an image processing apparatus including: the first acquisition module is used for acquiring a ground splice map around the target vehicle; the first semantic segmentation module is used for carrying out semantic segmentation on the ground mosaic to obtain at least one piece of ground semantic information; The image generation module is used for inputting the ground mosaic and the at least one piece of ground semantic information into a pre-trained generator of a generated type countermeasure network to generate a noiseless inverse perspective transformation diagram corresponding to the ground mosaic; and the second semantic segmentation module is used for carrying out semantic segmentation on the noiseless inverse perspective transformation diagram to obtain at least one item of label semantic information. In an exemplary embodiment, the image generation module includes: The first weight distribution module is used for distributing weights to the at least one piece of ground semantic information to obtain weights corresponding to each piece of ground semantic information; The first information synthesis module is used for obtaining weighted ground semantic information corresponding to each piece of ground semantic information based on the at least one piece of ground semantic information and the weight corresponding to each piece of ground semantic information; and the noiseless image generating module is used for inputting the ground mosaic and the weighted ground semantic information to a generator of the pre-trained generating type countermeasure network to generate the noiseless inverse perspective transformation map. In an exemplary embodiment, the apparatus further comprises a model training module for training the generative antagonism network, the model training module comprising: the second acquisition module is used for acquiring a sample ground splice graph and a noiseless ground splice graph corresponding to the sample ground splice graph; The sample segmentation module is used for carrying out semantic segmentation on the sample ground mosaic to obtain at least one piece of sample ground semantic information; The prediction module is used for inputting the sample ground splice graph and the at least one piece of sample ground semantic information to a generator of an init