CN-116543367-B - Method, device, equipment and medium for generating parking space information based on fisheye camera

Abstract

The embodiment of the invention discloses a method, a device, equipment and a medium for generating parking space information based on a fisheye camera. The method comprises the steps of obtaining a target parking space fisheye image, inputting the target parking space fisheye image into a backbone network layer included in a pre-trained parking space detection model to obtain a multi-scale feature map set, inputting the multi-scale feature map set into a feature fusion layer to obtain a multi-scale fusion feature map set, inputting the multi-scale fusion feature map set into a detection head layer to obtain a parking space detection information set, generating a parking space topological structure information set according to the parking space detection information set, and generating the parking space information set according to the parking space topological structure information set. According to the embodiment, the integrity of the extracted parking space information is improved, so that the accuracy of the generated parking space information is improved, the number of caused parking accidents is reduced, and the safety of a vehicle in parking is improved.

Inventors

• ZHANG YANG

Assignees

• 禾多科技（北京）有限公司

Dates

Publication Date

20260505

Application Date

20230413

Claims (9)

1. 1. A method for generating parking space information based on a fisheye camera comprises the following steps: acquiring a target parking space fisheye image, wherein the target parking space fisheye image is a parking space image shot by a fisheye camera loaded on a target vehicle; Inputting the target parking space fisheye image into a backbone network layer included in a pre-trained parking space detection model to obtain a multi-scale feature map set, wherein the parking space detection model further comprises a feature fusion layer and a detection head layer; inputting the multi-scale feature map set to the feature fusion layer to obtain a multi-scale fusion feature map set; Inputting the multi-scale fusion feature map set to the detection head layer to obtain a parking space detection information set; generating a parking space topological structure information set according to the parking space detection information set, wherein the generating comprises the following steps: for each parking space detection information in the parking space detection information set, executing the following steps: generating parking space center point coordinate information according to the parking space center grid point coordinate information and the parking space center grid point coordinate offset information included in the parking space detection information; generating a first vehicle position vertex information group according to a vehicle position vertex relative offset information group included in the vehicle position detection information and the vehicle position center point coordinate information; Generating a second vehicle position vertex information group according to the vehicle position vertex grid coordinate information group and the vehicle position vertex grid coordinate offset information group which are included in the vehicle position detection information, wherein second vehicle position vertex information in the second vehicle position vertex information group corresponds to first vehicle position vertex information in the first vehicle position vertex information group; determining the first vehicle position vertex information group as a vehicle position vertex information group in response to determining that at least one vehicle position vertex information in the vehicle position vertex information groups included in the vehicle position detection information is smaller than a preset confidence threshold; In response to determining that each of the parking stall vertex confidence information in the parking stall vertex confidence information set is greater than or equal to the preset confidence threshold, performing the following steps: Generating a circular region set according to the first vehicle position vertex information group, wherein the first vehicle position vertex information in the first vehicle position vertex information group corresponds to a circular region in the circular region set, and the second vehicle position vertex information in the second vehicle position vertex information group corresponds to a circular region in the circular region set; Determining coordinate positions corresponding to respective second-bit vertex information in the second-bit vertex information group as a second-bit vertex coordinate position group; in response to determining that each of the second vehicle-position vertex coordinate positions in the second vehicle-position vertex coordinate position group is within a corresponding circular region, determining the second vehicle-position vertex information group as a vehicle-position vertex information group; In response to determining that at least one of the second vehicle-level vertex coordinate positions in the second vehicle-level vertex coordinate position group is not within the corresponding circular region, determining the first vehicle-level vertex information group as a vehicle-level vertex information group; Determining a topological structure corresponding to each parking space vertex information included in the determined parking space vertex information group as a parking space topological structure; Determining each obtained parking space topological structure as a parking space topological structure information set; and generating a parking space information set according to the parking space topological structure information set.
2. 2. The method of claim 1, wherein the method further comprises: And sending the parking space information set to the target vehicle, so that the target vehicle executes parking operation according to the parking space information set.
3. 3. The method of claim 1, wherein the parking spot detection model is trained by: Acquiring a sample set, wherein a sample in the sample set comprises a sample parking space fisheye image and a sample parking space detection information set corresponding to the sample parking space fisheye image; The following training steps are performed based on the sample set: Respectively inputting sample parking space fisheye images of at least one sample in a sample set into an initial parking space detection model to obtain a parking space detection information set corresponding to each sample in the at least one sample; comparing the parking space detection information set corresponding to each sample in the at least one sample with the corresponding sample parking space detection information set; determining whether the initial parking space detection model reaches a preset optimization target according to the comparison result; and in response to determining that the initial parking space detection model reaches the optimization target, determining the initial parking space detection model as a trained parking space detection model.
4. 4. The method of claim 3, wherein training the parking spot detection model further comprises: And in response to determining that the initial parking space detection model does not reach the optimization target, adjusting network parameters of the initial parking space detection model, using unused samples to form a sample set, using the adjusted initial parking space detection model as the initial parking space detection model, and executing the training step again.
5. 5. The method of claim 1 wherein the stall detection information in the set of stall detection information further comprises stall type information, stall occupancy information, and a set of stall vertex visibility information, the stall vertex visibility information in the set of stall vertex visibility information corresponding to stall vertex grid coordinate information in the set of stall vertex grid coordinate information, and Generating the parking space information set according to the parking space topological structure information set comprises the following steps: determining a parking space topological structure area set according to the parking space topological structure information set; Generating a parking space topological area set according to the parking space type information, the preset parking space area condition and the parking space topological structure area set; Carrying out distortion region masking processing on each parking space topological region in the parking space topological region set to obtain each parking space topological region subjected to the distortion region masking processing as a parking space region set; And combining and processing the parking space occupation information and the parking space vertex visibility information group corresponding to the parking space areas in the parking space area set to obtain a parking space information set.
6. 6. The method of claim 1, wherein the set of multi-scale feature maps comprises a first scale feature map, a second scale feature map, and a third scale feature map, the data dimension of the first scale feature map being greater than the data dimension of the second scale feature map, the data dimension of the second scale feature map being greater than the data dimension of the third scale feature map, and Inputting the multi-scale feature map set to the feature fusion layer to obtain a multi-scale fusion feature map set, including: performing convolution processing on the first scale feature map to obtain a second scale convolution feature map; fusing the second scale convolution feature map and the second scale feature map to obtain a second scale convolution fusion feature map; Performing convolution processing on the second scale convolution fusion feature map to obtain a third scale convolution feature map; Fusing the third scale convolution feature map and the third scale feature map to obtain a third scale convolution fusion feature map as a third fusion feature map; performing up-sampling processing on the third-scale feature map to obtain a second-scale up-sampling feature map; Performing fusion processing on the second scale up-sampling feature map, the second scale feature map and the second scale convolution feature map to obtain a second fusion feature map; Performing up-sampling processing on the second-scale up-sampling feature map to obtain a first-scale up-sampling feature map; fusing the first scale up-sampling feature map and the first scale feature map to obtain a first fused feature map; And determining the first fusion feature map, the second fusion feature map and the third fusion feature map as a multi-scale fusion feature map set.
7. 7. A parking space information generating device based on a fisheye camera comprises: The apparatus being capable of carrying out the method of any one of claims 1-6; An acquisition unit configured to acquire a target parking space fisheye image, wherein the target parking space fisheye image is a parking space image captured by a fisheye camera loaded on a target vehicle; the first input unit is configured to input the target parking space fisheye image to a backbone network layer included in a pre-trained parking space detection model to obtain a multi-scale feature map set, wherein the parking space detection model further comprises a feature fusion layer and a detection head layer; The second input unit is configured to input the multi-scale feature map set to the feature fusion layer to obtain a multi-scale fusion feature map set; The third input unit is configured to input the multi-scale fusion feature map set to the detection head layer to obtain a parking space detection information set; The first generation unit is configured to generate a parking space topological structure information set according to the parking space detection information set; The second generating unit is configured to generate a parking space information set according to the parking space topological structure information set.
8. 8. An electronic device, comprising: One or more processors; a storage device having one or more programs stored thereon, When executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-6.
9. 9. A computer readable medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the method of any of claims 1-6.

Description

Method, device, equipment and medium for generating parking space information based on fisheye camera Technical Field The embodiment of the disclosure relates to the technical field of computers, in particular to a method, a device, equipment and a medium for generating parking space information based on a fisheye camera. Background With the development of computer technology and automatic driving technology, automatic parking space detection and automatic parking technology is mature. At present, when generating parking space information, a general parking space detection model (for example YOLOV) is adopted to detect the parking space picture information to obtain a parking space detection result so as to park the vehicle. However, the inventor has found that when the parking space information is generated in the above manner, there are often the following technical problems: firstly, in the process of generating parking space information, multi-scale feature extraction and multi-scale feature fusion are not carried out on the parking space pictures. The integrity of the extracted parking space information is lower when the parking space in the parking space image is larger or smaller. Therefore, the generated parking space information is low in accuracy, the number of caused parking accidents is large, and the safety of the vehicle in parking is low. Secondly, in the process of generating the parking space information, parking space type subdivision and parking space area condition judgment are not carried out on the predicted parking space, and relevant parking space information screening is not carried out on the basis of the judgment result, so that the generated parking space information is low in accuracy, more in number of parking accidents are caused, and the safety of a vehicle in parking is low. The above information disclosed in this background section is only for enhancement of understanding of the background of the inventive concept and, therefore, may contain information that does not form the prior art that is already known to those of ordinary skill in the art in this country. Disclosure of Invention The disclosure is in part intended to introduce concepts in a simplified form that are further described below in the detailed description. The disclosure is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Some embodiments of the present disclosure propose a fisheye camera-based parking space information generation method, apparatus, electronic device, and computer-readable medium to solve one or more of the technical problems mentioned in the background section above. According to the first aspect, some embodiments of the present disclosure provide a method for generating parking space information based on a fisheye camera, the method comprising the steps of obtaining a target parking space fisheye image, wherein the target parking space fisheye image is a parking space image shot by the fisheye camera loaded on a target vehicle, inputting the target parking space fisheye image to a backbone network layer included in a pre-trained parking space detection model to obtain a multi-scale feature map set, wherein the parking space detection model further comprises a feature fusion layer and a detection head layer, inputting the multi-scale feature map set to the feature fusion layer to obtain a multi-scale fusion feature map set, inputting the multi-scale fusion feature map set to the detection head layer to obtain a parking space detection information set, generating a parking space topological structure information set according to the parking space detection information set, and generating the parking space information set according to the parking space topological structure information set. In a second aspect, some embodiments of the present disclosure provide a device for generating parking space information based on a fisheye camera, where the device includes an acquisition unit configured to acquire a target parking space fisheye image, where the target parking space fisheye image is a parking space image captured by the fisheye camera loaded on a target vehicle, a first input unit configured to input the target parking space fisheye image to a backbone network layer included in a pre-trained parking space detection model to obtain a multi-scale feature map set, where the parking space detection model further includes a feature fusion layer and a detection head layer, a second input unit configured to input the multi-scale feature map set to the feature fusion layer to obtain a multi-scale fusion feature map set, a third input unit configured to input the multi-scale fusion feature map set to the detection head layer to obtain a parking space detection information set, and a first generation unit configured to generate a topology structure information set according to t