CN-121982512-A - Road ponding detection method, device, terminal equipment and computer program product

CN121982512ACN 121982512 ACN121982512 ACN 121982512ACN-121982512-A

Abstract

The application relates to a road ponding detection method, a device, a terminal device and a computer program product, wherein the method comprises the steps of obtaining a visible light image and a thermal infrared image of a road to be detected; the method comprises the steps of carrying out feature extraction and feature fusion on a visible light image and a thermal infrared image to obtain a multi-level fusion feature map, sampling the multi-level fusion feature map to obtain a plurality of query vectors and level codes of each query vector, wherein the level codes characterize the levels of the feature map corresponding to the query vectors in the multi-level fusion feature map, setting a confidence threshold for the query vectors based on the query vectors and the level codes of the query vectors, the confidence threshold and the level corresponding to the level codes are in positive correlation, and carrying out water accumulation confidence judgment based on the multi-level fusion feature map and the confidence threshold corresponding to each query vector to obtain a water accumulation detection result of a road to be detected. The application solves the problem of higher omission factor of accumulated water on the road in the related technology.

Inventors

GAO LIYA
WU JINYONG
YU XIAOTIAN
LI AIJUN

Assignees

深圳云天励飞技术股份有限公司

Dates

Publication Date: 20260505
Application Date: 20251202

Claims (10)

1. The road ponding detection method is characterized by comprising the following steps of: obtaining a visible light image and a thermal infrared image of a road to be detected; performing feature extraction and feature fusion on the visible light image and the thermal infrared image to obtain a multi-level fusion feature map; Sampling the multi-level fusion feature map to obtain a plurality of query vectors and a level code of each query vector, wherein the level code characterizes the level of the feature map corresponding to the query vector in the multi-level fusion feature map; setting a confidence threshold for the query vector based on the query vector and the hierarchical code of the query vector, wherein the confidence threshold has positive correlation with a hierarchy corresponding to the hierarchical code; And based on the multi-level fusion feature map and the confidence threshold corresponding to each query vector, executing water accumulation confidence judgment to obtain a water accumulation detection result of the road to be detected.
2. The method of claim 1, wherein performing feature extraction and feature fusion on the visible light image and the thermal infrared image to obtain a multi-level fusion feature map comprises: extracting texture features of the visible light image to obtain a first feature map; Extracting the temperature characteristics of the thermal infrared image to obtain a second characteristic diagram; And calculating feature weights based on the first feature map and the second feature map, and fusing the first feature map and the second feature map according to the feature weights to obtain the multi-level fusion feature map.
3. The method of claim 2, wherein the computing feature weights based on the first feature map and the second feature map and fusing the first feature map and the second feature map according to the feature weights to obtain the multi-level fused feature map comprises: splicing the first feature map and the second feature map in the channel dimension to obtain a spliced feature map; Inputting the spliced feature map into a first multi-layer perceptron MLP to obtain the feature weight output by the first MLP and corresponding to each pixel point on the spliced feature map, wherein the feature weight is greater than zero and less than 1, and the feature weight characterizes the trust degree of the first MLP on the texture feature or the temperature feature at the pixel point; and carrying out weighted fusion on the first feature map and the second feature map pixel by pixel according to the feature weights corresponding to each pixel to obtain the multi-level fusion feature map.
4. A method according to any one of claims 1 to 3, wherein the setting a confidence threshold for the query vector based on the query vector and the hierarchical encoding of the query vector comprises: fusing the query vector and the hierarchical code of the query vector to obtain a fusion vector; inputting the fusion vector into a second MLP to obtain a confidence prediction value output by the second MLP, wherein the confidence prediction value characterizes that the probability of water accumulation exists at a sampling point corresponding to the query vector; Normalizing the confidence coefficient predicted value to obtain a normalized predicted value, wherein the normalized predicted value is greater than zero and less than 1; Setting a confidence coefficient value interval based on the hierarchy corresponding to the hierarchy coding, wherein the length of the confidence coefficient value interval is a fixed value, and the minimum value and the maximum value of the confidence coefficient value interval are in positive correlation with the hierarchy; And setting the confidence threshold for the query vector based on the confidence predicted value and the confidence value interval.
5. The method of claim 4, wherein the setting the confidence threshold for the query vector based on the confidence prediction value and the confidence interval comprises: subtracting the minimum value of the confidence coefficient value interval from the maximum value of the confidence coefficient value interval to obtain a subtraction result; multiplying the subtraction result by the confidence prediction value of the query vector to obtain a multiplication result; And subtracting the multiplication result from the maximum value of the confidence coefficient value interval to obtain the confidence coefficient threshold value of the query vector.
6. A method according to any one of claims 1 to 3, wherein the performing a water accumulation confidence judgment based on the multi-level fusion feature map and the confidence threshold value corresponding to each query vector, to obtain a water accumulation detection result of the road to be detected comprises: Upsampling the multi-level fusion feature map to obtain an upsampled feature map; extracting semantic features of the up-sampling feature map to obtain an initial segmentation prediction map, wherein the initial segmentation prediction map is characterized in that at each pixel point on the up-sampling feature map, the probability of water accumulation exists; And executing the accumulated water confidence judgment based on the initial segmentation prediction graph and the confidence threshold corresponding to each query vector to obtain the accumulated water detection result.
7. A method according to any one of claims 1 to 3, wherein the acquiring a visible light image and a thermal infrared image of the road to be detected comprises: Acquiring an initial visible light image and an initial thermal infrared image of the road to be detected; and preprocessing and size standardization processing are carried out on the initial visible light image and the initial thermal infrared image, so that the registered visible light image and the registered thermal infrared image are obtained.
8. A road water accumulation detection device, comprising: the acquisition module is used for acquiring a visible light image and a thermal infrared image of the road to be detected; The feature fusion module is used for carrying out feature extraction and feature fusion on the visible light image and the thermal infrared image to obtain a multi-level fusion feature map; The sampling module is used for sampling the multi-level fusion feature map to obtain a plurality of query vectors and a level code of each query vector, wherein the level code characterizes the level of the feature map corresponding to the query vector in the multi-level fusion feature map; The setting module is used for setting a confidence threshold for the query vector based on the query vector and the hierarchical coding of the query vector, wherein the confidence threshold and a hierarchy corresponding to the hierarchical coding are in positive correlation; And the detection module is used for executing water accumulation confidence judgment based on the multi-level fusion feature map and the confidence threshold corresponding to each query vector to obtain a water accumulation detection result of the road to be detected.
9. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the road water detection method according to any one of claims 1 to 7 when executing the computer program.
10. A computer program product comprising a computer program which, when run, causes the road water detection method as claimed in any one of claims 1 to 7 to be performed.

Description

Road ponding detection method, device, terminal equipment and computer program product Technical Field The application belongs to the technical field of computer images, and particularly relates to a road ponding detection method, a device, terminal equipment and a computer program product. Background Road ponding detection is an important link for intelligent traffic and urban safety monitoring. The query vector based water detection model in the related art generates a prediction box for each potential target in the image and filters the prediction boxes with confidence below a fixed threshold. However, areas with less water accumulation (e.g., targets smaller than 10 x 10 pixels) tend to have weaker characteristic responses, and their confidence tends to be lower, and filtering with a fixed threshold tends to miss such scenes, and therefore, such methods have higher miss rates for road water accumulation. At present, no effective solution is proposed for the problem of higher omission rate of accumulated water on roads in the related art. Disclosure of Invention The embodiment of the application provides a road ponding detection method, a device, terminal equipment and a computer program product, which are used for at least solving the problem of higher omission rate of road ponding in the related technology. The embodiment of the application provides a road ponding detection method, which comprises the steps of obtaining a visible light image and a thermal infrared image of a road to be detected, carrying out feature extraction and feature fusion on the visible light image and the thermal infrared image to obtain a multi-level fusion feature map, sampling the multi-level fusion feature map to obtain a plurality of query vectors and a level code of each query vector, wherein the level code characterizes a level of the feature map corresponding to the query vector in the multi-level fusion feature map, setting a confidence threshold for the query vector based on the query vector and the level code of the query vector, wherein the confidence threshold and the level corresponding to the level code are in a positive correlation, and carrying out ponding confidence judgment based on the multi-level fusion feature map and the confidence threshold corresponding to each query vector to obtain a ponding detection result of the road to be detected. In some embodiments, the feature extraction and feature fusion are performed on the visible light image and the thermal infrared image to obtain a multi-level fusion feature map, wherein the feature extraction and feature fusion are performed on the visible light image and the thermal infrared image to obtain a multi-level fusion feature map, the multi-level fusion feature map is obtained by extracting texture features of the visible light image to obtain a first feature map, extracting temperature features of the thermal infrared image to obtain a second feature map, calculating feature weights based on the first feature map and the second feature map, and fusing the first feature map and the second feature map according to the feature weights. In some embodiments, the calculating the feature weight based on the first feature map and the second feature map, and fusing the first feature map and the second feature map according to the feature weight, so as to obtain the multi-level fusion feature map includes stitching the first feature map and the second feature map in a channel dimension to obtain a stitched feature map, inputting the stitched feature map into a first multi-layer perceptron MLP to obtain the feature weight output by the first MLP and corresponding to each pixel point on the stitched feature map, where the feature weight is greater than zero and less than 1, and the feature weight characterizes the confidence degree of the first MLP on the texture feature or the temperature feature at the pixel points, and weighting and fusing the first feature map and the second feature map pixel by pixel point according to the feature weight corresponding to each pixel point to obtain the multi-level fusion feature map. In some embodiments, the step of setting a confidence threshold for the query vector based on the query vector and the hierarchical code of the query vector includes fusing the query vector and the hierarchical code of the query vector to obtain a fused vector, inputting the fused vector into a second MLP to obtain a confidence prediction value output by the second MLP, wherein the confidence prediction value characterizes that a probability of water accumulation exists at a sampling point corresponding to the query vector, performing normalization processing on the confidence prediction value to obtain a normalized prediction value, wherein the normalized prediction value is greater than zero and less than 1, setting a confidence value interval based on the hierarchy corresponding to the hierarchical code, wherein a length of the confidence value inte