CN-115953678-B - Pavement damage detection method based on local gray extreme point and feature fusion

CN115953678BCN 115953678 BCN115953678 BCN 115953678BCN-115953678-B

Abstract

The invention relates to a pavement damage detection method based on local gray level extreme points and feature fusion, which comprises the steps of dividing a pavement image into a plurality of windows, calculating local extreme points by taking the windows as units, determining a plurality of local extreme points as feature points according to gray level distribution of neighborhood blocks of the extreme points, taking the local extreme points as centers, intercepting image blocks of the neighborhood blocks, establishing a block image data set, training a CNN block image feature extraction model, calculating feature vectors corresponding to all feature points of the image windows by using the trained CNN feature model, carrying out feature fusion to obtain composite feature vectors, classifying the composite feature vectors by using a support vector machine, and detecting pavement damage. The method has the beneficial effects that the method combines some priori image characteristics of the damaged target area with the combined deep learning algorithm, reduces the scale of a data set and the requirement of calculation capacity, and improves the detection effect of the road surface.

Inventors

CHEN GUOHONG
SUN YUNLEI
CHEN ZHUO
NI JIE
JIN WEIWEI

Assignees

浙大城市学院

Dates

Publication Date: 20260508
Application Date: 20221209

Claims (9)

1. The pavement damage detection method based on the fusion of the local gray extreme points and the characteristics is characterized by comprising the following steps: step 1, obtaining a pavement image, and preprocessing the pavement image; step 2, dividing the pavement image into a plurality of windows, and calculating local extremum points by taking the windows as units respectively; The step 3 comprises the following steps: Step 3.1, determining the gray level distribution condition of the extreme point neighborhood block, considering the gray level average value and the variance, wherein the gray level value of the pixel point of the ith block is There are N pixels in total, and the calculation formula is as follows: Wherein, the The average value reflects the whole brightness degree around the extreme point; the variance is used for reflecting the gray level distribution condition of the block pixels; step 3.2, setting critical values for the average value and the variance of the extreme point neighborhood gray level, and performing primary screening on local extreme points; step 3.3, sequencing the rest extreme points according to gray depth, selecting a certain number of extreme points as characteristic points of the window, and intercepting a neighborhood block of the characteristic points for analyzing the image window; step 4, taking the local extreme point as the center, intercepting the image blocks in the neighborhood of the local extreme point, establishing a block image dataset, and training a CNN block image feature extraction model; And 5, calculating feature vectors corresponding to each feature point of the image window by using the trained CNN feature model, carrying out feature fusion to obtain a composite feature vector, classifying the composite feature vector by using a support vector machine, and detecting the damage condition of the road surface.
2. The pavement damage detection method based on local gray-scale extremum point and feature fusion according to claim 1, wherein step 1 comprises: step 1.1, obtaining a pavement image, and performing Gaussian filtering on the pavement image; step 1.2, gamma correction is carried out on the pavement image, wherein the formula of the gamma correction is as follows: Wherein, the The formula for determining the value is: Wherein, the Is the gray distribution peak before correction, Is the corrected gray distribution peak.
3. The pavement damage detection method based on local gray-scale extremum point and feature fusion according to claim 2, wherein step 2 comprises: Step 2.1, dividing the pavement image into the following sizes Firstly inverting the gray level of the image, and then searching for a maximum value by taking the image window as a unit; Step 2.2, defining two parameters, namely a minimum distance P dist of peak detection and a threshold P th of peak detection, and converting the threshold P th into a peak detection amplitude P A of each window according to the following formula: wherein G max and G min are the gray-scale maximum and minimum values of the corresponding windows; Step 2.3, obtaining a first-order difference and a subscript position with a difference amplitude value of 0, wherein the adjacent 0 position states are in a horizontal state, and only a middle point is selected as a unique peak point; step 2.4, obtaining peak points meeting peak detection, wherein the judgment basis is that the gradient sign change is met and the gray value is larger than the peak detection amplitude P A ; And 2.5, screening invalid peak points by using the minimum peak distance P dist to obtain local extreme points.
4. The pavement damage detection method based on local gray-scale extremum point and feature fusion according to claim 3, wherein in step 2.5, the distance between the local extremum points is equivalent to the size of the intercepted extremum point neighborhood image block.
5. The pavement damage detection method based on local gray-scale extremum point and feature fusion according to claim 4, wherein step 4 comprises: Step 4.1, intercepting a block image from the public data set and the road surface image which is actually required to be detected, and establishing a block image data set; Step 4.2, construction The feature extraction model, CNN, is essentially to convolve and pool input image data layer by layer to extract data features, wherein the convolution operation formula of a single convolution kernel is as follows: Wherein, the Representing the convolved output pixel values, The activation function is represented as a function of the activation, Representing convolution kernel Line 1 The weight of the column is determined, For the size of the convolution kernel, Representing the offset of the convolution kernel, Representing the first of the feature map Line 1 Column elements.
6. The method for detecting the pavement damage based on the local gray extreme point and the feature fusion according to claim 5, wherein in the step 4.1, the specific method for establishing the data set by manually intercepting the block image comprises the steps of determining the local extreme point in an image window and marking a sequence number in the image, judging the extreme point in a damaged area, inputting the sequence number of the extreme point to intercept the image block and mark the image block as a positive sample, and marking the block corresponding to the local extreme point in a normal pavement image as a negative sample.
7. The pavement damage detection method based on local gray-scale extremum point and feature fusion according to claim 6, wherein step 5 comprises: Step 5.1, the CNN block image feature extraction model outputs feature vectors from the full connection layer; step 5.2, combining the feature vectors of all the feature points involved in the image window to form a feature vector for describing the whole window; and 5.3, taking the composite feature vector as a feature descriptor of the image window, utilizing a support vector machine to construct a classifier to perform feature recognition, and judging whether the image window contains a damaged area.
8. A pavement damage detection device based on local gray extreme point and feature fusion is characterized in that, a road surface breakage detection method for performing the local gray-scale extremum point and feature fusion-based road surface breakage detection method according to claim 1, comprising: The acquisition module is used for acquiring a pavement image and preprocessing the pavement image; the dividing module is used for dividing the pavement image into a plurality of windows, and calculating local extremum points by taking the windows as units respectively; The determining module is used for determining a plurality of local extremum points as characteristic points according to the gray level distribution of the extremum point neighborhood block; the training module is used for taking the local extreme point as the center, intercepting the image blocks in the neighborhood of the local extreme point, establishing a block image dataset and training a CNN block image feature extraction model; the detection module is used for calculating feature vectors corresponding to each feature point of the image window by using the trained CNN feature model, carrying out feature fusion to obtain a composite feature vector, and classifying the composite feature vector by using a support vector machine to detect the damage condition of the road surface.
9. A computer storage medium, wherein a computer program is stored in the computer storage medium, and the computer program when running on a computer causes the computer to execute the pavement damage detection method based on the local gray extreme point and the feature fusion according to claim 1.

Description

Pavement damage detection method based on local gray extreme point and feature fusion Technical Field The invention relates to the field of pavement damage detection, in particular to a pavement damage detection method based on local gray extreme point and feature fusion. Background The road surface maintenance integrating the front technologies such as image sensing, big data, artificial intelligence and the like can accurately and comprehensively sense and predict the road condition. Thereby providing reliable data analysis and decision basis for decision makers and improving the safety transportation efficiency of roads. The current road surface image acquisition technology has fast development, for example, a global shutter camera can realize rapid imaging, and has obvious advantages in terms of price compared with devices such as ground penetrating radar, a laser system and the like. There are still challenges in the research of automatic road surface image processing algorithms. In view of the differences of road conditions and economic levels of various places, the research on the road detection method with strong adaptability has more practical significance. Road surface image recognition technology has undergone a range from conventional image processing methods to detection methods based on deep learning. The traditional image processing method divides the image by artificial design features, including a threshold segmentation method, an edge detection method, a texture segmentation method, a multi-feature fusion-based method and the like, and the methods can obtain good detection effects on a specific data set. The deep learning algorithm greatly reduces links of feature description, extraction, recognition and the like based on artificial experience, and is beneficial to improving the accuracy and the universality of the pavement recognition algorithm. The deep learning algorithm essentially realizes detection through the strong fitting capability of the neural network, however, the deep learning neural network is trained by directly utilizing the high-resolution image, and the method has high requirements on a huge pavement data set and calculation capability. Disclosure of Invention The invention aims to overcome the defects in the prior art and provides a pavement damage detection method based on local gray extreme point and feature fusion. In a first aspect, a pavement damage detection method based on local gray level extreme point and feature fusion is provided, including: step 1, obtaining a pavement image, and preprocessing the pavement image; step 2, dividing the pavement image into a plurality of windows, and calculating local extremum points by taking the windows as units respectively; Step 3, determining a plurality of local extremum points as characteristic points according to the gray level distribution of the extremum point neighborhood blocks; step 4, taking the local extreme point as the center, intercepting the image blocks in the neighborhood of the local extreme point, establishing a block image dataset, and training a CNN block image feature extraction model; And 5, calculating feature vectors corresponding to each feature point of the image window by using the trained CNN feature model, carrying out feature fusion to obtain a composite feature vector, classifying the composite feature vector by using a support vector machine, and detecting the damage condition of the road surface. Preferably, step 1 includes: step 1.1, obtaining a pavement image, and performing Gaussian filtering on the pavement image; step 1.2, gamma correction is carried out on the pavement image, wherein the formula of the gamma correction is as follows: I(x,y)=I(x,y)γ the gamma value is determined by the following formula: Where I P is a gradation distribution peak before correction, and I 0 is a gradation distribution peak after correction. Preferably, step 2 includes: Step 2.1, dividing the pavement image into pavement image windows with the size of Nw multiplied by Nw, inverting the gray level of the image, and searching for a maximum value by taking the image window as a unit; Step 2.2, defining two parameters, namely a minimum distance P dist of peak detection and a threshold P th of peak detection, and converting the threshold P th into a peak detection amplitude P A of each window according to the following formula: PA＝Pth*(Gmax-Gmin)+Gmin wherein G max and G min are the gray-scale maximum and minimum values of the corresponding windows; Step 2.3, obtaining a first-order difference and a subscript position with a difference amplitude value of 0, wherein the adjacent 0 position states are in a horizontal state, and only a middle point is selected as a unique peak point; step 2.4, obtaining peak points meeting peak detection, wherein the judgment basis is that the gradient sign change is met and the gray value is larger than the peak detection amplitude P A; And 2.5, screening invalid peak points by