Search

CN-121639694-B - Wind turbine blade defect detection method based on improved RT-DETR

CN121639694BCN 121639694 BCN121639694 BCN 121639694BCN-121639694-B

Abstract

The invention belongs to the technical field of target detection, and discloses a wind turbine blade defect detection method based on improved RT-DETR, the method introduces SWRepBlock modules into the backbone network, thereby effectively solving the problem that the traditional network fixed receptive field is difficult to adaptively extract target features with different scales. In addition, the method also replaces AIFI modules in the encoder with DyT-AIFI modules so as to improve the semantic understanding effect of long-distance feature interaction. In addition, the method of the invention introduces a CAA-HSFPN module into the encoder, thereby effectively solving the problem of insufficient distinction between the semantic gap and the feature importance of the traditional feature pyramid. The defect detection method for the wind turbine blade not only remarkably improves the detection capability of micro damage under the complex surface texture background of the wind turbine blade, but also can effectively identify multi-type defect characteristics such as cracks, erosion, oil leakage and the like.

Inventors

  • LI XUJIAN
  • YE TONG

Assignees

  • 山东科技大学

Dates

Publication Date
20260508
Application Date
20260205

Claims (9)

  1. 1. A method for detecting defects of a wind turbine blade based on improved RT-DETR, comprising the steps of: Step 1, obtaining a defect image of a wind turbine blade, and constructing a data set for detecting the defect of the wind turbine blade; step 2, an improved RT-DETR model is built, and the model comprises a backbone network, an encoder, a decoder and a detection head; The improved RT-DETR model is obtained by introducing a displacement re-parameterized convolution SWRepBlock module into a backbone network, replacing a AIFI module in an encoder with a DyT-AIFI module, and replacing a AIFI module with a dynamic hyperbolic tangent normalization module to obtain a DyT-AIFI module, introducing a CAA-HSFPN module based on the attention of a context anchor point into the encoder; Step 3, training the improved RT-DETR model by utilizing the data set constructed in the step 1 to obtain a trained model; Step 4, performing defect detection on the image of the wind turbine blade by using the trained model obtained in the step 3; The SWRepBlock module includes a main branch and a residual branch; the processing flow of the signal in SWRepBlock module is as follows: input feature map of SWRepBlock module Sequentially passing through convolution kernels in main branches with the size of Convolution operation, batch normalization and Processing the activation function to obtain an intermediate feature map : ; Wherein, the Representation of The function is activated and the function is activated, A batch normalization operation is shown and, Indicating that the convolution kernel in the main branch is of size The convolution weights of the convolution operations of (a), Representing a convolution operation; then the intermediate feature map Send the data to RLKConv module for processing and output characteristic diagram of RLKConv module An output feature map as a main branch; input feature map of SWRepBlock module Judging whether the parameter shortcut is true in the residual branch, if so, executing identity mapping by the residual branch Obtaining an output characteristic diagram of residual branches ; If the parameter shortcut is false, further judging whether the parameter variant='d' is satisfied and stride=2; When the parameter shortcut is false and the parameters variant='d' and stride=2 are satisfied at the same time, the input feature map to SWRepBlock module Performing an average pooling of stride 2, convolution kernel size of Convolution operation and batch normalization processing of the residual branches to obtain output characteristic diagrams of residual branches : ; Wherein, the Representation of feature graphs An average pooling operation with a stride of 2 is performed, Indicating a convolution kernel size in the residual branch of Convolution weights of the convolution operations of (a); When the parameter shortcut is false and the parameters variant='d' and stride=2 are not satisfied at the same time, the input feature map to SWRepBlock module Performing convolution kernel of size Convolution operation and batch normalization processing of the residual branches to obtain output characteristic diagrams of residual branches : ; Output characteristic diagram of main branch And residual branching output feature map After addition, the result is processed by a ReLU activation function to obtain an output characteristic diagram of SWRepBlock modules : ; Wherein, the Representing the ReLU activation function.
  2. 2. The method for detecting defects of wind turbine blades based on the improved RT-DETR according to claim 1, wherein in step 1, the images of defects of wind turbine blades are obtained through daily operation and maintenance inspection reports of wind farm and image data collected by unmanned aerial vehicle aerial photography.
  3. 3. The improved RT-DETR based wind turbine blade defect detection method of claim 1, wherein said backbone network comprises three convolutional normalization layers, one max pooling layer and four SWRepBlock modules; Four SWRepBlock modules were defined as a first SWRepBlock module, a second SWRepBlock module, a third SWRepBlock module, and a fourth SWRepBlock module, respectively; The signal processing flow in the backbone network is as follows: The input to the backbone network is a signature in the dataset for wind turbine blade defect detection ; Feature map of input backbone network Sequentially processing by three convolution normalization layers and a maximum pooling layer, and then sending to a first SWRepBlock module to obtain an output characteristic diagram of the first SWRepBlock module ; Mapping the output characteristics of the first SWRepBlock module Sending the output characteristic diagram to a second SWRepBlock module for processing to obtain an output characteristic diagram of the second SWRepBlock module ; Output characteristic diagram of second SWRepBlock module Sending the output characteristic diagram to a third SWRepBlock module for processing to obtain an output characteristic diagram of the third SWRepBlock module ; Mapping the output characteristics of the third SWRepBlock module Sending the output characteristic diagram to a fourth SWRepBlock module for processing to obtain an output characteristic diagram of the fourth SWRepBlock module 。
  4. 4. The improved RT-DETR based wind turbine blade defect detection method of claim 1, wherein said RLKConv module comprises a LoRA decomposition path and an auxiliary convolution path; the processing flow of the signal in RLKConv module is as follows: the input feature map of RLKConv module is the intermediate feature map ; Input feature map of RLKConv module The processing in LoRA decomposition path is: ; Wherein, the Representing LoRA a break-up path; , Representing the number of branches of the convolution kernel that are split; And Representing the characteristic displacement and cutting operation along the horizontal and vertical directions respectively; representing a Sigmoid activation function; And Representing a matrix of a mask that can be learned, , , Representing the channel dimension of the mask matrix; Representing element-by-element multiplication; representing input feature graphs The ith sub-feature diagram is obtained after being split; Input feature map of RLKConv module The size of the convolution kernel passing through the auxiliary convolution path is To obtain an output characteristic diagram of the auxiliary convolution path Wherein Indicating that the convolution kernel in the auxiliary convolution path is of size Convolution weights of the convolution operations of (a); LoRA output characteristic diagram of decomposition path Output feature map with auxiliary convolution path After addition, the sum is subjected to batch normalization operation After the activation function is processed, an output characteristic diagram of the main branch is obtained : 。
  5. 5. The method for improved RT-DETR based wind turbine blade defect detection of claim 4, wherein after training, the main branches of SWRepBlock modules are merged into a convolution weight Biased to Is an equivalent convolution layer of (a); output characteristic diagram of SWRepBlock module The method is characterized in that the method is obtained by adding an output characteristic diagram of an equivalent convolution layer and an output characteristic diagram of a residual branch and then processing the obtained result through a ReLU activation function, and the process is expressed as follows: 。
  6. 6. The improved RT-DETR based wind turbine blade defect detection method of claim 3, wherein said encoder comprises two convolution layers of convolution kernel size 1 x1, one DyT-AIFI module, five CAA-HSFPN modules, three two-dimensional convolution layers, two transposed convolution layers and two re-parameterized C3 modules; Two convolution layers with the convolution kernel size of 1×1 are respectively defined as a first 1×1 convolution layer and a second 1×1 convolution layer; The five CAA-HSFPN modules are defined as a first CAA-HSFPN module, a second CAA-HSFPN module, a third CAA-HSFPN module, a fourth CAA-HSFPN module, and a fifth CAA-HSFPN module, respectively; respectively defining three two-dimensional convolution layers as a first two-dimensional convolution layer, a second two-dimensional convolution layer and a third two-dimensional convolution layer; defining two transposed convolutional layers as a first transposed convolutional layer and a second transposed convolutional layer, respectively; Defining two re-parameterized C3 modules as a first re-parameterized C3 module and a second re-parameterized C3 module, respectively; The processing flow of the signal in the encoder is as follows: Output characteristic diagram of second SWRepBlock module After being processed by a first CAA-HSFPN module, the processed data are sent into a first two-dimensional convolution layer to obtain an output characteristic diagram of the first two-dimensional convolution layer ; Output characteristic diagram of third SWRepBlock module After being processed by a second CAA-HSFPN module, the processed data are sent into a second two-dimensional convolution layer to obtain an output characteristic diagram of the second two-dimensional convolution layer ; Output characteristic diagram of fourth SWRepBlock module Sequentially processing by a first 1×1 convolution layer, dyT-AIFI module, a second 1×1 convolution layer and a third CAA-HSFPN module, and sending into a third two-dimensional convolution layer to obtain an output characteristic diagram of the third two-dimensional convolution layer ; Output feature map of third two-dimensional convolution layer After being processed by the first transfer convolution layer and the fourth CAA-HSFPN module, the output characteristic diagram of the second two-dimensional convolution layer Performing element-by-element multiplication operation, performing addition fusion with the output feature map of the first transposed convolutional layer, and sending to a first reparameterized C3 module to obtain the output feature map of the first reparameterized C3 module ; The output characteristic diagram of the first transposed convolutional layer is processed by the second transposed convolutional layer and the fifth CAA-HSFPN module and then is matched with the output characteristic diagram of the first two-dimensional convolutional layer Performing element-by-element multiplication operation, performing addition fusion with the output feature map of the second transposed convolution layer, and sending to a second parameterized C3 module to obtain the output feature map of the second parameterized C3 module ; Output characteristic diagram of first reparameterized C3 module Output feature map of second re-parameterized C3 module And an output feature map of a third two-dimensional convolution layer And splicing along the channel dimension to obtain an output characteristic diagram of the encoder.
  7. 7. The improved RT-DETR based wind turbine blade defect detection method of claim 6, wherein DyT-AIFI modules comprise N stacked fransformer encoder layers, each fransformer encoder layer comprising two dynamic hyperbolic tangent normalization modules; the processing flow of the signals in the DyT-AIFI module is as follows: input feature map for DyT-AIFI module I.e. the output characteristic diagram of the first 1 x1 convolution layer, in which The size of the batch is indicated and, The number of channels is indicated and the number of channels is indicated, And The height and width of the feature map are represented respectively; Compressing the space dimension into the sequence length dimension by flattening and transposing the input feature map of DyT-AIFI module to obtain the input sequence of the first transducer encoder layer, namely the serialized input feature map ; Construction of two-dimensional sine-cosine position codes The construction process is expressed as follows: ; Wherein, the And Grid coordinates in the width and height directions respectively, , ; Representing an outer product operation; The angular frequency vector is represented as such, ; And Respectively sine and cosine functions; the query and key matrix is obtained by adding the input sequence to the position code: ; Wherein, the Representing the query matrix and, Representing a key matrix; Value matrix Directly using input sequences, i.e. ; Multi-headed self-attention mechanism will input query matrix Key matrix Matrix of values Projected to Each of the attention heads independently calculates a scaled dot product attention: ; Wherein, the , Represent the first An output of the individual attention heads; representing a Softmax function; 、 、 Represent the first A projection matrix of the individual attention heads, , , ; Is a scaling factor; The output of each attention head is operated by splicing And output projection matrix Linear transformation fusion is carried out to obtain fusion output of multiple heads of self-attentions : ; Multi-headed self-attention fusion output After Dropout regularization, the obtained product is matched with an input feature map Residual connection is carried out to obtain intermediate characteristic representation : ; Wherein, the Representing Dropout regularization; Introducing dynamic hyperbolic tangent normalization operation, compressing the characteristic amplitude by adopting a hyperbolic tangent nonlinear function, and representing the signal processing process in a first dynamic hyperbolic tangent normalization module as follows: ; Wherein, the The characteristics after the normalization are represented, The output of the first dynamic hyperbolic tangent normalization module in the first transducer encoder layer; And Representing a channel-by-channel affine transformation parameter vector, , ; Representing the globally learnable adaptive scaling parameters, ; Representing hyperbolic tangent nonlinear function for compressing input values to A section; Normalized features Entering a feedforward neural network FFN to perform nonlinear characteristic transformation among channels; The FFN is composed of two full-connection layers, and the first full-connection layer is used for connecting the characteristic dimension from To extend to Then applying GELU activation functions to introduce nonlinearity, and compressing the characteristic dimension back through the second full-connection layer after Dropout regularization The process is expressed as: ; Wherein, the An output representing FFN; And Representing the weight matrix of two fully connected layers, , ; Representation GELU activates a function; And The offset vector is represented as such, , ; Output of FFN Features regularized by Dropout and normalized Residual connection is carried out to obtain output characteristics after residual connection : ; Output characteristics of second dynamic hyperbolic tangent normalization module after residual connection Applying dynamic hyperbolic tangent normalization operation to obtain the output of the second dynamic hyperbolic tangent normalization module : ; Wherein, the The output of the first transducer encoder layer; And Representing a channel-by-channel affine transformation parameter vector, , ; Representing the globally learnable adaptive scaling parameters, ; Output of the first transducer encoder layer As input to the next transducer encoder layer; After being processed by N stacked Transformer encoder layers, the output of the last Transformer encoder layer is restored into a space structure of the original characteristic diagram through inverse transformation, and the restored characteristic diagram is obtained; the finally obtained restored characteristic diagram is the output characteristic diagram of DyT-AIFI module.
  8. 8. The improved RT-DETR based wind turbine blade defect detection method of claim 1, wherein the signal processing in the CAA-HSFPN module is as follows: input feature graphs of the CAA-HSFPN module are noted as ; Input feature map of CAA-HSFPN module Carrying out average pooling operation to obtain a feature map after spatial downsampling : ; Wherein, the Representing a pooled window size as Is subjected to an average pooling operation; feature map after pooling, i.e. spatial downsampling By convolving a kernel of size Is processed by the convolution operation, the batch normalization layer and SiLU activation functions to obtain a transformed intermediate feature map : ; Wherein, the Representing the weight of the point convolution kernel, ; For transformed intermediate features Applying a depth convolution in the horizontal direction with a convolution kernel of size Obtaining a characteristic diagram after horizontal convolution : ; Wherein, the For the horizontal depth convolution kernel weights, ; Is a depth separable convolution operation; for characteristic diagram after convolution in horizontal direction Applying a depth convolution in the vertical direction with a convolution kernel of size Obtaining a characteristic diagram after convolution in the vertical direction : ; Wherein, the For the vertical depth convolution kernel weights, ; Convolving the feature map in the vertical direction By convolving a kernel of size Is processed by the convolution operation, the batch normalization layer and SiLU activation functions to obtain a transformed intermediate feature map : ; Wherein, the Representing the weight of the point convolution kernel, ; For transformed features, i.e. intermediate feature maps Mapping to by Sigmoid activation function Interval, generating a channel-by-channel pixel-by-pixel spatial attention weighting map : ; Wherein, the ; Spatial attention weighting map Each element of (3) Wherein 、 、 The height, width and channel index of the feature map are respectively represented, 、 、 ; Judging whether the parameter flag is true, if true, mapping the spatial attention weight Input feature map with CAA-HSFPN module Multiplying by element, and outputting the result as CAA-HSFPN module, if the parameter flag is false, then making the space attention weight map As an output of the CAA-HSFPN module.
  9. 9. A computer device comprising a memory and one or more processors, the memory having executable code stored therein, wherein the processor, when executing the executable code, performs the steps of the improved RT-DETR based wind turbine blade defect detection method as claimed in any one of claims 1 to 8.

Description

Wind turbine blade defect detection method based on improved RT-DETR Technical Field The invention belongs to the technical field of target detection, and particularly relates to a wind turbine blade defect detection method based on improved RT-DETR. Background In wind turbine blade defect detection, the defects have obvious scale differences in images due to complex detection environments and obvious shooting distance changes caused by the fact that the blades are usually huge in size and installed in high altitude or remote areas. Meanwhile, defects such as cracks and erosion on the surface of the blade tend to be small in size and limited in duty ratio, and the conditions of characteristic blurring and detail missing are easy to occur. In addition, in the blade image that unmanned aerial vehicle was gathered by plane, the defect area often highly similar with background such as blade surface texture, spot, has increased the degree of difficulty of discernment. In addition, the unmanned plane is easy to shake and blur in the flight process, and the blades can also generate motion blur in the running state, so that the definition of the image is reduced, and the detection uncertainty is further aggravated. Second, the challenges presented by environmental and application requirements are not negligible. Wind turbine blades are exposed to outdoor extreme environments throughout the year and are subject to many factors such as wind, rain, ice and snow, ultraviolet radiation, changes in illumination, weather conditions, etc., resulting in reduced visibility of defective features and reduced image quality stability. Meanwhile, the defect detection needs to be timely and high-precision to achieve early fault diagnosis and preventive maintenance, and the traditional manual detection method is low in efficiency and has potential safety hazards, and the novel detection technology faces the constraints of equipment cost, environmental noise interference, signal processing complexity and the like. These factors together limit the performance breakthroughs of wind turbine blade defect detection, particularly early detection of minor defects. Wind turbine blade defect detection has made significant progress in recent years as an important research direction in the area of energy engineering and computer vision intersection, but also faces many challenges. In terms of detection methods, traditional means comprise manual visual inspection, acoustic emission detection, vibration signal analysis, ultrasonic detection, infrared thermal imaging and the like, but the traditional means have the limitations that the manual visual inspection depends on experience and has safety risks, the acoustic emission detection is limited by noise interference and cost, the vibration signal analysis has insufficient precision on micro-crack detection, the ultrasonic detection signal processing is complex, and the infrared thermal imaging has limited defect type and depth identification. With the development of deep learning, an image detection method based on machine vision and unmanned aerial vehicle gradually becomes a research hot spot, but a large number of tiny defects exist in a wind turbine blade image, the pixel ratio is extremely small, the characteristics are not obvious, the image is easily covered by a complex background, and the image is easily missed or misdetected. Therefore, a detection method capable of identifying micro defect features in the context of complex blade surface textures is needed. Specifically, the method not only has the capability of multi-scale feature extraction and fusion so as to capture shallow texture detail information and deep semantic information at the same time, but also can keep robustness under the interference factors such as visual angle change, illumination difference, meteorological condition fluctuation, blade surface pollution and the like caused by aerial photography. In addition, the algorithm needs to be compatible with detection precision and calculation efficiency in practical industrial application, and can adapt to various forms of different defect types, such as cracks, erosion, dirt, oil leakage and the like, so that reliable technical support is provided for safe and stable operation and preventive maintenance of the wind driven generator. Disclosure of Invention The invention aims to provide a wind turbine blade defect detection method based on improved RT-DETR, which combines detection precision and calculation efficiency, can realize small target feature identification under a complex background, and can keep robustness under the interference factors of visual angle change, illumination difference, meteorological condition fluctuation, blade surface pollution and the like. In order to achieve the above purpose, the invention adopts the following technical scheme: a wind turbine blade defect detection method based on improved RT-DETR, comprising the steps of: Step 1, obtainin