CN-121998900-A - High-precision real-time photovoltaic panel multisource image end-to-end defect detection method and system

CN121998900ACN 121998900 ACN121998900 ACN 121998900ACN-121998900-A

Abstract

The invention discloses a high-precision real-time end-to-end defect detection method and system for a multi-source image of a photovoltaic panel, and relates to the technical field of photovoltaic detection and computer vision. The method aims to solve the problems of poor single-mode adaptability, insufficient multi-source fusion, poor real-time performance and no closed-loop optimization in the prior art. According to the method, infrared and visible light characteristics are extracted through a dual-branch network containing GRB and SGB modules, after the infrared and visible light characteristics are fused through an FB module, defect detection is completed through PANet and a YOLO detection head with an attention module, and an optimized closed loop is formed by combining edge deployment test and feedback regulation. The system comprises modules such as a double-branch encoder and the like, and improves the performance by adopting a lightweight network architecture and an inference optimization technology. On an edge computing platform, single-frame reasoning takes less than or equal to 35ms, the defect mAP reaches more than 93%, the false alarm rate of a strong light environment is less than or equal to 4%, and the method is suitable for automatic inspection of a large-scale photovoltaic power station and can reduce operation and maintenance cost.

Inventors

ZHU ZANMING
CHEN JUN
SUN WENHAO
QIN KAIXIN
XIE HAIFENG
LUO HAIPING
LIU LIANGJIE
HAN SONGCHEN
YUAN ZHAOYU
TANG MINGZE
LI CHENGBAO

Assignees

国能永福发电有限公司

Dates

Publication Date: 20260508
Application Date: 20251212

Claims (12)

1. The high-precision real-time photovoltaic panel multisource image end-to-end defect detection method is characterized by comprising the following steps of: step S1, respectively extracting input infrared image features and visible light image features through an infrared feature extraction backbone network and a visible light feature extraction backbone network based on GRB and SGB modules; S2, inputting the extracted infrared image features and visible light image features into a feature fusion network based on an FB module to perform multi-scale feature fusion, so as to obtain a fused multi-scale feature map; Step S3, inputting the multi-scale feature map to a feature decoder based on a path aggregation network for feature enhancement and aggregation, and respectively inputting a multi-scale feature pyramid obtained after aggregation to a corresponding YOLO detection head for defect positioning and classification, wherein a lightweight attention module is arranged in front of the YOLO detection head; S4, training a model comprising the feature extraction backbone network, the feature fusion network, the feature decoder and the detection head by using the registered visible light-infrared photovoltaic panel defect data set to obtain a trained detection model; step 5, deploying the trained detection model on an edge computing platform, detecting defects under different photovoltaic scenes and environmental conditions, collecting detection results and corresponding multi-source image data, forming a scene performance data set, and uploading the scene performance data set to a cloud model optimization center; Step S6, evaluating the scene performance data set, and triggering a model optimization flow when at least one index of detection precision, false alarm rate or omission rate in a specific scene does not reach a preset performance threshold, wherein the model optimization flow comprises the following steps: S6.1, analyzing scene data with unqualified performance, determining an optimization strategy, preferentially adjusting preprocessing algorithm parameters or detection head confidence threshold values if performance degradation is caused by specific environment interference, and starting model parameter updating if the performance degradation is caused by new scene features which are not fully learned; S6.2, the relevant data in the scene performance data set is used as a supplementary training sample and transmitted back to a training end, a currently deployed model is used as an initialization, and all or part of parameters of the model are finely adjusted by adopting the supplementary training sample or combined with an original training set for incremental training; And S6.3, evaluating the optimized model on a verification set, generating a new model file after confirming that the performance meets the standard, updating the new model file to an edge computing platform, and replacing the original detection model.
2. The high-precision real-time end-to-end defect detection method for the photovoltaic panel multisource image according to claim 1 is characterized in that in the step S1, the GRB module adopts a dense connection structure and integrates the SGB module in a residual path, and the SGB module uses a transverse Scharr operator and a longitudinal Scharr operator to respectively convolve an input feature map so as to extract and integrate edge gradient features in the horizontal direction and the vertical direction.
3. The method for detecting the end-to-end defect of the high-precision real-time photovoltaic panel multisource image according to claim 1, wherein in the step S2, the execution process of the FB module comprises the steps of carrying out splicing and preliminary fusion on infrared and visible light feature images of the same level, processing the infrared and visible light feature images through a semantic fusion branch and an edge fusion branch in parallel, wherein the edge fusion branch is integrated with the SGB module, and finally adding output features of the two branches to obtain fusion features of the level.
4. The method for detecting the end-to-end defects of the multi-source image of the photovoltaic panel with high precision and real time according to claim 1, wherein in the step S3, the lightweight attention module is an SE attention module or CBAM module, and the YOLO detection head adopts a decoupling structure and has independent classification branches and regression branches.
5. The method for detecting end-to-end defects of a multi-source image of a photovoltaic panel in real time with high precision according to claim 1, wherein in step S4, the data enhancement method adopted in the model training comprises at least one of random rotation, color disturbance, multi-scale scaling, horizontal random inversion and image translation.
6. The method for detecting the end-to-end defect of the multi-source image of the photovoltaic panel with high precision and real time according to claim 1, wherein in the step S5, when the defect detection is performed, the method further comprises preprocessing the input visible light image by adopting a reflection suppression algorithm, the algorithm identifies a reflection area based on self-adaptive dynamic threshold segmentation, and corrects the pixel value according to the formula of "corrected pixel value = original pixel value× (1-reflection coefficient) +infrared gray value× (reflection coefficient)", wherein the reflection coefficient is dynamically calculated by the illumination sensor data ", and in the step S6.1, the adjustment of the preprocessing algorithm parameters comprises dynamically adjusting the calculation parameters or the mapping curve of the reflection coefficient.
7. The high-precision real-time end-to-end defect detection method of a photovoltaic panel multi-source image according to claim 1, wherein the preset performance threshold comprises at least one of an average precision mean value threshold, a false positive rate threshold and a omission factor threshold.
8. A high precision real-time photovoltaic panel multisource image end-to-end defect detection system for implementing the method of any of claims 1-7, the system comprising: the feature coding module comprises an infrared feature extraction sub-network and a visible light feature extraction sub-network which are arranged in parallel, wherein each sub-network comprises a GRB module and an SGB module and is used for respectively extracting the infrared features and the visible light features of an input image; The feature fusion module is constructed based on the FB module, connected with the feature coding module and used for receiving and fusing the infrared features and the visible light features and outputting a multi-scale fusion feature map; The feature decoding and detecting module comprises a path aggregation network and a decoupled YOLO detecting head, wherein the input end of the path aggregation network is connected with the feature fusion module and used for enhancing and aggregating the multi-scale fusion feature map, and the YOLO detecting head is connected with the output end of the path aggregation network and used for carrying out defect positioning and classification on the aggregated features; The model training module is used for training an integral model formed by the feature encoding module, the feature fusion module and the feature decoding and detecting module by utilizing the registered visible light-infrared photovoltaic panel defect data set; The edge deployment and reasoning module is used for optimizing and deploying the trained model to an edge computing platform and executing real-time defect detection tasks under different photovoltaic scenes and environmental conditions; the data collection and performance monitoring module is arranged at the edge side and is used for collecting multi-source image data, detection results and environmental parameters in the reasoning process, calculating real-time performance indexes and forming a scene performance data set; The cloud model optimization center is in communication connection with the edge deployment and reasoning module and the data collection and performance monitoring module and is used for receiving and storing scene performance data sets, and the cloud model optimization center comprises: the performance evaluation unit is used for judging whether the performance of each scene meets the standard; The optimization decision unit is used for analyzing reasons and generating an optimization strategy when the performance does not reach the standard, wherein the strategy comprises the steps of adjusting the algorithm parameters of the edge side or starting model updating; The parameter updating unit is used for calling the model training module when the strategy is model updating, taking the latest model as a base line, and carrying out fine adjustment or incremental training by utilizing the related data in the scene performance data set; And the model distribution unit is used for transmitting the optimized model or parameter configuration file to the edge deployment and reasoning module to finish model updating.
9. The high-precision real-time photovoltaic panel multisource image end-to-end defect detection system according to claim 8, wherein the GRB module in the feature encoding module has a main flow path containing dense connections and a residual path integrated with the SGB module, and the SGB module is internally provided with a transverse Scharr operator convolution kernel and a longitudinal Scharr operator convolution kernel.
10. The high-precision real-time photovoltaic panel multisource image end-to-end defect detection system according to claim 8, wherein the FB module in the feature fusion module comprises a splicing and preliminary fusion unit, and a semantic fusion branch and an edge fusion branch which are connected in parallel, wherein the edge fusion branch comprises an SGB module.
11. The high-precision real-time photovoltaic panel multisource image end-to-end defect detection system according to claim 8, wherein the path aggregation network in the feature decoding and detection module is used for achieving semantic transfer from top to bottom and detail fusion from bottom to top, and the YOLO detection head comprises independent classification convolution branches and regression convolution branches.
12. The high-precision real-time photovoltaic panel multisource image end-to-end defect detection system according to claim 8, wherein the edge deployment and reasoning module further comprises a preprocessing unit, the preprocessing unit is integrated with a reflection suppression algorithm, and the optimization strategy generated by the optimization decision unit is to adjust the edge algorithm parameters, and the model distribution unit is used for transmitting the adjusted reflection coefficient calculation parameters or detection confidence threshold to the preprocessing unit or the detection module for thermal updating.

Description

High-precision real-time photovoltaic panel multisource image end-to-end defect detection method and system Technical Field The invention relates to the technical field of operation and maintenance of photovoltaic power stations and computer vision, in particular to a method and a system for intelligently detecting defects of a photovoltaic panel by fusing visible light and infrared images, which are particularly suitable for an edge computing platform carried by an unmanned aerial vehicle to realize real-time and accurate automatic inspection. Background With the rapid development of the photovoltaic power generation industry, the scale of photovoltaic power stations is increasingly enlarged, and the deployment environment is also increasingly complex (such as deserts, mountainous regions, roofs and the like). The traditional inspection method by means of manual visual inspection or handheld equipment has the problems of low efficiency, high cost, poor detection consistency, potential safety hazard and the like, and can not meet the operation and maintenance requirements of a large-scale power station. Therefore, an automatic inspection technology based on an unmanned aerial vehicle platform has become a mainstream development direction of the industry. In automated defect detection, machine vision technology plays a central role. Currently, two types of image data, visible light images and infrared thermography images, are relied upon. The visible light image can clearly show the surface state of the photovoltaic panel, such as cracking, dust accumulation, bird droppings and the like, has rich texture details, but cannot detect internal defects (such as hot spots) of the battery piece or bypass diode faults. The infrared thermal imaging image can effectively identify internal defects related to thermal characteristics such as hot spots, multiple spots, streaks, empty load and the like by capturing abnormal temperature distribution of the panel, but has lower spatial resolution, lacks texture information and is difficult to accurately classify and position the defects. Most of the prior art solutions detect based on a single image source, and have obvious limitations. Some methods attempt to fuse the detection results of two modes at the decision layer, but fail to realize deep fusion at the feature layer, and the complementary advantages are limited and are easy to be interfered by false detection. Other researches try to perform feature fusion, but have complex network structure, large calculation amount, and are difficult to realize real-time reasoning on unmanned aerial vehicle edge equipment with limited calculation power, and cannot meet the severe requirement of inspection operation on timeliness. In addition, the operation environment of the photovoltaic power station is complex and changeable (such as strong light in noon, cloudy days, sand dust and the like), and the generalization capability and robustness of a single model under different scenes face challenges. The current lack of an end-to-end detection scheme which can run in real time at the edge end and can perform self-adaptive optimization according to field detection feedback becomes a key technical bottleneck for restricting the further improvement of the intelligent operation and maintenance level of the photovoltaic. Disclosure of Invention The invention provides a high-precision real-time photovoltaic panel multisource image end-to-end defect detection method and system for solving the problems of incomplete defect detection, poor model instantaneity, insufficient environmental adaptability, lack of continuous optimization capability and the like in the background technology. According to the invention, through an innovative network structure design, efficient complementary fusion of visible light and infrared images is realized in a feature layer, and a complete technology closed loop from cloud training to edge deployment and then to feedback-based continuous optimization is constructed. In order to achieve the above purpose, the invention adopts the following technical scheme: In a first aspect, the present invention provides a method for detecting end-to-end defects of a multi-source image of a photovoltaic panel in real time with high precision, comprising the steps of: step S1, respectively extracting input infrared image features and visible light image features through an infrared feature extraction backbone network and a visible light feature extraction backbone network based on GRB and SGB modules; S2, inputting the extracted infrared image features and visible light image features into a feature fusion network based on an FB module to perform multi-scale feature fusion, so as to obtain a fused multi-scale feature map; Step S3, inputting the multi-scale feature map to a feature decoder based on a path aggregation network for feature enhancement and aggregation, and respectively inputting a multi-scale feature pyramid obtai