CN-122020477-A - Electric power inspection multi-mode large model fine adjustment method and device

CN122020477ACN 122020477 ACN122020477 ACN 122020477ACN-122020477-A

Abstract

The invention discloses a method and a device for fine tuning a multi-mode large model of electric power inspection, wherein the method carries out time sequence data mapping pretreatment on one-dimensional current signals through a gram angle field algorithm to obtain a two-dimensional current map; the method comprises the steps of combining visible light/infrared images and diagnostic texts to construct an instruction fine adjustment data set, designing a 1+N LoRA fine adjustment frame comprising a current spectrum encoder and an electrical semantic optimization LoRA module on the basis of a universal multi-mode backbone model to obtain an adjusted model, outputting preliminary diagnostic probability vectors through a visual branch and a current expert branch respectively, calculating cosine similarity of the two preliminary diagnostic probability vectors, triggering a disbelief logic if the cosine similarity is lower than a preset threshold or a physical paradox exists, adjusting the weight of the electrical semantic optimization LoRA module, recalculating a feature fusion result and outputting a final diagnostic conclusion. The invention solves the problems of insufficient diagnosis precision and the like caused by single-mode sensing limitation and environmental interference in the prior art.

Inventors

XU XING
GAO LIPING
ZENG JIAN
ZHAO JIANBIN
ZHANG BO
ZHANG PENGFEI
LIU YADUO
ZHAO XIAOXIANG
Geng Mingxi
YANG CHEN
WANG SHAOYING

Assignees

国网河北省电力有限公司信息通信分公司
北京西清能源科技有限公司

Dates

Publication Date: 20260512
Application Date: 20260202

Claims (10)

1. The method for fine-tuning the multi-mode large power inspection model is characterized by comprising the following steps of: Carrying out time sequence data mapping pretreatment on one-dimensional current signals acquired by a substation SCADA system through a Graham angle field algorithm to obtain a two-dimensional current map; Based on the two-dimensional current map, combining a visible light/infrared image and a diagnosis text to construct an instruction fine adjustment data set; Based on the instruction fine tuning data set, designing a 1+N LoRA fine tuning framework comprising a current spectrum encoder and an electrical semantic optimization LoRA module on the basis of a universal multi-mode backbone model to obtain an adjusted model; based on the adjusted model, the preliminary diagnosis probability vectors are respectively output through the visual branch and the current expert branch, cosine similarity of the two preliminary diagnosis probability vectors is calculated, if the cosine similarity is lower than a preset threshold or a physical paradox exists, an anti-thinking logic is triggered, the weight of the electrical semantic optimization LoRA module is adjusted, the feature fusion result is recalculated, and a final diagnosis conclusion is output.
2. The method for fine tuning a multi-modal large model for power inspection according to claim 1, wherein in the process of carrying out time-series data mapping pretreatment on one-dimensional current signals through the Gellan angle field algorithm, the pretreatment process comprises self-adaptive normalization and polar coordinate transformation, gellan angle and/or difference field generation and multi-scale GAF fusion.
3. The method for fine tuning a multi-mode large power inspection model according to claim 2, wherein the weight update formula of the 1+n lorea fine tuning framework is as follows: ; In the formula, The updated weight; Original weights are pre-trained models; 、 For LoRA low-rank matrix transforms; a diagonal constraint matrix for power terminology.
4. The method for fine tuning a multi-mode large power inspection model according to claim 3, wherein the cosine similarity calculation formula is as follows: ; In the formula, Cosine similarity; Diagnosing a probability vector for the visual branch; the probability vector is diagnosed for the current expert branch.
5. The method for fine tuning a multi-modal large model for power inspection according to claim 4, wherein the final diagnostic conclusion is expressed as: ; In the formula, To the final diagnostic conclusion; The corrected feature fusion result is obtained; is an activation function; A decoder module that is a multimodal large language model; To trigger conflicting context instructions entered at the time of the miss logic.
6. A power inspection multi-mode large model fine adjustment device, which adopts the power inspection multi-mode large model fine adjustment method according to any one of claims 1-5, and is characterized by comprising the following steps: The one-dimensional current signal processing unit is used for carrying out time sequence data mapping pretreatment on one-dimensional current signals acquired by the substation SCADA system through a Graham angle field algorithm to obtain a two-dimensional current map; The instruction fine tuning data set construction unit is used for constructing an instruction fine tuning data set based on the two-dimensional current map and combining a visible light/infrared image and a diagnosis text; The 1+N LoRA fine tuning framework design unit is used for designing a 1+N LoRA fine tuning framework comprising a current spectrum encoder and an electrical semantic optimization LoRA module on the basis of the universal multi-mode backbone model based on the instruction fine tuning data set to obtain an adjusted model; The self-verification reasoning framework construction unit is used for respectively outputting preliminary diagnosis probability vectors through a visual branch and a current expert branch based on the adjusted model, calculating cosine similarity of the two preliminary diagnosis probability vectors, triggering an anti-thinking logic if the cosine similarity is lower than a preset threshold or a physical paradox exists, adjusting the weight of the electrical semantic optimization LoRA module, recalculating a feature fusion result, and outputting a final diagnosis conclusion.
7. The multi-mode large-model fine tuning device for power inspection according to claim 6, wherein in the one-dimensional current signal processing unit, in the process of carrying out time sequence data mapping pretreatment on one-dimensional current signals through the gram angle field algorithm, the pretreatment process comprises self-adaptive normalization and polar coordinate transformation, gram angle and/or difference field generation and multi-scale GAF fusion.
8. The power inspection multimode large model fine tuning device according to claim 7, wherein in the 1+n lorea fine tuning framework design unit, a weight update formula of the 1+n lorea fine tuning framework is: ; In the formula, The updated weight; Original weights are pre-trained models; 、 For LoRA low-rank matrix transforms; a diagonal constraint matrix for power terminology.
9. The device for fine tuning a multi-mode large model for power inspection according to claim 8, wherein in the self-verification reasoning framework construction unit, the cosine similarity is calculated by the following formula: ; In the formula, Cosine similarity; Diagnosing a probability vector for the visual branch; the probability vector is diagnosed for the current expert branch.
10. The power inspection multimode big model fine tuning device according to claim 9, wherein in the self-verification reasoning framework construction unit, the expression of the final diagnosis conclusion is: ; In the formula, To the final diagnostic conclusion; The corrected feature fusion result is obtained; is an activation function; A decoder module that is a multimodal large language model; To trigger conflicting context instructions entered at the time of the miss logic.

Description

Electric power inspection multi-mode large model fine adjustment method and device Technical Field The invention relates to the technical field of power equipment detection and artificial intelligence, in particular to a method and a device for fine-tuning a multi-mode large model of power inspection. Background The electric power inspection is a core link for guaranteeing safe and stable operation of a power grid, and the core aim is to accurately identify defects of electric power equipment in time and prevent fault expansion. Along with the development of artificial intelligence technology, a single-mode automatic model is gradually applied to inspection scenes, such as a visual model based on CNN or a transducer, and the recognition of equipment surface defects is realized by analyzing visible light and infrared images, thereby playing an important role in the apparent anomaly detection of foreign matter attachment, shell damage, local overheating and the like. In recent years, the multi-Modal Large Language Model (MLLM) has a wide application prospect in the field of electric power inspection by virtue of strong multi-source data fusion capability, and provides possibility for comprehensive diagnosis of integrating multiple types of data such as vision, electric parameters and the like. On one hand, a single-mode model has obvious perception limitation, a visual mode can only capture external apparent characteristics of equipment, internal electrical problems without obvious visual characteristics such as transformer winding deformation, hidden faults of cables and the like cannot be perceived, a one-dimensional current data sequence acquired by a transformer substation at high frequency is overlong, a far-ultra-multi-mode large model processing window is too short, a pure numerical sequence lacks physical semantics and is difficult to be effectively understood by the model, on the other hand, the single-mode model is easily interfered by environmental factors such as sunlight refraction and the like, false hot spots and other 'visual illusions' are generated, and error diagnosis conclusion is caused by lack of electrical and physical constraints, and in addition, a natural gap exists between current data and visual images on data structure and semantic expression, an effective cross-mode alignment mechanism is lacked, so that multi-source data fusion fails, a 1+1>2 diagnosis effect cannot be realized, and high standard requirements of power grid operation on inspection precision and robustness are difficult to be met. Therefore, a method for fine tuning a multi-mode large model for power inspection is needed to solve the problems of insufficient diagnosis precision and the like caused by single-mode sensing limitation and environmental interference in the prior art. Disclosure of Invention Therefore, the invention provides a method and a device for fine-tuning a multi-mode large model for electric power inspection, which solve the problems that in the existing electric power inspection, single-mode sensing limitation is caused, current time sequence data is difficult to understand by the model, and the multi-mode data lacks an effective alignment mechanism and is easy to be interfered by environment, so that diagnosis precision is insufficient. In order to achieve the purpose, the invention provides the following technical scheme that the electric power inspection multi-mode large model fine adjustment method is characterized by comprising the following steps of: Carrying out time sequence data mapping pretreatment on one-dimensional current signals acquired by a substation SCADA system through a Graham angle field algorithm to obtain a two-dimensional current map; Based on the two-dimensional current map, combining a visible light/infrared image and a diagnosis text to construct an instruction fine adjustment data set; Based on the instruction fine tuning data set, designing a 1+N LoRA fine tuning framework comprising a current spectrum encoder and an electrical semantic optimization LoRA module on the basis of a universal multi-mode backbone model to obtain an adjusted model; based on the adjusted model, the preliminary diagnosis probability vectors are respectively output through the visual branch and the current expert branch, cosine similarity of the two preliminary diagnosis probability vectors is calculated, if the cosine similarity is lower than a preset threshold or a physical paradox exists, an anti-thinking logic is triggered, the weight of the electrical semantic optimization LoRA module is adjusted, the feature fusion result is recalculated, and a final diagnosis conclusion is output. As a preferable scheme of the electric power inspection multi-mode large model fine adjustment method, in the process of carrying out time sequence data mapping pretreatment on one-dimensional current signals through the Gellan angle field algorithm, the pretreatment process comprises self-adaptive normaliza