CN-121982488-A - Management method and system for detecting, identifying and tracing hidden dangers of dykes and dams

CN121982488ACN 121982488 ACN121982488 ACN 121982488ACN-121982488-A

Abstract

The invention discloses a dam hidden danger detection, identification and traceability management method and system, and belongs to the field of engineering intelligent detection and operation management. The method comprises the following steps of rapidly detecting hidden dangers of the dam, detecting hidden dangers in the dam and recording appearance structures, performing data standardization processing on images acquired in the detection process, reserving corresponding waveform channel positioning information through ID association, and establishing a linear relation with pixels of the images. And then adopting a multi-cascade multi-mode large model to identify hidden dangers, wherein the multi-mode large model comprises a hidden danger and noise background field intelligent identification multi-mode large model, a hidden danger attention time sequence multi-mode large model under the noise background field, and finally realizing hidden danger processing result visualization, hidden danger three-dimensional visualization and hidden danger traceability visualization. The method effectively solves the problem of inconsistent traceability of hidden danger data and engineering ontology in space-time scale and precision, and generates more comprehensive and accurate dam health state portraits.

Inventors

MING PAN
ZHANG XIANGWEI
HUANG TIANCHENG
LI HAO
WEI WEI
LU JUN
NIU XIN
YUAN QUAN
DONG MAOGAN
WANG BEI
Wu Jingshang
WANG SIYAO
LI HONGLIANG

Assignees

水利部交通运输部国家能源局南京水利科学研究院
深圳市赛盈地空技术有限公司

Dates

Publication Date: 20260505
Application Date: 20260113

Claims (10)

1. A dam hidden danger detection, identification and tracing management method is characterized by comprising the following steps: the method comprises the following steps of (1) rapidly detecting hidden danger of a dam, and simultaneously detecting hidden danger inside the dam and recording appearance structures in the detection process, wherein an internal hidden danger detection image and an appearance recording image share positioning information, and each waveform is endowed with one piece of positioning information in the internal hidden danger detection process, and the positioning information is associated through an ID; Step (2), carrying out data standardization processing on the images acquired in the detection process, and outputting standard-size result pictures, wherein corresponding waveform channel positioning information is reserved through ID association in the processing process of each result picture, and a linear relation is established between the waveform channel positioning information and pixels of the images; step (3), transmitting the result pictures of standard sizes and text information of positioning results to a hidden danger intelligent recognition window through an embedded RTP (real-time protocol), recognizing hidden danger by adopting a multi-cascade multi-mode large model, wherein the hidden danger intelligent recognition window comprises a hidden danger and noise background field intelligent recognition multi-mode large model, a hidden danger attention time sequence multi-mode large model and a hidden danger background field intelligent recognition multi-mode large model; And (4) displaying the result in each sub-window of the hidden danger visualization platform window in various functionalities, wherein the visualization sub-windows comprise a hidden danger processing result visualization window, a hidden danger three-dimensional visualization window and a hidden danger tracing window.
2. The method for detecting, identifying and tracing hidden dangers of a dam according to claim 1, wherein in the step (2), the waveform channel positioning information and the pixels of the image establish a linear relationship, and the following formula is shown: , Wherein, the The method is characterized in that the method is represented as a waveform positioning information and image pixel mapping function set, P represents a pixel matrix in a single image, L represents a positioning information matrix obtained by the same image waveform, i, j is a pixel point in a standard image, x, y is an actual positioning point, and LT is a starting point positioning of engineering field acquisition.
3. The dam hidden danger detection, identification and tracing management method according to claim 1 is characterized in that in the step (3), hidden danger and noise background fields intelligently identify a multi-mode large model, a data set with a multi-source background field is screened and prepared based on a result of synchronous acquisition of dam appearance mechanism records and hidden danger in the dam in the acquisition process; The double-flow transducer architecture comprises a visual depth feature extraction branch, a semantic text feature extraction branch and four continuous full-connection blocks, wherein the two branches extract different features from respective input modes, space related information is extracted from semantic text table space data, deep feature extraction is carried out from image vision, and the architecture of the visual depth feature extraction branch and the semantic text feature extraction branch comprises output shapes and parameter numbers; The visual depth feature extraction branch provides detailed hidden danger images, the semantic text feature extraction branch provides valuable text information, features are sent to a classifier with cross, the capability of detecting complex modes by utilizing visual and spatial data complementarity is enhanced, finally, based on feature fusion, an initial discrimination result is output, standard picture result samples are classified, whether electromagnetic field noise background exists in the images is judged, and noiseless image standard processing results are directly transmitted to a result visualization window through FTP and RTP transmission protocols and are stored.
4. The method for detecting, identifying and tracing dam hidden danger according to claim 3, wherein the visual depth feature extraction branch is adopted, the EFFICIENTNET-B0 feature extractor is adopted, the EFFICIENTNET-B0 comprises eight stages, each stage of the moving reverse convolution MBConv block is adopted to gradually extract deeper features, the overall resolution performance is improved, and meanwhile, the calculation load is reduced; The text definition feature description is provided in the semantic text feature extraction branch, and geographic information is provided systematically by using a geographic information system tool, wherein text information features comprise data such as spatial positions, shape features, energy features, frequency features and geographic information types, and finally the information is compiled into a table format and input into a feature extractor, wherein the semantic text feature extraction branch is divided into four stages, the sizes of middle layers are respectively 64, 128, 256 and 512 neurons, each stage is composed of a completely connected layer, and then a ReLU activation function and a dropout layer are connected.
5. The dam hidden danger detection, identification and tracing management method according to claim 1 is characterized by constructing a hidden danger intelligent identification multi-mode big model under a noise background field, introducing a multi-mode dynamic weight attention guiding module and a weighted key value attention guiding mechanism based on the hidden danger and the noise background field, selecting an image processing result containing background noise to intelligently identify hidden danger of an confusion area, focusing attention on a key area of the confusion area of noise and hidden danger in the identification process, fusing visual characteristics and spatial characteristics, and enhancing hidden danger identification of the confusion area; Based on hidden danger identification results overlapped and confused in a multi-mode large model output image result under a noise background field, hidden danger in a picture result is classified, specific positions of hidden danger in the image and feature description of text output hidden danger are delineated, the feature description comprises semantic description such as noise background, hidden danger definition, spatial feature and position information, and finally, visual and text results of hidden danger identification are transmitted to a result visualization window through FTP and RTP transmission protocols, and information of hidden danger features is stored.
6. The method for detecting, identifying and tracing hidden dangers of a dam according to claim 1, wherein in a multi-modal large model for intelligent hidden dangers identification under a noise background field, visual features are captured from standard result images by a ViT module, a [ CLS ] method is adopted for space morphological semantic feature vectors, roBERTa is adopted for extracting space feature semantic information of hidden dangers from unstructured texts, then outputs of RoBERTa and ViT are linearly transformed into the same dimension, layerNorm is adopted for standardization, and QWEN-VL is utilized for generating fused multi-modal vectors, wherein the following formula is adopted: , Wherein, X t and X v represent the same dimensional features generated by RoBERTa and ViT, respectively, MLLM is a QWEN 2-VL multimodal large model; When QWEN-VL fuses the features of visual and language information, in order to guide hidden danger features of a concerned confusion zone, a fusion module adopts a weighted fusion strategy to dynamically adjust the contribution ratio of two components of the visual and semantic information: , Where α is a weight parameter embedded by the semantic model for controlling its contribution to the final fusion result, β is a weight parameter embedded by the dual stream model, w t and w v are weight parameters of text and image features, respectively, and w V is a projection matrix corresponding to the image modality.
7. The method for detecting, identifying and tracing the hidden danger of the dam according to claim 1, wherein the hidden danger attention time sequence multi-mode big model adopts a knowledge storage mechanism memory_bank to store results of different time nodes, and then adopts a causal convolution and expansion convolution mechanism to realize cross-mode time domain embedding fusion; The knowledge storage mechanism memory_Bank is characterized in that when the hidden danger state of a current or predicted future time node is calculated, the memory_Bank and a multi-stage convolution path are fused in parallel, a text-image crossing time is extracted from two aspects of semantic and structural modes respectively, then in a text-image conversion layer, the front N main entity characterizations which are stored in the memory_Bank according to the current K value and V value and the historical K value and V value and are sequenced according to the current attention weight are searched to be used as candidate Memory entries, the contextual window information of the memory_Bank is reserved, the convolution path is adopted to model each local temporal continuity of the history, the memory_Bank is jointly encoded, a sliding window structural mode of dynamic depth is adopted by the memory_Bank, different window depths are defined according to different task states, and after the memory_Bank is called, the image characteristics are distinguished And text features Mapping to unified key value projection matrixes W v and W t , and enhancing the continuity of a causal chain, wherein the following formula is as follows: , where M v represents memory_bank of the image modality, M t represents memory_bank of the text modality, and N is the depth of memory_bank.
8. The method for detecting, identifying and tracing hidden danger of a dam according to claim 7, wherein the mechanism of causal convolution of the hidden danger attention time sequence multi-mode large model is that the hidden danger attention time sequence multi-mode large model is input through current and previous records, and then the result of the current time step is output after convolution; The extended convolution mechanism expands receptive fields when the structure is simplified through interval sampling, enhances modeling capability of long time intervals, overcomes the defect of large-span causal convolution time, and for a given input sequence X= { X 1 ,x 2 ,….,x T }∈R T×d , hidden layer output characteristics H l ∈R T×d are transmitted layer by layer through a multi-stage time convolution unit, and the mathematical expression is as follows: , where T represents the time step, d represents the feature dimension, For the input of the i-th sequence, For the convolution kernel weight matrix, For the convolution kernel size, For the nonlinear activation function ReLU, For bias, s is the current time step index.
9. The dam hidden danger detection, identification and tracing management method according to claim 10, wherein a learnable expansion factor allocation strategy is designed by dynamically adjusting K and d, the expansion rate of each layer is dynamically adjusted, the long-process causal modeling capability is enhanced while local details are maintained, interaction operation is performed through mapping matrixes of two modes of text and images, so that a fusion key value space after the mode alignment is constructed, a K value and a V value of each mode are respectively obtained, and finally a fused K fusion value and a V fusion value are obtained according to the fusion strategy: , , wherein K fusion and V fusion are different time nodes cross-modal fused vectors; And finally, taking K fusion and V fusion fused with the current and historical characteristics as inputs, executing attention calculation, adopting residual connection and normalization of an output layer to output a final judging result, transmitting the judging result to a result visualization window based on FTP and RTP transmission protocols, and storing hidden danger information and positioning information.
10. A dyke hidden danger detects, discerns, management system who traces to source, its characterized in that includes: The dam hidden danger detection module is used for simultaneously detecting hidden danger inside the dam and recording an appearance structure in the dam detection process, wherein the internal hidden danger detection image and the appearance recording image share positioning information, and each waveform is endowed with one piece of positioning information in the internal hidden danger detection process and is associated through an ID; The dam image preprocessing module is used for carrying out data standardization processing on the images acquired in the detection process and outputting result pictures with standard sizes, wherein corresponding waveform channel positioning information is reserved through ID association in the processing process of each result picture, and a linear relation is established with pixels of the images; The dam hidden danger identification module is used for transmitting a result picture of a standard size and text information of a positioning result to the hidden danger intelligent identification window through an embedded RTP transmission protocol, adopting a multi-cascade multi-mode large model to identify hidden danger, wherein the multi-mode large model comprises a hidden danger and noise background field intelligent identification multi-mode large model, a hidden danger attention time sequence multi-mode large model under a noise background field; The dam hidden danger identification display tracing module is used for displaying various functionalities of results on each sub-window of the hidden danger visualization platform window, wherein the visualization sub-windows comprise a hidden danger processing result visualization window, a hidden danger three-dimensional visualization window and a hidden danger tracing window.

Description

Management method and system for detecting, identifying and tracing hidden dangers of dykes and dams Technical Field The invention belongs to the field of engineering intelligent detection and operation management, and particularly relates to a dam hidden danger detection, identification and traceability management method and system. Background In the face of severe climate change of global short-time extremely heavy rainfall and drought and waterlogging, watershed floods with over-history numbers are more and more frequent, and the dam is used as a first defense line for protecting rivers, lakes and seas, so that the responsibility is great. However, the dam has long history, long line length, wide range and alternate soil structure, and point type hidden dangers are randomly distributed, so that the safe operation management task of the dam is difficult, and particularly the defense in flood season is realized. Conventional dike management relies on periodic inspection by personnel, and with the development of geophysical exploration technology and the application of land-based, water-based and aerospace equipment, the inspection of dikes is gradually developed to be rapid, automatic and intelligent. However, the operation and maintenance modes of the conventional dykes mainly depend on manual decision making, even if various technical equipment provides abundant dykes hidden danger information, the quick discrimination, tracing and decision making of hidden danger still need to be decided by personnel in the professional field at present, so that the operation and maintenance efficiency of a large amount of current detection data is low, the history information management automation and the intelligent rate of each element of the dykes are low, and the conditions of improper maintenance of the dykes in the later period, missed detection of key information, inaccurate flood season decision information and the like are easily caused. At present, aiming at the management of hidden dangers of a dam, a large amount of unstructured operation and maintenance data, such as historical inspection records, monitoring data, multi-source detection data and the like are independently archived, are not effectively converted into reusable domain knowledge, and limit the optimization of an operation and maintenance knowledge system, so that potential data value loss is caused. The judgment of hidden danger still depends on systematic accumulation of field expert experience, however, although expert systems can provide conventional experience storage, the workload is great and the efficiency is high. Therefore, various neural networks and deep learning models appear, but most of the existing models depend on image visual information, face complex electromagnetic environments of city embankments, are easy to cause confusion of hidden danger features, embed hidden danger feature information, ignore hidden danger knowledge semantics and image morphology information, often lead to lack of comprehensive and rich information of the constructed models, seriously influence construction quality of operation and maintenance knowledge graphs, further influence efficiency and quality of operation and maintenance work based on the knowledge graphs, and enable hidden danger identification accuracy to be low. Moreover, the center of gravity of the current research is on the identification of dam hidden danger, so that the dynamic association visual management requirement of dam entities in a flood season multimedia environment is difficult to adapt, and the visual hidden danger information in a media environment is traced to the source, so that association is more accurately established with the long-distance dam entities, and great challenges are still faced. Disclosure of Invention Aiming at the problems existing in the prior art, the invention provides a dam hidden danger detection, identification and tracing management method, which realizes hidden danger processing result visualization, hidden danger three-dimensional visualization and hidden danger tracing visualization by constructing a cloud-side-end cooperative system architecture by taking data and artificial intelligence as driving. In order to solve the technical problems, the invention provides a dam hidden danger detection, identification and tracing management method, which comprises the following steps: the method comprises the following steps of (1) rapidly detecting hidden danger of a dam, and simultaneously detecting hidden danger inside the dam and recording appearance structures in the detection process, wherein an internal hidden danger detection image and an appearance recording image share positioning information, and each waveform is endowed with one piece of positioning information in the internal hidden danger detection process, and the positioning information is associated through an ID; Step (2), carrying out data standardization processing on the images acq