CN-121561464-B - Port facility management and maintenance large model expertise base data labeling method and system

CN121561464BCN 121561464 BCN121561464 BCN 121561464BCN-121561464-B

Abstract

The invention provides a method and a system for labeling data in a major knowledge base of a large management and maintenance model of a port facility. And correcting and marking by using an expert collaborative auditing platform to obtain standardized and structured marking data of the harbor facility management and maintenance professional knowledge base. The method comprises the steps of acquiring data through a multi-mode sensing network, generating a three-dimensional damage body through an algorithm, solving the problem of labeling of traditional single-mode data, constructing and fine-adjusting an intelligent labeling model, realizing accurate association identification, overcoming the semantic gap of a general model, combining expert platform correction and labeling to form a quantitative evaluation closed loop, reducing labor cost, improving consistency, realizing knowledge reasoning, guiding labeling, settling field rules, improving high-difficulty sample processing efficiency and meeting the management and maintenance core requirements of port facilities through structural conversion and association index construction.

Inventors

WANG WEI
SHANG DONGFANG
WANG YONGCHAO
ZHANG JINHAO
QIN LIU
YANG DONGYUAN
HAN XUE

Assignees

交通运输部天津水运工程科学研究所

Dates

Publication Date: 20260508
Application Date: 20260121

Claims (10)

1. The method for labeling the data of the professional knowledge base of the large management and maintenance model of the port facilities is characterized by comprising the following steps: Deploying a multi-mode sensing network at a port facility to acquire structural damage images, vibration sensor data, temperature stress monitoring data and maintenance log text data, so as to form a time-space synchronous multi-source heterogeneous data set; constructing a port digital twin three-dimensional model based on the multi-source heterogeneous data set, and mapping a damage detection result in the structural damage image into the port digital twin three-dimensional model by using a point cloud registration algorithm and a semantic segmentation model to generate a three-dimensional damage body with attribute labels; constructing an intelligent annotation model based on the three-dimensional injury body, and performing instruction fine adjustment and parameter efficient fine adjustment by utilizing a corpus in the port field to form the intelligent annotation model with multi-mode understanding capability; Automatically marking the facility type, the damage type and the maintenance requirement by utilizing the multi-mode understanding capability of the intelligent marking model, and carrying out semiautomatic marking correction and uncertainty marking by combining an expert collaborative auditing platform to obtain standardized and structured marking data of a harbor facility management professional knowledge base; quantitatively evaluating the labeling result by adopting consistency verification, integrity check and field suitability verification to obtain an evaluation result; Based on the evaluation result, returning unqualified annotation data to the intelligent annotation model for secondary fine tuning training, and supplementing annotation rules and domain knowledge constraints by combining manual correction opinions of an expert collaborative auditing platform for secondary annotation; And carrying out structural conversion on the qualified annotation data according to the entity-relation-attribute triplet specification of the knowledge graph, and establishing an association index with the port facility management and maintenance large model.
2. The method for labeling data in a specialized knowledge base of a large management model for port facilities as claimed in claim 1, wherein the step of generating a three-dimensional lesion body with attribute labels comprises the steps of: based on the multi-source heterogeneous data set, laser point cloud data, unmanned aerial vehicle aerial survey orthographic images and BIM model basic frames, constructing an initial three-dimensional model by utilizing a three-dimensional reconstruction algorithm and a multi-view stereo matching algorithm; Performing point cloud rough registration by adopting a RANSAC algorithm and an FPFH feature descriptor based on the initial three-dimensional model, and performing large-scale space alignment by calculating an initial transformation matrix of source point cloud and target point cloud in the initial three-dimensional model to obtain a rough registration result; performing point cloud fine registration by utilizing an ICP algorithm based on the coarse registration result to obtain a fine registration result; Identifying a damaged area in the structural damage image by adopting a semantic segmentation model based on the fine registration result, and outputting a pixel level segmentation mask and a damage type; and converting the pixel coordinates of the image into three-dimensional model coordinates through camera calibration parameters based on the segmentation mask, and injecting dynamic attributes and static attributes to generate a three-dimensional damage body with attribute labels.
3. The method for labeling data in a specialized knowledge base of a large harbor facility management model according to claim 2, wherein the expression of the initial transformation matrix for calculating the source point cloud and the target point cloud in the initial three-dimensional model is as follows: , , , ; Wherein, the The transformation matrix is represented by a representation of the transformation matrix, Representing the orthogonal rotation matrix of the matrix, Representing the displacement of the source point cloud centroid to the target point cloud centroid, Representing a1 x 3 zero vector, for the panning term filling of the homogeneous transformation matrix, 、 And Representing covariance matrix Is used for the SVD decomposition result of (1), The transpose of the representation vector is performed, Representing the post-synchronization target point Yun Yangben centroid, Representing the centroid of the source point Yun Yangben after synchronization, Represents the corresponding pair number of points used to estimate the transformation matrix, Representing the origin of the center of mass removed after synchronization, Representing the post-synchronization center-of-mass target point.
4. The method for labeling data in a specialized knowledge base of a large management model for port facilities according to claim 1, wherein the forming of the intelligent labeling model with multi-modal understanding capability in step S3 comprises: based on the three-dimensional injury body, a convolutional neural network is used as a basic model, and a three-dimensional space encoder, an attribute reasoning module and an uncertainty quantification unit are added into the basic model to form an initial intelligent labeling model; constructing a port field corpus based on the initial intelligent labeling model, and preprocessing the port field corpus to obtain a training data set, wherein the port field corpus comprises maintenance logs, labeling images and expert experience texts; Designing a three-level instruction template comprising identification, classification and reasoning on the initial intelligent annotation model based on the training data set, performing instruction fine adjustment through supervision fine adjustment flow, and performing layered training by adopting an optimizer to generate a fine adjustment intelligent annotation model; adding a low-rank adapter to a weight matrix bypass of the fine-tuning intelligent annotation model, and carrying out parameter fusion on the low-rank adapter and an activation function through two full-connection layers to obtain an intelligent annotation model after parameter fusion; and verifying and optimizing the intelligent annotation model based on the parameter fusion by adopting a four-dimensional index system to obtain the intelligent annotation model with multi-mode understanding capability, wherein the four-dimensional index system comprises accuracy, recall rate, F1 value and reasoning speed.
5. The method for labeling data in a large harbor facility management model expertise base according to claim 1, wherein obtaining standardized, structured harbor facility management expertise base labeling data comprises: automatically labeling the multi-mode sensing network real-time acquired multi-source heterogeneous data based on the intelligent labeling model to obtain a preliminary three-dimensional attribute labeling result comprising a facility type, a damage type and a maintenance requirement; Correcting the preliminary three-dimensional attribute labeling result by using an expert collaborative auditing platform to form semiautomatic correction labeling data; Based on the semi-automatic correction marking data, performing confidence assessment on the marking result by adopting an uncertainty quantization module, marking a low confidence region and generating an uncertainty label; and combining the uncertainty label to manually recheck the low-confidence-coefficient region, and optimizing a decision boundary of the intelligent labeling model through an expert knowledge supplementing rule base to obtain standardized and structured labeling data of the harbor facility management and maintenance expert knowledge base.
6. The method for labeling data in a major knowledge base of a port facility management model according to claim 1, wherein obtaining the evaluation result comprises: Detecting whether the labeling attribute has conflict or abnormal fluctuation or not by comparing labeling data of the same facility at different time points based on the labeling result by utilizing a consistency check rule, and generating a consistency check report; Carrying out integrity check on the labeling results, verifying whether each labeling entity contains complete attribute information, including facility type, damage type and maintenance requirement, and marking labeling data with missing attributes to obtain an integrity check result; Constructing a field suitability verification model based on a port field knowledge base, evaluating whether the labeling data accords with the field specification or not by calculating the semantic similarity between the labeling result and the field knowledge, and outputting a field suitability score; And integrating the consistency check report, the integrity check result and the field suitability score to form a quantitative evaluation result, and classifying the evaluation result into three labeling data of qualification, correction and disqualification.
7. The method for labeling data in a specialized knowledge base of a large management model of a harbor facility according to claim 1, wherein the steps of structurally converting the labeled data qualified for evaluation according to a three-element specification of entity-relation-attribute of a knowledge graph, and establishing an associated index with the large management model of the harbor facility comprise: Based on the qualified annotation data, carrying out structural conversion according to the entity, relation and attribute triples, generating facility entity, damaged entity and core relation, and forming a triples data set conforming to the knowledge graph specification; storing the data set based on the triples in an entity-relation nested dictionary form to obtain a knowledge graph storage structure; And generating a triplet unique identifier by utilizing a hash algorithm based on the knowledge graph storage structure, and constructing an association index containing a time stamp and source information based on the unique identifier and a port facility management and maintenance model.
8. The method for labeling data in a large-scale specialized knowledge base of a port facility management model according to claim 1, further comprising: Constructing an expert knowledge graph based on expert experience; And carrying out knowledge reasoning guiding labeling based on the expert knowledge graph when the intelligent labeling model is labeled.
9. The method for labeling data in a large-scale expertise base of a port facility management model according to claim 8, wherein constructing an expert knowledge graph based on expert experience comprises: collecting expert experience documents in the port facility maintenance field, including historical maintenance records, fault diagnosis reports and technical manuals, and forming an original data set; Preprocessing the original data set, extracting key entities and relations, identifying domain terms and association rules thereof by adopting a natural language processing technology, and generating a preliminary knowledge graph frame; Carrying out semantic enhancement by utilizing a graph embedding algorithm based on the preliminary knowledge graph framework, optimizing the relation weight among entities, and carrying out multiple iterative verification to obtain a verified expert knowledge graph; And injecting the newly added expert experience and the real-time labeling result into the expert knowledge graph for updating.
10. A system for labeling data in a major knowledge base of a large management model of port facilities, the system comprising: A processor; A memory for storing processor-executable instructions; Wherein the processor is configured to implement the harbor facilities management model expertise base data tagging method of any one of claims 1 to 9 when executing the executable instructions.

Description

Port facility management and maintenance large model expertise base data labeling method and system Technical Field The invention relates to the technical field of data annotation, in particular to a method and a system for annotating data in a major knowledge base of a management and maintenance model of a port facility. Background The current data labeling in the field of port facility management mainly depends on a mode of combining manual experience with basic automation tools. In the traditional method, single-mode data (such as structural damage images or vibration sensor data) are adopted for independent analysis in structural damage detection, and the damage types are manually identified and maintenance requirements are marked. Some automated systems attempt to integrate multi-source data (e.g., images, sensors, maintenance logs), but stay at the data simple stitching level, lacking depth fusion and semantic association capabilities. For example, the three-dimensional model construction generally only realizes geometric mapping, does not effectively integrate dynamic attributes (such as damage evolution trend) and static attributes (such as facility material characteristics), and the intelligent labeling model mostly adopts a general architecture, does not conduct instruction fine adjustment and parameter optimization aiming at port field characteristics, so that multi-modal understanding capability is limited, and three-dimensional attribute association of facility types, damage types and maintenance requirements is difficult to accurately identify. Disclosure of Invention The invention aims at least solving the technical problem that three-dimensional attribute association of facility types, damage types and maintenance requirements are difficult to accurately identify in the prior art, and particularly creatively provides a method and a system for labeling data of a major knowledge base of a management and maintenance large model of port facilities. In order to achieve the above object of the present invention, the present invention provides a method for labeling data in a major knowledge base of a large management model of a harbor facility, the method comprising: S1, deploying a multi-mode sensing network at a port facility to acquire structural damage images, vibration sensor data, temperature stress monitoring data and maintenance log text data, so as to form a time-space synchronous multi-source heterogeneous data set; S2, constructing a port digital twin three-dimensional model based on the multi-source heterogeneous data set, and mapping a damage detection result in the structural damage image into the port digital twin three-dimensional model by using a point cloud registration algorithm and a semantic segmentation model to generate a three-dimensional damage body with attribute labels; s3, constructing an intelligent annotation model based on the three-dimensional injury body, and performing instruction fine adjustment and parameter efficient fine adjustment by utilizing a port field corpus to form the intelligent annotation model with multi-mode understanding capability; S4, automatically marking the facility type, the damage type and the maintenance requirement by utilizing the multi-mode understanding capability of the intelligent marking model, and carrying out semiautomatic marking correction and uncertainty marking by combining an expert collaborative auditing platform to obtain standardized and structured marking data of a harbor facility management professional knowledge base; s5, quantitatively evaluating the labeling result by adopting consistency verification, integrity check and field suitability verification to obtain an evaluation result; s6, returning unqualified annotation data to the intelligent annotation model for secondary fine adjustment training based on the evaluation result, and supplementing annotation rules and domain knowledge constraints by combining manual correction opinions of the expert collaborative auditing platform for secondary annotation; and S7, carrying out structural conversion on the qualified annotation data according to the entity-relation-attribute triplet specification of the knowledge graph, and establishing an association index with the port facility management and maintenance model. In another aspect, the invention also provides a system for labeling data in a major knowledge base of a large management model of port facilities, which comprises: A processor; A memory for storing processor-executable instructions; the processor is configured to implement the harbor facility management large model expertise base data labeling method when executing the executable instructions. The method has the advantages that a structure damage image, vibration sensor data, temperature stress monitoring data and maintenance log text data are collected through a multi-mode sensing network to form a space-time synchronous multi-source heterogeneous data set, a