CN-121980452-A - Electromechanical intelligent inspection defect identification system and method based on multi-mode large model

CN121980452ACN 121980452 ACN121980452 ACN 121980452ACN-121980452-A

Abstract

The invention belongs to the technical field of industrial intelligent inspection, and discloses an electromechanical intelligent inspection defect recognition system and method based on a multi-mode large model, comprising the steps of acquiring multi-source multi-mode data, extracting multi-granularity characteristics and binding across modes to generate deduction metadata, and forming a cognitive meta-domain of global perception through verification; the method comprises the steps of generating a core calibration hypothesis and verifying an optimal solution by a linkage physical rule base, adding traceability information to update a cognitive element domain to form an optimized cognitive element domain and high-quality fusion visual data, generating a preliminary defect hypothesis through a multi-mode large model, combining double evidence verification of physical consistency and time sequence to generate a defect depth diagnosis report, scheduling an inspection agent group to select an optimal agent combination, generating an executable insight report of collaborative task planning and refined action instructions aiming at the optimal agent combination, acquiring recheck data by the optimal agent combination execution instruction, optimizing evidence weight, updating the physical rule base to form a closed loop iteration link.

Inventors

JIA JIANWEI
XIE SHUKUI
XIONG XUEFENG
LIN TAO
CHEN JUNXING
YIN XIAOJIN
Liang Pengjie

Assignees

郑州龙华机电工程有限公司

Dates

Publication Date: 20260505
Application Date: 20260115

Claims (10)

1. Electromechanical intelligent inspection defect identification system based on multi-mode large model, which is characterized by comprising: The multi-source fusion and authentication metadomain module is used for acquiring multi-source multi-mode data, performing multi-granularity feature extraction and cross-mode binding, generating deduction metadata by related semantics and context deduction, and forming a globally perceived cognitive metadomain by verification; The data calibration and cognition optimization module is used for generating a core calibration hypothesis and verifying an optimal solution based on a cognition meta-domain, adding traceability information to update the cognition meta-domain, and forming optimized cognition meta-domain and high-quality fusion visual data; The defect hypothesis and double evidence verification module is used for generating a preliminary defect hypothesis through a multi-mode large model based on the optimized cognitive element domain and the fusion visual data, and generating a defect depth diagnosis report by combining double evidence verification of physical consistency and time sequence; The intelligent agent scheduling module schedules the inspection intelligent agent group to select an optimal intelligent agent combination based on the defect depth diagnosis report and the inspection intelligent agent resource attribute, and generates an executable insight report of collaborative task planning and refined action instructions aiming at the optimal intelligent agent combination; and the rechecking verification and iteration module is used for collecting rechecking data based on the executable insight report and the optimal agent combination execution instruction, optimizing evidence weight, and updating a physical rule base to form a closed loop iteration link.
2. The multi-modal large model based electromechanical intelligent inspection defect recognition system of claim 1, wherein the manner of performing multi-granularity feature extraction comprises: synchronously acquiring multi-mode visual data, equipment operation data, environment data and inspection platform state data, and combining the multi-mode visual data, the equipment operation data, the environment data and the inspection platform state data into multi-source multi-mode data; And further extracting the overall quality characteristics of the image and the point cloud as global characteristics, extracting the geometric and texture characteristics of the inspection interest area and the equipment assembly area as regional characteristics, extracting the semantic tags of the preliminary classification as semantic characteristics, and integrating to form complete multi-granularity characteristics.
3. The multi-modal large model based electromechanical intelligent inspection defect recognition system of claim 2, wherein the means for forming a cognitive metadomain of global perception comprises: Based on the multi-granularity characteristics, generating a cross-modal binding group by spatially matching multi-modal characteristics of the same physical entity associated with the characteristic correlation; combining the semantic tag with the context data, deducting according to a preset rule to generate deduction metadata containing confidence, and further constructing a defect hypothesis; the method comprises the steps of taking a device assembly, a visual ROI, a defect hypothesis and a cross-modal binding group as core entities, fusing multi-source multi-modal data and deduction metadata, filling core attributes for the entities, and constructing association relations to form a knowledge network; and executing integrity check on the knowledge network to form a high-quality global perception cognitive metadomain.
4. The multi-modal large model based electromechanical intelligent inspection defect recognition system of claim 3, wherein the means for generating core calibration assumptions and verifying optimal solutions comprises: based on the cognitive element domain, extracting the equipment type, the running state and the environmental parameters to form a core working condition label, and calling a unique corresponding physical model from a preset physical rule base to generate a calibration hypothesis of f groups of focusing dominant influence factors and a calibration target; and verifying the matching degree of the calibration hypothesis through the spatial consistency and the characteristic correlation, screening an optimal solution with the matching degree meeting a threshold value, and generating an optimal calibration hypothesis confirmation list.
5. The multi-modal large model-based electromechanical intelligent inspection defect recognition system according to claim 4, wherein the means for forming the optimized cognitive metadomain and the high-quality fusion visual data comprises: Based on the optimal calibration hypothesis validation sheet, performing targeted data calibration on the multi-modal visual data for a calibration target, and generating high-quality fusion visual data; and (3) adding calibration traceability information in the cognitive meta-domain, and updating the corresponding field of the cognitive meta-domain to form an optimized cognitive meta-domain.
6. The multi-modal large model based electromechanical intelligent inspection defect recognition system of claim 5, wherein the means for generating preliminary defect hypotheses comprises: based on the optimized cognitive metadomain and fusion visual data, extracting image, point cloud characteristics and text auxiliary information to construct multi-mode input by combining deduction metadata, and fusing to form a unified multi-mode characteristic vector input multi-mode large model; generating a preliminary defect candidate result through multi-mode big model reasoning, carrying out semantic mapping by combining deduction metadata, and optimizing the confidence coefficient of the preliminary defect candidate result; And screening results with the confidence reaching standards and reasonable areas, and generating a preliminary defect hypothesis list.
7. The multi-modal large model based electromechanical intelligent inspection defect recognition system of claim 6, wherein the means for generating the defect depth diagnostic report comprises: based on the preliminary defect hypothesis list, calling a physical model of corresponding equipment, and simultaneously calling recent inspection historical data of the same equipment component, and constructing a double evidence body of physical consistency and time sequence; respectively checking deviation of the defect physical characteristics corresponding to the preliminary defect hypothesis and the normal parameter range and recent evolution trend of the defect to obtain a physical consistency matching degree and a time sequence evidence support degree, carrying out weighted fusion, and combining the confidence degree of the preliminary defect hypothesis to obtain a final defect confidence degree; and screening and reserving a preliminary defect hypothesis result with the final confidence reaching the standard, and generating a defect depth diagnosis report.
8. The multi-modal large model-based electromechanical intelligent inspection defect recognition system of claim 7, wherein the executable insight report generation manner includes: analyzing the requirements of a defect inspection task based on the defect depth diagnosis report by combining the resource attribute of the inspection agent; screening available agents matched with task demands according to the states of the patrol agent groups, generating a bidding scheme based on the states of the available agents, and selecting an optimal agent combination through global arbitration; And formulating a collaborative task plan and a refined action instruction aiming at the optimal agent combination, and integrating to generate an executable insight report.
9. The system for identifying the electromechanical intelligent patrol defect based on the multi-mode large model according to claim 8, wherein the mode of collecting recheck data, optimizing evidence weight and updating a physical rule base by the optimal agent combination execution instruction comprises: Based on the executable insight report, the optimal agent combination executes the refined action instruction according to the coordinated time sequence to acquire the recheck data, so as to form a high-quality recheck data set, and the high-quality recheck data set is used as a verification basis to evaluate the effectiveness of the double evidence bodies and optimize the double evidence weighted fusion weight; and combining the double evidence body evaluation and optimization results, correcting physical model parameters in the physical rule base, supplementing association rules of new working conditions and new defects, and reversely pushing the updated physical rule base and the double evidence body fusion weights to corresponding preamble links.
10. The electromechanical intelligent inspection defect identification method based on the multi-mode large model is realized based on the electromechanical intelligent inspection defect identification system based on the multi-mode large model as claimed in claims 1 to 9, and is characterized by comprising the following steps: s1, acquiring multi-source multi-mode data, performing multi-granularity feature extraction and cross-mode binding, performing associated semantic and context deduction to generate deduction metadata, and forming a globally perceived cognitive meta-domain through verification; s2, generating a core calibration hypothesis and verifying an optimal solution based on a cognitive element domain and a linkage physical rule base, and updating the cognitive element domain by adding traceability information to form optimized cognitive element domain and high-quality fusion visual data; s3, generating a preliminary defect hypothesis through a multi-mode large model based on the optimized cognitive element domain and the fusion visual data, and generating a defect depth diagnosis report by combining double evidence verification of physical consistency and time sequence; S4, scheduling an inspection agent group to select an optimal agent combination based on the defect depth diagnosis report and the inspection agent resource attribute, and generating an executable insight report of collaborative task planning and refined action instructions aiming at the optimal agent combination; and S5, based on the executable insight report, the optimal agent combination execution instruction collects recheck data, optimizes evidence weight, and updates a physical rule base to form a closed loop iteration link.

Description

Electromechanical intelligent inspection defect identification system and method based on multi-mode large model Technical Field The invention relates to the technical field of industrial intelligent inspection, in particular to an electromechanical intelligent inspection defect identification system and method based on a multi-mode large model. Background Industrial intelligent upgrading promotes wide application of electromechanical equipment, the requirements on inspection accuracy of the electromechanical equipment are increasingly improved, traditional manual inspection is low in efficiency, high in risk and easy to miss inspection, and large-scale fine requirements are difficult to meet. The short plates of most of the existing intelligent inspection technologies are insufficient in multi-mode data fusion depth, cannot effectively correlate different mode characteristics to exert complementary advantages, cannot comprehensively capture defect characteristics, causes insufficient defect recognition accuracy, is difficult to meet industrial inspection requirements, and restricts improvement and long-term adaptability of the defect recognition accuracy. Disclosure of Invention In order to overcome the defects in the prior art and achieve the purposes, the invention provides the following technical scheme that the electromechanical intelligent inspection defect identification system based on the multi-mode large model comprises the following components: The multi-source fusion and authentication metadomain module is used for acquiring multi-source multi-mode data, performing multi-granularity feature extraction and cross-mode binding, generating deduction metadata by related semantics and context deduction, and forming a globally perceived cognitive metadomain by verification; The data calibration and cognition optimization module is used for generating a core calibration hypothesis and verifying an optimal solution based on a cognition meta-domain, adding traceability information to update the cognition meta-domain, and forming optimized cognition meta-domain and high-quality fusion visual data; The defect hypothesis and double evidence verification module is used for generating a preliminary defect hypothesis through a multi-mode large model based on the optimized cognitive element domain and the fusion visual data, and generating a defect depth diagnosis report by combining double evidence verification of physical consistency and time sequence; The intelligent agent scheduling module schedules the inspection intelligent agent group to select an optimal intelligent agent combination based on the defect depth diagnosis report and the inspection intelligent agent resource attribute, and generates an executable insight report of collaborative task planning and refined action instructions aiming at the optimal intelligent agent combination; and the rechecking verification and iteration module is used for collecting rechecking data based on the executable insight report and the optimal agent combination execution instruction, optimizing evidence weight, and updating a physical rule base to form a closed loop iteration link. Further, the method for extracting the multi-granularity characteristics comprises the following steps: synchronously acquiring multi-mode visual data, equipment operation data, environment data and inspection platform state data, and combining the multi-mode visual data, the equipment operation data, the environment data and the inspection platform state data to form multi-source multi-mode data; And further extracting the overall quality characteristics of the image and the point cloud as global characteristics, extracting the geometric and texture characteristics of the inspection interest area and the equipment assembly area as regional characteristics, extracting the semantic tags of the preliminary classification as semantic characteristics, and integrating to form complete multi-granularity characteristics. Further, the forming the cognitive metadomain of the global perception includes: Based on the multi-granularity characteristics, generating a cross-modal binding group by spatially matching multi-modal characteristics of the same physical entity associated with the characteristic correlation; combining the semantic tag with the context data, deducting according to a preset rule to generate deduction metadata containing confidence, and further constructing a defect hypothesis; the method comprises the steps of taking a device assembly, a visual ROI, a defect hypothesis and a cross-modal binding group as core entities, fusing multi-source multi-modal data and deduction metadata, filling core attributes for the entities, and constructing association relations to form a knowledge network; and executing integrity check on the knowledge network to form a high-quality global perception cognitive metadomain. Further, the means for generating the core calibration hypothesis and verifying