CN-122019812-A - Bridge member information retrieval method and system based on computer vision and BIM

CN122019812ACN 122019812 ACN122019812 ACN 122019812ACN-122019812-A

Abstract

The invention discloses a bridge component information retrieval method and system based on computer vision and BIM, and relates to the technical field of bridge engineering digitization, wherein the method comprises the steps of obtaining image data of a bridge to be retrieved, inputting the image data into a computer vision cascade model to obtain class labels and appearance defect characteristics of candidate components; the method comprises the steps of associating candidate components with components in a BIM model of a bridge to be searched, screening out target components through preset screening rules, extracting structured data, constructing a search request based on the structured data, category labels of the target components and appearance defect characteristics, inputting the search request into a preset vector knowledge base, obtaining unstructured knowledge segments related to the search request, fusing the structured data and the unstructured knowledge segments, inputting the fused structured data and unstructured knowledge segments into a large language model, and generating a comprehensive decision report. According to the invention, the decision report is generated through the closed loop of visual identification, BIM association and knowledge retrieval, so that the retrieval efficiency and the decision accuracy are improved.

Inventors

HUANG FEI
MEI DAPENG
YANG CANWEN
SU CHUANHAI
Wen Zihao

Assignees

中铁大桥勘测设计院集团有限公司

Dates

Publication Date: 20260512
Application Date: 20260410

Claims (10)

1. The bridge component information retrieval method based on computer vision and BIM is characterized by comprising the following steps of: obtaining image data of a bridge to be searched, and inputting the image data into a computer vision cascade model to obtain class labels and appearance defect characteristics of candidate components; Associating the candidate components with components in a BIM model of the bridge to be retrieved, screening target components corresponding to the candidate components from the associated components through preset screening rules, and extracting structural data of the target components; Constructing a search request based on the structured data, the category labels and the appearance defect characteristics of the candidate components corresponding to the target components, inputting the search request into a preset vector knowledge base, and obtaining unstructured knowledge segments related to the search request; Fusing the structured data and the unstructured knowledge segments, inputting the fused structured data and the unstructured knowledge segments into a large language model, and generating a comprehensive decision report containing target component state description, compliance analysis and treatment suggestions; wherein the computer vision cascade model includes a component classification model and a defect detection model.
2. The method for retrieving information of bridge components based on computer vision and BIM according to claim 1, wherein the step of obtaining image data of the bridge to be retrieved includes: collecting multi-angle high-definition image data covering a bridge to be searched, wherein the image data comprises a high-definition video stream and a static picture; Binding each frame of the high-definition video stream and each static picture with spatial position information and attitude angle parameters at the acquisition time, wherein the spatial position information comprises longitude and latitude coordinates and elevation values; And transmitting the image data carrying the spatial position information and the attitude angle parameters to an edge calculation server or carrying out local storage through wireless communication.
3. The method for retrieving bridge component information based on computer vision and BIM according to claim 2, wherein inputting the image data into a computer vision cascade model to obtain a class label of a candidate component includes: extracting a single frame image from the image data or extracting frames from the video stream to obtain an image, and carrying out pretreatment of adjusting the obtained image to a fixed resolution and normalizing pixel values, and organizing the image into a batch tensor format; carrying out multi-scale feature extraction and fusion on the preprocessed image by adopting a component classification model, carrying out bounding box regression and class prediction based on the fused features, and outputting parameters of a plurality of candidate components, wherein the parameters comprise bounding box coordinates, class labels and confidence scores of the candidate components in the image; Filtering parameters of a plurality of candidate components according to a first preset confidence threshold, reserving candidate components with confidence scores higher than the first preset confidence threshold, and outputting class labels of the filtered candidate components as class labels of the plurality of candidate components.
4. The method for retrieving information of bridge components based on computer vision and BIM according to claim 3, wherein inputting the image data into a computer vision cascade model to obtain the appearance defect feature of the candidate component includes: Cutting out candidate component areas corresponding to the boundary frames in the image data based on the boundary frame coordinates of each candidate component in the image to obtain local images of each candidate component; preprocessing for adjusting each local image to a fixed resolution and normalizing pixel values, and organizing the local images into a batch tensor format; calling a special defect detection model corresponding to the category of each candidate component from a model library according to the category label of each candidate component; Performing defect detection on the local image of each candidate member by adopting a special defect detection model of each candidate member, and outputting defect parameters of a plurality of candidate members, wherein the defect parameters comprise defect types, defect positions and defect confidence scores of each candidate member; And filtering defect parameters of the plurality of candidate components according to a second preset confidence threshold, reserving candidate components with confidence scores higher than the second preset confidence threshold, and outputting the defect types and defect positions of the filtered candidate components as appearance defect characteristics of the plurality of candidate components.
5. The method for retrieving information of bridge components based on computer vision and BIM according to claim 4, wherein the component classification model adopts a target detection model trained based on YOLOv models, and the special defect detection models all adopt defect detection models trained based on YOLOv models.
6. The method for retrieving bridge component information based on computer vision and BIM according to claim 2, wherein the associating the candidate component with the component in the BIM model of the bridge to be retrieved, screening the target component corresponding to the candidate component from the associated component by a preset screening rule, and extracting the structured data of the target component includes: According to the category labels of the candidate components and the spatial position information of the images of the candidate components, positioning all BIM components with the same category as the current candidate components in a BIM model of the bridge to be searched through a preset mapping relation, and establishing an association relation; calculating the space distance between the candidate component and each BIM component associated with the candidate component, and screening out a BIM component with the minimum Euclidean distance and lower than the preset distance threshold as a target component based on the preset distance threshold; And acquiring the unique component identifier of the target component, accessing a database associated with the BIM model, and extracting the structured data of the target component according to the unique component identifier.
7. The method for retrieving information of bridge components based on computer vision and BIM according to claim 6, wherein the locating all BIM components with the same category as the current candidate component in the BIM model of the bridge to be retrieved according to the category label of the candidate component and the spatial position information of the image thereof through the preset mapping relation includes: According to the category labels of the candidate components and the spatial position information and attitude angle parameters when the images are acquired, obtaining the direction vector of the candidate components under the camera coordinate system through the direct geographic reference principle of photogrammetry; The direction vector is converted into three-dimensional geographic coordinates under a WGS-84 coordinate system through rigid transformation of coordinate rotation and translation; according to the coordinate conversion parameters of the bridge, converting the three-dimensional geographic coordinates into three-dimensional space coordinates under a bridge local engineering coordinate system adopted by the BIM model of the bridge to be searched; determining a space partition of the current candidate component in the BIM according to the preset mapping relation and the three-dimensional space coordinate; and positioning all BIM components which are the same as the category of the current candidate component in the space partition corresponding to the current candidate component according to the category label of the current candidate component.
8. The method for retrieving bridge component information based on computer vision and BIM according to claim 6, wherein the constructing a retrieval request based on the structured data, the class label and the appearance defect feature of the candidate component corresponding to the target component, inputting the retrieval request into a preset vector knowledge base, and obtaining an unstructured knowledge segment related to the retrieval request includes: Combining the category labels of the candidate components, the appearance defect characteristics and the structural data of the target components to generate query sentences in a natural language format; converting the query sentence into a query vector by using a text vector embedding model, inputting the query vector into a preset vector knowledge base, and calculating the similarity between the query vector and each knowledge segment vector stored in the preset vector knowledge base; selecting at least one knowledge segment with similarity higher than a preset threshold value as unstructured knowledge segments related to the retrieval request to be output according to the sequence of the similarity from high to low; Wherein the unstructured knowledge segments include industry specification terms, technical manual content, and historical engineering cases.
9. The method for retrieving bridge component information based on computer vision and BIM according to claim 8, wherein the fusing the structured data and the unstructured knowledge segments and inputting the fused data and the unstructured knowledge segments into a large language model to generate a comprehensive decision report including target component state descriptions, compliance analysis and treatment suggestions includes: constructing an input prompt word of a large language model based on the structured data and the unstructured knowledge segments; Inputting the input prompt words into a large language model, and generating a comprehensive decision report containing target component state description, compliance analysis and treatment suggestion by the large language model; the structural data comprise componentID, design bearing capacity, material, construction and maintenance records of the bridge to be searched.
10. The bridge component information retrieval system based on computer vision and BIM is characterized by comprising: The acquisition module is used for acquiring image data of the bridge to be searched, and inputting the image data into the computer vision cascade model to obtain class labels and appearance defect characteristics of candidate components; The association module is used for associating the candidate components with the components in the BIM model of the bridge to be retrieved, screening target components corresponding to the candidate components from the associated components through a preset screening rule, and extracting structural data of the target components; The retrieval module is used for constructing a retrieval request based on the structured data, the category labels and the appearance defect characteristics of the candidate components corresponding to the target components, inputting the retrieval request into a preset vector knowledge base and obtaining unstructured knowledge segments related to the retrieval request; The generation module is used for inputting the structured data and the unstructured knowledge segments into a large language model after fusing, and generating a comprehensive decision report containing target component state description, compliance analysis and treatment suggestion; wherein the computer vision cascade model includes a component classification model and a defect detection model.

Description

Bridge member information retrieval method and system based on computer vision and BIM Technical Field The application relates to the technical field of bridge engineering digitization, in particular to a bridge member information retrieval method and system based on computer vision and BIM. Background In the field of traditional bridge engineering, the query and management of component information have the problems of low efficiency and data splitting for a long time. The method comprises the following core pain points that an engineer needs to manually review paper drawings, electronic accounts or scattered databases, retrieval flow is complex and takes long time, data splitting is serious, unstructured knowledge such as field detection data (e.g. photos), BIM model structured data, industry standards and historical cases and the like is stored in a scattered mode, an effective association link is lacking, collaborative calling cannot be achieved, retrieval efficiency and accuracy are low, under complex working conditions (e.g. high altitude, shielding and backlight), manual identification components are prone to error, traditional retrieval results are single data and cannot be combined with full-dimension knowledge to form decision support, emergency response is delayed, when sudden disasters or faults occur, time is long for manually retrieving component key information (e.g. bearing capacity and reinforcing scheme), and real-time requirements of emergency repair are difficult to meet. Therefore, an intelligent retrieval technology capable of opening up on-site visual information, model structured data and industry knowledge base is needed. Disclosure of Invention The application provides a bridge member information retrieval method and system based on computer vision and BIM, which can solve the technical problems of low efficiency and insufficient decision support caused by the fact that the traditional bridge information retrieval in the prior art depends on manual work and data fracture. In a first aspect, an embodiment of the present application provides a bridge member information retrieval method based on computer vision and BIM, where the bridge member information retrieval method based on computer vision and BIM includes: obtaining image data of a bridge to be searched, and inputting the image data into a computer vision cascade model to obtain class labels and appearance defect characteristics of candidate components; Associating the candidate components with components in a BIM model of the bridge to be retrieved, screening target components corresponding to the candidate components from the associated components through preset screening rules, and extracting structural data of the target components; Constructing a search request based on the structured data, the category labels and the appearance defect characteristics of the candidate components corresponding to the target components, inputting the search request into a preset vector knowledge base, and obtaining unstructured knowledge segments related to the search request; Fusing the structured data and the unstructured knowledge segments, inputting the fused structured data and the unstructured knowledge segments into a large language model, and generating a comprehensive decision report containing target component state description, compliance analysis and treatment suggestions; wherein the computer vision cascade model includes a component classification model and a defect detection model. With reference to the first aspect, in an implementation manner, the acquiring image data of the bridge to be retrieved includes: collecting multi-angle high-definition image data covering a bridge to be searched, wherein the image data comprises a high-definition video stream and a static picture; Binding each frame of the high-definition video stream and each static picture with spatial position information and attitude angle parameters at the acquisition time, wherein the spatial position information comprises longitude and latitude coordinates and elevation values; And transmitting the image data carrying the spatial position information and the attitude angle parameters to an edge calculation server or carrying out local storage through wireless communication. With reference to the first aspect, in an implementation manner, inputting the image data into a computer vision cascade model to obtain a class label of the candidate component includes: extracting a single frame image from the image data or extracting frames from the video stream to obtain an image, and carrying out pretreatment of adjusting the obtained image to a fixed resolution and normalizing pixel values, and organizing the image into a batch tensor format; carrying out multi-scale feature extraction and fusion on the preprocessed image by adopting a component classification model, carrying out bounding box regression and class prediction based on the fused features, and outputtin