CN-122023301-A - Industrial assembly quality detection planning method based on engineering drawing understanding

CN122023301ACN 122023301 ACN122023301 ACN 122023301ACN-122023301-A

Abstract

An industrial assembly quality detection planning method based on engineering drawing understanding belongs to the field of industrial assembly quality detection and intelligent manufacturing. The invention solves the problems of low efficiency and poor accuracy of the existing industrial assembly quality detection method. The invention provides a CAD model-based skeleton template view generation, engineering drawing structure analysis, annotation information mapping association and view matching method based on a graph neural network, wherein the topological association of two-dimensional annotation information and three-dimensional CAD model elements of an engineering drawing can be established according to view matching results, and high-precision matching of two-dimensional and three-dimensional information is realized. The invention can automatically check whether each installation part reaches the tolerance precision required by the engineering drawing in the assembly process, greatly reduces the workload of manual measurement and comparison of the engineering drawing, improves the assembly quality detection efficiency and reliability, and ensures the accuracy of assembly quality detection through high-precision matching of two-dimensional and three-dimensional information. The method can be applied to industrial assembly quality detection.

Inventors

LIU GUODONG
CHEN GUANHUA
CHEN FENGDONG
LU BINGHUI
ZOU LU

Assignees

哈尔滨工业大学

Dates

Publication Date: 20260512
Application Date: 20260121

Claims (10)

1. The industrial assembly quality detection planning method based on engineering drawing understanding is characterized by comprising the following steps of: Firstly, reading a three-dimensional CAD model of a part to be detected, and respectively blanking and projecting the three-dimensional CAD model to each determined projection direction to generate each framework template; backtracking the projection edges in the framework template, and establishing mapping from the projection edges in the framework template to the three-dimensional CAD model; step two, extracting each sub-view in the engineering drawing, and executing the step three to the step six for each extracted sub-view; Step three, for the first From sub-view, from Extracting all identifiers from the sub-views, and extracting the first The personal marks are marked as , , The representation is from the first The total number of the identifiers extracted from the sub-views; and respectively determining the target line pointed by each mark, and then carrying out the first step Personal identification The indicated target line is marked as ; Step four, selecting and removing Skeleton templates matched with the sub-views; step five, mapping the projection edges in the skeleton template selected in the step four to a three-dimensional CAD model, and carrying out the step four The target lines pointed by the marks in the sub-views are related to the faces and edges in the three-dimensional CAD model; and step six, calculating the spatial pose of the surface and the edge associated in the step five, and obtaining the sensor observation pose according to the calculated spatial pose so as to realize industrial assembly quality detection.
2. The industrial assembly quality detection planning method based on engineering drawing understanding according to claim 1, wherein the blanking projection is performed on the three-dimensional CAD model to each determined projection direction, so as to generate each skeleton template, specifically: Step one, sampling is carried out on the surface of the three-dimensional CAD model to obtain a sampling point set ; Wherein, the Represent the first The three-dimensional coordinates of the individual sampling points, Representing the total number of sampling points; step two, calculating the mean value and covariance matrix of all sampling points: Wherein: Representing the mean; representing a covariance matrix; upper corner mark Representing a transpose; Step one, three, pair covariance matrix Performing feature decomposition to obtain principal axis unit vector ; Step one, constructing a projection direction set , ; Step one and five, to the first The blanking projections are performed in the respective projection directions, Generate to the first Two-dimensional wire frame set after projection in each projection direction 。
3. The industrial assembly quality detection planning method based on engineering drawing understanding according to claim 2, wherein backtracking is performed on projection edges in a framework template, and mapping from the projection edges in the framework template to a three-dimensional CAD model is established, specifically: Step six, traversing the B-Rep model data topology of the three-dimensional CAD, and establishing an adjacent index of the three-dimensional edge and the three-dimensional surface, namely for any three-dimensional edge , Is set of adjacent faces of (a) The method comprises the following steps: step seven, is a collection Line cells within Assigning unique identifiers , According to the result of step one, the line primitive Backtracking and establishing line primitives Mapping relation with three-dimensional CAD model : Wherein: representing line primitives A set of corresponding edges in the three-dimensional CAD model; representing line primitives A set of facets in the three-dimensional CAD model; representing line primitives The type of (2); and eighthly, combining the collinear and the co-circular line primitives, namely ensuring that the collinear and the co-circular line primitives correspond to the same identifier, and obtaining a mapping relation corresponding to the identifier.
4. The industrial assembly quality detection planning method based on engineering drawing understanding according to claim 3, wherein each sub-view in the engineering drawing is extracted, specifically: analyzing the engineering drawing to obtain a line set consisting of all line segments and curves in the engineering drawing ; Step two, setting a line width threshold value Then according to the line width threshold value Pair aggregation The lines in the (3) are screened out to meet the requirements of Lines of (2) , Representation lines Is used for the line width of the (c) line, ; The set of the screened lines is recorded as ; Step two, three, pair collection The lines in the tree are clustered, and the tree is obtained according to the clustering result A connected cluster, the first The connected clusters are marked as , ; Will be the first The area corresponding to the circumscribed rectangle of each connected cluster is taken as the first area Sub-view area, will be The sub-view is marked as ; And respectively extracting the wire frame set of each sub-view, and putting the first The collection of wireframes of the sub-view is denoted as 。
5. The industrial assembly quality detection planning method based on engineering drawing understanding according to claim 4, wherein the determining the target line pointed by each identifier respectively specifically comprises: In the first place Personal identification The following are examples: Step III, from the first Wire frame set of sub-view Candidate lines meeting both distance constraint and direction consistency constraint are screened out ; Step three, selecting marks from all candidate lines through geometric consistency The target line is referred to.
6. The industrial assembly quality inspection planning method based on engineering drawing understanding according to claim 5, wherein the distance constraint in the third step is: Identification mark Arrow point of (2) Coordinate to candidate line Distance of (2) Or a logo Is a line candidate and a line candidate Distance of (2) ; Wherein: And A set distance threshold; Indicating the arrow point; Representing arrow points to candidate lines Is a distance of (2); Representation identifier Is a line candidate and a line candidate Is a distance of (2); The direction consistency constraint in the third step is that the identification is carried out Is a candidate for the line direction of the lead wire Included angle of direction ; Wherein: indicating a set angle threshold; Representation identifier Is a candidate for the line direction of the lead wire The included angle of the direction.
7. The industrial assembly quality detection planning method based on engineering drawing understanding of claim 6, wherein the specific process of the third step is as follows: Wherein: Representation identifier The target line that refers to; Representation identifier Is a candidate line set; Is a weight coefficient.
8. The industrial assembly quality detection planning method based on engineering drawing understanding of claim 7, wherein the specific process of the fourth step is as follows: step four, one, the first Wire frame set of sub-views built as a band attribute map structure : Wherein: Represent the first Vertex sets in the individual sub-views; Represent the first A set of edges in the individual sub-views; Will be the first Wire frame set of individual skeleton templates is constructed into a structure with an attribute map , : Wherein: Represent the first Vertex sets in the individual skeleton templates; Represent the first A set of edges in a skeleton template; Step four, two, will map the structure As the input of the first GINE graph neural network, the output of the first GINE graph neural network sequentially passes through the first attention pooling layer and the first MLP, and the first MLP outputs the first image Features of the sub-view; structure of picture As the input of the second GINE graph neural network, the output of the second GINE graph neural network sequentially passes through the second attention pooling layer and the second MLP, and the first is output through the second MLP Features of the individual skeleton templates; step IV, respectively calculating the characteristics and the fourth of each framework template The similarity of the features of the sub-views, taking the skeleton template corresponding to the maximum similarity as the first template And a skeleton template matched with the sub-views.
9. The industrial assembly quality detection planning method based on engineering drawing understanding of claim 8, wherein the working process of GINE-figure neural network is as follows: step four, two, one and initializing layer number ; Step IV, two by two, initialize the first Vertex characteristics of layer output: Wherein: Represent the first Vertex of layer output Is characterized by (2); Represent the first Vertex of layer output Is characterized by (2); representing vertices And a vertex Edge features in between; And Is a learnable function; representing vertices Is a neighbor vertex set of (1); Represent the first Layer output node Is a neighborhood aggregation message; Represent the first Vertex of layer output Is characterized by (2); judging whether the method meets the following conditions , Representing the total number of layers; If it meets Obtaining node characteristics of GINE graph neural network output ; If it does not meet Order in principle And returning to the fourth step and the second step.
10. The industrial assembly quality detection planning method based on engineering drawing understanding according to claim 9, wherein the specific process of the step six is as follows: Step six, calculating a conversion matrix from a part coordinate system to be detected to a detection robot base coordinate system ; Step six, marking the pose of the surface or the line associated in the step five under the coordinate system of the part to be detected as According to the conversion matrix Pose the position Converting to a detection robot base coordinate system to obtain the pose of the associated surface or line under the detection robot base coordinate system ; Sixthly, observing the pose according to the expected sensor Obtaining a transformation matrix from the detection robot base coordinate system to the industrial camera coordinate system : Wherein, the Representation of An inverse matrix of (a); Step six, four, according to the transformation matrix Projecting the associated surface or line onto the image plane of the industrial camera to obtain a projection point : Wherein, the Representation of Is used for the matching of the coordinate system, Is a matrix of units which is a matrix of units, For a matrix of 0's, Is a camera internal reference matrix; Step six, five, according to the projection point And setting a region of interest (ROI) in an image plane of an industrial camera with the actual scale of the part to be detected, and performing assembly quality detection in the region of the ROI.

Description

Industrial assembly quality detection planning method based on engineering drawing understanding Technical Field The invention belongs to the field of industrial assembly quality detection and intelligent manufacturing, and particularly relates to an industrial assembly quality detection planning method based on engineering drawing understanding. Background The traditional industrial assembly quality detection method generally relies on manual reading of engineering drawings, and plans the measurement position, the visual angle and the qualification threshold according to the understanding of the engineering drawings by manpower, so that the detection efficiency is low, the subjectivity is strong, and the large-scale multiplexing is difficult to realize. Even though the pose matching method based on the CAD model exists, most of the output poses are integral poses, it is still difficult to automatically acquire whether a certain position mark on the engineering drawing corresponds to which face or side of the three-dimensional model, a specific observation position of a detection site, a specific allowed deviation and the like, so that the detection accuracy is still poor, and in fact, the root cause of the problems is that a computable semantic and topology mapping link is lacking between the view, mark, cross-section information and the three-dimensional model of the two-dimensional engineering drawing. Therefore, how to relate the two-dimensional view and the labeling information in the mechanical engineering drawing with the corresponding three-dimensional CAD model, and plan the high-precision and automatic detection method of the industrial assembly quality based on the robot vision means, so as to improve the efficiency and accuracy of the industrial assembly quality detection is an urgent need at present. Disclosure of Invention The invention aims to solve the problems of low efficiency and poor accuracy of the existing industrial assembly quality detection method, and provides an industrial assembly quality detection planning method based on engineering drawing understanding. The technical scheme adopted by the invention for solving the technical problems is as follows: An industrial assembly quality detection planning method based on engineering drawing understanding specifically comprises the following steps: Firstly, reading a three-dimensional CAD model of a part to be detected, and respectively blanking and projecting the three-dimensional CAD model to each determined projection direction to generate each framework template; backtracking the projection edges in the framework template, and establishing mapping from the projection edges in the framework template to the three-dimensional CAD model; step two, extracting each sub-view in the engineering drawing, and executing the step three to the step six for each extracted sub-view; Step three, for the first From sub-view, fromExtracting all identifiers from the sub-views, and extracting the firstThe personal marks are marked as,,The representation is from the firstThe total number of the identifiers extracted from the sub-views; and respectively determining the target line pointed by each mark, and then carrying out the first step Personal identificationThe indicated target line is marked as; Step four, selecting and removingSkeleton templates matched with the sub-views; step five, mapping the projection edges in the skeleton template selected in the step four to a three-dimensional CAD model, and carrying out the step four The target lines pointed by the marks in the sub-views are related to the faces and edges in the three-dimensional CAD model; and step six, calculating the spatial pose of the surface and the edge associated in the step five, and obtaining the sensor observation pose according to the calculated spatial pose so as to realize industrial assembly quality detection. Further, blanking projection is performed on the three-dimensional CAD model in each determined projection direction to generate each framework template, which specifically comprises the following steps: Step one, sampling is carried out on the surface of the three-dimensional CAD model to obtain a sampling point set ; Wherein, the Represent the firstThe three-dimensional coordinates of the individual sampling points,Representing the total number of sampling points; step two, calculating the mean value and covariance matrix of all sampling points: Wherein: Representing the mean; representing a covariance matrix; upper corner mark Representing a transpose; Step one, three, pair covariance matrix Performing feature decomposition to obtain principal axis unit vector; Step one, constructing a projection direction set,; Step one and five, to the firstThe blanking projections are performed in the respective projection directions,Generate to the firstTwo-dimensional wire frame set after projection in each projection direction。 Further, backtracking is performed on the projec