EP-4737846-A1 - ONLINE FULL DIMENSIONAL INSPECTION METHOD AND SYSTEM BASED ON MULTI-VIEW VISION

EP4737846A1EP 4737846 A1EP4737846 A1EP 4737846A1EP-4737846-A1

Abstract

An online full-dimension inspection method and system based on multi-view vision are provided. The method first determines correspondences between cameras in the camera array and the holes to be measured, performs three-dimensional reconstruction on all holes to be measured using a triangulation method based on the hole center coordinates and the correspondence to obtain three-dimensional coordinates of the holes to be measured in a workpiece coordinate system, and then transforms the three-dimensional coordinates of each hole to be measured from the workpiece coordinate system to a reconstructed measurement coordinate system and determines dimensional information of each hole to be measured in the measurement coordinate system. In this way, the three-dimensional coordinates of the holes to be measured are determined through joint calibration using the camera array, and the dimensional information of the holes to be measured is determined based on the measurement coordinate system and the three-dimensional coordinates of the holes to be measured, which achieves non-contact inspection, and is suitable for high-precision dimensional inspection of online inspection requirements on an assembly line, and the method has the characteristics of high efficiency, high precision, and high stability.

Inventors

LI, JI
CHEN, JIE
DENG, JUNJIE

Assignees

SpeedBot Robotics Co., Ltd

Dates

Publication Date: 20260506
Application Date: 20230904

Claims (9)

An online full-dimension inspection method based on multi-view vision, characterized by comprising: S1: acquiring images of holes to be measured on a workpiece to be inspected through a camera array; S2: detecting hole center coordinates of the holes to be measured, wherein the hole center coordinates are two-dimensional coordinates of each hole to be measured in each image; S3: saving the hole center coordinates in groups, and determining a correspondence between each camera in the camera array and the holes to be measured based on the hole center coordinates; S4: performing three-dimensional reconstruction on all of the holes to be measured using a triangulation method based on the hole center coordinates and the correspondence to obtain three-dimensional coordinates of the holes to be measured in a workpiece coordinate system; and S5: transforming the three-dimensional coordinates of each hole to be measured from the workpiece coordinate system to a reconstructed measurement coordinate system, and determining dimensional information of each hole to be measured in the measurement coordinate system.
The online full-dimension inspection method based on multi-view vision according to claim 1, characterized in that the S1 comprises: controlling the camera array to acquire an image of the workpiece to be inspected in a time-division manner under set general lighting and local lighting.
The online full-dimension inspection method based on multi-view vision according to claim 2, characterized in that the S2 comprises: generating, through projection, 2D template images of the holes to be measured under the camera array and template hole center coordinates corresponding to the holes to be measured based on intrinsic and extrinsic parameters of cameras in the camera array and CAD three-dimensional coordinates of the holes; obtaining edge feature point information from the template images; and cropping a region of interest, ROI image of each hole to be measured from the image of the workpiece to be inspected, performing edge extraction on the ROI image to obtain an edge image, extracting gradient information of each pixel from the edge image, and performing sliding window search and matching between the gradient information and the edge feature point information in the template images to obtain the hole center coordinates in the ROI image.
The online full-dimension inspection method based on multi-view vision according to claim 1, characterized in that the correspondence between each camera and the holes to be measured in the S3 refers to which holes can be captured by each camera and which cameras can capture each hole.
The online full-dimension inspection method based on multi-view vision according to claim 1, characterized in that the S4 comprises: determining optimized camera intrinsic parameters and optimized camera extrinsic parameters; and calculating the three-dimensional coordinates of the holes to be measured using the triangulation method based on the camera intrinsic parameters, the camera extrinsic parameters, and the correspondence, wherein the calculation formula satisfies the following relation: X = K R T x , where X represents reconstructed hole center coordinates after triangulation, K represents the camera intrinsic parameters, [ R | T ] represents the camera extrinsic parameters, and x represents the hole center coordinates.
The online full-dimension inspection method based on multi-view vision according to claim 1, characterized in that before the S5, the method further comprises: determining one datum plane and two datum holes of the workpiece to be inspected, and aligning the one datum plane and the two datum holes with theoretical coordinates according to a drawing requirement of an actual workpiece to be inspected to perform coordinate system establishment and obtain the measurement coordinate system.
The online full-dimension inspection method based on multi-view vision according to claim 6, characterized in that the S5 comprises: determining a position tolerance of each hole to be measured, wherein the position tolerance satisfies the following relation: position = 2 ∗ sqrt x 1 − x 0 2 + y 1 − y 0 2 + z 1 − z 0 2 , where position represents the position tolerance, reconstructed measurement coordinates are (x1, y1, z1), the theoretical coordinates are (x0, y0, z0), and sqrt represents the square root expression; and determining the dimensional information of each hole to be measured based on the three-dimensional coordinates of each hole to be measured in the workpiece coordinate system and the position tolerance.
An online full-dimension inspection system based on multi-view vision, characterized by comprising: a darkroom, a camera array, and a master control unit, wherein the camera array is communicatively connected to the master control unit, the camera array comprises a plurality of first cameras, a plurality of second cameras, and a plurality of third cameras, the plurality of first cameras are spaced apart from each other and arranged at top of the darkroom, the plurality of second cameras are spaced apart from each other and arranged at bottom of the darkroom, and the plurality of third cameras are spaced apart from each other on side walls of the darkroom in a circumferential direction of the darkroom; the camera array is configured to acquire images of holes to be measured on a workpiece to be inspected; and the master control unit is configured to: detect hole center coordinates of the holes to be measured, wherein the hole center coordinates are hole center coordinates of the holes to be measured under the cameras; save the hole center coordinates in groups, and determine a correspondence between each camera in the camera array and the holes to be measured based on the hole center coordinates; perform three-dimensional reconstruction on all of the holes to be measured based on the correspondence and a triangulation method, and determine three-dimensional coordinates of all holes to be measured in a workpiece coordinate system based on a three-dimensional reconstruction result and the hole center coordinates; and transform the three-dimensional coordinates of the holes to be measured in the workpiece coordinate system into dimensional information in a reconstructed measurement coordinate system.
The online full-dimension inspection system based on multi-view vision according to claim 8, characterized by further comprising a plurality of light sources arranged on inner walls of the darkroom at intervals.

Description

The present application claims priority to Chinese patent application No. 202310958248.8, titled "ONLINE FULL DIMENSIONAL INSPECTION METHOD AND SYSTEM BASED ON MULTI-VIEW VISION", filed on August 1, 2023, the entire content of which is incorporated herein by reference. TECHNICAL FIELD The present application relates to the field of multi-view vision dimensional measurement technologies, and in particular, to an online full-dimension inspection method and system based on multi-view vision. BACKGROUND As an indispensable part of industrial manufacturing, dimensional inspection can accomplish a wide range of measurement items, such as position tolerance, flatness, profile tolerance, linear distance, aperture, or the like, depending on specific inspection requirements. Currently, companies often rely on conventional measuring tools such as micrometers, vernier calipers, mating parts, gauge blocks, etc., for dimensional inspection. Alternatively, for some large and precision parts, inspection equipment like coordinate measuring machines or flexible arms are used for measurement. During the measurement, the structural information of holes is obtained to determine whether the measured parts meet specifications. The former mostly involves manual operation, requiring substantial manpower and resources to detect while offering relatively low inspection accuracy, making it unsuitable for high-precision measurement. The latter, although providing good inspection accuracy and repeatability, typically requires contact-based measurement at each measuring point, with inspection times generally calculated in hours, which is difficult to meet the production cycle of assembly lines and unsuitable for online measurement. Thus, the related art struggles to balance high-precision measurement with efficient inspection. SUMMARY The present application provides an online full-dimension inspection method and system based on multi-view vision. To achieve the above objectives, the present application is realized by the following technical solutions. In a first aspect, the present application provides an online full-dimension inspection method based on multi-view vision, including: S1: acquiring images of holes to be measured on a workpiece to be inspected through a camera array;S2: detecting hole center coordinates of the holes to be measured, wherein the hole center coordinates are two-dimensional coordinates of each hole to be measured in each image;S3: saving the hole center coordinates in groups, and determining a correspondence between each camera in the camera array and the holes to be measured based on the hole center coordinates;S4: performing three-dimensional reconstruction on all of the holes to be measured using a triangulation method based on the hole center coordinates and the correspondence to obtain three-dimensional coordinates of the holes to be measured in a workpiece coordinate system; andS5: transforming the three-dimensional coordinates of each hole to be measured from the workpiece coordinate system to a reconstructed measurement coordinate system, and determining dimensional information of each hole to be measured in the measurement coordinate system. Optionally, the S1 includes: controlling the camera array to acquire an image of the workpiece to be inspected in a time-division manner under set general lighting and local lighting. Optionally, the S2 includes: generating, through projection, 2D template images of the holes to be measured under the camera array and template hole center coordinates corresponding to the holes to be measured based on intrinsic and extrinsic parameters of cameras in the camera array and CAD three-dimensional coordinates of the holes;obtaining edge feature point information from the template images; andcropping a region of interest, ROI image of each hole to be measured from the image of the workpiece to be inspected, performing edge extraction on the ROI image to obtain an edge image, extracting gradient information of each pixel from the edge image, and performing sliding window search and matching between the gradient information and the edge feature point information in the template images to obtain the hole center coordinates in the ROI image. Optionally, the correspondence between each camera and the holes to be measured in the S3 refers to which holes can be captured by each camera and which cameras can capture each hole. Optionally, the S4 includes: determining optimized camera intrinsic parameters and optimized camera extrinsic parameters; andcalculating the three-dimensional coordinates of the holes to be measured using the triangulation method based on the camera intrinsic parameters, the camera extrinsic parameters, and the correspondence, wherein the calculation formula satisfies the following relation: X=KRTx,where X represents reconstructed hole center coordinates after triangulation, K represents the camera intrinsic parameters, [R|T] represents the camera extrinsic p