CN-122015651-A - Optical lens installation detection method and system based on sensor

CN122015651ACN 122015651 ACN122015651 ACN 122015651ACN-122015651-A

Abstract

The invention relates to the technical field of optical lens detection, in particular to a sensor-based optical lens installation detection method and a sensor-based optical lens installation detection system, by arranging an image sensor device, a lens mounting station and a calibration device in an optical lens detection scene, and arranging a checkerboard calibration reference plate and a plurality of fixed reference balls in fixed positions in the calibration device. And acquiring calibration information in the camera by setting up a detection datum point, and then calculating and storing three-dimensional datum coordinates of the camera under a workpiece coordinate system by combining pixel coordinates of the fixed datum sphere. After the optical lens calibration process with the set number is completed, the verification coordinates of the fixed reference ball are calculated, and whether the image sensor device or the checkerboard calibration reference plate is subjected to position deviation is judged by comparing the reference coordinates with the verification coordinates, so that the problems of calibration data failure, inaccurate detection results and misjudgment of inferior products/superior products caused by fine position deviation of the image sensor device or the checkerboard calibration reference plate in the prior art are solved.

Inventors

CHEN ZHONGFEI
HE MINGLIN
YE ZHENXIANG
DU JIANGTING
Jiang Shuaitian
XIONG KUN

Assignees

深圳市通用核心光电有限公司

Dates

Publication Date: 20260512
Application Date: 20260416

Claims (10)

1. A sensor-based optical lens installation inspection method for identifying whether a positional shift has occurred in an image sensor device or a checkerboard calibration reference plate used in an optical lens inspection application, comprising the steps of: step S1, providing an optical lens detection scene, and setting an image sensor device, a lens mounting station and a calibration device in the optical lens detection scene, wherein the calibration device comprises a checkerboard calibration reference plate and a plurality of fixed reference balls which are arranged on the periphery of the checkerboard calibration reference plate and are fixed in position; Step S2, setting up a detection datum point on the checkerboard calibration reference plate, and acquiring camera internal calibration information in the image sensor device based on the detection datum point, wherein the camera internal calibration information comprises internal parameters of an internal reference matrix and distortion coefficients and external parameters comprising a rotation matrix and a translation vector; S3, acquiring an image containing a plurality of fixed reference balls, and identifying and obtaining pixel coordinates of each fixed reference ball from the image containing the plurality of fixed reference balls; S4, calculating three-dimensional coordinates of each fixed reference ball under a workpiece coordinate system by using pixel coordinates of each fixed reference ball and camera internal calibration information, and taking the three-dimensional coordinates as reference coordinates; Step S5, after the set number of optical lens calibration procedures are completed, calculating three-dimensional coordinates of each fixed reference ball in the workpiece coordinate system after the set number of optical lens calibration procedures are completed by using pixel coordinates of each fixed reference ball and camera internal calibration information, and taking the three-dimensional coordinates as verification coordinates; and S6, comparing the reference coordinates with the verification coordinates, obtaining a comparison result, and judging whether the image sensor device or the checkerboard calibration reference plate is subjected to position deviation or not according to the comparison result.
2. The method according to claim 1, wherein the step S2 specifically comprises: step S21, using a checkerboard calibration reference plate and setting angular points on the checkerboard calibration reference plate as detection datum points; Step S22, acquiring images of the checkerboard calibration reference plate under different poses by using an industrial camera, and identifying detection datum points in each acquired image of the checkerboard calibration reference plate by using a checkerboard angular point detection function in an OpenCV library to obtain corresponding pixel coordinates; Step S23, solving internal parameters comprising an internal reference matrix and distortion coefficients and external parameters comprising a rotation matrix and a translation vector by using a least square method by using pixel coordinates corresponding to detection datum points in each image of the checkerboard calibration reference plate.
3. The method according to claim 1, wherein the step S3 specifically comprises: step S31a, acquiring images containing a plurality of fixed reference balls; And step S32a, identifying the edge contour of each fixed reference ball in the image containing a plurality of fixed reference balls by adopting a Canny edge detection algorithm, and calculating the pixel coordinates of the center of each fixed reference ball by adopting a least square circle fitting algorithm based on the edge contour of each fixed reference ball.
4. The optical lens mounting inspection method according to claim 3, further comprising, in the step S32 a: and correcting the edge profile of each fixed reference sphere by using a Zernike moment fitting method.
5. The method according to claim 1, wherein the step S4 specifically comprises: Step S41a, pixel coordinates of each fixed reference sphere are used for calculating by combining the internal calibration information of the camera, and the pixel coordinates of each fixed reference sphere are converted into three-dimensional coordinates of each fixed reference sphere under a camera coordinate system; Step S42a, recognizing and obtaining the position of each fixed reference ball in the image and the posture of the camera according to the pixel coordinates of each fixed reference ball and external parameters in the internal calibration information of the camera; And step S43a, adopting a triangulation method, reversely calculating to obtain the three-dimensional coordinates of each fixed reference ball in the workpiece coordinate system according to the positions of each fixed reference ball in the image and the camera gestures and the three-dimensional coordinates of each fixed reference ball in the camera coordinate system, and taking the three-dimensional coordinates as the reference coordinates.
6. The method according to claim 1, wherein the step S3 specifically comprises: and acquiring continuous multi-frame images containing a plurality of fixed reference balls, and respectively identifying the images containing the plurality of fixed reference balls in each frame to obtain pixel coordinates of each fixed reference ball in the images containing the plurality of fixed reference balls in each frame.
7. The method according to claim 6, wherein the step S4 specifically includes: Step S41b, calculating by using pixel coordinates of each fixed reference sphere in the image containing a plurality of fixed reference spheres in each frame and combining internal calibration information of the camera, and converting the pixel coordinates of each fixed reference sphere in the image containing a plurality of fixed reference spheres in each frame into three-dimensional coordinates of each fixed reference sphere in the camera coordinate system in the image containing a plurality of fixed reference spheres in each frame; Step S42b, respectively identifying and obtaining the position and the camera posture of each fixed reference ball in the corresponding image in each frame of the images containing the fixed reference balls according to the pixel coordinates of each fixed reference ball in the images containing the fixed reference balls and the external parameters in the internal calibration information of the camera; Step S43b, adopting a triangulation method, and reversely calculating to obtain the three-dimensional coordinates of each fixed reference ball in the workpiece coordinate system in each frame of images containing a plurality of fixed reference balls according to the positions of each fixed reference ball in the corresponding images and the camera gestures of each frame of images containing a plurality of fixed reference balls in combination with the three-dimensional coordinates of each fixed reference ball in the camera coordinate system in each frame of images containing a plurality of fixed reference balls; Step S44b, extracting coordinates of three-dimensional coordinates of each fixed reference sphere in the workpiece coordinate system in each frame of image containing a plurality of fixed reference spheres according to a one-to-one correspondence relation of the plurality of fixed reference spheres in each frame of image, and obtaining an X coordinate value sequence, a Y coordinate sequence and a Z coordinate sequence of each fixed reference sphere; Step S45b, applying median filtering processing to the X coordinate value sequence, the Y coordinate sequence, and the Z coordinate sequence for each fixed reference sphere and extracting the median value to obtain the X coordinate value, the Y coordinate, and the Z coordinate for each fixed reference sphere, and taking the obtained X coordinate value, Y coordinate, and Z coordinate for each fixed reference sphere as the reference coordinates.
8. The method according to claim 1, wherein the step S6 specifically includes: Step S61, calculating Euclidean distance between the three-dimensional coordinates of each fixed reference sphere under the workpiece coordinate system and the reference coordinates after the completion of the optical lens calibration process with set quantity, and structurally storing the Euclidean distance to form a distance data set; And step S62, analyzing the distance data set to obtain an analysis result of whether the image sensor device or the checkerboard calibration reference plate is subjected to position deviation, wherein when each numerical value in the distance data set is larger than a set distance threshold value, the result of whether the image sensor device or the checkerboard calibration reference plate is subjected to position deviation is obtained.
9. The optical lens mounting inspection method according to claim 1, further comprising, after the step S6: And S7, when judging that the position of the image sensor device or the checkerboard calibration reference plate is offset, acquiring feedback information according to the comparison result, wherein the feedback information comprises a recommended lens detection stop indication, a recommended image sensor device recalibration indication or abnormal occurrence time providing and deviation data indicated by using a fixed reference ball.
10. A sensor-based optical lens installation detection system applied to an optical lens detection scene in which an image sensor device, a lens installation station, and a calibration device are disposed, wherein the calibration device includes a checkerboard calibration reference plate and a plurality of fixed reference balls arranged on the periphery of the checkerboard calibration reference plate and fixed in position, characterized by comprising: A camera internal calibration information acquisition module for acquiring camera internal calibration information in the image sensor device based on a detection reference point by setting up the detection reference point on the checkerboard calibration reference plate, the camera internal calibration information including internal parameters of an internal reference matrix and a distortion coefficient and external parameters including a rotation matrix and a translation vector; the first pixel coordinate acquisition module is used for acquiring images containing a plurality of fixed reference balls and identifying the pixel coordinates of each fixed reference ball from the images containing the plurality of fixed reference balls; The reference coordinate acquisition module is used for calculating three-dimensional coordinates of each fixed reference sphere under a workpiece coordinate system by using pixel coordinates of each fixed reference sphere and camera internal calibration information, and taking the three-dimensional coordinates as reference coordinates; The verification coordinate acquisition module is used for calculating three-dimensional coordinates of each fixed reference sphere under a workpiece coordinate system after the set number of optical lens calibration procedures are completed by using pixel coordinates of each fixed reference sphere and camera internal calibration information, and taking the three-dimensional coordinates as verification coordinates; And the judging and analyzing module is used for comparing the reference coordinates with the verification coordinates, obtaining a comparison result and judging whether the image sensor device or the checkerboard calibration reference plate is subjected to position deviation or not according to the comparison result.

Description

Optical lens installation detection method and system based on sensor Technical Field The application relates to the technical field of optical lens detection, in particular to a sensor-based optical lens installation detection method and a sensor-based optical lens installation detection system. Background In precision optical device manufacturing, precise mounting of optical lenses is critical. The conventional detection method is difficult to meet the requirement of modern industry on high precision, so that a detection system based on a sensor is commonly adopted in the industry to evaluate the installation quality of an optical lens accurately in real time. Such systems typically include an image sensor device, a lens mounting station, and a calibration device, wherein the calibration device comprises a checkerboard calibration reference plate. The system determines the internal calibration information of the camera of the image sensor device, including internal parameters (such as internal reference matrix and distortion coefficient) and external parameters (such as rotation matrix and translation vector), by image acquisition and analysis of the checkerboard calibration reference plate, which are the basis for all subsequent accurate measurements. However, in an actual industrial production environment, the image sensor device or the checkerboard calibration reference plate may undergo a fine positional shift which is hardly noticeable to the naked eye due to an unexpected collision or maintenance operation. Such an offset may cause the camera internal calibration information of the image sensor device to no longer accurately reflect its true spatial pose, disabling the mathematical transformation relationship that the device relies on to make accurate measurements. When the image sensor device or the checkerboard calibration reference plate is shifted, systematic inaccuracy can occur in the conversion from the image point location to the actual space location by the device according to old and failed calibration data, and it is difficult to distinguish between the qualified component and the unqualified component. In view of the above problems, no effective technical solution is currently available. Disclosure of Invention The application aims to solve the technical problems in the background art, and provides a sensor-based optical lens installation detection method and a sensor-based optical lens installation detection system, which are used for solving the technical problems that in the manufacturing of precision optical equipment, an image sensor device or a checkerboard calibration reference plate is subjected to fine position deviation caused by accidental collision or maintenance operation, so that calibration data is invalid, and further defective products/superior products are misjudged. In order to solve the problems, the scheme of the application is as follows: As an aspect of the present application, there is provided a sensor-based optical lens installation detecting method for recognizing whether a positional shift of an image sensor device or a checkerboard calibration reference plate used in an optical lens detection application occurs, comprising the steps of: step S1, providing an optical lens detection scene, and setting an image sensor device, a lens mounting station and a calibration device in the optical lens detection scene, wherein the calibration device comprises a checkerboard calibration reference plate and a plurality of fixed reference balls which are arranged on the periphery of the checkerboard calibration reference plate and are fixed in position; Step S2, setting up a detection datum point on the checkerboard calibration reference plate, and acquiring camera internal calibration information in the image sensor device based on the detection datum point, wherein the camera internal calibration information comprises internal parameters of an internal reference matrix and distortion coefficients and external parameters comprising a rotation matrix and a translation vector; S3, acquiring an image containing a plurality of fixed reference balls, and identifying and obtaining pixel coordinates of each fixed reference ball from the image containing the plurality of fixed reference balls; S4, calculating three-dimensional coordinates of each fixed reference ball under a workpiece coordinate system by using pixel coordinates of each fixed reference ball and camera internal calibration information, and taking the three-dimensional coordinates as reference coordinates; Step S5, after the set number of optical lens calibration procedures are completed, calculating three-dimensional coordinates of each fixed reference ball in the workpiece coordinate system after the set number of optical lens calibration procedures are completed by using pixel coordinates of each fixed reference ball and camera internal calibration information, and taking the three-dimensional coordinates as veri