CN-121982114-A - Camera calibration method based on deep learning, electronic equipment, medium and product

CN121982114ACN 121982114 ACN121982114 ACN 121982114ACN-121982114-A

Abstract

The application discloses a camera calibration method, electronic equipment, media and products based on deep learning, and relates to the technical field of image processing, wherein the method comprises the steps of determining a first coordinate corresponding to at least one calibration point in a three-dimensional model corresponding to a product to be detected; the method comprises the steps of determining a first image pixel coordinate corresponding to at least one calibration point in a product image of a product to be detected according to a deep learning model, determining a homography matrix representing a coordinate conversion relation according to the first coordinate and the first image pixel coordinate which belong to the same calibration point, determining a Euclidean distance from at least one calibration point to a camera according to the homography matrix to determine the distance from the camera to the product to be detected, adjusting a position parameter of the camera in response to the fact that the distance is not matched with the optimal position of the depth of field of the camera, and re-executing acquisition of the product image of the product to be detected until the latest obtained distance is matched with the optimal position of the depth of field of the camera. The application can accurately determine the distance between the camera and the calibration object so as to adjust the position of the camera and further improve the shooting effect of the camera.

Inventors

LV JIANTAO
HOU CHUANYONG
ZHAO SHIWEN
LIU KUN
QI XINKAI

Assignees

歌尔股份有限公司

Dates

Publication Date: 20260505
Application Date: 20251229

Claims (10)

1. The camera calibration method based on the deep learning is characterized by comprising the following steps of: determining a three-dimensional model corresponding to a product to be tested marked with at least one calibration point, and determining a first coordinate corresponding to the at least one calibration point in the three-dimensional model; acquiring a product image of the product to be detected, and determining a first image pixel coordinate corresponding to the at least one standard point in the product image according to a pre-trained deep learning model; Determining a homography matrix representing a coordinate conversion relation according to a first coordinate and a first image pixel coordinate which belong to the same calibration point; determining the Euclidean distance from the at least one calibration point to a camera according to the homography matrix, and determining the distance from the camera to the product to be tested according to the Euclidean distance; And adjusting the position parameters of the camera in response to the fact that the distance is not matched with the optimal position of the depth of field of the camera, and re-executing the acquisition of the product image of the product to be detected until the latest obtained distance is matched with the optimal position of the depth of field of the camera.
2. The depth learning based camera calibration method of claim 1, wherein the homography matrix characterizes coordinate conversion relationships among a product coordinate system corresponding to the first coordinate, an image coordinate system corresponding to the first image pixel coordinate, and a camera coordinate system of the camera.
3. A depth learning based camera calibration method as defined in claim 2, wherein the at least two calibration points are located on the same plane of the product under test, The step of determining the euclidean distance between the at least one calibration point and the camera according to the homography matrix comprises the following steps: determining a rotation matrix and a translation vector of the camera according to the homography matrix; Determining a second coordinate of any one calibration point in a preset world coordinate system; Determining a conversion relation between the world coordinate system and the camera coordinate system, and determining a third coordinate of the calibration point in the camera coordinate system according to the rotation matrix, the translation vector and the second coordinate by utilizing the conversion relation; And carrying out Euclidean distance calculation according to the third coordinate and a camera coordinate system origin in the camera coordinate system to obtain the Euclidean distance from the standard point to the camera, wherein the camera coordinate system origin is the camera optical center.
4. The camera calibration method based on deep learning of claim 1, the camera calibration method based on the deep learning is characterized by further comprising the following steps: inputting a preset training product image into a preset neural network model, and determining second image pixel coordinates corresponding to a first prediction calibration point according to an output result of the neural network model; Determining a first loss function value according to a preset focus loss function and the second image pixel coordinates, reversely updating model parameters of the neural network model according to the first loss function value until a preset training cut-off condition is met, obtaining a pre-trained neural network model, and taking the pre-trained neural network model as a pre-trained deep learning model.
5. The camera calibration method based on deep learning of claim 4, wherein the step of determining the second image pixel coordinates corresponding to the first prediction calibration point according to the output result of the neural network model comprises: Mapping the training product image to a high-dimensional feature space according to an encoder of the neural network model to obtain a first feature image, carrying out convolution processing on the first feature image by utilizing a plurality of convolution layers, and fusing convolution results of each convolution layer to obtain a fused convolution feature image; performing deconvolution processing on the fusion convolution characteristic map by using two parallel deconvolution layers according to a decoder of the neural network model to obtain a central thermodynamic diagram and an offset thermodynamic diagram; -determining said second image pixel coordinates from said central thermodynamic diagram and said offset.
6. The depth learning based camera calibration method of claim 5, wherein the step of taking a thermodynamic diagram to determine the second image pixel coordinates from the central thermodynamic diagram and the offset comprises: aiming at each central thermodynamic diagram, carrying out Gaussian distribution detection on the central thermodynamic diagram, and determining third image pixel coordinates corresponding to the first prediction standard point in the training product image according to the Gaussian distribution detection result; and updating the third image pixel coordinate according to the offset thermodynamic diagram corresponding to the central thermodynamic diagram to obtain the second image pixel coordinate.
7. The depth learning based camera calibration method of claim 6, wherein the step of performing gaussian distribution detection on the central thermodynamic diagram and determining third image pixel coordinates corresponding to the first predicted calibration point in the training product image according to the gaussian distribution detection result comprises: determining a preset Gaussian distribution detection condition, wherein the Gaussian distribution detection condition comprises the steps of determining that the circle center in a key feature area corresponding to a real number matrix in a training product image is a third image pixel coordinate corresponding to a first prediction standard point when a result generated by Gaussian distribution processing of the central thermodynamic diagram contains the real number matrix which is non-zero; Carrying out Gaussian distribution detection on the central thermodynamic diagram according to the Gaussian distribution detection conditions; Determining a first real number greater than a preset real number threshold value in a real number matrix in response to the Gaussian distribution detection result comprising the existence of the non-zero real number matrix; And taking a pixel region where the pixel corresponding to each first real number in the training product image is located as a standard point characteristic region, and determining the pixel coordinate of the third image according to the standard point characteristic region.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the depth learning based camera calibration method according to any one of claims 1 to 7.
9. A storage medium, characterized in that the storage medium is a computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the steps of the deep learning based camera calibration method according to any one of claims 1 to 7.
10. A computer program product, characterized in that the computer program product comprises a computer program which, when executed by a processor, implements the steps of the depth learning based camera calibration method according to any one of claims 1 to 7.

Description

Camera calibration method based on deep learning, electronic equipment, medium and product Technical Field The present application relates to the field of image processing technologies, and in particular, to a camera calibration method based on deep learning, an electronic device, a storage medium, and a computer program product. Background In the field of visual calibration, it is generally required to accurately measure the spatial distance between a camera and a calibration object (such as a product component with a calibration point, such as a product accessory, a plastic component, etc.), for example, the Z-direction distance (i.e., the distance in the Z-axis direction), so as to determine the positional relationship between the camera and the calibration object, and further enable the camera to accurately capture a high-precision image of the calibration object, but at present, the camera height is manually adjusted through manual experience, so that the distance between the camera and the calibration object cannot be accurately determined, and further the capturing effect of the camera is affected. Therefore, how to accurately determine the distance between the camera and the calibration object to adjust the position of the camera, so as to support the improvement of the shooting effect of the camera is an urgent problem to be solved at present. The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present application and is not intended to represent an admission that the foregoing is prior art. Disclosure of Invention The application mainly aims to provide a camera calibration method, electronic equipment, media and products based on deep learning, which aim at solving the technical problem of how to accurately determine the distance between a camera and a calibration object so as to adjust the position of the camera and further support the improvement of the shooting effect of the camera. In order to achieve the above object, the present application provides a camera calibration method based on deep learning, the camera calibration method based on deep learning includes the following steps: determining a three-dimensional model corresponding to a product to be detected, which is marked with at least one calibration point, and determining a first coordinate corresponding to at least one calibration point in the three-dimensional model; acquiring a product image of a product to be detected, and determining first image pixel coordinates corresponding to at least one standard point in the product image according to a pre-trained deep learning model; determining a homography matrix representing a coordinate conversion relation according to a first coordinate and a first image pixel coordinate which belong to the same calibration point; Determining the Euclidean distance from at least one calibration point to the camera according to the homography matrix, and determining the distance from the camera to the product to be tested according to the Euclidean distance; And in response to the fact that the distance is not matched with the optimal position of the depth of field of the camera, adjusting the position parameters of the camera, and re-executing the acquisition of the product image of the product to be detected until the latest obtained distance is matched with the optimal position of the depth of field of the camera. Optionally, the homography matrix characterizes a coordinate conversion relationship among a product coordinate system corresponding to the first coordinate, an image coordinate system corresponding to the first image pixel coordinate, and a camera coordinate system of the camera. Optionally, at least two of the calibration points are located on the same plane as the product to be tested, Determining the Euclidean distance between at least one standard point and the camera according to the homography matrix, wherein the step comprises the following steps: determining a rotation matrix and a translation vector of the camera according to the homography matrix; Determining a second coordinate of the calibration point in a preset world coordinate system aiming at any calibration point; determining a conversion relation between the world coordinate system and the camera coordinate system, and determining a third coordinate of the marked point in the camera coordinate system according to the rotation matrix, the translation vector and the second coordinate by utilizing the conversion relation; and carrying out Euclidean distance calculation according to the third coordinate and the origin of the camera coordinate system in the camera coordinate system to obtain the Euclidean distance from the standard point to the camera, wherein the origin of the camera coordinate system is the optical center of the camera. Optionally, the camera calibration method based on deep learning further comprises: Inputting a preset training product image in