Search

CN-121304806-B - Panoramic camera and turntable movement measurement matching method and system

CN121304806BCN 121304806 BCN121304806 BCN 121304806BCN-121304806-B

Abstract

The invention relates to the technical field of mobile measurement, in particular to a panoramic camera and turntable mobile measurement calibration method and system, comprising the steps of initializing calibration, obtaining an internal reference of a camera and an initial rotation center of a turntable, and taking the internal reference and the initial rotation center of the turntable as initial references of a deviation model; and acquiring the position deviation between the rotating shaft of the turntable and the optical center of the camera, the attitude angle deviation of the camera and the environmental data in real time through a dual-mode dynamic deviation sensing module. The method comprises the steps of obtaining an initial rotation center of a participating turntable in a camera through initialization calibration to serve as a reference, laying a foundation for subsequent dynamic deviation calculation, collecting position deviation, attitude angle deviation and environment data in real time by a bimodal dynamic deviation sensing module, constructing a deep learning model by combining self-adaptive deviation modeling, dynamically learning the change rule of the deviation along with the environment and the operation time, outputting a predicted value, and solving the limitation that the dynamic deviation cannot be captured in real time by the traditional static calibration.

Inventors

  • Tian Guanghou

Assignees

  • 北京酷车易美网络科技有限公司

Dates

Publication Date
20260508
Application Date
20251210

Claims (10)

  1. 1. A panoramic camera and turntable movement measurement matching method is characterized by comprising the following steps: Initializing and calibrating, namely acquiring an internal reference of a camera and an initial rotation center of a turntable, and taking the internal reference and the initial rotation center of the turntable as initial references of a deep learning deviation association model; Real-time dynamic deviation sensing, wherein radial position deviation, camera attitude angle deviation and environment data between the rotating shaft of the turntable and an optical center of a camera are acquired in real time through a dual-mode dynamic deviation sensing module, and the environment data comprise environment temperature change data and turntable operation vibration data; Self-adaptive deviation modeling, namely constructing a deep learning deviation association model based on the radial position deviation, the attitude angle deviation, the environmental temperature change data and the turntable operation vibration data, and outputting a dynamic deviation predicted value in real time according to the change rule of the dynamic learning deviation along with environmental factors and operation time; The two-dimensional real-time compensation comprises a mechanical control compensation step and an image processing compensation step, wherein the dynamic deviation predicted value is converted into a mechanical control instruction, the rotation center of the turntable and the angle of a camera mounting bracket are adjusted in real time through a servo motor, and a mechanical centering error is corrected; And (3) precision closed loop feedback optimization, namely quantifying the current matching precision through spliced image edge overlap ratio calculation and three-dimensional point cloud heavy projection error analysis, and if the precision is lower than a preset threshold value, adjusting the sampling frequency of the bimodal dynamic deviation sensing module and the weight coefficient of the deep learning deviation correlation model.
  2. 2. The panoramic camera and carousel movement measurement calibration method of claim 1, wherein the bimodal dynamic bias sensing module comprises: The deviation monitoring unit is used for integrating a laser displacement sensor and a micro-inertia measuring unit, wherein the laser displacement sensor is used for collecting the radial position deviation between the rotating shaft of the turntable and the optical center of the camera in real time, and the micro-inertia measuring unit is used for capturing the attitude angle deviation of the camera; and the environment association unit is used for additionally arranging a temperature sensor and a mechanical vibration sensor, collecting the environment temperature change data and the turntable operation vibration data, and providing environment influence factor input for the deviation association model.
  3. 3. The panoramic camera and turret movement measurement calibration method of claim 1, wherein the adaptive bias modeling includes: Constructing a deep learning deviation association model based on a time sequence convolution network, wherein the model takes radial position deviation, attitude angle deviation and environmental data as training samples and is used for dynamically learning the change rule of the deviation along with environmental factors and operation time length and outputting the dynamic deviation predicted value in real time; Setting a model iteration mechanism, and automatically updating parameters of the deviation association model by newly acquired deviation data every interval preset period.
  4. 4. The panoramic camera and carousel motion measurement calibration method of claim 1, wherein the mechanical control compensation step in the two-dimensional real-time compensation is as follows: Converting the radial position deviation output by the dynamic deviation predicted value into a turntable control instruction, and adjusting the rotation center of the turntable in real time through a servo motor; and converting the attitude angle deviation output by the dynamic deviation predicted value into a camera installation instruction, and adjusting the angle of the camera installation bracket in real time through a servo motor so as to correct the centering error of the mechanical motion layer.
  5. 5. The panoramic camera and carousel motion measurement calibration method of claim 1, wherein the image processing compensation step in the two-dimensional real-time compensation is as follows: Before panoramic image stitching, performing coordinate transformation on a single-frame panoramic image according to the dynamic deviation predicted value, wherein the coordinate transformation comprises translation correction and rotation correction so as to eliminate image pixel dislocation caused by the mechanical deviation; and after the coordinate transformation correction is executed, performing feature matching and stitching on the single-frame panoramic image, so as to ensure the geometric consistency of the panoramic image.
  6. 6. The panoramic camera and turret movement measurement calibration method of claim 1, wherein the precision closed loop feedback optimization comprises: detecting pixel deviation at the joint of adjacent images through edge coincidence ratio calculation of the joint images, and quantifying current matching accuracy; comparing the corresponding relation between the three-dimensional point cloud and the image through the reprojection error analysis of the three-dimensional point cloud, and quantifying the current calibration precision; if the quantized current matching accuracy is lower than a preset threshold value, the sampling frequency of the bimodal dynamic deviation sensing module is automatically adjusted, or the weight coefficient of the deep learning deviation association model is automatically adjusted.
  7. 7. The panoramic camera and carousel movement measurement calibration method of claim 6, wherein adjusting the sampling frequency of the bimodal dynamic bias perception module and the weight coefficient of the deep learning bias association model comprises: If the matching accuracy is lower than a preset threshold, the sampling frequency of the bimodal dynamic deviation sensing module is increased so as to acquire deviation data more frequently; and if the matching accuracy is lower than a preset threshold, adjusting the weight coefficient of the deep learning deviation association model, and adjusting the calculated duty ratio of the influence of the enhanced environment temperature change on the deviation.
  8. 8. The panoramic camera and carousel movement measurement calibration method of claim 1, wherein the initializing calibration comprises: Under standard environmental conditions, acquiring an internal reference of a camera and an initial rotation center of a turntable by a traditional static calibration method; and taking the acquired camera internal parameters and the initial rotation center of the turntable as initial references of the deep learning deviation correlation model.
  9. 9. The panoramic camera and carousel movement measurement calibration method of claim 1, wherein the method comprises: After the panoramic measurement system is started, a real-time dynamic deviation sensing step is circularly executed to continuously acquire the latest deviation data and environment data; Then, a self-adaptive deviation modeling step is circularly executed, a deep learning deviation correlation model is iteratively updated, and a dynamic deviation value at the current moment is output; then, circularly executing a two-dimensional real-time compensation step, and synchronously completing deviation compensation of a machine and a data layer; And then, circularly executing a precision closed-loop feedback optimization step, and if the matching precision does not reach the standard, feeding back and adjusting parameters of the sensing, modeling and compensating links until the measurement is finished.
  10. 10. A panoramic camera and carousel movement measurement calibration system for use in a panoramic camera and carousel movement measurement calibration method as defined in any one of claims 1-9, comprising: the dual-mode dynamic deviation sensing module is used for collecting radial position deviation between the rotating shaft of the turntable and the optical center of the camera, camera attitude angle deviation and environmental data in real time, wherein the environmental data comprises environmental temperature change data and turntable operation vibration data; The self-adaptive deviation modeling unit is used for constructing a deep learning deviation association model based on radial position deviation, attitude angle deviation, environmental temperature change data and turntable operation vibration data, and outputting a dynamic deviation predicted value in real time according to the change rule of dynamic learning deviation along with environmental factors and operation time; The system comprises a two-dimensional real-time compensation unit, a single-frame panoramic image, a characteristic matching and splicing unit, a dynamic deviation prediction unit and a two-dimensional real-time compensation unit, wherein the two-dimensional real-time compensation unit is used for converting the dynamic deviation prediction value into a mechanical control instruction, adjusting the rotation center of a turntable and the angle of a camera mounting bracket in real time through a servo motor, correcting mechanical centering errors, carrying out coordinate transformation correction on the single-frame panoramic image according to the dynamic deviation prediction value before image splicing, eliminating image pixel dislocation caused by mechanical deviation, and carrying out characteristic matching and splicing; the precision closed loop feedback unit is used for quantifying the current calibration precision through the edge overlap ratio calculation of the spliced image and the three-dimensional point cloud re-projection error analysis, and adjusting the sampling frequency of the bimodal dynamic deviation sensing module and the weight coefficient of the deep learning deviation correlation model if the precision is lower than a preset threshold value.

Description

Panoramic camera and turntable movement measurement matching method and system Technical Field The invention relates to the technical field of mobile measurement, in particular to a panoramic camera and turntable mobile measurement calibration method and system. Background Panoramic camera and carousel movement measurement technique are often used in fields such as three-dimensional modeling, virtual reality. The panoramic camera is responsible for gathering the image information of surrounding environment, and the carousel is then used for controlling the rotary motion of camera in the horizontal direction, and the scene coverage of full visual angle can be realized to two collaborative work. Accurate calibration is critical to ensure the quality of the acquired data, which involves a high match of the position, pose, and motion parameters between the camera and the turntable. If the matching is inaccurate, the problems of dislocation of spliced images, distortion of three-dimensional point clouds and the like are caused. However, the prior art has the problem that the rotation axis of the turntable and the optical center of the camera are subjected to non-ideal centering deviation in the matching process, and the deviation is dynamically changed by factors such as ambient temperature, mechanical abrasion and the like, so that the error cannot be corrected in real time by the traditional static calibration method. Such dynamic bias can gradually accumulate during long-period, wide-range scene scanning, resulting in reduced geometric consistency of panoramic image stitching. The existing solution depends on preset rigid body transformation parameters, and does not consider on-line compensation of dynamic centering deviation, so that the measurement result is subjected to precision attenuation which is difficult to correct in a medium-long distance scene. Disclosure of Invention Aiming at the defects of the prior art, the invention provides a panoramic camera and turntable mobile measurement matching method and system, which solve the problems that static calibration cannot correct dynamic deviation in real time, image splicing dislocation and three-dimensional point cloud distortion caused by deviation accumulation and on-line compensation is lacking in comparison with the prior art. In order to achieve the purpose, the invention is realized by the following technical scheme that the panoramic camera and turntable movement measurement matching method comprises the following steps: initializing and calibrating, namely acquiring an internal reference of a camera and an initial rotation center of a turntable, and taking the internal reference and the initial rotation center as initial references of a deviation model; Real-time dynamic deviation sensing, namely acquiring position deviation between a rotating shaft of the turntable and an optical center of a camera, camera attitude angle deviation and environmental data in real time through a dual-mode dynamic deviation sensing module, wherein the environmental data comprises environmental temperature change data and turntable operation vibration data; Self-adaptive deviation modeling, namely constructing a deep learning deviation association model based on the position deviation, the attitude angle deviation, the environmental temperature change data and the turntable operation vibration data, and outputting a dynamic deviation predicted value in real time according to the change rule of the dynamic learning deviation along with environmental factors and operation time; The system comprises a turntable, a camera mounting bracket, a dynamic deviation prediction value, a two-dimensional real-time compensation, a single-frame panoramic image, a characteristic matching and splicing, wherein the turntable is used for rotating at least one of the turntable and the camera mounting bracket, and the dynamic deviation prediction value is used for converting the dynamic deviation prediction value into a mechanical control instruction, and adjusting the rotation center of the turntable and the camera mounting bracket angle in real time through a servo motor to correct mechanical centering errors; And (3) precision closed loop feedback optimization, namely quantifying the current matching precision through spliced image edge overlap ratio calculation and three-dimensional point cloud heavy projection error analysis, and if the precision is lower than a preset threshold value, adjusting the sampling frequency of the bimodal dynamic deviation sensing module and the weight coefficient of the deep learning deviation correlation model. Further, the bimodal dynamic deviation sensing module comprises: The deviation monitoring unit is used for integrating a laser displacement sensor and a micro-inertia measuring unit, wherein the laser displacement sensor is used for collecting the radial position deviation between the rotating shaft of the turntable and the optical center of the c