Search

CN-117649453-B - Event camera external parameter calibration method and device, electronic equipment and storage medium

CN117649453BCN 117649453 BCN117649453 BCN 117649453BCN-117649453-B

Abstract

The invention provides an event camera external parameter calibration method, an event camera external parameter calibration device, electronic equipment and a storage medium, which comprehensively consider the kinematic characteristics (linear correlation of two groups of linear velocities) of a system and the limited precision (time offset t d ) of a visual odometer based on an event camera, utilize two groups of linear velocity estimates respectively from the event camera and a sensor on a carrier to estimate the external parameters of the event camera by using a first-order kinematic model based on a CCA scheme.

Inventors

  • ZHOU YI
  • MAI JIAYAO

Assignees

  • 湖南大学

Dates

Publication Date
20260512
Application Date
20231212

Claims (10)

  1. 1. An event camera external parameter calibration method is characterized by comprising the following steps: During the linear motion of the carrier, determining the translational linear velocity v e of the event camera on the carrier, determining the translational linear velocity v o of the specified sensor on the carrier, and obtaining two synchronous linear velocities And Where N represents the number of samples and t i represents the sampling time; The cross-covariance and auto-covariance of the two sets of linear velocities at time offset t d are calculated by equation (1): Wherein, the And Mean values of v e and v o are shown respectively, And Mean values of v e and v o after time compensation are shown, respectively; Based on the first order kinematic model v o ＝R oe v e of the two sets of linear velocities, the identity matrix I 3×3 ＝[s 1 s 2 s 3 is recorded, and the matrix is externally rotated The trace correlation between the two sets of linear velocities is represented by equation (3) based on equation (2): Wherein Tr (·) represents the trace of the matrix; Calculating an optimal solution of the time offset t d by the formula (4): Calculating an external rotation matrix according to an optimal solution of the time offset t d and a formula (5): wherein U, V is obtained by SVD decomposition of the following matrix:
  2. 2. The method for calibrating an external parameter of an event camera according to claim 1, wherein said determining a translational linear velocity v e of the event camera on the carrier further comprises: Acquiring event data from the event camera, converting the event data into a similar image representation form, and obtaining an event image, wherein the event image accords with the speed-unchanged representation; Extracting feature points from the image through an Arc angular point detection algorithm, and matching the feature points through BRIEF descriptors; calculating an essential matrix by a five-point method based on the matched characteristic points; And obtaining the translation linear velocity v e from the essence matrix.
  3. 3. The method for calibrating an external parameter of an event camera according to claim 2, wherein said converting the event data into a representation of a similar image to obtain an event image, further comprises: representing the event data as a TS image and a TOS image; and performing logical AND operation on pixel values of corresponding points of the TS image and the TOS image to form an event image.
  4. 4. The method of claim 1, wherein determining the translational linear velocity v o of the specified sensor on the carrier further comprises: acquiring data of the specified sensor, and recovering a corresponding track according to the data; and acquiring the translational linear velocity v o from the track.
  5. 5. An event camera external parameter calibration device, comprising: The speed sampling module is used for determining the translational linear speed v e of the event camera on the carrier and determining the translational linear speed v o of the specified sensor on the carrier in the linear motion process of the carrier to obtain two synchronous groups of linear speeds And Where N represents the number of samples and t i represents the sampling time; The variance calculation module is used for calculating the cross covariance and the autocovariance of the two groups of linear velocities under the time offset t d according to the formula (1): Wherein, the And Mean values of v e and v o are shown respectively, And Mean values of v e and v o after time compensation are shown, respectively; The trace correlation definition module is used for recording an identity matrix I 3×3 ＝[s 1 s 2 s 3 and an external rotation matrix according to the first-order kinematic model v o ＝R oe v e of the two groups of linear velocities The trace correlation between the two sets of linear velocities is represented by equation (3) based on equation (2): Wherein Tr (·) represents the trace of the matrix; a time offset calculation module, configured to calculate an optimal solution of the time offset t d according to formula (4): the external parameter calculation module is configured to calculate an external rotation matrix according to the optimal solution of the time offset t d and formula (5): wherein U, V is obtained by SVD decomposition of the following matrix:
  6. 6. the device for calibrating an external parameter of an event camera according to claim 5, wherein said determining a translational linear velocity v e of the event camera on said carrier further comprises: Acquiring event data from the event camera, converting the event data into a similar image representation form, and obtaining an event image, wherein the event image accords with the speed-unchanged representation; Extracting feature points from the image through an Arc angular point detection algorithm, and matching the feature points through BRIEF descriptors; calculating an essential matrix by a five-point method based on the matched characteristic points; And obtaining the translation linear velocity v e from the essence matrix.
  7. 7. The device for calibrating an external parameter of an event camera according to claim 6, wherein said converting the event data into a representation of a similar image to obtain an event image, further comprises: representing the event data as a TS image and a TOS image; and performing logical AND operation on pixel values of corresponding points of the TS image and the TOS image to form an event image.
  8. 8. The event camera extrinsic calibration apparatus according to claim 5, wherein said determining a translational linear velocity v o of a given sensor on said carrier further comprises: acquiring data of the specified sensor, and recovering a corresponding track according to the data; and acquiring the translational linear velocity v o from the track.
  9. 9. An electronic device comprising a memory module including instructions loaded and executed by a processor, which when executed, cause the processor to perform an event camera extrinsic calibration method according to any one of claims 1 to 4.
  10. 10. A computer readable storage medium storing one or more programs, which when executed by a processor, implement an event camera extrinsic calibration method according to any of claims 1-4.

Description

Event camera external parameter calibration method and device, electronic equipment and storage medium Technical Field The invention belongs to the technical field of event cameras, and particularly relates to an event camera external parameter calibration method, an event camera external parameter calibration device, electronic equipment and a storage medium. Background For physical carriers equipped with cameras, such as robots, autopilots, etc., it is important to determine the mounting position and orientation of each camera relative to the carrier, a problem called the camera's external parameter calibration problem. The external calibration scheme designed for the traditional camera is mainly a hand-eye calibration method (hand-eye calibration), namely, two tracks are respectively restored through visual information provided by a camera and other sensors (such as a wheel odometer and the like) on a carrier, and external parameters are estimated based on the alignment of the two tracks, however, the method must ensure that the tracks restored from the visual information are reliable. Unlike traditional frame camera, the event camera is used as a novel sensor inspired by biology, asynchronously measures the brightness change of pixel level reaching the trigger condition, and outputs a series of events, wherein the events are the time t k, the position (x k,yk) and the code of the symbol p k E { 1, +1} of the brightness change, and the events are recorded asThis particular principle of operation brings about better performance in terms of time resolution and dynamic range. However, due to the different output formats, the processing technology for the traditional camera cannot be directly applied to the event camera, so that a mature method capable of effectively eliminating accumulated errors, such as beam method adjustment (Bundle Adjustment, BA), is still not available in the event camera-based visual odometer, so that the trace recovered from visual information cannot be ensured to be reliable in the process of calibrating the external parameters of the event camera, and a new reliable event camera external parameter calibration method needs to be found. Disclosure of Invention Based on the above, an event camera external parameter calibration method, an event camera external parameter calibration device, an electronic device and a storage medium are provided for the technical problems. The technical scheme adopted by the invention is as follows: as a first aspect of the present invention, there is provided an event camera extrinsic parameter calibration method, including: During the linear motion of the carrier, determining the translational linear velocity v e of the event camera on the carrier, determining the translational linear velocity v o of the specified sensor on the carrier, and obtaining two synchronous linear velocities AndWhere N represents the number of samples and t i represents the sampling time; The cross-covariance and auto-covariance of the two sets of linear velocities at time offset t d are calculated by equation (1): Wherein, the AndMean values of v e and v o are shown respectively,AndMean values of v e and v o after time compensation are shown, respectively; Based on the first order kinematic model v o＝Roeve of the two sets of linear velocities, the identity matrix I 3×3＝[s1 s2 s3 is recorded, and the matrix is externally rotated The trace correlation between the two sets of linear velocities is represented by equation (3) based on equation (2): Wherein Tr (·) represents the trace of the matrix; Calculating an optimal solution of the time offset t d by the formula (4): Calculating an external rotation matrix according to an optimal solution of the time offset t d and a formula (5): wherein U, V is obtained by SVD decomposition of the following matrix: As a second aspect of the present invention, there is provided an event camera external parameter calibration apparatus comprising: The speed sampling module is used for determining the translational linear speed v e of the event camera on the carrier and determining the translational linear speed v o of the specified sensor on the carrier in the linear motion process of the carrier to obtain two synchronous groups of linear speeds AndWhere N represents the number of samples and t i represents the sampling time; The variance calculation module is used for calculating the cross covariance and the autocovariance of the two groups of linear velocities under the time offset t d according to the formula (1): Wherein, the AndMean values of v e and v o are shown respectively,AndMean values of v e and v o after time compensation are shown, respectively; The trace correlation definition module is used for recording an identity matrix I 3×3＝[s1 s2 s3 and an external rotation matrix according to the first-order kinematic model v o＝Roeve of the two groups of linear velocities The trace correlation between the two sets of linear velocities