Search

JP-7855017-B2 - Automatic calibration from epipolar line distance within the projection pattern

JP7855017B2JP 7855017 B2JP7855017 B2JP 7855017B2JP-7855017-B2

Inventors

  • シンドラー，パトリク
  • シック，フリードリヒ
  • レンナルツ，クリスティアン
  • シレン，ペーター
  • ウンガー，ヤコブ

Assignees

  • トリナミクス ゲゼルシャフト ミット ベシュレンクテル ハフツング

Dates

Publication Date
20260507
Application Date
20220530
Priority Date
20210531

Claims (15)

  1. A detector (110) for determining the position of at least one object (112), - At least one projector (122) for illuminating the object (112) with at least one irradiation pattern (124), wherein the irradiation pattern (124) includes a plurality of irradiation features (125), - At least one sensor element (114) having a matrix (116) of optical sensors (118), each of the optical sensors having a photosensitive area (120), each optical sensor (118) is designed to generate at least one sensor signal in response to illumination of each photosensitive area (120) by a reflected light beam propagating from the object (112) to the detector (110), the sensor element is configured to determine at least one reflection image (142) including a plurality of reflection features, each of the reflection features including a beam profile, at least one sensor element (114) and - At least one evaluation device (144) configured to determine the initial distance information of the reflection features by analyzing the beam profiles of each of the reflection features, wherein the analysis of the beam profiles includes evaluating the coupled signal Q from the sensor signal , and the evaluation device (144) a) (150) Matching the reflection features to the reference features of the reference image, taking into account the initial distance information, thereby determining the matched pair of reflection features and reference features, b) (152) For each of the matching reflective feature and reference feature pairs, determine the epipolar line of the matching reference feature in the reference image, c) (154) Determining the epipolar line distance d of the matched reflection feature to the epipolar line, d) (156) Evaluate the epipolar line distance d as a function of the image position (x, y) in the reference image, and thereby determine the geometric pattern, e) (158) Determine at least one correction for the rotation and/or translation of the reflected image (142) according to the geometric pattern. An evaluation device (144) configured to perform a calibration method including, A detector (110) is provided.
  2. The detector (110) according to claim 1, wherein the evaluation device (144) is configured to correct the reflected image (142) based on the determined correction.
  3. The detector (110) according to claim 1, wherein the evaluation device (144) is configured to determine at least one triangulation distance information of the reflection feature by using triangulation, taking into account the determined correction.
  4. The detector (110) according to claim 3 , wherein the evaluation device (144) is configured to perform the calibration method on the fly while determining the triangulation distance information.
  5. The detector (110) according to claim 1, wherein the evaluation device (144) is configured to determine at least one external parameter of the detector, the external parameter being at least one parameter selected from the group consisting of a rotation angle between the coordinates of the projector (122) and the sensor element (114), a translation component between the coordinates of the projector (122) and the sensor element (114), an aperture angle, the center of the sensor element, an aperture, and a focal length.
  6. The detector (110) according to claim 1, wherein the evaluation device (144) is configured to perform steps b) to e) even for pairs of reflection features and reference features that were incorrectly determined as matching pairs of reflection features and reference features in step a).
  7. The detector (110) according to claim 1, wherein the evaluation device (144) is configured to determine the correction of the reflected image by evaluating one or more of the shape, repeatability, steepness, discontinuity, and curvature of the geometric pattern.
  8. The detector (110) according to claim 1, wherein the irradiation pattern includes at least one periodic and regular pattern selected from a group consisting of at least one periodic and regular point pattern, at least one hexagonal pattern, and at least one rectangular pattern.
  9. The detector (110) according to claim 1, wherein the evaluation device (144) is configured to derive the combined signal Q by one or more of the following: dividing the sensor signal, dividing by a multiple of the sensor signal, and dividing by a linear combination of the sensor signals, and the evaluation device (144) is configured to use at least one predetermined relationship between the combined signal Q and a y-coordinate to determine the initial distance information.
  10. The detector (110) according to claim 1, wherein the evaluation device (144) is configured to perform image analysis of the reflected image (142) and thereby identify the reflection characteristics of the reflected image (142).
  11. The detector (110) according to claim 1, wherein the evaluation device (144) is configured to determine the longitudinal region of each reflection feature, the longitudinal region being given by the initial distance information and error interval ±ε of the reflection feature determined from the combined signal Q, and the evaluation device (144) is configured to determine at least one displacement region in the reference image corresponding to the longitudinal region.
  12. The detector (110) according to claim 11, wherein the evaluation device (144) is configured to match each of the reflection features with each of the reference features in the displacement region by using at least one linear scaling algorithm.
  13. A method for calibrating at least one detector (110) according to any one of claims 1 to 12, the method comprising the following steps: i) (148) Initial distance information, - Irradiating an object ( 112) with at least one irradiation pattern (124) generated by at least one projector (122) of the detector (110) , wherein the irradiation pattern (124) includes a plurality of irradiation features (125), - To generate at least one sensor signal for each reflected light beam that collides with the photosensitive area (120) of the optical sensor (118) of the sensor element (114) having a matrix (116) of the optical sensor (118) in response to irradiation. - To determine at least one reflection image (142) including a plurality of reflection features by using the sensor element (114) , wherein each of the reflection features includes a beam profile. - To evaluate the sensor signal using at least one evaluation device (144), thereby determining the combined signal Q, and to determine the initial distance information of the reflection features by analyzing the beam profiles of each of the reflection features, wherein the analysis of the beam profiles includes evaluating the combined signal Q from the sensor signal . ii) (150) Matching the reflection features to the reference features of the reference image, taking into account the initial distance information, thereby determining the matched pair of reflection features and reference features, iii) (152) For each of the matching pairs of reflection features and reference features, determine the epipolar line of the matching reference feature in the reference image, iv) (154) Determining the epipolar line distance d of the matched reflection feature to the epipolar line, v) (156) Evaluate the epipolar line distance d as a function of the image position (x, y) in the reference image, and thereby determine the geometric pattern, vi) (158) Determine at least one correction for the rotation and/or translation of the reflected image (142) according to the geometric pattern, Methods that include...
  14. The method according to claim 13, comprising correcting the reflected image (142) based on the determined correction, and determining at least one triangulation distance information of the reflected feature by using triangulation considering the determined correction.
  15. A method of using a detector (110) according to any one of claims 1 to 12 for an intended use selected from the group consisting of location measurement in traffic technology, entertainment applications, security applications, surveillance applications, safety applications, human-machine interface applications, logistics applications, tracking applications, outdoor applications, mobile applications, communication applications, photography applications, machine vision applications, robotics applications, quality control applications, and manufacturing applications.

Description

This invention relates to a detector for determining the position of at least one object and a method for calibrating the aforementioned detector. The invention further relates to various uses of the detector. The devices, methods, and uses according to the present invention can be employed, for example, in various areas of daily life, gaming, traffic technology, production technology, security technology, photography such as digital or video photography for art, documentation or technical purposes, medical technology, or in science. Furthermore, the invention can be used, for example, in the fields of architecture, metrology, archaeology, art, medicine, engineering, or manufacturing, to scan one or more objects and/or scenery, such as to generate depth profiles of objects or scenery. However, other applications are also possible. An active triangulation system typically includes at least one camera and at least one optical projector, e.g., a structured optical system. Other triangulation systems, such as stereo cameras, may include at least two cameras. Knowledge of the position and rotation of components such as cameras and projectors is essential for proper three-dimensional reconstruction by triangulation. In addition, three-dimensional reconstruction by triangulation also requires resolved correspondences of key points on the scene, e.g., laser spots, projector light spots, or detected edges captured by the cameras. The three-dimensional position can be calculated from the known translational movement and relative rotation of the camera to the projector. This parameter defines the external calibration of the triangulation system. Therefore, the quality of the three-dimensional measurement results depends on the external calibration. Depending on the hardware, an already calibrated system may degrade due to physical stress or temperature shifts, i.e., rotational changes in relative position and time. This can lead to erroneous measurement results. Obviously, the system may be restored by additional new calibration processes. Depending on the application, this can be time-consuming and impractical. Calibration can be based on capturing a static scene with a defined target at a known distance. The concept of recalibration algorithms, for example, E. Rehder et al., "Online Stereo Camera Calibration From Scratch," June 2017, Conference: 2017 IEEE Intelligent Vehicles Symposium, DOI: 10.1109/IVS.2017.7995952, and T. Dang, "Continuous Stereo Self-Calibration by Camera Parameter Tracking," IEEE Transactions on Image Processing 8(7):1536-50, August 2009, DOI:10.1109/TIP. 2009.2017824 already exists. These recalibration approaches are based on discovering feature correspondences that should satisfy a system of equations (e.g., an epipolar condition) with respect to external parameters. A well-known example is the eight-point algorithm. For example, in the case of a three-dimensional measurement system with one camera and a laser dot projector, it can be assumed that the laser spots on the captured camera image are precisely assigned to the laser grid. This means that the correspondences can be precisely found. The relationship between the position of the laser spots on the camera image and the reference laser grid can be used to obtain a system of linear equations. These linear equations require at least eight corresponding laser spots on the reference grid. These equations can be solved by least-squares fitting. The adept utilization rate of the singular value decomposition can determine the rotation and translation of the camera and laser projector. However, if the correspondences are incorrect, the results can be very poor. Proper outlier detection of incorrect correspondences can be very important for this type of method. This procedure can also be used for stereo measurement. However, it may be necessary to find corresponding features in both camera images, such as edges or corners in the images. The location of these features can be used, for example, to determine an eight-point method. Generally, a third reconstruction method for structured light or stereo may require an externally calibrated system. This means that the search for correspondences is based on epipolar lines. However, this type of search is only one-dimensional. In the case of an uncalibrated system, the search for correspondences can no longer operate on one-dimensional epipolar lines. Conventional recalibration methods should search for correspondences within a two-dimensional image domain. This may follow an additional search algorithm for correspondences at the top of the three-dimensional reconstruction. E. Rehder et al., "Online Stereo Camera Calibration From Scratch," June 2017, Conference: 2017 IEEE Intelligent Vehicles Symposium, DOI: 10.1109/IVS.2017.7995952T. Dang, “Continuous Stereo Self-Calibration by Camera Parameter Tracking”, August 2009 IEEE Transactions on Image Processing8(7):1536-50, DOI:10.1109/TIP. 2009.2017824 An embo