Search

EP-4000005-B1 - METHOD FOR DETERMINING THE POSITION OF A FIRST IMAGE REGION IN A CORRESPONDENCE IMAGE, SOC AND CONTROL DEVICE, AND SYSTEM FOR CARRYING OUT THE METHOD, AND COMPUTER PROGRAM

EP4000005B1EP 4000005 B1EP4000005 B1EP 4000005B1EP-4000005-B1

Inventors

  • SIMON, STEPHAN

Dates

Publication Date
20260513
Application Date
20200707

Claims (16)

  1. Method for determining a position (58) of a first image region (50) of an initial image (24) in a correspondence image (52), comprising at least the following method steps: • providing the first image region (50) from the initial image (24) comprising at least one reference position (60), wherein the reference position (60) is arbitrarily selected, • providing the correspondence image (52), • determining a plurality of signature values s i (64) for different image positions (x i, y i) (62) in the first image region (50), wherein a signature value s j (64) characterizes surroundings of an image position (x i ,y i ) (62) in the first image region (50) and wherein an image position (x i ,y i ) (62) has a relative offset (Δx i ,Δy i ) = (x r ,y r ) - (x i ,y i ) (66) with respect to the reference position (x r ,y r ) (60), • assigning relative offsets (Δx i ,Δy i ) (66) of a plurality of image positions (x i ,y i ) (62) in the first image region (50) to respectively determined signature values s i (64) in an assignment specification (70), in particular in a lookup table, • determining a plurality of correspondence signature values s j (78) for different image positions (u j ,v j ) (80) in the correspondence image (52), wherein a correspondence signature value s j (78) characterizes surroundings of an image position (u j ,v j ) (80) in the correspondence image (52), • determining relative offsets (Δu j ,Δv j ) (66) for determined correspondence signature values s j (78) using the assignment and • calculating target positions (u 0 ,v 0 ) = (u j ,v j ) + (Δu j ,Δv j ) in the correspondence image (52) by summation of the relative offset (Δu j ,Δv j ) (66) and the respective image position (u j ,v j ) (80), • incrementally assigning a weighting value w (74) to a calculated target position by reading out from the assignment specification (70), wherein an incremental weighting value w' (86) is formed, • determining the position (u 0 ,v 0 )* (58) in the correspondence image (52) which corresponds to the reference position (u r ,v r ) (58) in the first image region (50), by finding an extremum (90) in the incremental weighting values w' (86).
  2. Method according to Claim 1, wherein the plurality of signature values s i (64) for different image positions (x j ,y j ) (62) in the first image region (50) are determined in the form of a matrix or table and/or wherein the plurality of correspondence signature values s j (78) for different image positions (u j ,v j ) (80) of the correspondence image (52) are determined in the form of a matrix or table.
  3. Method according to either of the preceding claims, wherein a respective signature value s i (64) defines an address (72) of the assignment specification (70), in particular of the lookup table.
  4. Method according to any of the preceding claims, wherein writing the assignment specification (70) comprises the following steps: • element-wise and synchronous iteration through all the matrix elements of a matrix with size 5 x 5 matrix elements; • signature value matrix (68), matrix of relative offsets (66) and matrix of weighting values (74); • reading out signature values (64) from the signature value matrix (68), relative offset (66) from the matrix of relative offsets (66) and weighting value (74) from the matrix of weighting values (74); • jumping to an address (72) of the assignment specification (70) which is defined using the signature value (64), and entering relative offset (66) and weighting value (74).
  5. Method according to any of the preceding claims, wherein the weighting values w (74) are assigned to target positions in the correspondence image (52) in the form of a matrix or table.
  6. Method according to any of the preceding claims, wherein a correspondence signature value s j (78) for an image position (u j ,v j ) (80) in the correspondence image (52) is formed analogously to, in particular according to the same specification as, a signature value s i (64) for an image position (x i ,y i ) (62) in the first image region (50).
  7. Method according to any of the preceding claims, wherein the assigned weighting value w (74) is selected depending on the relative offset (Δu j ,Δv j ) (66), in particular increases or decreases as the relative offset (66) increases.
  8. Method according to any of the preceding claims, wherein a position (u 0 ,v 0 )* (58) of the first image region (50) in the correspondence image (52) is determined if the incremental weighting value w' (86) assigned to the position (58) reaches a relative or absolute threshold value.
  9. Method according to any of the preceding claims, wherein a relative offset (Δu j , Δv j ) (66) for determined correspondence signature values s j (78) is calculated using the assignment and a scaling variable.
  10. Method according to any of the preceding claims, wherein ambiguities in the plurality of determined signature values s i (64) are eliminated by deleting and/or by merging the ambiguous signature values s i (64), wherein the merging is carried out by weighted averaging of relative offsets (66).
  11. Method according to any of the preceding claims, wherein the first image region (50) from the initial image (24) and/or the correspondence image (52) are/is captured and provided using at least one imaging sensor device (12a).
  12. SoC (System-on-Chip) configured to carry out a method according to any of Claims 1 to 11, wherein the SoC (300) is configured to capture at least one first image region (50) and a correspondence image (52) and to generate an output signal (302) depending on the determined position (u 0 ,v 0 )* (58) of the first image region (50) in the correspondence image (52).
  13. Control device configured to carry out a method according to any of Claims 1 to 11.
  14. System comprising at least one control device according to Claim 13 and at least one imaging sensor device (12a) configured to capture and provide a first image region (50) and a correspondence image (52).
  15. Computer program comprising instructions which, when executed by means of a processor device of a control device, cause the control device to carry out a method according to any of the preceding Claims 1 to 11.
  16. Computer-readable storage medium on which a computer program according to Claim 15 is stored.

Description

The invention relates to a method for determining a first image region in a correspondence image. The invention further relates to a system-on-a-chip (SoC) and a control device for carrying out the method. The invention also relates to a system, a computer-readable storage medium, and a computer program, each configured to execute the proposed methods. State of the art Several methods are known in the art for determining the position of a first image region in a corresponding image, for example, so-called "block matching." In this method, the first image region is compared with all potentially matching second image regions of the same size from the corresponding image. In each of these comparisons, all pixel pairs are compared. Each comparison evaluates the similarity or difference of the pixels under consideration, for example, an absolute difference in grayscale values. By summing the absolute differences, a score is obtained for each potential second image region, with the sum being higher the greater the difference between the image regions. Block matching is computationally very intensive, as the computational effort increases with both the dimensions of the first image region and the dimensions of the corresponding image. Furthermore, block matching is not very reliable because the result is not very discriminative; that is, no clear extremum and therefore no unambiguous position can be determined. Revised description page 2 - Final version Another possibility is the so-called "feature comparison," i.e., the comparison of at least one feature extracted from the first image region with features extracted from the corresponding image. This generates sequences of binary values, referred to as signatures, which characterize the corresponding image regions. The approach is based on the assumption that a similar signature of image regions indicates similar image regions. The signatures are then compared by bitwise comparing the first signature with the second signature (XOR comparison) and counting the bits that differ. The smaller the difference, the higher the correspondence between the image regions. However, comparing signatures can also be computationally intensive, as a comparison of signatures is required for each possible position in the corresponding image, with two signatures (for example, of 256 bits in length) having to be compared bitwise. Furthermore, the problem can also arise that the result of the feature comparison is not sufficiently discriminatory. The writing DE 103 51 778 A1 This reveals a method for processing image data of moving scenarios. For this purpose, corresponding pixels or image areas are identified in individual, temporally successive image datasets. From the WO 2005/048195 A2 The processing of image data from moving scenes, particularly for recognizing and tracking objects within them, is a known technique. This requires identifying corresponding pixels or image areas within the individual, sequentially generated image datasets. DE 10 2010 006522 A1 describes a method for analyzing corresponding pixels (B1, B2) in at least two image datasets, in which signature information is determined for each pixel (B1, B2) of the image datasets, whereby a correspondence hypothesis is created to determine corresponding pixels (B1, B2) for signature information that matches in both image datasets. Revised description page 2a - Final version In the DE 103 56 090 B3 A method for computer-aided determination of image areas in a digitized image is disclosed, wherein a first image area (B1) in a reference image (RB) is compared with several second image areas (B2) in a search area (S) of a search image (SB) and as a result a second image area (B2) is determined which has the highest similarity with the first image area (B1), and wherein the determined second image area (B2) is marked if the determined second image area (B2) is located at the edge of the search area (S). Disclosure of the invention The invention relates in a first aspect to a method for determining a position of a first image area in a correspondence image according to claim 1. The invention solves the problem of locating a position of a first image area in a corresponding image. This problem frequently arises in the fields of image processing, image analysis, machine vision, computer vision, etc. For example, the method according to the invention can be used for the following applications: Indirect distance measurement, wherein distances to at least two distant points are successively measured using a handheld laser distance meter. The laser distance meter indirectly determines the distance between the two points from the two distances and synchronously recorded camera images, and outputs this distance. According to the inventive method, it is possible to locate the first image area captured when aiming at the first measuring point in the corresponding image captured when aiming at the second measuring point and to determine