Search

DE-112024002486-T5 - IMAGE PROCESSING DEVICE

DE112024002486T5DE 112024002486 T5DE112024002486 T5DE 112024002486T5DE-112024002486-T5

Abstract

An image processing device is provided that appropriately filters the displacement amount of a corresponding point between images, which is incorrectly calculated due to a mismatch in the image acquisition environment between two images or a low level of texture on an object. This image processing device comprises: a search unit that looks for a corresponding point between two images; a reliability calculation unit that, within a search area for corresponding points, calculates a field similarity that indicates the degree of similarity between a luminance field of a pixel group in one image and a luminance field of a pixel group in the other image, and calculates the reliability with respect to the displacement amount (parallax) of a corresponding point between the two images based on the calculated field similarity; and a correction unit that invalidates or corrects the displacement amount (parallax) if the calculated reliability falls below a predetermined threshold.

Inventors

  • UCHIDA YUSUKE
  • INATA KEISUKE
  • TAKADA MASASHI

Assignees

  • ASTEMO LTD

Dates

Publication Date
20260507
Application Date
20240823
Priority Date
20230912

Claims (7)

  1. Image processing device comprising: a search unit that searches for corresponding points between two images; a reliability calculation unit that calculates a field similarity, which indicates a similarity between a luminance field in a pixel group in one image and a luminance field in a pixel group in the other image within a search area of the corresponding points, and calculates the reliability for a displacement amount of the corresponding points between the two images based on the calculated field similarity; and a correction unit that invalidates or corrects the displacement amount if the calculated reliability falls below a predetermined threshold.
  2. Image processing device according to Claim 1 , wherein the reliability calculation unit calculates a correction value between the luminance field in one image and the luminance field in the other image in the search area as field similarity and calculates the reliability based on the calculated correction value.
  3. Image processing device according to Claim 1 , wherein the reliability calculation unit calculates a correction value between the luminance field in one image and the luminance field in the other image in the search area as field similarity, and calculates the reliability based on a difference value between a maximum value and a minimum value of the calculated correction value.
  4. Image processing device according to Claim 1 , wherein the reliability calculation unit calculates a correction value between the luminance field in one image and the luminance field in the other image in the search area as field similarity, calculates a difference value between a maximum value and a minimum value of the calculated correction value, calculates a first reliability as reliability based on the calculated correction value and a second reliability as reliability based on the calculated difference value, and the correction unit invalidates or corrects the shift amount if the first reliability falls below a first threshold or if the second reliability falls below a second threshold.
  5. Image processing device according to Claim 4 , further comprising a determination unit that performs a first determination to ascertain whether the first reliability falls below the first threshold and performs a second determination to ascertain whether the second reliability falls below the second threshold, the determination unit performing the first determination with priority over the second determination.
  6. Image processing device according to Claim 1 , where the two images are images taken by a multitude of cameras whose fields of view overlap at least partially, and the displacement amount is the parallax between the two images.
  7. Image processing device according to Claim 1 , where the two images are images taken at different times by the same camera, and the shift amount is a time-dependent shift amount between the two images.

Description

Technical field The present invention relates to an image processing device. Technical background A stereo camera system is known that measures the distance to an object located within a field of view using the principle of triangulation, based on two images taken by two cameras whose fields of view at least partially overlap. The principle of triangulation is to calculate the distance from the cameras to the object based on the positional deviation (shift) of images of the same object taken by the two cameras, i.e., based on the parallax. The parallax is calculated by finding the point where an image of the object appears in one image but not in the other. Several methods have been proposed for calculating parallax. One classical method, for example, is block matching, which searches for a region with the highest similarity (a region with low dissimilarity) in the other image with respect to a region consisting of a large number of pixels in one image. In an ideal imaging environment, the exact parallax can be calculated from two images. In reality, however, misalignment in the imaging environment, such as blurring or lighting between two images, or low texture on an object, such as a road surface, can lead to miscalculation and thus an incorrect parallax. This can result in object detection errors, such as failing to detect an object lying on a road or the condition of the road surface. It's important to note that low texture means the object's texture has little variation, making it difficult to discern a texture edge in an image of the object. As a countermeasure to performing position detection of an object with high accuracy independent of the imaging environment, patent literature 1, for example, describes a method for checking parallaxes with low reliability using a dissimilarity index between two images used to calculate the parallax. Citation list Patent literature Patent literature 1: JP 2016-38886 A Summary of the invention Technical In patent literature 1, the reliability of the parallax is defined as a change in an index representing similarity or dissimilarity, such as SAD (Sum of Absolute Difference), which is used to search for corresponding points between two images. Specifically, patent literature 1 defines the reliability of the parallax as sharpness, indicating the degree to which a maximum value of the index is expressed. Consequently, patent literature 1 suggests the possibility that a parallax with low reliability might not be filtered out if a misalignment in the imaging environment results in a high sharpness misalignment between two images. Furthermore, patent literature 1 uses a dissimilarity index derived from comparing individual pixels in each image as the parallax reliability. Consequently, patent literature 1 cannot account for a feature of an object represented by an arrangement of a large number of pixels in each image when calculating parallax reliability. Therefore, patent literature 1 makes it difficult to detect a matching error resulting from a low level of object texture, and even if this matching error leads to an incorrect parallax calculation, there is a possibility that this incorrect parallax cannot be filtered out. Such a problem applies not only to a stereo camera system, but similarly to a monocular camera system that searches for corresponding points between two images taken at different times by the same camera and calculates a time shift of the corresponding points. The present invention was developed taking into account the foregoing and aims to reduce the displacement of a corresponding point between images, which is caused by a misalignment in the imaging environment between two images or by a To filter out objects with low texture that were incorrectly calculated, it is necessary to do so appropriately. Solution to the problem To solve the above-mentioned problem, an image processing device of the present invention comprises: a search unit that searches for corresponding points between two images, a reliability calculation unit that calculates a field similarity indicating a similarity between a luminance field in a pixel group in one image and a luminance field in a pixel group in the other image in a search area of the corresponding points, and calculates the reliability for a displacement amount of the corresponding points between the two images based on the calculated field similarity, and a correction unit that invalidates or corrects the displacement amount if the calculated reliability falls below a predetermined threshold. Advantageous effects of the invention According to the present invention, the displacement amount of a corresponding point between images, which was incorrectly calculated due to a mismatch in the imaging environment between two images or due to a low texture of an object, can be appropriately filtered out. Problems, configurations and effects other than those described above will become apparent from the following descriptio