Search

EP-4246467-B1 - ELECTRONIC INSTRUMENT, MOVABLE APPARATUS, DISTANCE CALCULATION METHOD, AND STORAGE MEDIUM

EP4246467B1EP 4246467 B1EP4246467 B1EP 4246467B1EP-4246467-B1

Inventors

  • TANAKA, SHIN
  • NOBAYASHI, KAZUYA
  • OIGAWA, MAKOTO
  • KODAMA, SHIGEO
  • KAWAMOTO, TOMOYUKI

Dates

Publication Date
20260506
Application Date
20230303

Claims (16)

  1. An electronic instrument (150) comprising: at least one processor or circuit configured to function as: a first distance information acquisition unit (322) configured to acquire first distance information (D2) corresponding to an object included in an image signal; a second distance information acquisition unit (322) configured to acquire second distance information (D1) based on information of an end position of the object included in the image signal; and a third distance information acquisition unit (322) configured to acquire third distance information (D3) based on information of a size of the object included in the image signal; characterized by further comprising a distance information integration unit (322) configured to generate integrated distance information by combining and integrating the first distance information (D2), the second distance information (D1), the third distance information (D3), and a ranging value (Dm) from a kinetic model estimated from relative velocity parameters with respect to the object.
  2. The electronic instrument (150) according to claim 1, wherein the distance information integration unit (322) is configured to generate the integrated distance information by performing weighting addition of the first distance information (D2), the second distance information (D1) and the third distance information (D3).
  3. The electronic instrument (150) according to claim 1, wherein the first distance information acquisition unit (322) is configured to acquire the first distance information (D2) by a phase difference ranging method.
  4. The electronic instrument (150) according to claim 3, wherein the first distance information acquisition unit (322) is configured to acquire the first distance information (D2) by the phase difference ranging method on the basis of signals from a first photoelectric conversion unit (411) and a second photoelectric conversion unit (412) disposed within a pixel of an imaging element (302).
  5. The electronic instrument (150) according to claim 3, wherein the first distance information acquisition unit (322) is configured to acquire the first distance information (D2) by the phase difference ranging method on the basis of two image signals from a stereo camera.
  6. The electronic instrument (150) according to claim 1, wherein the third distance information acquisition unit (322) is configured to acquire the third distance information (D3) on the basis of information of a width or a height of the object included in the image signal.
  7. The electronic instrument (150) according to claim 1, wherein the distance information integration unit (322) is configured to generate the integrated distance information also on the basis of a kind of the object.
  8. The electronic instrument (150) according to claim 1, wherein the distance information integration unit (322) is configured to generate the integrated distance information on the basis of histories of the respective distance information of the object.
  9. The electronic instrument (150) according to claim 1, wherein at least one processor or circuit is further configured to function as: a fourth distance information acquisition unit (120) configured to acquire fourth distance information with respect to the object using a radar, and wherein the distance information integration unit (322) is configured to generate the integrated distance information on the basis of the fourth distance information.
  10. The electronic instrument (150) according to claim 1, wherein the distance information integration unit (322) is configured to generate the integrated distance information by using a variance (V) in the probability distribution with respect to the first distance information (D2), the second distance information (D1), and the third distance information (D3).
  11. The electronic instrument (150) according to claim 10, wherein the variance (V) in the probability distribution with respect to the first distance information (D2) is determined based on a block size of block matching in a case where the first distance information (D2) is acquired by a phase difference ranging method on the basis of signals from a first photoelectric conversion unit (411) and a second photoelectric conversion unit (412) disposed within a pixel of an imaging element (302).
  12. A movable apparatus (100), wherein at least one processor or circuit is further configured to function as: a path generation unit (130, 330) configured to generate path information on the basis of the integrated distance information acquired by the electronic instrument (150) according to claim 1.
  13. The movable apparatus (100) according to claim 12, wherein at least one processor or circuit is further configured to function as: a path generation unit (130, 330) configured to generate the path information on the basis of the integrated distance information and a speed of the movable apparatus (100).
  14. The movable apparatus (100) according to claim 12, wherein at least one processor or circuit is further configured to function as: a driving control unit (170) configured to control driving of the movable apparatus (100) on the basis of the path information generated by the path generation unit (130, 330).
  15. A distance calculation method comprising: a first distance information acquiring step (S1404) of acquiring first distance information (D2) corresponding to an object included in an image signal; a second distance information acquiring step (D1402) of acquiring second distance information (D1) based on information of an end position of the object included in the image signal; and a third distance information acquiring step (D1402) of acquiring third distance information (D3) based on information of a size of the object included in the image signal; characterized by a distance information integrating step (D1406) of generating integrated distance information by combining and integrating the first distance information (D2), the second distance information (D1), the third distance information (D3), and a ranging value (Dm) from a kinetic model estimated from relative velocity parameters with respect to the object.
  16. A non-transitory computer-readable storage medium configured to store a computer program comprising instructions for executing following processes: a first distance information acquiring step (D1404) of acquiring first distance information (D2) corresponding to an object included in an image signal; a second distance information acquiring step (D1402) of acquiring second distance information (D1) based on information of an end position of the object included in the image signal; and a third distance information acquiring step (D1406) of acquiring third distance information based on information of a size of the object included in the image signal; characterized by a distance information integrating step (S1406) of generating integrated distance information by combining and integrating the first distance information (D2), the second distance information (D1), the third distance information (D3), and a ranging value (Dm) from a kinetic model estimated from relative velocity parameters with respect to the object.

Description

BACKGROUND OF THE INVENTION Field of the Invention The present invention relates to an electronic instrument, a movable apparatus, a distance calculation method, a storage medium, and the like for acquiring a distance from the movable apparatus to an object. Description of the Related Art There is a photographing device that has a sensor in which a plurality of pixel regions having a photoelectric conversion function are two-dimensionally disposed and is able to acquire an image signal and distance information in each of the pixel regions. In a solid-state imaging element disclosed in Japanese Patent Laid-Open No. JP 2007-281 296 A, pixels having a ranging function are disposed for some or all pixels in the imaging element, and a subject distance is detected on the basis of a phase difference detected on an imaging surface (imaging surface phase difference system). That is, positional deviation is computed on the basis of a correlation between two image signals based on images generated by luminous fluxes which have passed through different pupil regions in an imaging optical system provided in the photographing device, and a distance is acquired on the basis of the positional deviation. A correlation between two image signals is evaluated using a technique such as a region-based matching technique of evaluating a correlation by cutting out an image signal included in a predetermined collation region from each of the image signals. However, erroneous evaluation of a correlation may occur due to a subject or photographing conditions, such as a case in which a subject included in an image signal has little change in contrast, a case in which an image signal includes a large amount of noise, and the like. If occurrence of erroneous evaluation of a correlation included therein reaches or exceeds a certain level, there is concern that a computed amount of positional deviation between two image signals may have an error and accuracy of an acquired distance may deteriorate. Document WO 2020/213 386 A1 discloses an object position detection device, a travel control system and a travel control method. The object position detection device has: a first distance detection unit that detects the distance to an object from an image acquired from a first and a second imaging unit; a first position detection unit that detects the position of the object on the basis of the distance detected by the first distance detection unit; a processing area setting unit that sets a processing area, in which the object detected by the first distance detection unit is included, in an image obtained from any of a plurality of imaging units including the first and second imaging units; a second position detection unit that detects the position of the object in the processing area; a second distance detection unit that detects the distance to the object on the basis of the position of the object detected by the second position detection unit; an orientation detection unit that detects the orientation of the object on the basis of the position of the object detected by the second position detection unit; and a determination unit that determines the distance to and the position of the object in accordance with the information detected by each of the first and second distance detection units, the first and second position detection units, and the orientation detection unit. Document US 2021/ 097 707 A1 discloses an information processing device and a movement device. An object distance is calculated on the basis of a captured image of a camera that captures an image in a direction that is orthogonal to a movement direction of a movement device or in a direction having an orthogonal component. A data processing unit detects a point at infinity from a captured image of a camera that captures an image in a direction that is orthogonal to a movement direction of a movement device or in a direction having an orthogonal component, and calculates an object distance by using information relating to a positional relationship between a position of the detected point at infinity and a position of an object in the captured image. The data processing unit detects, from the captured image, a plurality of parallel lines on a real world that extends in a direction away from a camera position, and determines an intersection point on extended lines of the detected plurality of parallel lines to be the point at infinity. Alternatively, an intersection point on respective extended lines of the straight lines in an image frame unit is determined to be the point at infinity, directions of the straight lines changing on the captured image in accordance with a movement of the camera. Document US 2015/ 015 673 A1 discloses a distance calculator which includes a monocular estimated distance calculation section, a stereo estimated distance calculation section, and an output distance calculation section. The monocular estimated distance calculation section calculates the