US-12620132-B2 - Depth estimation apparatus, calibration method, and non-transitory computer readable recording medium

US12620132B2US 12620132 B2US12620132 B2US 12620132B2US-12620132-B2

Abstract

A depth estimation apparatus executes a first process of calculating a calibration value for an estimated depth and a second process of calibrating the estimated depth based on the calibration value. The first process includes: specifying a plane area in the image in which a horizontal plane or vertical plane is reflected; setting a plurality of partial regions in the image; calculating a regression plane representing the horizontal plane or the vertical plane for each partial region; and calculating the calibration for each partial region by comparing an installation position of the camera with a position of the camera with respect to the regression plane. The second process includes calibrating the estimated depth for each partial region based on the calibration value corresponding to each partial region.

Inventors

Quan KONG
Mustafa ERDOGAN
Norimasa Kobori

Assignees

TOYOTA JIDOSHA KABUSHIKI KAISHA

Dates

Publication Date: 20260505
Application Date: 20240321
Priority Date: 20230519

Claims (5)

1 . A depth estimation apparatus comprising: processing circuitry configured to analyze an image captured by a camera to estimate a depth in the image; and a memory storing information on an installation position of the camera with respect to a horizontal plane or a vertical plane, wherein the processing circuitry is further configured to execute: a first process of calculating a calibration value for an estimated depth; and a second process of calibrating the estimated depth based on the calibration value, the first process includes: specifying a plane area in the image in which the horizontal plane or the vertical plane is reflected; setting a plurality of partial regions in the image; calculating a regression plane, for each of the plurality of partial regions, representing the horizontal plane or the vertical plane based on the estimated depth of the plane area included in the each partial region; and calculating the calibration value for each of the plurality of partial regions by comparing the installation position with a position of the camera with respect to the regression plane, and the second process includes calibrating the estimated depth for each of the plurality of partial regions based on the calibration value corresponding to the each partial region.
2 . The depth estimation apparatus according to claim 1 , wherein each of the plurality of partial regions is a region surrounded by sides with a curvature, and the curvature is larger with respect to the each partial region which is farther from a center of the image.
3 . The depth estimation apparatus according to claim 1 , wherein a size of each of the plurality of partial regions is smaller as the each partial region is farther from a center of the image.
4 . A calibration method for calibrating an estimated depth in an image captured by a camera, the calibration method, which is executed by a computer, comprising: acquiring information on an installation position of the camera with respect to a horizontal plane or a vertical plane; specifying a plane area in the image in which the horizontal plane or the vertical plane is reflected; setting a plurality of partial regions in the image; calculating a regression plane, for each of the plurality of partial regions, representing the horizontal plane or the vertical plane based on the estimated depth of the plane area included in the each partial region; calculating the calibration value for each of the plurality of partial regions by comparing the installation position with a position of the camera with respect to the regression plane; and calibrating the estimated depth for each of the plurality of partial regions based on the calibration value corresponding to the each partial region.
5 . A non-transitory computer readable recording medium on which a computer program for calibrating an estimated depth in an image captured by a camera, the computer program, when executed by a computer, causing the computer to execute: acquiring information on an installation position of the camera with respect to a horizontal plane or a vertical plane; specifying a plane area in the image in which the horizontal plane or the vertical plane is reflected; setting a plurality of partial regions in the image; calculating a regression plane, for each of the plurality of partial regions, representing the horizontal plane or the vertical plane based on the estimated depth of the plane area included in the each partial region; calculating the calibration value for each of the plurality of partial regions by comparing the installation position with a position of the camera with respect to the regression plane; and calibrating the estimated depth for each of the plurality of partial regions based on the calibration value corresponding to the each partial region.

Description

CROSS-REFERENCE TO RELATED APPLICATION The present disclosure claims priority to Japanese Patent Application No. 2023-083343, filed on May 19, 2023, the contents of which application are incorporated herein by reference in their entirety. TECHNICAL FIELD The present disclosure relates to a technique for analyzing an image captured by a camera to estimate a depth in the image. BACKGROUND ART Patent Literature 1 discloses, regarding a ranging apparatus including a plurality of imaging systems, a calibration method for estimating a camera parameter representing characteristics of each of the plurality of imaging systems. In the calibration method disclosed in Patent Literature 1, the camera parameter is estimated using information acquired by capturing an image of a reference chart positioned to be in a predetermined positional relationship with the ranging apparatus. In addition, there are the following Patent Literature 2 and Patent Literature 3 as documents showing the technical level of this technical field. LIST OF RELATED ART Patent Literature 1: WO 2009/141998 A1Patent Literature 2: JP 2021/117130 APatent Literature 3: JP 2013/187822 A SUMMARY The technique disclosed in Patent Literature 1 is applied to an apparatus (e.g., an apparatus using a stereo camera) that recognizes a depth in an image by detecting disparity between multiple images. On the other hand, a technique of estimating a depth in an image by analyzing one image captured by a monocular camera or the like has been considered. Such a technique has the advantage of low cost and space saving because it requires less equipment. Further, it has also the advantage of being applicable to cameras used for other purposes. Particularly in recent years, it has been considered to analyze an image using a trained machine learning model. By the way, the appearance of an image to be analyzed changes depending on the characteristics of a camera that captures the image. Therefore, in order to ensure estimation accuracy, it is required to perform calibration according to the appearance of the image and the characteristics of the camera. However, in the related art, performing calibration requires manual work or an additional equipment, resulting in considerable effort and cost. For example, when analyzing an image using a trained machine learning model, calibration is performed by retraining the parameters. An object of the present disclosure is to provide a technique for analyzing an image to estimate a depth in the image, which is capable of realizing highly accurate depth estimation without requiring manual work or an additional equipment. A first aspect of the present disclosure relates to a depth estimation apparatus. The depth estimation apparatus comprises: one or more processors configured to analyze an image captured by a camera to estimate a depth in the image; anda memory storing information on an installation position of the camera with respect to a horizontal plane or a vertical plane. The one or more processors are further configured to execute: a first process of calculating a calibration value for an estimated depth; anda second process of calibrating the estimated depth based on the calibration value. The first process includes: specifying a plane area in the image in which the horizontal plane or the vertical plane is reflected;setting a plurality of partial regions in the image;calculating a regression plane, for each of the plurality of partial regions, representing the horizontal plane or the vertical plane based on the estimated depth of the plane area included in the each partial region; andcalculating the calibration value for each of the plurality of partial regions by comparing the installation position with a position of the camera with respect to the regression plane. The second process includes calibrating the estimated depth for each of the plurality of partial regions based on the calibration value corresponding to the each partial region. A second aspect of the present disclosure relates to a calibration method for calibrating an estimated depth in an image captured by a camera. The calibration method, which is executed by a computer, comprises: acquiring information on an installation position of the camera with respect to a horizontal plane or a vertical plane;specifying a plane area in the image in which the horizontal plane or the vertical plane is reflected;setting a plurality of partial regions in the image;calculating a regression plane, for each of the plurality of partial regions, representing the horizontal plane or the vertical plane based on the estimated depth of the plane area included in the each partial region;calculating the calibration value for each of the plurality of partial regions by comparing the installation position with a position of the camera with respect to the regression plane; andcalibrating the estimated depth for each of the plurality of partial regions based on the calibration value correspo