US-12623667-B2 - Driver monitoring camera

US12623667B2US 12623667 B2US12623667 B2US 12623667B2US-12623667-B2

Abstract

A driver monitoring camera comprises an optical sensor configured to take an image of a driver of a vehicle, an inclination sensor configured to determine an inclination of the driver monitoring camera with respect to a vehicle coordinate system and a processor configured to determine a pose of the driver in a driver monitoring camera coordinate system from the image and configured to transform the pose of the driver in the driver monitoring camera coordinate system to a pose of the driver in the vehicle coordinate system using the determined inclination.

Inventors

KAH SOON ANG
Florin Burza

Assignees

Continental Automotive Technologies GmbH

Dates

Publication Date: 20260512
Application Date: 20221020
Priority Date: 20211022

Claims (13)

1 . A driver monitoring camera mounted directly on a steering column of a vehicle, said driver monitoring camera comprising: an optical sensor mounted on a camera printed circuit board and surrounded by a lens stack, said optical sensor being configured to obtain a sequence of images of a driver of the vehicle; an inclination sensor mounted on a processor printed circuit board, said inclination sensor being configured to determine an inclination value of the driver monitoring camera with respect to a vehicle coordinate system; and a processor mounted on the processor printed circuit board, said processor being in operative communication with the optical sensor and the inclination sensor via first and second connectors which interconnect the camera printed circuit board and the processor printed circuit board; wherein the inclination sensor is further configured to monitor the inclination value of the driver monitoring camera with respect to the vehicle coordinate system and repeatedly indicate the inclination value of the driver monitoring camera with respect to the vehicle coordinate system to the processor via a signal which is filtered to eliminate or reduce an influence of jitter on the signal; wherein the processor is further configured to determine a pose of the driver in a driver monitoring camera coordinate system from each image of the sequence of images and is further configured to transform the pose of the driver in the driver monitoring camera coordinate system to a pose of the driver in the vehicle coordinate system via a first inclination value indicated by the inclination sensor until the inclination sensor indicates a second inclination value which is different from the first inclination value by more than a predetermined threshold and to subsequently transform the pose of the driver in the driver monitoring camera coordinate system to the pose of the driver in the vehicle coordinate system via the second inclination value; and wherein the processor is further configured to perform a control action depending on a determined pose of the driver of the vehicle; and wherein the driver monitoring camera determines the pose of the driver in the vehicle coordinate system when vehicle interior landmarks are unavailable.
2 . The driver monitoring camera claim 1 , wherein the processor is further configured to determine a transformation matrix from the determined inclination value and transform the pose of the driver in the driver monitoring camera coordinate system to the pose of the driver in the vehicle coordinate system via the transformation matrix.
3 . The driver monitoring camera of claim 2 , wherein the inclination sensor is further configured to trigger an update of the transformation matrix by signaling a changed inclination value of the driver monitoring camera with respect to the vehicle coordinate system.
4 . The driver monitoring camera of claim 1 , wherein the processor is further configured to determine a transformation matrix from the determined inclination value and transform the pose of the driver in the driver monitoring camera coordinate system to the pose of the driver in the vehicle coordinate system via the transformation matrix; and wherein the processor is further configured to update a current transformation matrix when a change of value from the first inclination value based on which the processor has determined the current transformation matrix to the second inclination value determined by the inclination sensor is above a predetermined threshold.
5 . The driver monitoring camera of claim 1 , further comprising: a housing holding the optical sensor and the inclination sensor.
6 . The driver monitoring camera of claim 5 , wherein the housing further holds the processor.
7 . A vehicle comprising: a driver monitoring camera mounted directly on a steering column of the vehicle, the driver monitoring camera including: an optical sensor mounted on a camera printed circuit board and surrounded by a lens stack, said optical sensor being configured to obtain a sequence of images of a driver of the vehicle; an inclination sensor mounted on a processor printed circuit board, said inclination sensor being configured to determine an inclination value of the driver monitoring camera with respect to a vehicle coordinate system; and a processor mounted on the processor printed circuit board, said processor being in operative communication with the optical sensor and the inclination sensor via first and second connectors which interconnect the camera printed circuit board and the processor printed circuit board; wherein the inclination sensor is further configured to monitor the inclination value of the driver monitoring camera with respect to the vehicle coordinate system and repeatedly indicate the inclination value of the driver monitoring camera with respect to the vehicle coordinate system to the processor via a signal which is filtered to eliminate or reduce an influence of jitter on the signal; wherein the processor is further configured to determine a pose of the driver in a driver monitoring camera coordinate system from each image of the sequence of images and is further configured to transform the pose of the driver in the driver monitoring camera coordinate system to a pose of the driver in the vehicle coordinate system via a first inclination value indicated by the inclination sensor until the inclination sensor indicates a second inclination value which is different from the first inclination value by more than a predetermined threshold and to subsequently transform the pose of the driver in the driver monitoring camera coordinate system to a pose of the driver in the vehicle coordinate system via the second inclination value; wherein the processor is further configured to perform a control action depending on a determined pose of the driver in the vehicle coordinate system; and wherein the driver monitoring camera determines the pose of the driver in the vehicle coordinate system when vehicle interior landmarks are unavailable.
8 . The vehicle of claim 7 , wherein the vehicle is a convertible.
9 . The vehicle of claim 7 , wherein the processor is further configured to determine a transformation matrix from the determined inclination value and transform the pose of the driver in the driver monitoring camera coordinate system to the pose of the driver in the vehicle coordinate system via the transformation matrix.
10 . The vehicle of claim 9 , wherein the inclination sensor is further configured to trigger an update of the rotation matrix by signaling a changed inclination value of the driver monitoring camera with respect to the vehicle coordinate system.
11 . The vehicle of claim 7 , wherein the processor is further configured to determine a transformation matrix from the determined inclination value and transform the pose of the driver in the driver monitoring camera coordinate system to the pose of the driver in the vehicle coordinate system via the transformation matrix; and wherein the processor is further configured to update a current transformation matrix when a change of inclination from the first inclination based on which the processor has determined the current transformation matrix to the second inclination determined by the inclination sensor is above a predetermined threshold.
12 . The vehicle of claim 7 , further comprising: a housing holding the optical sensor and the inclination sensor.
13 . The vehicle of claim 12 , wherein the housing further holds the processor.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS The present application claims the benefit and/or priority of European Patent Application No. 21465553.2 filed on Oct. 22, 2021, the content of which is incorporated by reference herein. TECHNICAL FIELD The present invention relates to driver monitoring cameras. BACKGROUND In a vehicle, a driver monitoring camera may be used to monitor the driver and, by determining the pose of the driver, for example determine whether the driver is distracted or tired. A good place for a driver monitoring camera is the steering column because the driver typically adjusts the steering wheel such that the drivers head is in the field of view of a driver monitoring camera mounted to the steering column. However, this means that the position of the driving monitoring camera changes when the driver adjusts the steering wheel. This needs to be taken into account when determining the driver's pose. Image analysis may be used to compute the tilt angle of camera relative to world coordinate (in this case, the vehicle coordinate) via a transformation matrix. Image analysis of environmental components in camera coordinate system is correlated to the estimated tilt angle after the steering column tilt angle is adjusted. However, image analysis is dependent on known and predictable features, such as straight edges projected onto image plane, otherwise inaccuracies will be high. Accordingly, the lack of vehicle interior landmarks limits the ability of the driver monitoring camera to judge its position relative to the vehicle coordinate when it is moved. One such situation is if the camera is used in a convertible car with the roof removed and thus the number of vehicle interior landmarks is limited since most of the landmarks are situated on the roof. Moreover, image analysis driver monitoring for determining the camera position camera is computationally expensive (high computation resource consumption, in particular when the image analysis needs to function in case of a lack of clear vehicle landmarks). Since it is not known when the driver will adjust the steering column tilt, the camera position determination typically needs to run continuously and thus continuously occupies computational resources. SUMMARY Hence, the provision of efficient approaches which allow taking into account a change of position of a driver monitoring camera in case of a lack of vehicle landmarks is needed. Accordingly, a vehicle cabin sensing system includes an optical sensor configured to take an image of a driver of a vehicle, an inclination sensor configured to determine an inclination of the driver monitoring camera with respect to a vehicle coordinate system and a processor configured to determine a pose of the driver in a driver monitoring camera coordinate system from the image and configured to transform the pose of the driver in the driver monitoring camera coordinate system to a pose of the driver in the vehicle coordinate system using the determined inclination. According to one embodiment, the processor is configured to perform a control action depending on the pose of the driver in vehicle coordinate system. According to one embodiment, the optical sensor is configured to take a sequence of images of the driver of the vehicle, wherein the inclination sensor is configured to monitor the inclination of the driver monitoring camera with respect to the vehicle coordinate system and repeatedly indicate the inclination of the driver monitoring camera with respect to the vehicle coordinate system to the processor and wherein the processor is configured to determine a pose of the driver in a driver monitoring camera coordinate system from each image of the sequence of images and is configured to transform the pose of the driver in the driver monitoring camera coordinate system to a pose of the driver in the vehicle coordinate system using a first inclination indicated by the inclination sensor until the inclination sensor indicates a second inclination which is different from the first inclination by more than a predetermined threshold and then to transform the pose of the driver in the driver monitoring camera coordinate system to a pose of the driver in the vehicle coordinate system using the second inclination. According to one embodiment, the sensor is configured to only indicate the second indication if it is different from the first indication by more than the predetermined threshold. According to one embodiment, the processor is configured to determine a transformation matrix from the determined inclination and transform the pose of the driver in the driver monitoring camera coordinate system to the pose of the driver in the vehicle coordinate system using the transformation matrix. According to one embodiment, the sensor is configured to trigger an update of the rotation matrix by signalling a changed inclination of the driver monitoring camera with respect to a vehicle coordinate system. According to one