US-12617339-B2 - Driver monitoring device, driver monitoring method, and non-transitory recording medium

US12617339B2US 12617339 B2US12617339 B2US 12617339B2US-12617339-B2

Abstract

The driver monitoring device includes a judgment part configured to judge whether a driver of a vehicle is driving distracted by a predetermined judgment criteria based on an image generated by an imaging device capturing a face of the driver, a notification part configured to issue a warning to the driver when it is judged that the driver is driving distracted, and a change part configured to change the judgment criteria when an emergency warning light mounted in the vehicle is turned on.

Inventors

Masataka Sano
Hiromi NEMOTO
Yutaro TSUGAWA

Assignees

TOYOTA JIDOSHA KABUSHIKI KAISHA
DENSO CORPORATION

Dates

Publication Date: 20260505
Application Date: 20231129
Priority Date: 20230207

Claims (9)

1 . A driver monitoring device comprising a processor configured to: train a machine learning classifier to output an angle of a facial orientation or line of sight of a driver of a vehicle from an image data of a monitoring image so as to detect the angle of the facial orientation or line of sight of the driver; change a predetermined judgment criteria for judging whether the driver of the vehicle is driving distracted when an emergency warning light mounted in the vehicle is turned on; receive a monitoring image capturing a face of the driver of the vehicle, judge whether the driver is distracted based on the changed predetermined judgment criteria and by detecting the angle of the facial orientation or line of sight of the driver output from the trained machine learning classifier to determine whether the angle satisfies a predetermined distracted driving judgment condition; and issue a warning to the driver when it is judged that the driver is driving distracted.
2 . The driver monitoring device according to claim 1 , wherein the distracted driving judgment condition is that the angle is outside a threshold value range for greater than or equal to a threshold value time period, and the processor is configured to lengthen the threshold value time period or broaden the threshold value range when the emergency warning light is turned on.
3 . The driver monitoring device according to claim 1 , wherein the processor is configured to change the predetermined judgment criteria so that whether the driver is driving distracted is not judged when the emergency warning light is turned on.
4 . The driver monitoring device according to claim 1 , wherein the processor is configured to change the predetermined judgment criteria based on at least one of positional information and peripheral information of the vehicle when the emergency warning light is turned on.
5 . The driver monitoring device according to claim 4 , wherein the processor is configured to change the predetermined judgment criteria so that it becomes harder to judge that the driver is driving distracted when the vehicle is running on a highway compared to when the vehicle is not running on the highway.
6 . The driver monitoring device according to claim 4 , wherein the processor is configured to change the predetermined judgment criteria so that it becomes harder to judge that the driver is driving distracted when the number of peripheral vehicles located around the vehicle is large compared to when the number of the peripheral vehicles is small.
7 . The driver monitoring device according to claim 4 , wherein the processor is configured to change the predetermined judgment criteria so that it becomes harder to judge that the driver is driving distracted when there is a stationary object present ahead of the vehicle compared to when there is no stationary object present ahead of the vehicle.
8 . A driver monitoring method performed by a computer, comprising: training a machine learning classifier to output an angle of a facial orientation or line of sight of a driver of a vehicle from an image data of a monitoring image so as to detect the angle of the facial orientation or line of sight of the driver; changing a predetermined judgment criteria for judging whether the driver of the vehicle is driving distracted when an emergency warning light mounted in the vehicle is turned on; receiving a monitoring image capturing a face of the driver of the vehicle, judging whether the driver is driving distracted based on the changed predetermined judgment criteria and by detecting the angle of the facial orientation or line of sight of the driver output from the trained machine learning classifier to determine whether the angle satisfies a predetermined distracted driving judgment condition; and issuing a warning to the driver when it is judged that the driver is driving distracted.
9 . A non-transitory recording medium having recorded thereon a computer program, the computer program causing a computer to: train a machine learning classifier to output an angle of a facial orientation or line of sight of a driver of a vehicle from an image data of a monitoring image so as to detect the angle of the facial orientation or line of sight of the driver; change a predetermined judgment criteria for judging whether the driver of the vehicle is driving distracted when an emergency warning light mounted in the vehicle is turned on; receive a monitoring image capturing a face of the driver of the vehicle; judge whether the driver is driving distracted based on the changed predetermined judgment criteria and by detecting the angle of the facial orientation or line of sight of the driver output from the trained machine learning classifier to determine whether the angle satisfies a predetermined distracted driving judgment condition; and issue a warning to the driver when it is judged that the driver is driving distracted.

Description

This application claims priority to Japanese Patent Application No. 2023-016924 filed Feb. 7, 2023, the entire contents of which are herein incorporated by reference. FIELD The present disclosure relates to a driver monitoring device, a driver monitoring method, and a non-transitory recording medium. BACKGROUND PTL 1 describes to issue a warning to a driver by a buzzer sound etc., when it is judged that the driver is in a distracted driving state based on an image of a CCD camera capturing the driver. PTL 2 describes to judge that the driver is not driving distracted when the vehicle is in a state with the steering wheel being operated, in a state moving backward, or in a stopped state. PTL 3 describes to temporarily switch a judgment condition for judging distracted driving of a driver during an operating period of a turn signal. PTL 4 describes judging that a vehicle is liable to drive off a running section when a condition showing that the driver is intentionally trying to change a running lane does not stand and it is judged that the vehicle is approaching a solid lane line, and judging whether a condition showing that the driver is intentionally trying to change a running lane stands based on an on/off state of a winker and hazard light. CITATION LIST Patent Literature [PTL 1] Japanese Unexamined Patent Publication No. 03-167698[PTL 2] Japanese Unexamined Patent Publication No. 06-243367[PTL 3] Japanese Unexamined Patent Publication No. 2019-091275[PTL 4] Japanese Unexamined Patent Publication No. 2000-020897 SUMMARY Technical Problem As described in PTL 1, by issuing a warning about distracted driving of a driver, it is possible to prompt the driver to pay attention to the road ahead and in turn improve the safety of a vehicle. However, in a situation where an emergency warning light is turned on in a vehicle, the driver has to look in the side direction to determine the surrounding situation. If a warning is issued for such a suitable action, determination of the surrounding situation by the driver will be obstructed or the driver is liable to be irritated by the warning. Therefore, in view of this problem, an object of the present disclosure is to suppress unnecessary warnings when issuing warnings about distracted driving of a driver. Solution to Problem The summary of the present disclosure is as follows. (1) A driver monitoring device comprising a processor configured to: judge whether a driver of a vehicle is driving distracted by a predetermined judgment criteria based on an image generated by an imaging device capturing a face of the driver; issue a warning to the driver when it is judged that the driver is driving distracted; and change the judgment criteria when an emergency warning light mounted in the vehicle is turned on.(2) The driver monitoring device described in above (1), wherein the processor is configured to detect an angle of a facial orientation or line of sight of the driver based on the image, judge that the driver is driving distracted when the angle satisfies a predetermined distracted driving judgment condition, and change the distracted driving judgment condition when the emergency warning light is turned on.(3) The driver monitoring device described in above (2), wherein the distracted driving judgment condition is that the angle is outside a threshold value range for greater than or equal to a threshold value time period, and the processor is configured to lengthen the threshold value time period or broaden the threshold value range when the emergency warning light is turned on.(4) The driver monitoring device described in above (1), wherein the processor is configured to change the judgment criteria so that whether the driver is driving distracted is not judged when the emergency warning light is turned on.(5) The driver monitoring device described in any one of above (1) to (4), wherein the processor is configured to change the judgment criteria based on at least one of positional information and peripheral information of the vehicle when the emergency warning light is turned on.(6) The driver monitoring device described in above (5), wherein the processor is configured to change the judgment criteria so that it becomes harder to judge that the driver is driving distracted when the vehicle is running on a highway compared to when the vehicle is not running on a highway.(7) The driver monitoring device described in above (5), wherein the processor is configured to change the judgment criteria so that it becomes harder to judge that the driver is driving distracted when the number of peripheral vehicles located around the vehicle is large compared to when the number of the peripheral vehicles is small.(8) The driver monitoring device described in above (5), wherein the processor is configured to change the judgment criteria so that it becomes harder to judge that the driver is driving distracted when there is a stationary object present ahead of the vehicle compared to wh