US-12617420-B2 - Driver warning system

Abstract

A vehicle includes a ranged sensor that generates time-series data indicating positions of objects in an environment surrounding the vehicle, a user interface configured to warn the driver of a predicted collision between the vehicle and one of the objects in the environment, and at least one processor including an ECU operatively connected to the ranged sensor and the user interface. The processor records control inputs by the driver driving the vehicle, and develops a driver behavior model associated with the driver driving the vehicle based on the control inputs. The processor also predicts trajectories of the objects and the vehicle based on the time-series data and the driver behavior model, and predicts a collision between the vehicle and one of the objects based on the predicted trajectories. The processor also generates a warning indicating the predicted collision to the driver.

Inventors

• Aolin XU
• Chenran LI
• Enna Sachdeva
• Teruhisa Misu
• Behzad Dariush
• Kentaro Yamada
• Kikuo Fujimura

Assignees

• HONDA MOTOR CO., LTD.

Dates

Publication Date

20260505

Application Date

20240327

Claims (20)

1. 1 . A vehicle comprising: a ranged sensor that generates time-series data indicating positions of objects in an environment surrounding the vehicle; a user interface configured to warn a driver of a predicted collision between the vehicle and one of the objects in the environment; and at least one processor including an electronic control unit operatively connected to the ranged sensor and the user interface, wherein the at least one processor: records control inputs by the driver driving the vehicle; develops a driver behavior model associated with the driver driving the vehicle based on the control inputs; predicts trajectories of the objects and the vehicle based on the time-series data and the driver behavior model, and predicts a collision between the vehicle and one of the objects based on the predicted trajectories; and generates a warning indicating the predicted collision to the driver.
2. 2 . The vehicle of claim 1 , wherein the at least one processor: predicts a future vehicle state based on the driver behavior model, wherein the future vehicle state occurs after generating the warning, and indicates that the driver noticed the warning; records a present vehicle state associated with the control inputs; and controls a predetermined intensity of warnings by the user interface when a present vehicle state matches the future vehicle state.
3. 3 . The vehicle of claim 1 , wherein the at least one processor: predicts a future vehicle state based on the driver behavior model, wherein the future vehicle state occurs after generating the warning, and indicates that the driver did not notice the warning; records a present vehicle state associated with the control inputs; and controls an intensity of warnings by the user interface when the present vehicle state matches the future vehicle state.
4. 4 . The system of claim 3 , wherein controlling the intensity of the warning includes at least one of: reducing a safety threshold for generating the warning; increasing a duration of time the warning is generated; increasing a number of times the warning is generated during a period of time before the predicted collision; adding at least one of an audio output and a visual output to the warning; and increasing an intensity of at least one of the audio output and the visual output.
5. 5 . The vehicle of claim 1 , wherein the control inputs are first control inputs, the driver behavior model is a first driver behavior model, the predicted collision is a first predicted collision, the warning is a first warning, and after generating the first warning, the at least one processor: records second control inputs by the driver driving the vehicle; develops a second driver behavior model associated with the driver driving the vehicle based on the second control inputs; predicts trajectories of the objects and the vehicle based on the time-series data and the second driver behavior model, and predicts a collision between the vehicle and one of the objects based on the predicted trajectories; and generates a second warning indicating the predicted collision to the driver.
6. 6 . The vehicle of claim 1 , wherein for a duration of the driver driving the vehicle, the at least one processor repeatedly: records the control inputs by the driver driving the vehicle, and a present vehicle state associated with the control inputs; develops the driver behavior model associated with the driver driving the vehicle based on the control inputs and the present vehicle state; predicts the trajectories of the objects and the vehicle based on the time-series data and the driver behavior model, and predicts a collision between the vehicle and one of the objects based on the predicted trajectories; generates a warning indicating the predicted collision to the driver, wherein the warning has a predetermined intensity; predicts future vehicle states based on the driver behavior model and the generated warning; and adjusts the predetermined intensity of the warning when the present vehicle state matches one of the future vehicle states.
7. 7 . The vehicle of claim 1 , wherein the at least one processor: records a present vehicle state associated with the control inputs; develops the driver behavior model based on the control inputs and the present vehicle state; determines whether the driver noticed the generated warning based on the present vehicle state; and controls a predetermined intensity of warnings by the user interface based whether the driver noticed the generated warning.
8. 8 . The vehicle of claim 1 , wherein the at least one processor: predicts first future vehicle states based on the driver behavior model, wherein the first future vehicle states occur after generating the warning, and indicate whether the driver noticed the warning; predicts second future vehicle states that are each subsequent to, and depend from one of the first future vehicle states; records a present vehicle state; and controls a predetermined intensity of warnings by the user interface when the present vehicle state matches one of the first future vehicle states, and then matches one of the second future vehicle states subsequent to the first future vehicle state.
9. 9 . The vehicle of claim 8 , wherein the at least one processor predicts iterations of subsequent future vehicle states, including the second future vehicle states, to a time horizon corresponding with the predicted trajectories, wherein the iterations of subsequent future vehicle states each depend from one of the first future vehicle states or an intermediate iteration of subsequent future vehicle states, and indicate whether the driver noticed the warning; and controls the predetermined intensity of warnings by the user interface when the present vehicle state matches one of the first future vehicle states, and then matches a plurality of the iterations of subsequent future vehicle states that depend from the matched future vehicle state.
10. 10 . A method for generating a warning to a driver of a vehicle, the method comprising: generating time-series data indicating positions of objects in an environment surrounding the vehicle; recording control inputs by the driver driving the vehicle; developing a driver behavior model associated with the driver driving the vehicle based on the control inputs; predicting trajectories of the objects and the vehicle based on the time-series data and the driver behavior model, and predicting a collision between the vehicle and one of the objects based on the predicted trajectories; and generating a warning indicating the predicted collision to the driver.
11. 11 . The method of claim 10 , further comprising: predicting a future vehicle state based on the driver behavior model, wherein the future vehicle state occurs after generating the warning, and indicates that the driver noticed the warning; recording a present vehicle state associated with the control inputs; and controlling a predetermined intensity of warnings generated when a present vehicle state matches the future vehicle state.
12. 12 . The method of claim 10 , further comprising: predicting a future vehicle state based on the driver behavior model, wherein the future vehicle state occurs after generating the warning, and indicates that the driver did not notice the warning; recording a present vehicle state associated with the control inputs; and controlling a predetermined intensity of warnings generated when a present vehicle state matches the future vehicle state.
13. 13 . The system of claim 12 , wherein controlling the predetermined intensity of the warning includes at least one of: reducing a safety threshold associated with the predicted collision for generating the warning; increasing a duration of time the warning is generated; increasing a number of times the warning is generated during a period of time before the predicted collision; adding at least one of an audio output and a visual output to the warning; and increasing an intensity of at least one of the audio output and the visual output.
14. 14 . The method of claim 10 , wherein the control inputs are first control inputs, the driver behavior model is a first driver behavior model, the predicted collision is a first predicted collision, the warning is a first warning, and after generating the first warning, the method further comprises: recording second control inputs by the driver driving the vehicle; developing a second driver behavior model associated with the driver driving the vehicle based on the second control inputs; predicting trajectories of the objects and the vehicle based on the time-series data and the second driver behavior model, and predicting a collision between the vehicle and one of the objects based on the predicted trajectories; and generating a second warning indicating the predicted collision to the driver.
15. 15 . The method of claim 10 , wherein for a duration of the driver driving the vehicle, the method comprises repeatedly: recording the control inputs by the driver driving the vehicle, and a present vehicle state associated with the control inputs; developing the driver behavior model associated with the driver driving the vehicle based on the control inputs and the present vehicle state; predicting the trajectories of the objects and the vehicle based on the time-series data and the driver behavior model, and predicting a collision between the vehicle and one of the objects based on the predicted trajectories; generating a warning indicating the predicted collision to the driver, wherein the warning has a predetermined intensity; predicting future vehicle states based on the driver behavior model and the generated warning; and adjusting the predetermined intensity of the warning when the present vehicle state matches one of the future vehicle states.
16. 16 . A non-transitory computer readable storage medium storing instructions that, when executed by at least one processor, causes the at least one processor to perform a method, the method comprising: generating time-series data indicating positions of objects in an environment surrounding a vehicle; recording control inputs by a driver driving the vehicle; developing a driver behavior model associated with the driver driving the vehicle based on the control inputs; predicting trajectories of the objects and the vehicle based on the time-series data and the driver behavior model, and predicting a collision between the vehicle and one of the objects based on the predicted trajectories; and generating a warning indicating the predicted collision to the driver.
17. 17 . The non-transitory computer readable storage medium of claim 16 , wherein the method further comprises: predicting a future vehicle state based on the driver behavior model, wherein the future vehicle state occurs after generating the warning, and indicates that the driver noticed the warning; recording a present vehicle state associated with the control inputs; and controlling a predetermined intensity of warnings generated when a present vehicle state matches the future vehicle state.
18. 18 . The non-transitory computer readable storage medium of claim 16 , wherein the method further comprises: predicting a future vehicle state based on the driver behavior model, wherein the future vehicle state occurs after generating the warning, and indicates that the driver did not notice the warning; recording a present vehicle state associated with the control inputs; and controlling a predetermined intensity of warnings generated when a present vehicle state matches the future vehicle state.
19. 19 . The non-transitory computer readable storage medium of claim 16 , wherein the control inputs are first control inputs, the driver behavior model is a first driver behavior model, the predicted collision is a first predicted collision, the warning is a first warning, and after generating the first warning, the method further comprises: recording second control inputs by the driver driving the vehicle; developing a second driver behavior model associated with the driver driving the vehicle based on the second control inputs; predicting trajectories of the objects and the vehicle based on the time-series data and the second driver behavior model, and predicting a collision between the vehicle and one of the objects based on the predicted trajectories; and generating a second warning indicating the predicted collision to the driver.
20. 20 . The non-transitory computer readable storage medium of claim 16 , wherein for a duration of the driver driving the vehicle, the method comprises repeatedly: recording the control inputs by the driver driving the vehicle, and a present vehicle state associated with the control inputs; developing the driver behavior model associated with the driver driving the vehicle based on the control inputs and the present vehicle state; predicting the trajectories of the objects and the vehicle based on the time-series data and the driver behavior model, and predicting a collision between the vehicle and one of the objects based on the predicted trajectories; generating a warning indicating the predicted collision to the driver, wherein the warning has a predetermined intensity; predicting future vehicle states based on the driver behavior model and the generated warning; and adjusting the predetermined intensity of the warning when the present vehicle state matches one of the future vehicle states.

Description

BACKGROUND Road traffic plays an important role in people's lives. With the development of the complexity of city road networks in particular, it is crucial for an advanced driver assistance system to be able to alert a human driver to potential risks while driving a vehicle. Studies of human driver behavior show that existing driver warning technologies, mainly including forward collision warning systems and unsafe lane change warning systems, can reduce a risk of collision caused by human error. However, studies show that the human drivers' reactions to warnings vary with different types of warnings and drivers. For example, studies indicate that driver age and years of driving experience, collision type, and warning type can affect driving performance. In this regard, most methods in relevant literature mainly generate warnings in a one-shot manner without modeling an ego driver's reactions and surrounding objects. Meanwhile, triggering conditions of generating warnings are mostly rule-based threshold-checking based on a current state of the vehicle, such as a time-to-collision (TTC) and the minimum safety distance. As a consequence, studies have emphasized the importance of executing smoother and more comfortable braking maneuvers to assist drivers in avoiding not only identified obstacles but also collisions with subsequent vehicles. BRIEF DESCRIPTION According to one aspect, a vehicle includes a ranged sensor that generates time-series data indicating positions of objects in an environment surrounding the vehicle, and a user interface configured to warn the driver of a predicted collision between the vehicle and one of the objects in the environment. The vehicle also includes at least one processor including an electronic control unit operatively connected to the ranged sensor and the user interface. The at least one processor records control inputs by the driver driving the vehicle, and develops a driver behavior model associated with the driver driving the vehicle based on the control inputs. The at least one processor also predicts trajectories of the objects and the vehicle based on the time-series data and the driver behavior model, and predicts a collision between the vehicle and one of the objects based on the predicted trajectories. The at least one processor also generates a warning indicating the predicted collision to the driver. According to another aspect, a method for generating a warning to a driver of a vehicle includes generating time-series data indicating positions of objects in an environment surrounding the vehicle, and recording control inputs by the driver driving the vehicle. The method also includes developing a driver behavior model associated with the driver driving the vehicle based on the control inputs. The method also includes predicting trajectories of the objects and the vehicle based on the time-series data and the driver behavior model, and predicting a collision between the vehicle and one of the objects based on the predicted trajectories. The method also includes generating a warning indicating the predicted collision to the driver. According to another aspect, a non-transitory computer readable storage medium stores instructions that, when executed by at least one processor, causes the at least one processor to perform a method. The method includes generating time-series data indicating positions of objects in an environment surrounding the vehicle, and recording control inputs by the driver driving the vehicle. The method also includes developing a driver behavior model associated with the driver driving the vehicle based on the control inputs. The method also includes predicting trajectories of the objects and the vehicle based on the time-series data and the driver behavior model, and predicting a collision between the vehicle and one of the objects based on the predicted trajectories. The method also includes generating a warning indicating the predicted collision to the driver. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is an exemplary operating environment of a vehicle. FIG. 2 is a top view of the vehicle in an environment. FIG. 3 is a diagram of a learning framework. FIG. 4 is a diagram of a Markov decision process (MDP) state tree supported by the learning framework. FIG. 5 is an algorithm set. FIG. 6A is a simulation of the vehicle supported by the learning framework in a hard braked front vehicle scenario. FIG. 6B is a simulation of the vehicle supported by the learning framework in a lane change scenario. FIG. 7 shows Table I and Table II, which summarize results from the simulation. FIG. 8 is a plot depicting results from the simulation with respect to driver behavior model development. FIG. 9 is a plot depicting results from the simulation with respect to a speed profile in a front hard brake scenario. FIG. 10 is an exemplary process for generating a warning to a driver. FIG. 11 is an illustration of a computer-readable medium or computer-readable devi