CN-121973805-A - AR (augmented reality) glasses-based driver safe driving prompt system and method

CN121973805ACN 121973805 ACN121973805 ACN 121973805ACN-121973805-A

Abstract

The application relates to a driver safe driving prompt method based on AR glasses, which comprises the steps of obtaining a forward video of a first visual angle of a driver, which is collected by the AR glasses, tracking a fixed marker in the forward video based on a target tracking algorithm of relevant filtering to form a dynamic anchor frame, intercepting a front view sub-graph in the video frame of the forward video according to the dynamic anchor frame, obtaining a vehicle transverse deviation state, a vehicle angle deviation state and a sight attention deviation state according to the front view sub-graph, determining a vehicle risk level, and outputting prompt information according to the vehicle risk level. The application reduces the interference to the driving task while ensuring the effective transmission of the prompt information and the navigation information, thereby improving the overall driving safety.

Inventors

LIU YOUQUAN
DONG HAOYU
MA JUNYAN
ZHAO XIANGMO

Assignees

长安大学

Dates

Publication Date: 20260505
Application Date: 20251230

Claims (9)

1. The driver safe driving prompting method based on the AR glasses is characterized by comprising the following steps of: in the running process of a vehicle, acquiring a forward video of a first visual angle of a driver, which is acquired by AR glasses; Tracking the fixed markers in the forward video based on a target tracking algorithm of the related filtering to form a dynamic anchor frame; intercepting a front view sub-graph in a video frame of the forward video according to the dynamic anchor frame; Determining a lane line according to the front view subgraph by adopting a lane line detection method based on a line anchor, and determining a lane center line according to the lane line; Calculating the distance between the center point of the vehicle and the center line of the lane to obtain the transverse offset of the vehicle, calculating the angular offset of the vehicle according to the current directions of the center line of the lane and the vehicle, and calculating the offset of the sight line attention point of the driver relative to the sight line calibration point to obtain the sight line attention offset; determining a vehicle transverse offset state according to the vehicle transverse offset, the vehicle current speed and the vehicle current acceleration; determining a vehicle angle offset state according to the vehicle angle offset, the current speed of the vehicle and the current acceleration of the vehicle; Inputting the sight line attention offset into a sight line attention analysis model to obtain a sight line attention offset state, wherein the sight line attention analysis model comprises a double-layer bidirectional gating circulation unit and a multi-head self-attention mechanism which are sequentially connected; determining a vehicle risk level according to the vehicle transverse deviation state, the vehicle angle deviation state and the sight line attention deviation state; And outputting prompt information according to the vehicle risk level.
2. The method of claim 1, wherein intercepting a forward view sub-graph in a video frame of the forward video according to the dynamic anchor frame comprises: And intercepting a front view sub-graph at a preset position relative to the dynamic anchor frame, wherein the front view sub-graph comprises road core information.
3. The method of claim 1, wherein calculating a vehicle angular offset from the lane centerline and a current vehicle orientation comprises: When the lane center line is a straight line, calculating an included angle between the direction of the lane center line and the current direction of the vehicle, and taking the included angle as the vehicle angle offset; and when the central line of the lane is a curve, determining a tangent line of the starting point position of the central line of the lane, and calculating an included angle between the tangent line and the current direction of the vehicle to be used as the vehicle angle offset.
4. The method of claim 1, wherein determining a vehicle lateral offset state based on the vehicle lateral offset, the vehicle current speed, and the vehicle current acceleration comprises: calculating comprehensive index of vehicle lateral deviation state : Wherein, the 、、 Are all the weight coefficients of the two-dimensional space model, For the current speed of the vehicle, For the current acceleration of the vehicle, Is the vehicle lateral offset; Calculating dynamic threshold times : Wherein, the As a function of the base threshold time, 、 Is the attenuation coefficient; If it is The vehicle lateral offset state is a normal state; A timer value from the detected lateral offset of the vehicle to a current time; If it is And is also provided with The vehicle lateral offset state is a normal dangerous state; a discrimination value for discriminating a normal dangerous state from an emergency dangerous state; If it is And is also provided with The vehicle lateral offset state is an emergency dangerous state.
5. The method of claim 1, wherein determining a vehicle angle offset state based on the vehicle angle offset, the vehicle current speed, and the vehicle current acceleration comprises: Calculating vehicle angle deviation state comprehensive index : Wherein, the 、、 Are all the weight coefficients of the two-dimensional space model, For the current speed of the vehicle, For the current acceleration of the vehicle, Is the vehicle angular offset; Calculating dynamic threshold times : Wherein, the As a function of the base threshold time, 、 Is the attenuation coefficient; If it is The vehicle angle offset state is a normal state; A timer value for starting from the detected vehicle angular offset to the current time; If it is And is also provided with The vehicle angle offset state is a normal dangerous state; a discrimination value for discriminating a normal dangerous state from an emergency dangerous state; If it is And is also provided with The vehicle angle offset state is an emergency dangerous state.
6. The method of claim 1, wherein determining a vehicle risk level based on the vehicle lateral offset state, the vehicle angular offset state, the line of sight attention offset state comprises: the vehicle lateral deviation state comprises a normal state, a common dangerous state and an emergency dangerous state, and the corresponding state dangerous scores are 0,1,3 respectively; The vehicle angle deviation state comprises a normal state, a common dangerous state and an emergency dangerous state, and the corresponding state dangerous scores are 0,1,3 respectively; The sight line attention offset state comprises a normal state, a common dangerous state and an emergency dangerous state, and the corresponding state dangerous scores are respectively 0,2 and 4; Setting weights of the vehicle transverse deviation state, the vehicle angle deviation state and the sight line attention deviation state, multiplying the weights with corresponding state danger scores respectively to obtain 3 weighted values, and accumulating to obtain a final state danger score; and determining the vehicle risk level according to the final state risk score.
7. The driver safe driving prompt system based on the AR glasses is characterized by comprising the AR glasses, a computing unit, a vehicle-mounted sensing module and a communication module; the AR glasses are used for collecting forward videos of a first visual angle of a driver and sending the forward videos to the computing unit through the communication module; the computing unit is used for realizing the driver safe driving prompt method based on the AR glasses according to any one of claims 1-6, and sending the vehicle risk level prompt information to the AR glasses through the communication module; the vehicle-mounted sensing module is used for acquiring the current speed and the current acceleration of the vehicle and sending the current speed and the current acceleration to the computing unit through the communication module.
8. The system of claim 7, wherein the AR glasses present vehicle risk level cues in the form of text and/or speech.
9. The system of claim 7, wherein the communication module is further configured to receive roadside device information and other intelligent networking car information and send to AR glasses that present the received information in a driver's view.

Description

AR (augmented reality) glasses-based driver safe driving prompt system and method Technical Field The application relates to the field of man-machine interaction, in particular to a driver safe driving prompt system and method based on AR (augmented reality) glasses. Background The man-machine interaction technology is a technology for researching information instruction interaction between a user and a system, and along with the development of man-machine interaction and artificial intelligence, the man-machine interaction is changed from simple single-channel information interaction to multi-channel information collaborative interaction. In the field of intelligent transportation, a driving auxiliary system is a typical multi-channel information interaction system, an automobile senses the surrounding environment through a multi-source sensor and extracts relevant information to output instruction information to a driver, and the driver can control the automobile through instructions and operations to realize an interaction mode of man-machine co-driving. The common man-machine interaction mode in the prior art mainly comprises (1) central control screen Display, head-Up Display (HUD) and (3) audible and visual alarm equipment, wherein the central control screen Display is used for displaying prompt characters, figures and vehicle state information on a central control liquid crystal screen, (2) the Head-Up Display is used for projecting brief key information such as speed, navigation and the like on a front windshield, and the audible and visual alarm equipment is used for prompting through a buzzer, an instrument panel indicator lamp and voice broadcasting. The interaction mode has the following defects that (1) the sight migration is obvious, the secondary risk is increased, a central control screen is usually positioned below the sight side of a driver, the driver can acquire information by obviously shifting the sight and even turning the head, and the prompt system is easy to introduce new distraction. The HUD is located in a region in front of the line of sight, but is easy to block the road view when the display range is limited and the content is complex. (2) The HUD projection effect is obviously affected by external illumination, and the visibility and contrast are easy to be reduced in extreme weather such as strong light, backlight, night, rain and fog, so that key information is not clear. (3) The traditional ADAS system focuses on the motion state of the vehicle relative to the lane and surrounding obstacles, and focuses on the attention distribution, sight line deviation and distraction of the driver, so that the high-risk working conditions caused by long-time sight line deviation, mobile phone playing, talking with passengers and the like are difficult to recognize in time. (4) Eye tracker-based vision detection is high in cost and complex in deployment, and eye tracker tracking equipment is used for collecting the vision track of a driver in part of research, but the eye tracker is high in price, high in wearing threshold and deviation from daily driving habits, so that the eye tracker is difficult to popularize in a large scale in common vehicles and masses. Disclosure of Invention In order to overcome at least one of the defects in the prior art, the application provides a driver safe driving prompt system and a method based on AR glasses. In a first aspect, a driver safe driving prompting method based on AR glasses is provided, including: in the running process of a vehicle, acquiring a forward video of a first visual angle of a driver, which is acquired by AR glasses; tracking a fixed marker in the forward video based on a target tracking algorithm of related filtering to form a dynamic anchor frame; Determining a lane line according to the front view subgraph by adopting a lane line detection method based on a line anchor, and determining a lane center line according to the lane line; Calculating the offset of the sight line attention point of the driver relative to the sight line calibration point to obtain the sight line attention offset; determining a vehicle lateral offset state according to the vehicle lateral offset, the vehicle current speed and the vehicle current acceleration; Determining a vehicle angle offset state according to the vehicle angle offset, the current speed and the current acceleration of the vehicle; The line of sight attention offset is input into a line of sight attention analysis model to obtain a line of sight attention offset state, wherein the line of sight attention analysis model comprises a double-layer bidirectional gating circulation unit and a multi-head self-attention mechanism which are sequentially connected; Determining a vehicle risk level according to the vehicle lateral offset state, the vehicle angle offset state and the sight line attention offset state; And outputting prompt information according to the vehicle risk level. In one embo