CN-121987207-A - Driver cognitive load assessment method and electronic equipment

CN121987207ACN 121987207 ACN121987207 ACN 121987207ACN-121987207-A

Abstract

The application relates to the technical field of intelligent driving, in particular to a driver cognitive load assessment method and electronic equipment. The method comprises the steps of obtaining a visual signal, a voice signal and a steering wheel grip signal of a driver, respectively extracting characteristic information from the visual signal, the voice signal and the steering wheel grip signal, carrying out complementary verification on the signals, determining initial weights corresponding to the signals, identifying a current driving scene, adjusting the initial weights according to the driving scene to obtain dynamic weights, carrying out fusion processing on the characteristic information according to the dynamic weights to obtain fusion characteristics, calculating a cognitive load index through a pre-trained machine learning model based on the fusion characteristics, determining a cognitive load level according to the cognitive load index, and outputting an evaluation result. The robustness, the evaluation accuracy and the reliability of the evaluation result are obviously improved, and the fine grading evaluation of the cognitive state of the driver can be realized.

Inventors

WANG XING
DING GUOLIANG
WANG HONGXIN
YANG NAN
XU RUI

Assignees

武汉江夏楚能汽车技术研发有限公司

Dates

Publication Date: 20260508
Application Date: 20260126

Claims (10)

1. A driver cognitive load assessment method, comprising: Acquiring a visual signal, a voice signal and a steering wheel grip strength signal of a driver; Extracting characteristic information from the visual signal, the voice signal and the steering wheel grip signal respectively; performing complementary verification on each signal, and determining the corresponding primary weight of each signal; identifying a current driving scene, and adjusting the preliminary weight according to the driving scene to obtain a dynamic weight; Carrying out fusion processing on the characteristic information according to the dynamic weight to obtain fusion characteristics; calculating a cognitive load index through a pre-trained machine learning model based on the fusion features; and determining the cognitive load grade according to the cognitive load index, and outputting an evaluation result.
2. The method of claim 1, wherein the performing complementary verification on each signal to determine the preliminary weight corresponding to each signal comprises: Performing vision-voice complementary verification based on the association relationship between the vision signal and the voice signal, and determining preliminary weights corresponding to the vision signal and the voice signal; and performing grip-vision collaborative analysis based on the association relation between the grip signal of the steering wheel and the vision signal, and determining the preliminary weights corresponding to the grip signal and the vision signal.
3. The method of claim 2, wherein the performing the visual-to-speech complementary check based on the association between the visual signal and the speech signal comprises: When detecting that the sight line deviation exceeds a preset duration threshold, judging whether the sight line deviation is in a voice interaction state or not; If the voice interaction state is in, the preliminary weight of the visual signal is reduced, and the preliminary weight of the voice signal is improved; if the voice interaction state is not in, the preliminary weight of the visual signal is increased, and the preliminary weight of the voice signal is reduced.
4. The method of claim 2, wherein the grip-visual collaborative analysis based on an association between the steering wheel grip signal and the visual signal comprises: when the grip strength reduction amplitude is detected to exceed a preset amplitude threshold, judging whether the visual signal detects that the hand leaves the steering wheel area or not; if yes, the preliminary weights of the grip strength signal and the visual signal are improved.
5. The method of claim 1, wherein the driving scenario comprises at least one of a city congestion scenario, a high-speed cruise scenario, a voice interaction scenario, a complex road condition scenario, and a parking scenario; The step of adjusting the preliminary weight according to the driving scene to obtain a dynamic weight comprises the following steps: obtaining basic weight configuration corresponding to each driving scene, wherein different driving scenes configure different basic weights for visual signals, voice signals and grip strength signals; When a single driving scene is identified, fusing the basic weight corresponding to the scene with the preliminary weight to obtain the dynamic weight; When a plurality of driving scenes are identified to exist simultaneously, the basic weights of the scenes are weighted and averaged according to the confidence coefficient of each scene, and then are fused with the preliminary weights, so that the dynamic weights are obtained.
6. The method of claim 1, wherein the calculating a cognitive load index based on the fusion features by a pre-trained machine learning model comprises: Preprocessing the fusion features, wherein the preprocessing comprises at least one of feature standardization, time sequence feature construction and cross-modal feature crossing; inputting the preprocessed features into a pre-trained machine learning model for forward propagation, and outputting an original load index; and carrying out smoothing filtering treatment on the original load index to obtain the cognitive load index.
7. The method of claim 1, wherein said determining a cognitive load class from said cognitive load index comprises: Comparing the cognitive load index with a plurality of preset thresholds, and determining a class interval to which the cognitive load index belongs; and determining a corresponding cognitive load grade according to the grade interval, wherein the cognitive load grade comprises an extremely low load grade, a normal load grade, a higher load grade, a high load grade and a dangerous load grade.
8. The method of claim 1, wherein after determining a cognitive load level from the cognitive load index, further comprising: When the cognitive load level is an extremely low load level or a normal load level, pushing information according to a default interaction strategy; when the cognitive load level is a higher load level, simplifying information pushing content; when the cognitive load level is a high load level, delaying or suspending non-critical information pushing; and outputting safety early warning information when the cognitive load level is a dangerous load level.
9. The method of any of claims 1-8, wherein the visual characteristic information comprises at least one of average blink frequency, average eye closure duration, gaze point spread, duration of line of sight off road center area duty cycle, head pitch angle, and head yaw angle; the voice characteristic information comprises at least one of voice instruction response delay, voice speed, pronunciation definition, fundamental frequency standard deviation and non-fluency pause times; the grip characteristic information includes at least one of an average grip value, a grip variance, a number of grip mutations, and a symmetry of grip between the hands.
10. An electronic device comprising a processor, a memory and a computer program stored in the memory and executable on the processor, wherein execution of the computer program by the processor causes the electronic device to perform the method of any one of claims 1-9.

Description

Driver cognitive load assessment method and electronic equipment Technical Field The application relates to the technical field of intelligent driving, in particular to a driver cognitive load assessment method and electronic equipment. Background Along with the continuous promotion of the intelligent degree of car, functions such as on-vehicle infotainment system, navigation system, voice assistant are increasingly abundant, and information processing burden in the driving process has also been showing to increase when providing convenience for the driver. The cognitive load refers to the total amount of psychological activities born by the working memory of an individual when the individual executes a specific task, and when the cognitive load of a driver is too high, attention resources of the driver are excessively occupied, so that the perception capability of the road environment is reduced, the reaction time is prolonged, and the cognitive load becomes an important risk factor for causing traffic accidents. Therefore, the cognitive load state of the driver is accurately estimated in real time, and the method has important practical significance for improving driving safety and optimizing man-machine interaction experience. Existing driver cognitive load assessment techniques are largely divided into three categories. The first type is an evaluation method based on a subjective scale, such as a NASA-TLX scale, and collects self-report data of a driver through a post-questionnaire. The second type is a detection method based on physiological signals, and the cognitive state is deduced by collecting physiological indexes such as an electroencephalogram, an electrocardiogram, skin conductance and the like. The third type is a detection method based on behavior signals, including face state analysis based on vision, interaction feature analysis based on voice, driving behavior analysis based on control, etc., but the existing scheme generally only depends on a single type of behavior signal source, for example, a pure vision scheme is easy to be interfered by illumination change and shielding in a vehicle, a pure voice scheme cannot acquire effective data when a driver does not perform voice interaction, and the response of the pure control scheme to the state change of the driver has hysteresis. In summary, the existing cognitive load assessment technology has the defects that an effective compensation mechanism is lacking when the signal quality is damaged or the signal is lost due to the fact that a detection scheme relying on a single behavior signal source lacks, the robustness of an assessment result is insufficient, the existing scheme is difficult to maintain consistent assessment precision and reliability when facing different driving conditions such as urban congestion, high-speed cruising and parking, and in addition, the existing technology still lacks a systematic processing framework in the aspect of converting multidimensional behavior characteristics into quantifiable cognitive load indexes, so that the fine classification assessment of the cognitive state of a driver is difficult to realize. Disclosure of Invention In view of the above, the embodiment of the application provides a method, a device and an electronic device for evaluating cognitive load of a driver, which remarkably improve the robustness, the evaluation precision and the reliability of an evaluation result and can realize the fine and hierarchical evaluation of the cognitive state of the driver. A first aspect of an embodiment of the present application provides a driver cognitive load assessment method, including: Acquiring a visual signal, a voice signal and a steering wheel grip strength signal of a driver; Extracting characteristic information from the visual signal, the voice signal and the steering wheel grip signal respectively; performing complementary verification on each signal, and determining the corresponding primary weight of each signal; identifying a current driving scene, and adjusting the preliminary weight according to the driving scene to obtain a dynamic weight; Carrying out fusion processing on the characteristic information according to the dynamic weight to obtain fusion characteristics; calculating a cognitive load index through a pre-trained machine learning model based on the fusion features; and determining the cognitive load grade according to the cognitive load index, and outputting an evaluation result. A second aspect of an embodiment of the present application provides a driver cognitive load assessment apparatus, including: The data acquisition module is used for acquiring visual signals, voice signals and steering wheel grip strength signals of a driver; The feature extraction module is used for respectively extracting feature information from the visual signal, the voice signal and the steering wheel grip strength signal; the multi-mode fusion analysis module is used for carrying out com