CN-121997117-A - Cognitive load assessment method based on multi-modal data and edge computing equipment

CN121997117ACN 121997117 ACN121997117 ACN 121997117ACN-121997117-A

Abstract

The embodiment of the invention provides a cognitive load assessment method and edge computing equipment based on multi-modal data. The method comprises the steps of determining individual cognitive baseline data of a user, determining an individual cognitive load classification threshold of the user according to the individual cognitive baseline data of the user, obtaining multi-modal data of the user in a working environment, wherein the multi-modal data comprises human physiological signal data, human behavior data and interface characteristic data of a human-computer interaction interface of the user, and determining a cognitive load evaluation result of the user according to the multi-modal data and the individual cognitive load classification threshold of the user. According to the technical scheme provided by the embodiment of the invention, cognitive load evaluation can be effectively performed, and the operation efficiency and safety are improved.

Inventors

Request for anonymity
Request for anonymity
Request for anonymity
Request for anonymity

Assignees

北京津发科技股份有限公司

Dates

Publication Date: 20260508
Application Date: 20251225

Claims (10)

1. A cognitive load assessment method based on multimodal data, comprising: Determining individual cognitive baseline data for the user; Determining a personalized cognitive load classification threshold of the user according to the individual cognitive baseline data of the user; Acquiring multi-mode data of the user in a working environment, wherein the multi-mode data comprises human physiological signal data, human behavior data and interface characteristic data of a human-computer interaction interface of the user; and determining a cognitive load evaluation result of the user according to the multimodal data and the personalized cognitive load classification threshold of the user.
2. The method as recited in claim 1, further comprising: and adjusting the display content and/or the interaction mode of the human-computer interaction interface when the cognitive load evaluation result of the user indicates that the cognitive load level of the user exceeds the set cognitive load level threshold.
3. The method of claim 1, wherein the determining individual cognitive baseline data for the user comprises: determining the cognitive portraits of the users according to the acquired attribute information of the users and the cognitive load standardized test result; Acquiring a general cognitive baseline corresponding to the cognitive image of the user in a general cognitive model acquired in advance; updating the universal cognitive baseline to obtain individual cognitive baseline data for the user.
4. The method of claim 3, wherein the updating the universal cognitive baseline to obtain individual cognitive baseline data for the user comprises: Constructing at least two kinds of tasks with different human-computer interaction complexity aiming at the cognitive portraits of the users; Acquiring personnel capability data of the user in the execution process of the at least two kinds of tasks; and updating the universal cognitive baseline according to the personnel capacity data corresponding to the at least two kinds of tasks to obtain individual cognitive baseline data of the user.
5. The method of claim 4, wherein the at least two classes of tasks include a first task, a second task, and a third task that sequentially increase in complexity of human-machine interaction; the step of updating the universal cognitive baseline according to the personnel capacity data corresponding to the at least two kinds of tasks to obtain individual cognitive baseline data of the user, comprising the following steps: determining the difference of corresponding personnel capacity data of the user when the first task and the third task are executed, and updating the acquired universal cognitive baseline according to the difference; and verifying the validity of the adjusted universal cognitive baseline according to personnel capacity data corresponding to the user when the second task is executed, and determining that the adjusted universal cognitive baseline is individual cognitive baseline data of the user when verification is passed.
6. The method as recited in claim 1, further comprising: after the individual cognitive baseline data of the user are changed, the personalized cognitive load classification threshold value is adjusted according to a set proportion; If the deviation between the personalized cognitive load classification threshold value and the cognitive load standardized test result is larger than the set deviation threshold value, updating the individual cognitive baseline data of the user, and continuing to execute the step of determining the personalized cognitive load classification threshold value of the user according to the individual cognitive baseline data of the user.
7. The method of claim 1, wherein the cognitive load assessment results for the user comprise a cognitive load rating for the user, and wherein the determining the cognitive load assessment results for the user based on the multimodal data and the personalized cognitive load rating threshold for the user comprises: determining time sequence change characteristics of the human body physiological signal data and the human body behavior data according to the human body physiological signal data and the human body behavior data in the multi-mode data; Determining the spatial distribution characteristics of the interface characteristic data of the man-machine interaction interface according to the interface characteristic data of the man-machine interaction interface; Determining a weight distribution strategy based on the acquired task stage where the user is currently located according to the time sequence change characteristics and the spatial distribution characteristics, wherein the weight distribution strategy is a weight distribution strategy of the human physiological signal data, the human behavior data and the interface characteristic data in the multi-mode data, and the weight distribution strategy comprises characteristic vectors of each multi-mode data and weight coefficients corresponding to each characteristic vector; Generating a feature matrix according to the weight coefficient corresponding to each feature vector and each feature vector; inputting the feature matrix to the acquired fully-connected network to output a cognitive load value; And determining the cognitive load grade of the user according to the cognitive load value and the personalized cognitive load grading threshold value of the user.
8. The method of claim 1, wherein the cognitive load assessment result of the user comprises a cognitive load rating, and wherein the determining the cognitive load assessment result of the user based on the multimodal data and the personalized cognitive load rating threshold of the user comprises: According to interface adjustment rules corresponding to the cognitive load level, adjusting the man-machine interaction interface; Continuously monitoring the cognitive load level, and if the cognitive load level is not reduced in a preset time range, acquiring an interface adjustment rule corresponding to a higher-level cognitive load level of the cognitive load level; and adjusting the man-machine interaction interface according to the interface adjustment rule corresponding to the higher-level cognitive load level.
9. A computer readable storage medium, characterized in that the computer readable storage medium comprises a stored program, wherein the program when run controls a device in which the computer readable storage medium is located to perform the method according to any one of claims 1 to 8.
10. An edge computing device comprising a memory for storing information including program instructions and a processor for controlling execution of the program instructions, wherein the program instructions when loaded and executed by the processor implement the steps of the method of any one of claims 1 to 8.

Description

Cognitive load assessment method based on multi-modal data and edge computing equipment Technical Field The invention relates to the technical field of Human-Computer Interaction (HCI for short), in particular to a cognitive load assessment method and edge computing equipment based on multi-mode data. Background With the rapid development of intelligent terminals and complex HCI interfaces, scenes such as intelligent cabins, virtual Reality (VR)/augmented Reality (Augmented Reality AR) immersive systems, medical surgical robots, industrial central control rooms, and the like are becoming increasingly popular. The HCI interfaces generally have the characteristics of multi-mode (such as visual, auditory and tactile fusion) interaction, high information density (such as multi-source data parallel display), dynamic task switching (such as navigation and danger avoidance processing in driving), and the like, and users need to continuously distribute attention to process information and execute decisions in the operation process, so that the generated cognitive load directly affects the operation efficiency and safety. Too high a cognitive load may lead to user response delays and decision errors (e.g., the pilot misjudges meter data), while too low a cognitive load may lead to distraction (e.g., the monitor is missing an abnormal signal). How to perform effective cognitive load assessment to further improve operation efficiency and safety is a technical problem to be solved. Disclosure of Invention In view of the above, the embodiment of the invention provides a cognitive load evaluation method and edge computing equipment based on multi-modal data, which are used for effectively performing cognitive load evaluation and improving operation efficiency and safety. In one aspect, an embodiment of the present invention provides a cognitive load assessment method based on multimodal data, including: Determining individual cognitive baseline data for the user; Determining a personalized cognitive load classification threshold of the user according to the individual cognitive baseline data of the user; Acquiring multi-mode data of the user in a working environment, wherein the multi-mode data comprises human physiological signal data, human behavior data and interface characteristic data of a human-computer interaction interface of the user; and determining a cognitive load evaluation result of the user according to the multimodal data and the personalized cognitive load classification threshold of the user. In another aspect, an embodiment of the present invention provides a computer readable storage medium, where the computer readable storage medium includes a stored program, where when the program runs, the device in which the computer readable storage medium is located is controlled to execute the above method. In another aspect, an embodiment of the present invention provides an edge computing device, including a memory for storing information including program instructions, and a processor for controlling execution of the program instructions, where the program instructions, when loaded and executed by the processor, implement the steps of the method described above. According to the technical scheme provided by the embodiment of the invention, the individual cognitive baseline data of the user is determined, the individual cognitive load classification threshold of the user is determined according to the individual cognitive baseline data of the user, the multi-mode data of the user in a working environment is obtained, the multi-mode data comprises the human body physiological signal data, the human body behavior data and the interface characteristic data of a human-computer interaction interface of the user, and the cognitive load evaluation result of the user is determined according to the multi-mode data and the individual cognitive load classification threshold of the user. According to the technical scheme provided by the embodiment of the invention, cognitive load evaluation can be effectively performed, and the operation efficiency and safety are improved. Drawings In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. Fig. 1 is a schematic diagram of a cognitive load assessment system based on multi-modal data according to an embodiment of the present invention; fig. 2 is a flowchart of a cognitive load assessment method based on multi-modal data according to an embodiment of the present invention; FIG. 3 is a flow chart of determining individual cognitive baseline data for a user in accordance with one embodiment of the present inventio