CN-122024559-A - Audiovisual interaction cognitive training system under multitasking situation

CN122024559ACN 122024559 ACN122024559 ACN 122024559ACN-122024559-A

Abstract

The invention discloses an audiovisual interaction cognitive training system under a multi-task situation, which is characterized in that a training scene with high immersion is built through an intelligent cabin training end, a hierarchical design of a basic cognitive training unit and a flight cruising situation training unit is combined, systematic progression from core cognitive resource base capability to multi-task situation application is realized, strategies such as audiovisual complementation priority, redundancy control and concurrency inhibition are adopted, visual and auditory channel resource congestion is effectively avoided, interference of auditory preemption effect on a main task is reduced, the defect of conflict of audiovisual resources of the existing system is overcome, repeated, quantized and progressive training modes of the basic cognitive training unit can stabilize the base capability of core cognitive resources of students, and a multi-task structure and priority scheduling rule of the flight cruising situation training unit can simulate task cooperative requirements in a real scene to enhance the multi-task processing efficiency of the students.

Inventors

DONG CHENGLI
HUANG YANQUN
HUANG XUEQIN
ZHANG JIAN

Assignees

天津大学

Dates

Publication Date: 20260512
Application Date: 20260312

Claims (10)

1. The audio-visual interaction cognitive training system under the multi-task situation comprises an intelligent cabin training end, a coach management end, a training module, an evaluation module and a migration module, and is characterized in that: The intelligent cabin training end faces training execution and immersion experience and is used for bearing task presentation, interactive operation and instant feedback; the training management end is used for providing training mode selection, process monitoring and result duplication for course organization and assessment duplication; the training module comprises a basic cognitive training unit and a flight cruising situation training unit, wherein the basic cognitive training unit performs repeated, quantized and progressive training aiming at attention, working memory and base capacity training of execution control core cognitive resources based on a flight cruising task scene; the intelligent cabin training end is connected with the coach management end through a data processing and self-adaptive control module, and the data processing and self-adaptive control module is used for realizing real-time synchronization of data at two ends and transmission of interactive instructions, and cleaning, integrating and extracting features of student operation data and cognitive performance data acquired by the intelligent cabin training end; the evaluation module is used for carrying out multidimensional comprehensive evaluation on cognitive ability growth tracks, multitasking processing efficiency and audiovisual interaction suitability of students; the migration module is used for effectively migrating training results to real flight cruising tasks and similar complex multi-task scenes.
2. The system for audio-visual interactive cognitive training under a multitasking situation according to claim 1, wherein said intelligent cabin training terminal selects an intelligent cabin as a training implementation carrier.
3. The system for learning and learning audio-visual interaction under the multi-task situation according to claim 2 is characterized in that the interface and interaction of the intelligent cabin training end follow a multi-resource theory, namely that a main task occupies visual and cognitive resources for a long time, and key prompts reduce the congestion risk of resources in the same channel in a mode of audio-visual complementation priority, redundancy control and concurrency inhibition on the premise of ensuring the stability of the main task.
4. The system of claim 1, wherein the basic cognitive training unit comprises attention training, memory training and executive function training, wherein the attention training comprises selective attention training, persistent attention training and distributive attention training, the memory training comprises speech working memory training, vision space working memory training and context buffer training, and the executive function training comprises suppression control training, cognitive flexibility training and refreshing function training.
5. The system of claim 1, wherein the audio visual cues in the flight cruise context training unit are of a differentiated design using 5Hz visual flicker for visual capture of high value anomalies and 2kHz audible tones for audible cues requiring a quick response.
6. The audio-visual interactive cognitive training system under the multi-task situation according to claim 1, wherein the multi-task structure comprises a main task layer, an auxiliary task layer and a dynamic trigger task layer, wherein the main task layer is a core flight cruising operation task, the auxiliary task layer is a periodic routine task, and the dynamic trigger task layer is a burst abnormal task.
7. The system of claim 6, wherein the main task layer, the auxiliary task layer and the dynamic trigger task layer are linked by a priority scheduling rule, the priority is dynamic trigger task layer, main task layer and auxiliary task layer from high to low, and the duration of the dynamic trigger task does not exceed a preset threshold.
8. The audiovisual interactive cognitive training system in a multitasking scenario of claim 1, wherein said training module comprises the steps of: Customizing a training plan before training; preparing equipment and rules; training execution (split basis cognitive training and flight cruise situational training); Real-time evaluation in training (synchronously displaying an intelligent cabin training end and a coach management end); reporting and repeating after training.
9. The system for learning and training the audio-visual interaction under the multitasking situation according to claim 1, wherein the evaluation module comprises a cognitive growth evaluation unit, a multitasking performance evaluation unit and an audio-visual interaction suitability evaluation unit, wherein the cognitive growth evaluation unit is used for constructing a dynamic growth curve by combining a Bayesian network model through longitudinal comparison of repeated test data of students in tasks of a basic cognitive training unit, quantifying the lifting amplitude of a core cognitive resource, the multitasking performance evaluation unit is used for evaluating the cooperative processing capacity of a main/auxiliary/dynamic task layer by adopting a task completion rate, a resource conflict rate and a response delay standard deviation index, and simultaneously introducing a task switching cost coefficient to measure the flexibility of execution control, the audio-visual interaction suitability evaluation unit is used for analyzing the response accuracy and the error rate of the students to different types of audio-visual prompts based on a multitasking theory, and the evaluation module is further integrated with a real-time feedback interface to synchronize a staged evaluation result to the coach management end.
10. The audio-visual interaction cognitive training system under the multi-task situation according to claim 1, wherein the migration module comprises a migration design unit, a migration verification unit and a migration adaptation unit, wherein the migration design unit builds a core element mapping relation between a training scene and a target scene based on a task similarity theory, extracts key cognitive requirements in a real flight cruising task and similar complex scenes, and accurately associates the key cognitive requirements with corresponding task modules in the training module to ensure migration relevance and effectiveness of the training elements, the migration verification unit adopts a three-stage comparison experiment to acquire data such as task completion rate, response delay and resource conflict rate of a learner in a real flight simulator or an actual task scene, quantifies migration effects through a pairing sample T test, and simultaneously evaluates migration experience in combination with subjective feedback of the learner, and the migration adaptation unit dynamically optimizes training module parameters according to verification results to enhance matching degree of training and the target scene.

Description

Audiovisual interaction cognitive training system under multitasking situation Technical Field The invention relates to the technical field of audiovisual interaction cognitive training, in particular to an audiovisual interaction cognitive training system under a multi-task situation. Background In the high-risk occupation fields of aviation, navigation and the like, the cognitive ability of operators is directly related to the operation safety and efficiency. Research shows that the core cognitive resources such as working memory, attention allocation, execution control and the like are obviously related to human errors. Pilots without human error history perform significantly better in working memory, situational awareness and distraction. The existing cognitive training technology mainly has the following defects: The existing system mostly adopts isolated basic cognitive training (such as NeuroTracker and other 3D multi-target tracking systems) or a simple task simulator, the training migration effect is limited due to the lack of systematic advanced design from basic capability to situation application, the training level is single, the traditional system cannot dynamically adjust training content according to training expression, closed-loop optimization of 'evaluation-training-migration-re-evaluation' is difficult to realize, self-adaptive closed-loop is lacking, the existing system does not fully consider multi-resource theory in the audiovisual prompt design, visual and auditory channel resource congestion is easy to be caused, or interference of auditory preemption effect on a main task is ignored, and audiovisual resource conflict exists. Therefore, the invention provides an audiovisual interaction cognitive training system under a multitasking situation. Disclosure of Invention Aiming at the defects of the prior art, the invention provides an audiovisual interaction cognitive training system under a multi-task situation so as to solve the problems. In order to achieve the aim, the invention is realized by the following technical scheme that the audiovisual interaction cognitive training system under the multi-task situation comprises an intelligent cabin training end, a coach management end, a training module, an evaluation module and a migration module; The intelligent cabin training end faces training execution and immersion experience and is used for bearing task presentation, interactive operation and instant feedback; the training management end is used for providing training mode selection, process monitoring and result duplication for course organization and assessment duplication; the training module comprises a basic cognitive training unit and a flight cruising situation training unit, wherein the basic cognitive training unit performs repeated, quantized and progressive training aiming at attention, working memory and base capacity training of execution control core cognitive resources based on a flight cruising task scene; the intelligent cabin training end is connected with the coach management end through a data processing and self-adaptive control module, and the data processing and self-adaptive control module is used for realizing real-time synchronization of data at two ends and transmission of interactive instructions, and cleaning, integrating and extracting features of student operation data and cognitive performance data acquired by the intelligent cabin training end; the evaluation module is used for carrying out multidimensional comprehensive evaluation on cognitive ability growth tracks, multitasking processing efficiency and audiovisual interaction suitability of students; the migration module is used for effectively migrating training results to real flight cruising tasks and similar complex multi-task scenes. Preferably, the intelligent cabin training terminal selects an intelligent cabin as a training implementation carrier. Preferably, the interface and interaction of the intelligent cabin training end follow a multi-resource theory that the main task occupies visual and cognitive resources for a long time, and key prompts reduce the congestion risk of resources in the same channel in a mode of audio-visual complementation priority, redundancy control and concurrent inhibition on the premise of ensuring the stability of the main task. Preferably, the basic cognitive training unit comprises attention training, memory training and executive function training, wherein the attention training comprises selective attention training, persistent attention training and distributive attention training, the memory training comprises speech working memory training, vision space working memory training and scene buffer training, and the executive function training comprises suppression control training, cognitive flexibility training and refreshing function training. Preferably, the audio-visual cues in the flight cruise context training unit are of a differentiated design, usi