Search

CN-122023075-A - Safety education immersion teaching method based on virtual simulation technology

CN122023075ACN 122023075 ACN122023075 ACN 122023075ACN-122023075-A

Abstract

The invention discloses a security education immersive teaching method based on a virtual simulation technology, which comprises the steps of S1, constructing a virtual simulation security environment, determining visual and auditory rendering parameters and rendering a first security scene by a multimedia learning cognition theory, S2, collecting head gestures, sight trajectories, action trajectories, walking paths and voice signal data, adding time indexes, S3, performing multi-mode preprocessing to generate a multi-mode behavior data sequence, S4, extracting visual, action and voice characteristics and forming a behavior characteristic vector sequence according to the time indexes, S5, calculating operation performance indexes, S6, triggering visual prompts, voice prompts and tactile feedback in the first security scene according to the operation performance indexes, S7, inputting a teaching path adjustment model, determining the number and rendering difficulty of the next security scene and loading the next security scene. The invention realizes scene presentation, behavior analysis, prompt generation and path adjustment in virtual security teaching.

Inventors

  • ZHU CHUNHUA

Assignees

  • 苏州华敏软件科技有限公司

Dates

Publication Date
20260512
Application Date
20260128

Claims (8)

  1. 1. The security education immersive teaching method based on the virtual simulation technology is characterized by comprising the following steps of: S1, constructing a virtual simulation safety environment, determining a visual channel rendering parameter and an auditory channel rendering parameter according to a multimedia learning cognition theory, and rendering a first safety scene in display equipment; s2, acquiring head posture data, sight line track data, action track data, walking path data and voice signal data of a user, and adding a time index; S3, performing multi-mode preprocessing on the data with the additional time index, and dividing continuous data into a plurality of data segments according to the data segmentation rule determined by the multi-media learning cognition theory to form a multi-mode behavior data sequence; s4, performing feature extraction on the multi-mode behavior data sequence, wherein the feature extraction comprises the steps of constructing a visual space-time feature block, a behavior event manifold and a voice energy contour sequence, generating corresponding features, and splicing according to time indexes to form a behavior feature vector sequence; S5, constructing a time distance tensor according to the behavior feature vector sequence, aligning the time distance tensor with a preset expert behavior reference manifold, and extracting and aggregating behavior differences to generate an operation expression index; s6, triggering preset parameters of a visual prompt graph, a voice prompt content and a touch feedback signal corresponding to the multimedia learning cognition theory in a first safety scene according to the operation performance index; S7, inputting the operation performance index into a teaching path adjustment model, determining a scene number and rendering difficulty parameters of a next safety scene, loading the next safety scene in the virtual simulation safety environment, and terminating the first safety scene.
  2. 2. The security education immersive teaching method based on virtual simulation technology according to claim 1, wherein the S1 comprises: S11, loading safety accident three-dimensional model data in computing equipment, wherein the safety accident three-dimensional model data is generated by three-dimensional modeling software and comprises a fire source object, a smoke particle object, a dangerous mark object and a scene building structure object; s12, performing scene initialization on the three-dimensional model data in a rendering engine, and setting a scene coordinate system, an illumination source position, shadow parameters and environment textures; s13, determining visual channel rendering parameters and auditory channel rendering parameters according to limiting conditions of visual channel and auditory channel information presentation density in a multimedia learning cognition theory, wherein the visual channel rendering parameters comprise frame rendering intervals, picture brightness thresholds and visual prompt graph superposition positions, and the auditory channel rendering parameters comprise prompt voice output intervals, background volume amplitudes and audio trigger thresholds; S14, processing the safety accident three-dimensional model data according to the visual channel rendering parameters, performing frame rendering and generating a continuous visual frame sequence corresponding to the first safety scene; s15, processing audio resources according to the auditory channel rendering parameters to generate an audio signal sequence which is time-synchronous with the continuous visual frame sequence; S16, loading the continuous visual frame sequence in a display device, loading the audio signal sequence in an audio playing device, and presenting a first security scene in a virtual simulation security environment constructed by a rendering engine.
  3. 3. The security education immersive teaching method based on the virtual simulation technology according to claim 1, wherein the S2 comprises: S21, acquiring head posture data of a user through a head-mounted display device, wherein the head posture data is output by a posture sensor and comprises pitch angle data, roll angle data and yaw angle data; S22, acquiring sight line track data of a user through a sight line tracking module, wherein the sight line track data is output by an infrared optical tracking system and comprises pupil position coordinates and sight line pointing vectors; s23, acquiring motion track data of a user through a motion capture device, wherein the motion capture device consists of an inertial measurement unit and depth camera equipment, and outputting a three-dimensional track point set of a hand and position information of body skeleton nodes; S24, acquiring walking path data of a user through a position tracker, wherein the walking path data is a continuous position coordinate sequence and is added with a uniform time index; s25, collecting user voice signal data through an audio collection device, wherein the user voice signal data is output by a microphone array and is an original voice waveform sequence with a time index; s26, integrating the head gesture data, the sight line track data, the action track data, the walking path data and the voice signal data into a multi-mode original data set according to the time index sequence, and recording corresponding acquisition time.
  4. 4. The security education immersive teaching method based on the virtual simulation technology according to claim 1, wherein the S3 comprises: S31, performing coordinate system conversion on the head posture data, and uniformly converting pitch angle data, roll angle data and yaw angle data into a global coordinate system of the virtual simulation safety environment; S32, performing noise removal processing on the video track data, and processing pupil position coordinates and sight line pointing vectors by adopting a sliding window-based smoothing method; S33, performing filtering processing on the motion track data, performing multi-channel filtering on the three-dimensional acceleration data and the angular velocity data output by the inertia measurement unit, and performing missing point interpolation on the body bone node position information output by the depth image pickup device; S34, carrying out path point resampling processing on the walking path data, and generating a continuous position coordinate sequence at uniform time intervals; s35, performing framing processing on voice signal data, dividing an original voice waveform sequence into voice frame sequences according to preset frame lengths and frame shifts, and reserving corresponding time indexes; S36, dividing the preprocessed data into a plurality of data segments according to a data segmentation rule set by a multimedia learning cognition theory according to time indexes, and generating a multi-modal behavior data sequence; s37, storing the multi-mode behavior data sequence into a data buffer area, and recording a start time index and an end time index as segment boundary information.
  5. 5. The security education immersive teaching method based on the virtual simulation technology according to claim 1, wherein the S4 comprises: s41, constructing vision related data in the multi-mode behavior data sequence into vision space-time feature blocks, performing space-time neighborhood search on the head gesture sequence and the sight track sequence to generate a vision neighborhood matrix, and extracting vision feature subsequences; S42, constructing action related data in the multi-mode action data sequence into an action event manifold, executing local geometric neighborhood preserving mapping on the three-dimensional track point set of the hand and the position information of the body skeleton nodes to generate an action manifold structure, and extracting action characteristic subsequences; S43, constructing voice related data in the multi-modal behavior data sequence into a voice energy profile sequence, performing local energy distribution calculation on a voice frame sequence to generate a voice energy profile and extracting a voice characteristic subsequence; s44, constructing a cross-mode synchronous diagram by taking the visual characteristic subsequence, the action characteristic subsequence and the voice characteristic subsequence as nodes and taking a time index difference value corresponding to each node as an edge weight; S45, performing synchronous extraction operation on the cross-modal synchronous graph, and performing time registration on different characteristic subsequences according to a minimum side weight path to form a behavior characteristic vector sequence; s46, constructing a feature continuity maintaining matrix for the behavior feature vector sequence, and recording a start time index and an end time index of the vector sequence.
  6. 6. The method for immersion teaching of security education based on virtual simulation technique according to claim 1, wherein S5 comprises: s51, constructing the behavior feature vector sequence into a time distance tensor according to the time index sequence, wherein the time distance tensor is generated by combining time index difference values of adjacent feature vectors and feature difference values of each dimension element by element; s52, constructing preset expert behavior feature data into expert behavior reference manifolds, and executing local neighborhood preserving mapping among expert behavior feature nodes to generate the expert behavior reference manifolds with continuous node structures; s53, constructing a time sequence alignment matrix according to the time distance tensor and the expert behavior reference manifold, wherein the time sequence alignment matrix is generated by calculating a corresponding time difference value and a characteristic difference value between a behavior characteristic vector sequence and an expert behavior reference manifold node; S54, performing behavior difference mapping on the time sequence alignment matrix, arranging characteristic difference values of the alignment nodes according to a time index sequence to generate a behavior difference vector sequence, and recording index positions of the difference vectors; s55, segment aggregation is carried out on the behavior difference vector sequence, and vector summation and vector average are carried out on the difference vectors in the continuous time index range to generate a behavior difference aggregation sequence; S56, generating an operation performance index according to the vector summation result of each aggregation segment of the behavior difference aggregation sequence, and recording a time index corresponding to the operation performance index.
  7. 7. The method for immersion teaching of security education based on virtual simulation technique according to claim 1, wherein S6 comprises: S61, inputting the operation performance index into a prompt trigger analysis module to generate a prompt trigger index sequence, wherein the prompt trigger index sequence is generated by a time index corresponding to the operation performance index; S62, constructing a visual prompt event diagram according to the prompt triggering index sequence, and calculating the difference between the prompt triggering interval and the prompt superposition position between the visual prompt nodes to generate the visual prompt event diagram; S63, determining a visual cue trigger node in the visual cue event map, determining a visual cue pattern type according to the multimedia learning cognition theory, generating a visual cue pattern superposition instruction, and injecting the visual cue pattern superposition instruction into a rendering buffer zone of a first safety scene; s64, constructing a voice prompt event sequence according to the prompt trigger index sequence, and generating the voice prompt event sequence by calculating the prompt trigger interval and the prompt tone intensity difference between voice prompt nodes; s65, determining a voice prompt trigger frame in the voice prompt event sequence, determining prompt voice content according to the multimedia learning cognition theory, and outputting a prompt voice playing instruction to an audio playing module of the first safety scene; S66, constructing a haptic feedback trigger table according to the prompt trigger index sequence, generating haptic feedback parameters by dividing the trigger index execution time period and performing intensity grading operation, determining a haptic feedback mode according to the multimedia learning cognition theory, and outputting a haptic feedback instruction to a haptic execution module of the first safety scene.
  8. 8. The method for immersion teaching of security education based on virtual simulation technique according to claim 1, wherein S7 comprises: S71, inputting the operation performance index into a teaching path adjustment model, wherein the teaching path adjustment model comprises a scene transfer sub-module, a difficulty generation sub-module and a path decision sub-module; s72, in the scene transfer submodule, calculating transfer difference among preset scene nodes to generate a scene transfer graph, determining a target scene node in the scene transfer graph according to the operation performance index, and recording a corresponding next safety scene number; S73, writing the operation performance index into a difficulty regulation matrix in the difficulty generation sub-module, and performing index screening operation on a difficulty level index to generate a rendering difficulty parameter; S74, in the path decision sub-module, writing the number of the next safety scene and the rendering difficulty parameter into a path decision table, and inquiring a corresponding item in the path decision table to generate a scene scheduling instruction; S75, sending the scene scheduling instruction to a scene management module of the virtual simulation safety environment, loading the next safety scene, writing the rendering difficulty parameter into a rendering regulation parameter set, and executing rendering initialization operation; S76, terminating the rendering process of the first safety scene, and starting the rendering process of the next safety scene.

Description

Safety education immersion teaching method based on virtual simulation technology Technical Field The invention relates to the technical field of machine learning and safety education simulation, in particular to a safety education immersion teaching method based on a virtual simulation technology. Background With the rapid development of virtual reality technology, sensor technology and immersive teaching modes, the field of security education gradually changes from traditional text explanation and video demonstration to an interactive and experiential virtual simulation training system. The existing virtual simulation safety teaching platform generally relies on basic three-dimensional modeling, scene rendering and operation flow simulation, presents scene information to learners through head-mounted display equipment or multi-screen projection, and tracks a learning process by combining a simple behavior recording means. However, such systems generally have obvious technical limitations in practical applications, and it is difficult to meet the requirements of high-risk industries for refined security training, immersive risk perception and multimodal behavior assessment. In the prior art, a data acquisition mode of a single mode or a small number of modes is relied on, for example, only the operation track and the walking path of a learner are recorded or sight line information is selectively acquired, the dimension of the overall behavior data is insufficient, and the cognitive state, the reaction mode and the operation normalization of the learner in a complex dangerous environment are difficult to comprehensively reflect. The time difference and the inconsistent sampling frequency between different sensors can also cause that the multisource data is difficult to realize high-precision alignment, and the common time synchronization method can only complete coarse-granularity time matching and can not meet the requirement of tight coupling of 'instantaneous behavior-environment feedback' in a safety education scene. In addition, the processing of the multi-modal data stays in a simple splicing or independent analysis stage, and unified modeling means are often lacking for the incidence relation, the time sequence dependence characteristic and the behavior evolution trend among the cross-modalities, so that a training system is difficult to form measurable and interpretable behavior characterization. In the aspect of teaching feedback mechanisms, the existing virtual simulation system generally adopts an immobilized prompt mode, such as a fixed audio prompt, a static text prompt or a scene highlighting mark. The prompt design lacks response capability to real-time behavior of a learner, and also does not combine the principle of controlling the load of a visual channel and an auditory channel in cognitive psychology, especially in a multimedia learning cognitive theory, so that the problems of shielding a picture by the visual prompt, excessively disturbing the auditory prompt or excessively high information density and the like influence immersion experience and increase learning burden are caused. The system association among rendering parameters, prompt modes and learning behaviors is lacking, the prompt events cannot be accurately matched with the time index of behavior data, and a stable and reliable behavior-feedback link cannot be constructed. Therefore, how to provide a security education immersive teaching method based on a virtual simulation technology is a problem to be solved by those skilled in the art. Disclosure of Invention The invention aims to provide a security education immersive teaching method based on a virtual simulation technology, which is characterized in that a multi-modal behavior data sequence, a cross-modal synchronization chart and an expert behavior reference manifold are adopted to realize high-precision behavior alignment and performance quantification through deep fusion of a multimedia learning cognition theory and the virtual simulation technology, time synchronization feedback is realized through a visual, voice and touch prompt event structure, and self-adaptive regulation of scene sequences and rendering difficulty is realized by combining a scene transition chart, a difficulty regulation matrix and a path decision table, so that a complete real-time closed-loop system of virtual security teaching is constructed, and the accuracy and individuation degree of training are greatly improved. According to the embodiment of the invention, the security education immersive teaching method based on the virtual simulation technology comprises the following steps of: S1, constructing a virtual simulation safety environment, determining a visual channel rendering parameter and an auditory channel rendering parameter according to a multimedia learning cognition theory, and rendering a first safety scene in display equipment; s2, acquiring head posture data, sight line track d