CN-122018701-A - VR platform control method and system for synchronous interaction of multiple education terminals

CN122018701ACN 122018701 ACN122018701 ACN 122018701ACN-122018701-A

Abstract

The invention discloses a VR platform control method and system for synchronous interaction of multiple education terminals, the method comprises the steps of receiving user gesture data and operation instructions uploaded by multiple education terminal devices in real time, generating initial interaction instruction sets of the education terminals based on the gesture data and the operation instructions, performing space-time consistency check on the initial interaction instruction sets, constructing a synchronous interaction feature matrix, decomposing the synchronous interaction feature matrix into space position feature vectors and time sequence feature vectors, performing dynamic weight distribution and delay optimization on the space position feature vectors and the time sequence feature vectors by utilizing a multi-target collaborative optimization algorithm, generating an optimized collaborative control strategy, generating multiple education terminal synchronization instructions based on the collaborative control strategy, and uniformly scheduling rendering content and interaction feedback time sequences of the education terminal devices. By utilizing the embodiment of the invention, the interaction synchronism and the operation consistency of the multi-education terminal in VR collaborative training can be improved, and the immersion and teaching effect of education training can be enhanced.

Inventors

LU YANG
WANG TENG

Assignees

北京国人通教育科技有限公司

Dates

Publication Date: 20260512
Application Date: 20260410

Claims (10)

1. The VR platform control method for synchronous interaction of the multi-education terminals is characterized by comprising the following steps: receiving user gesture data and operation instructions uploaded by a plurality of education terminal devices in real time, and generating initial interaction instruction sets of the education terminals based on the gesture data and the operation instructions; Performing space-time consistency check on an initial interaction instruction set, constructing a synchronous interaction feature matrix, and decomposing the synchronous interaction feature matrix into a space position feature vector and a time sequence feature vector so as to capture the space-time characteristics of multi-education terminal interaction; performing dynamic weight allocation and delay optimization on the spatial position feature vector and the time sequence feature vector by utilizing a multi-target collaborative optimization algorithm to generate an optimized collaborative control strategy; and generating a multi-education terminal synchronous instruction based on the cooperative control strategy, and uniformly scheduling rendering contents and interactive feedback time sequences of all education terminal equipment.
2. The method of claim 1, wherein receiving user gesture data and operation instructions uploaded by a plurality of educational terminal devices in real time, generating an initial interaction instruction set for each educational terminal based on the gesture data and operation instructions, comprises: Receiving user gesture data and operation instructions uploaded by a plurality of education terminal devices through a wireless communication module, wherein the gesture data comprise head positions, hand actions and body orientations, and the operation instructions comprise button clicking and gesture recognition results to generate an original data stream; Noise filtering and outlier removing processing are carried out on the original data stream, format standardization is carried out on the operation instruction, data consistency and integrity are ensured, and a preprocessed gesture data stream and an operation instruction stream are generated; Based on the preprocessed gesture data stream and the operation instruction stream, extracting key interaction features including user intention labels, action tracks and operation time stamps to generate an interaction feature vector set; According to the interaction feature vector set, converting the features into specific interaction instructions through an instruction mapping algorithm, and generating an initial interaction instruction set of each education terminal.
3. The method of claim 2, wherein performing a space-time consistency check on the initial interaction instruction set to construct a synchronous interaction feature matrix, and decomposing the synchronous interaction feature matrix into a spatial position feature vector and a time sequence feature vector to capture space-time characteristics of interaction of the multi-education terminal, comprises: performing space-time consistency check on the initial interaction instruction set, calculating the time stamp difference and the space coordinate deviation of each education terminal instruction, identifying inconsistent instructions, marking, and generating a consistency check report; Based on the consistency check report, organizing an initial interaction instruction set according to time sequences and space positions to construct a synchronous interaction feature matrix, wherein rows of the synchronous interaction feature matrix represent education terminal equipment, columns represent time points, and elements represent space coordinates and operation types; Singular value decomposition is carried out on the synchronous interaction feature matrix, a main space mode and a time sequence mode are extracted, and a space position feature vector and a time sequence feature vector are generated; And carrying out normalization processing on the spatial position feature vector and the time sequence feature vector to ensure that feature scales are consistent and generate a normalized spatial position feature vector and a normalized time sequence feature vector.
4. A method according to claim 3, wherein the performing dynamic weight allocation and delay optimization on the spatial location feature vector and the time series feature vector by using a multi-objective collaborative optimization algorithm to generate an optimized collaborative control strategy comprises: setting an objective function of a multi-objective collaborative optimization algorithm, wherein the objectives comprise minimizing interaction delay, maximizing spatial synchronicity and minimizing energy consumption, and generating an optimization objective configuration; Based on the optimization target configuration, dynamic weight distribution is carried out on the space feature vector and the time sequence feature vector, and weight coefficients are adjusted according to real-time network conditions and equipment performance, so that dynamic weight vectors are generated; Performing delay optimization on the time sequence feature vector by using the dynamic weight vector, and compensating network delay by using a prediction algorithm to generate an optimized time sequence feature vector; And outputting a cooperative control strategy comprising an instruction scheduling sequence and a resource allocation scheme through a strategy generator by combining the spatial position feature vector and the optimized time sequence feature vector.
5. The method of claim 4, wherein generating the multi-education terminal synchronization instruction based on the cooperative control strategy uniformly schedules the rendering content and the interactive feedback timing of each education terminal device, comprises: analyzing the cooperative control strategy, extracting an instruction scheduling sequence and a resource allocation scheme, and generating a multi-education terminal synchronous instruction template; generating a specific rendering instruction and an interactive feedback instruction for each education terminal device according to the multi-education terminal synchronous instruction template, and generating a specific instruction set of the education terminal; based on a specific instruction set of the education terminals, the rendering content and the interactive feedback time sequence of each education terminal device are uniformly scheduled through a time sequence coordinator, so that all the education terminals are ensured to execute corresponding operations at the same time point, and a synchronous scheduling plan is generated; Executing the synchronous scheduling plan, sending synchronous instructions to the education terminal devices, monitoring the execution state and outputting synchronous interaction results of the multi-education terminal.
6. A VR platform control system for synchronized interaction of multiple educational terminals, said system comprising: the receiving module is used for receiving user gesture data and operation instructions uploaded by the plurality of education terminal devices in real time and generating initial interaction instruction sets of the education terminals based on the gesture data and the operation instructions; the construction module is used for carrying out space-time consistency check on the initial interaction instruction set, constructing a synchronous interaction feature matrix, and decomposing the synchronous interaction feature matrix into a space position feature vector and a time sequence feature vector so as to capture the space-time characteristics of multi-education terminal interaction; The optimization module is used for carrying out dynamic weight allocation and delay optimization on the spatial position feature vector and the time sequence feature vector by utilizing a multi-target collaborative optimization algorithm, and generating an optimized collaborative control strategy; and the generation module is used for generating a multi-education terminal synchronous instruction based on the cooperative control strategy and uniformly scheduling the rendering content and the interactive feedback time sequence of each education terminal device.
7. The system according to claim 6, wherein the receiving module is specifically configured to: Receiving user gesture data and operation instructions uploaded by a plurality of education terminal devices through a wireless communication module, wherein the gesture data comprise head positions, hand actions and body orientations, and the operation instructions comprise button clicking and gesture recognition results to generate an original data stream; Noise filtering and outlier removing processing are carried out on the original data stream, format standardization is carried out on the operation instruction, data consistency and integrity are ensured, and a preprocessed gesture data stream and an operation instruction stream are generated; Based on the preprocessed gesture data stream and the operation instruction stream, extracting key interaction features including user intention labels, action tracks and operation time stamps to generate an interaction feature vector set; According to the interaction feature vector set, converting the features into specific interaction instructions through an instruction mapping algorithm, and generating an initial interaction instruction set of each education terminal.
8. The system according to claim 7, characterized in that said construction module is in particular adapted to: performing space-time consistency check on the initial interaction instruction set, calculating the time stamp difference and the space coordinate deviation of each education terminal instruction, identifying inconsistent instructions, marking, and generating a consistency check report; Based on the consistency check report, organizing an initial interaction instruction set according to time sequences and space positions to construct a synchronous interaction feature matrix, wherein rows of the synchronous interaction feature matrix represent education terminal equipment, columns represent time points, and elements represent space coordinates and operation types; Singular value decomposition is carried out on the synchronous interaction feature matrix, a main space mode and a time sequence mode are extracted, and a space position feature vector and a time sequence feature vector are generated; And carrying out normalization processing on the spatial position feature vector and the time sequence feature vector to ensure that feature scales are consistent and generate a normalized spatial position feature vector and a normalized time sequence feature vector.
9. A storage medium having a computer program stored therein, wherein the computer program is arranged to perform the method of any of claims 1-5 when run.
10. An electronic device comprising a memory and a processor, wherein the memory has stored therein a computer program, the processor being arranged to run the computer program to perform the method of any of claims 1-5.

Description

VR platform control method and system for synchronous interaction of multiple education terminals Technical Field The invention belongs to the technical field of VR interaction, and particularly relates to a VR platform control method and system for synchronous interaction of multiple education terminals. Background Currently, virtual Reality (VR) technology has been gradually applied in the field of educational training, capable of providing an immersive, scenic learning experience. In a multi-person cooperative training scene, a plurality of VR education terminal devices are often required to be accessed into the same virtual environment, so that synchronous interaction and unified teaching guidance are realized. However, the existing VR multi-education terminal synchronization control method generally adopts a centralized instruction distribution mechanism, so that it is difficult to effectively coordinate the problem of asynchronous interaction caused by network delay, device performance difference and inconsistent operation time sequence of different education terminal users. The method is easy to cause dislocation of the cooperative actions of multiple persons in the virtual scene and delay of feedback response, and seriously affects the presence and teaching effect of cooperative training. Especially in the scene that the operation consistency and the real-time feedback requirements are higher such as teaching demonstration, team practice training, the present system lacks the unified modeling and dynamic optimization ability to the space-time interaction characteristics of the multi-education terminal, leads to the inefficiency of cooperation, and is difficult to support the standardized training application of high immersion and strong interaction. Disclosure of Invention The invention aims to provide a VR platform control method and a VR platform control system for synchronous interaction of multiple education terminals, so as to solve the defects in the prior art, improve interaction synchronism and operation consistency of the multiple education terminals in VR collaborative training, and enhance immersion and teaching effect of the education training. An embodiment of the present application provides a VR platform control method for synchronous interaction of multiple education terminals, the method comprising: receiving user gesture data and operation instructions uploaded by a plurality of education terminal devices in real time, and generating initial interaction instruction sets of the education terminals based on the gesture data and the operation instructions; Performing space-time consistency check on an initial interaction instruction set, constructing a synchronous interaction feature matrix, and decomposing the synchronous interaction feature matrix into a space position feature vector and a time sequence feature vector so as to capture the space-time characteristics of multi-education terminal interaction; performing dynamic weight allocation and delay optimization on the spatial position feature vector and the time sequence feature vector by utilizing a multi-target collaborative optimization algorithm to generate an optimized collaborative control strategy; and generating a multi-education terminal synchronous instruction based on the cooperative control strategy, and uniformly scheduling rendering contents and interactive feedback time sequences of all education terminal equipment. Optionally, the receiving user gesture data and operation instructions uploaded by the multiple education terminal devices in real time, generating an initial interaction instruction set of each education terminal based on the gesture data and the operation instructions, includes: Receiving user gesture data and operation instructions uploaded by a plurality of education terminal devices through a wireless communication module, wherein the gesture data comprise head positions, hand actions and body orientations, and the operation instructions comprise button clicking and gesture recognition results to generate an original data stream; Noise filtering and outlier removing processing are carried out on the original data stream, format standardization is carried out on the operation instruction, data consistency and integrity are ensured, and a preprocessed gesture data stream and an operation instruction stream are generated; Based on the preprocessed gesture data stream and the operation instruction stream, extracting key interaction features including user intention labels, action tracks and operation time stamps to generate an interaction feature vector set; According to the interaction feature vector set, converting the features into specific interaction instructions through an instruction mapping algorithm, and generating an initial interaction instruction set of each education terminal. Optionally, the performing space-time consistency check on the initial interaction instruction set, constructing a