CN-121502363-B - Interaction model training method and device based on augmented reality and sensing analysis

CN121502363BCN 121502363 BCN121502363 BCN 121502363BCN-121502363-B

Abstract

The application relates to the technical field of electroencephalogram signal monitoring and data processing, in particular to an interactive model training method and device based on augmented reality and sensing analysis, wherein the method comprises the steps of constructing a dynamic virtual world according to virtual world generation parameters; and in the cyclic iteration process, when the interactive training effect does not meet the convergence condition, adjusting virtual world generation parameters according to a preset prediction model, and continuously constructing a new dynamic virtual world according to the adjusted virtual world generation parameters until the interactive training effect meets the convergence condition, ending the cyclic iteration process and training to obtain the interactive model. The interactive training scheme based on the virtual reality and the electroencephalogram can trigger the interactive reaction of the user in the immersed and personalized environment, so that the effect of 'virtual adjustment' is achieved.

Inventors

LI AO
Ge Wenya
HUANG MINQIANG
WANG WEI
MA WENJUAN
WANG YI

Assignees

上海术理智能科技有限公司
上海通用术理智慧医疗脑科学研究院

Dates

Publication Date: 20260512
Application Date: 20260112

Claims (6)

1. An interactive model training method based on augmented reality and sensing analysis, which is characterized by comprising the following steps: Determining virtual world generation parameters and user characteristic parameters; the method comprises the steps of establishing an interactive training paradigm according to interactive training sampling time and a condition combination, obtaining interactive response intensity distribution of each group of testees, which represents the interactive training effect of a user, when the testees receive interactive training of the corresponding condition combination at the corresponding interactive training sampling time, according to the interactive training paradigm, wherein the condition combination comprises situation characteristic parameters and negative characteristic parameters in virtual world generation parameters, obtaining a first condition combination with the strongest interactive training quantification index according to the interactive response intensity distribution, removing the negative condition combination from the first condition combination to obtain a target safety condition combination, taking the target safety condition combination as a base line value of the virtual world generation parameters, defining safety constraint conditions, under the safety constraint conditions, taking the base line value of the virtual world generation parameters as a reference, iteratively searching parameters corresponding to the strongest interactive training effect and taking the parameters as final virtual world generation parameters, respectively initializing a random process model according to the target safety condition combination as a target function and each safety constraint condition by using a clustering algorithm, predicting according to the random process model to obtain a random generation parameter, wherein the random generation parameter combination is used for representing the safety condition combination, and the random process model is used for describing the safety condition combination, and the safety constraint model is used for describing the safety condition combination; The method comprises the steps of removing a negative condition combination from a first condition combination to obtain a target safety condition combination, carrying out structuring treatment on feedback information of a tested person in an interaction training normal form according to a pre-training language model to obtain a negative reaction vector, carrying out clustering treatment on the negative reaction vector to obtain a negative cluster, extracting the negative reaction vector from a plurality of condition combinations in front of the interaction intensity distribution, matching the extracted negative reaction vector with the negative cluster, if the extracted negative reaction vector is successfully matched with the negative cluster, obtaining a matched condition combination, removing the matched condition combination from a plurality of condition combinations in front of the interaction intensity distribution to obtain the first condition combination, and if the negative reaction vector is not successfully matched with the negative cluster, taking the plurality of condition combinations in front of the interaction intensity distribution as the first condition combination; The construction step of the safety constraint condition comprises the steps of determining a first safety constraint condition according to the distance relation between a negative reaction vector of a user and the negative cluster, wherein the first safety constraint condition is related to clustered negative information; carrying out emerging processing on the reaction vectors corresponding to the specified condition combinations, expanding the discrete or continuous punctiform reaction vectors to obtain continuous spatial reaction vectors which surround the punctiform vectors and form semantic clusters, and determining a second safety constraint condition according to the distance relation between the negative reaction vectors of the user and the continuous spatial reaction vectors, wherein the second safety constraint condition is related to personalized negative information; Constructing a dynamic virtual world according to the virtual world generation parameters; acquiring electroencephalogram data and at least one of physiological sensing data and behavioral sensing data of a user when performing interactive training in the dynamic virtual world based on the user characteristic parameters; determining an interactive training effect of a user according to the electroencephalogram data and at least one of physiological sensing data and behavior sensing data; And in the cyclic iteration process, when the interactive training effect does not meet the convergence condition, adjusting the virtual world generation parameters according to a preset prediction model, and continuously constructing a new dynamic virtual world according to the adjusted virtual world generation parameters until the interactive training effect meets the convergence condition, ending the cyclic iteration process, and training to obtain an interactive model.
2. The method of claim 1, wherein said determining the interactive training effect of the user based on the electroencephalogram data in combination with at least one of physiological sensing data and behavioral sensing data comprises: Dividing continuous electroencephalogram data into a plurality of single test times according to trigger labels when events are set in an experimental paradigm, wherein each test time covers electroencephalogram data before the start of the event and within a period of time after the start of the event; Corresponding physiological sensing data of the user before, after or during the interactive training is collected, and physiological index change and the difference of the physiological index change of the user before, after or during the interactive training are determined according to a corresponding detection algorithm; And/or the number of the groups of groups, Acquiring behavior sensing data or scale data matched with a user before, after or during interactive training, and determining the change and the difference of the cognitive process of the user before, after or during the interactive training according to the corresponding reaction theory and the cognitive meaning thereof; And determining the interactive training effect of the user according to the event-related potential, the physiological index change and the variability thereof and/or the change of the cognitive process and the variability thereof.
3. The method of claim 1, wherein adjusting the virtual world generation parameters according to the preset predictive model and continuing to construct a new dynamic virtual world according to the adjusted virtual world generation parameters comprises: inputting the virtual world generation parameters into a preset prediction model, and predicting according to the preset prediction model to obtain an interactive training effect; When the difference between the predicted interactive training effect and the current actual interactive training effect of the user meets the preset condition, predicting a new virtual world generation parameter according to the preset prediction model, and continuously constructing a new dynamic virtual world according to the new virtual world generation parameter.
4. The method of claim 1, wherein predicting virtual world generation parameters from the stochastic process model comprises: Determining a confidence upper bound of each safety constraint condition according to the acquisition function and the safety constraint conditions, and predicting to obtain a next virtual world generation parameter, wherein the confidence upper bound of the safety constraint conditions is used for constraining a next acquisition point to reach a state of a target interaction training effect on the premise of conforming to the safety constraint; In each loop iteration process, if the predicted next virtual world generation parameter meets the safety constraint condition, continuously updating the objective function data set according to the predicted next virtual world generation parameter, and re-fitting the random process function according to the updated objective function data set until reaching the target convergence condition, thereby obtaining the final virtual world generation parameter.
5. An interactive model training device based on augmented reality and sensing analysis, the device comprising: The parameter determining module is used for determining virtual world generation parameters and user characteristic parameters, and the determining step of the virtual world generation parameters comprises the steps of constructing an interactive training paradigm according to the interactive training sampling time and the condition combination; obtaining interactive response intensity distribution of each group of testees representing interactive training effects of users when the testees receive interactive training of corresponding condition combinations at corresponding interactive training sampling time according to the interactive training paradigm, wherein the condition combinations comprise situation characteristic parameters and negative characteristic parameters in virtual world generation parameters, obtaining a first condition combination with the strongest interactive training quantification index according to the interactive response intensity distribution, removing the negative condition combination from the first condition combination to obtain a target safety condition combination, taking the target safety condition combination as a base line value of the virtual world generation parameters, defining safety constraint conditions, iteratively searching the corresponding parameters as final virtual world generation parameters by taking the base line value of the virtual world generation parameters as a base line value of the virtual world generation parameters under the safety constraint conditions, respectively initializing a random process model according to the target safety condition combination as an objective function and each safety constraint condition, predicting to obtain virtual world generation parameters according to the random process model, wherein the random process model characterizes part of condition combinations screened from the target safety condition combination and is used as a base line value of the virtual world generation parameters, defining safety constraint conditions, taking the target safety condition combination as a basic line value of the virtual world generation parameters as a basic value of the virtual world generation parameters, performing the maximum interaction training parameters, obtaining a negative safety condition combination from the target safety condition combination by using the clustering algorithm, the method comprises the steps of carrying out structuring treatment on feedback information of a tested person in an interactive training normal form according to a pre-training language model to obtain a negative reaction vector, carrying out clustering treatment on the negative reaction vector to obtain a negative cluster, extracting the negative reaction vector from a plurality of condition combinations which are located in front of the interactive reaction intensity distribution, matching the extracted negative reaction vector with the negative cluster, if the extracted negative reaction vector is successfully matched with the negative cluster to obtain a matching condition combination, removing the matching condition combination from the plurality of condition combinations which are located in front of the interactive reaction intensity distribution to obtain a first condition combination, if the negative reaction vector is not successfully matched with the negative cluster, taking the plurality of condition combinations which are located in front of the interactive reaction intensity distribution as the first condition combination, continuing to remove the specified condition combination from the first condition combination to obtain a target safety condition combination, wherein the specified condition combination is a condition combination which enables a user to have negative reaction, if the extracted negative reaction vector is successfully matched with the negative cluster, removing the matching condition combination from the plurality of condition combinations which are located in front of the interactive reaction intensity distribution, removing the negative reaction vector from the first condition combination according to the matching condition combination, wherein the negative reaction vector of the user has a continuous correlation with the first condition combination, the fact that the safety constraint vector is formed by a continuous spatial relation between the negative reaction vector and the corresponding to the first condition clusters, or the point-like safety constraint vector is formed, and the continuous space relation between the negative reaction vector is formed, the second security constraint is associated with personalized negative information; The world generation module is used for constructing a dynamic virtual world according to the virtual world generation parameters; The virtual reality module is used for acquiring the electroencephalogram data of a user during interactive training in the dynamic virtual world based on the user characteristic parameters through the electroencephalogram acquisition unit, and acquiring at least one of physiological sensing data and behavior sensing data through the physiological sensing data acquisition unit and the behavior sensing data acquisition unit; the calculation module is used for determining the interactive training effect of the user according to the acquired electroencephalogram data and at least one of physiological sensing data and behavior sensing data; and the adjusting module is used for adjusting the virtual world generation parameters according to a preset prediction model in the cyclic iteration process when the interactive training effect does not meet the convergence condition, and continuously constructing a new dynamic virtual world according to the adjusted virtual world generation parameters until the interactive training effect meets the convergence condition, ending the cyclic iteration process, and training to obtain the interactive model.
6. A computer device comprising a memory and a processor, the memory having stored therein a computer program, which when executed implements the augmented reality and sensory analysis based interaction model training method of any one of the preceding claims 1 to 4.

Description

Interaction model training method and device based on augmented reality and sensing analysis Technical Field The application relates to the technical field of electroencephalogram signal monitoring and data processing, in particular to an interactive model training method and device based on augmented reality and sensing analysis. Background In a multi-dimensional interaction scenario, individuals need to deal with potential threats through dynamic strategies, which typically merge the two core mechanisms of cognitive regulation and status response. The cognitive regulation mechanism focuses on optimizing a behavior mode through experience learning and risk assessment so as to reduce adverse exposure, and the state response mechanism relies on real-time state feedback to adjust response strength. In the prior art, the collaboration of two mechanisms mainly depends on a preset rule or a static model, namely, cognitive regulation is based on a fixed experience library generation strategy, adaptation to the real-time state of an individual is lacking, state response depends on single-dimensional data (such as a simple physiological index), and dynamic change under a complex scene is difficult to capture. The existing interaction strategy construction technology generally has the core limitations that parameter adjustment depends on manual experience, data utilization is limited to single mode, scene simulation and reality disconnection, and the like, so that the existing interaction strategy has obvious defects in response speed, flexibility and individuation adaptation, and the requirement of the existing interaction strategy on high-efficiency interaction strategy in a complex environment is difficult to meet. Disclosure of Invention The application aims to provide an interactive model training method and device based on augmented reality and sensing analysis, which can trigger interactive reaction of a user in an immersed and personalized environment based on an interactive training scheme of virtual reality and electroencephalogram, thereby achieving the effect of virtual adjustment. And parameters in the interactive training are adjusted in a self-adaptive manner according to indexes such as related nerves, physiology and behaviors of the interactive response, so that the effect of adjusting the interactive training dosage is achieved. In some embodiments, the application provides an interaction model training method based on augmented reality and sensing analysis, which comprises the steps of determining virtual world generation parameters and user characteristic parameters, constructing a dynamic virtual world according to the virtual world generation parameters, acquiring electroencephalogram data and at least one of physiological sensing data and behavior sensing data when a user performs interaction training in the dynamic virtual world based on the user characteristic parameters, determining an interaction training effect of the user according to the electroencephalogram data and at least one of the physiological sensing data and the behavior sensing data, adjusting the virtual world generation parameters according to a preset prediction model when the interaction training effect does not meet a convergence condition in a cyclic iteration process, and continuing to construct a new dynamic virtual world according to the adjusted virtual world generation parameters until the interaction training effect meets the convergence condition, and ending the cyclic iteration process and training to obtain the interaction model. In some embodiments, the determining the interactive training effect of the user according to the electroencephalogram data and at least one of physiological sensing data and behavioral sensing data comprises: Dividing continuous electroencephalogram data into a plurality of single test times according to trigger labels when events are set in an experimental paradigm, wherein each test time covers electroencephalogram data before the start of the event and within a period of time after the start of the event; Corresponding physiological sensing data of the user before, after or during the interactive training is collected, and physiological index change and the difference of the physiological index change of the user before, after or during the interactive training are determined according to a corresponding detection algorithm; And/or the number of the groups of groups, Acquiring behavior sensing data or scale data matched with a user before, after or during interactive training, and determining the change and the difference of the cognitive process of the user before, after or during the interactive training according to the corresponding reaction theory and the cognitive meaning thereof; and determining the interactive training effect of the user according to the event-related potential, the physiological index change and the variability thereof and/or the change of the cognitive pro