CN-121724812-B - Classroom teaching behavior analysis method and system based on multi-mode data fusion

CN121724812BCN 121724812 BCN121724812 BCN 121724812BCN-121724812-B

Abstract

The invention provides a classroom teaching behavior analysis method and system based on multi-mode data fusion, and relates to the technical field of data analysis, wherein the method comprises the following steps: collecting a multi-mode class behavior data set, identifying interaction behavior events for discretization coding, establishing a behavior chain dependency graph model based on the interaction behavior event coding set, identifying a chain dependency path based on each target interaction behavior event, determining class behavior data of a mode to be fused, generating fusion window weights according to the chain propagation time of the chain dependency path, weighting and fusing the class behavior data of the mode to be fused, obtaining class behavior fusion data, and obtaining interaction behavior scoring indexes of each target interaction behavior event. The method solves the technical problems of low accuracy of quantitative assessment of classroom teaching quality caused by lack of causal time sequence analysis in classroom multi-modal behavior fusion in the prior art. The technical effect of accurately and reliably evaluating the teaching quality of the class is achieved.

Inventors

LI SHAN
LI QIANYI
PENG FANGMING

Assignees

江苏腾权信息科技有限公司

Dates

Publication Date: 20260508
Application Date: 20260213

Claims (7)

1. The classroom teaching behavior analysis method based on multi-mode data fusion is characterized by comprising the following steps of: Collecting a multi-modal classroom behavior data set, identifying interactive behavior events of the multi-modal classroom behavior data set, and performing discretization coding on the interactive behavior events to obtain an interactive behavior event coding set; establishing a behavior chain dependency graph model based on the interactive behavior event coding set, and identifying a chain dependency path based on each target interactive behavior event according to the behavior chain dependency graph model; determining class behavior data of a mode to be fused according to the chain dependency path, generating fusion window weights according to the chain propagation time of the chain dependency path, and carrying out weighted fusion on the class behavior data of the mode to be fused according to the fusion window weights to obtain class behavior fusion data; Analyzing the classroom behavior fusion data to obtain interactive behavior scoring indexes of each target interactive behavior event; The method for collecting the multi-mode class behavior data set further comprises the following steps: identifying attribute information of current classroom teaching behaviors, including class types, teaching mode types and teaching object numbers; Extracting a class key time sequence feature according to the attribute information, acquiring a first group of multi-mode class behavior data sets of a target class under the class key time sequence feature by using a first data acquisition frequency, and acquiring a second group of multi-mode class behavior data sets which are not under the class key time sequence feature by using a second data acquisition frequency, wherein the first data acquisition frequency is greater than the second data acquisition frequency; Based on the first multi-mode class behavior data set and the second multi-mode class behavior data set, respectively obtaining a first interactive behavior event coding set and a second interactive behavior event coding set; After the first set of interaction behavior event codes and the second set of interaction behavior event codes are obtained respectively, the method further comprises: respectively constructing a local behavior chain dependency graph model and a global behavior chain dependency graph model according to the first group of interaction behavior event coding sets and the second group of interaction behavior event coding sets; Identifying a local chain dependency path and a global chain dependency path based on each target interaction behavior event according to the local behavior chain dependency graph model and the global behavior chain dependency graph model; acquiring local classroom behavior fusion data and global classroom behavior fusion data according to the local chain dependency path and the global chain dependency path; recalculating the interactive behavior scoring index of each target interactive behavior event according to the local classroom behavior fusion data and the global classroom behavior fusion data; the method for respectively constructing the local behavior chain dependency graph model and the global behavior chain dependency graph model comprises the following steps: The local behavior chain dependency graph model is used for analyzing a short-time window behavior triggering relationship of the first group of interaction behavior event coding sets, and analyzing edge weight composition of the first group of interaction behavior event coding sets according to the short-time window behavior triggering relationship; The global behavior chain dependency graph model is used for analyzing a long-term window behavior propagation relationship of the second group of interaction behavior event coding sets, and analyzing edge weight composition of the second group of interaction behavior event coding sets according to the long-term window behavior propagation relationship.
2. The classroom teaching behavior analysis method based on multi-modal data fusion according to claim 1, wherein the discrete coding is performed on the interactive behavior events to obtain an interactive behavior event code set, and the method comprises the following steps: the interactive behavior event is obtained by analyzing the characteristic change rate of the multi-mode class behavior data set and comparing the characteristic change rate with a preset change rate threshold value to divide time sequence behavior fragments of the multi-mode class behavior data set, wherein the multi-mode class behavior data set at least comprises visual behavior data, voice behavior data, gesture characteristics and equipment interactive characteristics; And mapping multidimensional semantic codes for the interactive behavior events to construct an interactive behavior event code set, wherein the multidimensional semantic codes comprise behavior event types, behavior roles, behavior intensities and behavior durations.
3. The classroom teaching behavior analysis method based on multi-modal data fusion according to claim 2, wherein a chain dependency path based on each target interactive behavior event is identified according to the behavior chain dependency graph model, each graph node of the behavior chain dependency graph model corresponds to one interactive behavior event, and the edge weight is a weighted result of the execution history transfer probability based on a time delay attenuation factor, a semantic association factor and a role dependency factor; Traversing the behavior chain dependency graph model to obtain forward dependency graph nodes with the weight larger than the preset weight based on each target interaction behavior event; and linking the forward dependency graph nodes based on time constraint to obtain a chain dependency path based on each target interaction behavior event.
4. The method for analyzing classroom teaching behavior based on multi-modal data fusion according to claim 1, wherein generating fusion window weights according to the chain propagation time of the chain dependent path comprises: calculating the chain propagation time of each target interaction behavior event and other nodes on the chain dependency path; Generating a fusion window according to the chain propagation time, and acquiring corresponding class behavior data of the mode to be fused under the fusion window; Constructing a time attenuation function according to the chain propagation time, and outputting a time weight based on the time attenuation function; And fusing the corresponding class behavior data of the mode to be fused under the fusion window by using the time weight.
5. The method for analyzing classroom teaching behavior based on multi-modal data fusion according to claim 4, wherein the time weight is used to fuse the corresponding modal class behavior data to be fused under the fusion window, the method further comprising: Calculating the modal association degree of the to-be-fused modal class behavior data set and each target interaction behavior event, wherein the modal association degree is obtained through calculation of mutual information values; configuring a modal weight according to the modal relevance, and calculating a joint weight of the modal weight and the time weight; And fusing the corresponding modal class behavior data to be fused under the fusion window according to the joint weight.
6. The method for analyzing classroom teaching behaviors based on multi-modal data fusion according to claim 1, wherein analyzing the classroom behavior fusion data to obtain the interactive behavior score index of each target interactive behavior event includes: analyzing the interactive event trigger quantity increasing rate, the interactive event semantic consistency and the interactive event quality change fluctuation of the classroom behavior fusion data under a continuous fusion window; And calculating according to the increment rate of the triggering quantity of the interaction events, the semantic consistency of the interaction events and the quality change fluctuation of the interaction events to obtain an interaction behavior scoring index.
7. A classroom teaching behavior analysis system based on multi-mode data fusion is characterized in that, a step for implementing the multi-modal data fusion-based classroom teaching behavior analysis method of any one of claims 1 to 6, comprising: The event processing module is used for collecting the multi-mode classroom behavior data set, identifying the interactive behavior event of the multi-mode classroom behavior data set, and performing discretization coding on the interactive behavior event to obtain an interactive behavior event coding set; The path identification module is used for establishing a behavior chain dependency graph model based on the interactive behavior event coding set and identifying a chain dependency path based on each target interactive behavior event according to the behavior chain dependency graph model; The fusion data acquisition module is used for determining the class behavior data of the mode to be fused according to the chain-dependent path, generating fusion window weights according to the chain propagation time of the chain-dependent path, and carrying out weighted fusion on the class behavior data of the mode to be fused according to the fusion window weights to obtain class behavior fusion data; And the scoring index acquisition module is used for analyzing the classroom behavior fusion data to acquire the interactive behavior scoring index of each target interactive behavior event.

Description

Classroom teaching behavior analysis method and system based on multi-mode data fusion Technical Field The invention relates to the technical field of data analysis, in particular to a classroom teaching behavior analysis method and system based on multi-mode data fusion. Background With the development of artificial intelligence and education technology, classroom teaching behavior analysis becomes an important means for improving teaching quality. At present, multi-modal data is widely collected in a classroom scene and used for understanding interaction modes of teachers and students and learning processes, however, the prior art still has obvious limitations in multi-modal behavior fusion. First, classroom behavior typically has complex causal timing features, such as teacher questions to elicit student answers or discussions, but existing analysis methods rely heavily on static features or simple fusion strategies, lacking fine modeling of causal relationships between behaviors and time series dependencies. Secondly, the actual contribution degree of various interaction behaviors to the teaching effect is difficult to accurately evaluate by the existing quantification method, so that the accuracy and reliability of the classroom quality evaluation result are insufficient, and the real classroom interaction condition is difficult to accurately reflect. The prior art has the technical problems that due to the lack of causal time sequence analysis in classroom multi-mode behavior fusion, the contribution degree of interaction behaviors is difficult to accurately quantify, and the accuracy of classroom teaching quality quantification assessment is low. Disclosure of Invention The application aims to provide a classroom teaching behavior analysis method and system based on multi-mode data fusion, which are used for solving the technical problems that in the prior art, the classroom multi-mode behavior fusion lacks causal time sequence analysis, the contribution degree of interaction behaviors is difficult to accurately quantify, and the accuracy of classroom teaching quality quantification assessment is low. In view of the above problems, the application provides a classroom teaching behavior analysis method and system based on multi-modal data fusion. The application provides a classroom teaching behavior analysis method based on multi-mode data fusion, which comprises the steps of collecting a multi-mode classroom behavior data set, identifying interaction behavior events of the multi-mode classroom behavior data set, performing discretization coding on the interaction behavior events to obtain an interaction behavior event coding set, establishing a behavior chain dependency graph model based on the interaction behavior event coding set, identifying a chain dependency path based on each target interaction behavior event according to the behavior chain dependency graph model, determining to-be-fused modal classroom behavior data according to the chain dependency path, generating fusion window weights according to the chain propagation time of the chain dependency path, performing weighted fusion on the to-be-fused modal classroom behavior data according to the fusion window weights to obtain classroom behavior fusion data, and analyzing the classroom behavior fusion data to obtain interaction behavior scoring indexes of each target interaction behavior event. Optionally, attribute information of current classroom teaching behaviors including class types, teaching mode types and teaching object numbers is identified, key time sequence characteristics of the classes are extracted according to the attribute information, a first group of multi-mode class behavior data sets of a target class under the key time sequence characteristics of the classes are obtained through a first data collection frequency, a second group of multi-mode class behavior data sets which are not under the key time sequence characteristics of the classes are obtained through a second data collection frequency, wherein the first data collection frequency is larger than the second data collection frequency, and a first group of interactive behavior event coding sets and a second group of interactive behavior event coding sets are respectively obtained through the first group of multi-mode class behavior data sets and the second group of multi-mode class behavior data sets. Optionally, a local behavior chain dependency graph model and a global behavior chain dependency graph model are respectively built according to the first group of interaction behavior event code sets and the second group of interaction behavior event code sets, a local chain dependency path and a global chain dependency path based on each target interaction behavior event are identified according to the local behavior chain dependency graph model and the global behavior chain dependency graph model, local classroom behavior fusion data and global classroom behavior fu