CN-122020566-A - Risk assessment model construction method based on multi-modal data fusion and neural network

CN122020566ACN 122020566 ACN122020566 ACN 122020566ACN-122020566-A

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a risk assessment model construction method based on multi-modal data fusion and a neural network, which comprises the following steps: the method comprises the steps of video acquisition, detection of clothing color extraction, audio frequency domain feature statistics, text emotion keyword construction, mapping to a causal matrix, back propagation correction to generate risk conduction weight data, judging conflict according to weight change to generate risk conflict identification information, reconstructing causal chain extraction key factors to construct a risk propagation path network, inputting LSTM (link state model) evaluation, classifying to generate a multi-mode fusion risk evaluation model, constructing composite features through multi-source signals, combining weight change to form an adjustable conduction relation, triggering self-correction to clear conflict component salient key factors according to node symbol difference, generating a conduction path according to strength descending order to form evolution expression in time sequence modeling, and enabling the model to have stable, interpretability and practical guiding significance in cultural heritage protection and cultural scene risk management.

Inventors

HE MANRONG
MA ZIYU

Assignees

云南大学

Dates

Publication Date: 20260512
Application Date: 20260408

Claims (10)

1. The risk assessment model construction method based on multi-mode data fusion and the neural network is characterized by comprising the following steps: S1, acquiring a digital video stream of a performance scene through video acquisition equipment, detecting clothing color channel values, performing differential operation based on adjacent pixel channel components, acquiring audio signals, performing discrete Fourier transform to extract frequency domain features, counting text emotion keyword frequencies, and weighting and splicing to construct a multi-modal risk feature set; S2, calling the multi-mode risk feature set, mapping the dimension components to causal relation matrix nodes, performing back propagation correction on the connection weight, weighting and fusing the weight and the feature components and linearly combining the weight and the weight variation to generate risk conduction weight data; s3, based on the change trend of the weight value of the risk conduction weight data, judging a node change symbol, triggering conflict detection when the opposite direction of the node change is detected, and generating risk conflict identification information; S4, starting causal chain reconstruction according to the risk conflict identification information, clearing nodes and downstream paths related to conflict components, extracting key risk factor intensities, and arranging in a descending order to construct a risk propagation path network; And S5, inputting the risk propagation path network into a long-term and short-term memory network for risk assessment, normalizing the risk probability distribution, classifying and judging according to the risk threshold value, and generating a multi-mode fusion risk assessment model.
2. The method for constructing a risk assessment model based on multi-modal data fusion and neural network according to claim 1, wherein the multi-modal risk feature set comprises a visual feature quantity, an audio feature quantity and a text emotion quantity, the risk conduction weight data comprises a node weight coefficient, a path influence coefficient and an overall fluctuation coefficient, the risk conflict identification information comprises a direction conflict identification, a state conflict identification and a structural abnormality identification, the risk propagation path network comprises a factor ordering sequence, a path organization structure and a node hierarchy structure, and the multi-modal fusion risk assessment model comprises a risk probability distribution structure, a grading structure and a model output structure.
3. The risk assessment model construction method based on multi-modal data fusion and neural network according to claim 1, wherein the specific steps of S1 are as follows: S101, acquiring a video stream frame column of a performance scene through video acquisition equipment, detecting a color channel numerical sequence in a traditional dancewear area of a bronze drum dancer in a Wenshan, performing difference on the basis of adjacent pixel channel components, aggregating the component sequences according to a difference result, and recombining the aggregated sequences into a matrix structure to generate a clothing pixel color matrix; The traditional dancewear area comprises a drum pattern, ethnic totem color blocks and ceremony clothing pattern area which have non-symbolic significance in the bronze drum dance performance; s102, based on the clothing pixel color matrix, acquiring an audio signal sequence formed by bronze drum beating sound, accompaniment instrument sound and dance step pedal in the bronze drum performance process, performing discrete Fourier transform to obtain a frequency domain amplitude sequence, comparing adjacent frequency point amplitude values, segmenting amplitude segments according to comparison results, combining the segmented segments into a structure set according to indexes, and generating a frequency domain amplitude segment structure set; s103, calling the frequency domain amplitude segment structure set, counting text emotion keyword sequences related to non-genetic inheritance, performance evaluation and cultural acceptance of the bronze drum dance, comparing word frequency values with word frequency reference values, performing weighted splicing on the word frequency values and the frequency band values based on comparison results, and encoding the spliced sequences into feature matrixes to obtain the multi-mode risk feature set.
4. The risk assessment model construction method based on multi-mode data fusion and neural network according to claim 3, wherein the specific steps of S2 are as follows: S201, based on dimension components in the multi-mode risk feature set, searching causal relation matrix node numbers corresponding to the components, performing matching comparison operation on the numbers and the component amplitudes, and performing sequential adjustment in the matching process according to the number sequence and the amplitude variation to generate node mapping sequence data; S202, invoking node indexes of the node mapping sequence data, calculating a difference value according to an error signal aiming at a connecting weight data frame and comparing the difference value with an error reference value, and executing a corresponding weight of linear offset correction on a weight position with the difference value smaller than the error reference value to obtain a weight correction matrix; s203, invoking the associated feature components of the multi-mode risk feature set according to the correction index in the weight correction matrix, performing weighted fusion on the component amplitude and the correction amplitude, and performing linear combination on the fusion result and the matrix variation to generate risk conduction weight data.
5. The method for constructing a risk assessment model based on multi-modal data fusion and neural network according to claim 4, wherein the error reference value is a fixed value obtained by performing linear weighted summation on the error signal raw average value and the variance calculation result by combining the error signal variance calculation result in the same training period with the error signal raw average value obtained by statistics in a preset training period of a connection weight data frame.
6. The method for constructing a risk assessment model based on multi-modal data fusion and neural network according to claim 4, wherein the specific steps of S3 are as follows: S301, acquiring adjacent node symbol change data based on the risk conduction weight data weight change trend, performing direction comparison on the symbol change data, and judging symbol attributes according to symbol direction reference values to obtain a node symbol difference matrix; S302, calling the node symbol difference matrix, judging symbol difference values in the matrix, comparing the difference values according to a node direction conflict threshold value, recording and aggregating the difference indexes exceeding the threshold value, and generating a node conflict index set; S303, according to the node conflict index set, searching the node pairs of the index records, obtaining the bronze drum dance performance elements, the non-genetic carrier elements and the digitalized operation risk factor names corresponding to the node pairs, mapping, and carrying out serialization coding on the mapped risk factor pairs to obtain risk conflict identification information.
7. The method for constructing the risk assessment model based on multi-modal data fusion and neural network according to claim 6, wherein the symbol direction reference value performs statistical analysis on the direction deviation distribution of the risk conduction weight data in the original continuous sampling period, and a zero value of a direction change mean value is used as a direction judgment threshold point in the direction deviation distribution; And the node direction conflict threshold value carries out interval discretization processing on the joint distribution of the difference frequency and the difference amplitude in the difference value sample of the node symbol difference matrix, and an interval boundary with the occurrence probability accumulation of the difference amplitude not lower than ninety-five percent is selected from discretization results as a conflict judgment threshold.
8. The method for constructing a risk assessment model based on multi-modal data fusion and neural network according to claim 6, wherein the specific steps of S4 are as follows: S401, searching a node parameter set in a causal link structure body based on the risk conflict identification information, performing difference calculation on conflict component marking values in the node parameter set and reference marking values in the same node, and loading node indexes into a reject list when a difference result is non-zero to generate a conflict node removal matrix; S402, invoking an effective node index sequence in the conflict node clearing matrix to execute index mapping on a risk factor intensity value set in a causal link structure body, executing descending order processing on the mapped intensity value set according to intensity, recording ordering positions and executing serialization coding to obtain a risk factor intensity sequence; S403, calling the ordering position of the risk factor intensity sequence to aggregate the node connection parameter set in the causal link structure, performing path topology reconstruction and index arrangement on the aggregated connection relation, performing structure coding, and establishing a risk propagation path network.
9. The method for constructing a risk assessment model based on multi-modal data fusion and neural network according to claim 8, wherein the specific step of S5 is: S501, transmitting states to a time sequence unit of a long-period memory network based on the risk propagation path network, superposing the node risk values and the state values at the previous moment, calculating gating weights according to superposition results, mapping the risk values, and generating a time sequence risk probability array; s502, performing proportional operation according to probability values in the time sequence risk probability array and maximum probability values in the array, performing position rearrangement on corresponding positions of the sequence based on probability value groups obtained by the proportional operation, and performing vector merging on rearranged probability value groups to obtain normalized probability distribution vectors; S503, invoking the probability value and the risk threshold value in the normalized probability distribution vector to execute relation judgment, and based on the probability index meeting the requirement of the risk threshold value, performing category marking and synchronous aggregation on the associated time sequence output vector sequence to obtain a multi-mode fusion risk assessment model for copper-encouraging non-genetic digital protection, performance propagation and country text travel fusion application.
10. The method for constructing the risk assessment model based on the multi-mode data fusion and the neural network according to claim 9, wherein the decision of the relation between the probability value and the risk threshold in the normalized probability distribution vector is that a probability index not lower than the risk threshold is selected from the normalized probability distribution vector; and the risk threshold value is determined by extracting a median probability value from a joint probability sample set formed by the original risk sample data and the time sequence risk probability array.

Description

Risk assessment model construction method based on multi-modal data fusion and neural network Technical Field The invention relates to the technical field of artificial intelligence, in particular to a risk assessment model construction method based on multi-modal data fusion and a neural network. Background The technical field of artificial intelligence relates to carrying out representation modeling and association analysis on multi-source information by utilizing the capabilities of perception learning reasoning and the like, and the core matters comprise acquisition and preprocessing of multi-mode data such as visual voice text and the like, expression mode construction of multi-mode characteristics, consistency characterization and fusion strategy construction of cross-mode information and mode recognition and decision method formation based on a neural network. The traditional risk assessment model construction method is characterized in that single-mode data is adopted as input in a risk analysis scene, discrete features are extracted according to a fixed quantitative index system, and then association modeling is carried out on risk factors by using a rule aggregation method or a weight training mode based on a shallow neural network. In the prior art, risk characteristics are generated based on a single data channel, the characteristic structure depends on fixed indexes and a simplified statistical mode, the difference correlation among multi-source signals is difficult to reflect, an adjusting mechanism is lacking in single-path modeling when the direction of risk factors diverges, the risk relation is easy to be pulled by local characteristics to cause the deviation of a conducting structure, abnormal changes lack of independent test basis, the output result is difficult to present risk interactivity and influence judgment stability in complex situations, the sensitivity of a risk level to the strong and weak changes is influenced by local peak interference, and the overall expression is difficult to support continuity assessment requirements. Disclosure of Invention In order to solve the technical problems that in the prior art, risk characteristics are generated based on a single data channel, a characteristic structure depends on fixed indexes and a simplified statistical mode, the difference correlation among multi-source signals is difficult to reflect, a single-path modeling lacks an adjusting mechanism when the directions of risk factors are divergent, the risk relation is easy to be pulled by local characteristics to cause the deviation of a conducting structure, abnormal changes lack independent checking basis, the output result is difficult to present risk interactivity and influence judgment stability in complex situations, the sensitivity of a risk level to the strong and weak changes is influenced by local peak interference, and the integral expression is difficult to support the continuity evaluation requirement, the embodiment of the invention provides a risk evaluation model construction method based on multi-mode data fusion and a neural network. In order to achieve the above purpose, the present invention adopts a risk assessment model construction method based on multi-modal data fusion and neural network, comprising the following steps: S1, acquiring a digital video stream of a performance scene through video acquisition equipment, detecting clothing color channel values, performing differential operation based on adjacent pixel channel components, acquiring audio signals, performing discrete Fourier transform to extract frequency domain features, counting text emotion keyword frequencies, and weighting and splicing to construct a multi-modal risk feature set; S2, calling the multi-mode risk feature set, mapping the dimension components to causal relation matrix nodes, performing back propagation correction on the connection weight, weighting and fusing the weight and the feature components and linearly combining the weight and the weight variation to generate risk conduction weight data; s3, based on the change trend of the weight value of the risk conduction weight data, judging a node change symbol, triggering conflict detection when the opposite direction of the node change is detected, and generating risk conflict identification information; S4, starting causal chain reconstruction according to the risk conflict identification information, clearing nodes and downstream paths related to conflict components, extracting key risk factor intensities, and arranging in a descending order to construct a risk propagation path network; And S5, inputting the risk propagation path network into a long-term and short-term memory network for risk assessment, normalizing the risk probability distribution, classifying and judging according to the risk threshold value, and generating a multi-mode fusion risk assessment model. As a further scheme of the invention, the multi-modal risk featu