CN-122002053-A - Intelligent sensing and decision system for distributed multi-mode data fusion

CN122002053ACN 122002053 ACN122002053 ACN 122002053ACN-122002053-A

Abstract

The invention relates to the technical field of distributed artificial intelligence and collaborative awareness, and discloses an intelligent awareness and decision-making system for distributed multi-mode data fusion, which comprises the steps of acquiring multi-domain real-time data streams in a live broadcast process, aligning the multi-domain real-time data streams according to uniform time granularity, and storing the multi-domain real-time data streams in a sliding window buffer area; the method comprises the steps of detecting abnormal points of a deviation degree sequence of each field by using an accumulation and detection algorithm, outputting abnormal alarm information when the accumulated deviation exceeds a dynamic threshold value, outputting the type of current abnormality to the field triggering abnormal alarm, calculating the occurrence probability and the urgency degree of each downstream event when a trend type abnormal event is detected, and generating early warning information and intervention suggestions. The problem of prejudging hysteresis caused by single-field detection is solved, and a time window is striven for operation intervention.

Inventors

SUN ZHIMING
CHEN HAITAO
LEI TONG
WANG BIN

Assignees

南京汇智互娱网络科技有限公司

Dates

Publication Date: 20260508
Application Date: 20260105

Claims (10)

1. The intelligent perception and decision method for the distributed multi-mode data fusion is characterized by comprising the following steps of: acquiring multi-domain real-time data streams in a live broadcast process, aligning the multi-domain real-time data streams according to a uniform time dimension, and storing the multi-domain real-time data streams in a sliding window buffer area; extracting features of the data in each field to generate a feature vector sequence in each field; Acquiring historical reference feature distribution of each field, calculating the deviation degree between the feature vector of each field and the corresponding historical reference distribution, and generating a deviation degree time sequence of each field; Detecting abnormal points by applying an accumulation and detection algorithm to the deviation sequences of all the fields, and outputting abnormal alarm information when the accumulated deviation exceeds a dynamic threshold value; Extracting feature change vectors of time periods before and after the abnormality in the field of triggering the abnormality alarm, performing similarity matching with a historical abnormality event library, and outputting the type pre-judgment of the current abnormality; Carrying out time sequence causal analysis on the historical abnormal event sequence to generate a cross-domain abnormal signal causal correlation map; When a trend type abnormal event is detected, inquiring the causal correlation map, acquiring a downstream correlation event list and an expected occurrence time window, calculating the occurrence probability and the urgency degree of each downstream event, and generating early warning information and intervention suggestions; And calculating the deviation by using a Markov distance, and calculating through the difference value between the current feature vector and the historical reference mean value vector and the inverse matrix of the historical reference covariance matrix.
2. The intelligent perception and decision-making method for the distributed multi-modal data fusion according to claim 1, wherein the multi-domain real-time data stream comprises anchor side data, content side data, audience side data and platform side data, wherein the anchor side data comprises peripheral operation frequency sequences and face video frame data, the content side data comprises picture quality indexes and topic keyword sequences, the audience side data comprises an online population sequence, a bullet screen text stream and an interaction rate sequence, and the platform side data comprises push distribution weight change data.
3. The intelligent sensing and decision-making method for the distributed multi-mode data fusion according to claim 1, wherein the application of an accumulation and detection algorithm to the deviation sequences of each field for outlier detection comprises setting a tolerance deviation reference value and an alarm threshold, for a positive deviation accumulation trend, adding a positive accumulation value of a previous moment to a current deviation degree minus the tolerance deviation reference value and taking a non-negative value as a current positive accumulation value, for a negative deviation accumulation trend, adding a negative accumulation value of a previous moment to a negative deviation degree minus the tolerance deviation reference value and taking a non-negative value as a current negative accumulation value, and outputting abnormal alarm information when the positive accumulation value or the negative accumulation value exceeds the alarm threshold.
4. The intelligent perception and decision-making method for distributed multi-mode data fusion according to claim 3, wherein the dynamic threshold is adaptively adjusted according to live broadcast type and historical false alarm rate, the threshold is increased when the false alarm rate is higher than a preset value, and the threshold is decreased when the false alarm rate is higher than the preset value.
5. The intelligent perception and decision-making method for the distributed multi-modal data fusion according to claim 1, wherein the type pre-judgment of the anomalies comprises pulse type and trend type, wherein an observation waiting strategy is adopted for the pulse type anomalies, and the trend type anomalies enter a causality association analysis and event pre-judgment flow.
6. The intelligent perception and decision-making method for distributed multi-modal data fusion according to claim 1, wherein the time-series causal analysis of the historical abnormal event sequence includes applying a glabelle causal test to the abnormal event sequence of any two domains, calculating causal intensity coefficients, wherein the nodes of the causal association graph are abnormal types of the domains, directed edges between the nodes represent causal relation directions, and the edge weights include causal intensity coefficients and average time differences.
7. The intelligent perception and decision-making method for the distributed multi-modal data fusion according to claim 1, wherein calculating the occurrence probability and the urgency of each downstream event comprises obtaining a causal intensity coefficient and an average time difference between the downstream event and the current anomaly, calculating the occurrence probability of each downstream event through a nonlinear increasing function based on the causal intensity coefficient and the current anomaly deviation intensity, and taking the ratio of the occurrence probability and the average time difference as the urgency.
8. The intelligent perception and decision-making method for the distributed multi-modal data fusion according to claim 1, further comprising the steps of recording the matching condition of the actual event evolution track and the pre-judgment result, updating the evolution mode classification in the historical abnormal event library and the causal strength coefficient in the causal correlation map by utilizing the matching result, enhancing the weight of the corresponding edge if the pre-judgment event actually occurs, and weakening the weight of the corresponding edge if the pre-judgment event does not occur.
9. The intelligent perception and decision-making method for the distributed multi-modal data fusion according to claim 1, wherein the early warning information comprises a pre-determined event type, a predicted occurrence time range and a comprehensive risk level, and the intervention advice is obtained by retrieving from a preset intervention policy library according to the event type.
10. A distributed multi-modal data fusion intelligent awareness and decision making system for performing the method of any of claims 1-9, comprising: the data acquisition and alignment module is used for acquiring multi-field real-time data streams in the live broadcast process and aligning the multi-field real-time data streams according to the unified time dimension; The feature extraction module is used for extracting features of the data in each field and generating a feature vector sequence in each field; the deviation degree calculation module is used for calculating the deviation degree between the characteristic vector of each current field and the corresponding historical reference distribution; the anomaly detection module is used for detecting anomaly points by applying an accumulation and detection algorithm to the deviation sequences of all the fields; the abnormality classification module is used for matching the characteristic change vectors before and after abnormality with the historical abnormal event library and outputting abnormality type pre-judgment; The causal analysis module is used for carrying out time sequence causal analysis on the historical abnormal event sequence and generating a cross-domain abnormal signal causal association map; the event prejudging module is used for inquiring the causal correlation map and calculating the occurrence probability and the urgency degree of the downstream event; And the early warning generation module is used for generating early warning information and intervention suggestions.

Description

Intelligent sensing and decision system for distributed multi-mode data fusion Technical Field The invention relates to the technical field of distributed artificial intelligence and collaborative awareness, in particular to an intelligent awareness and decision-making system for distributed multi-mode data fusion. Background In a digital live broadcast or a live broadcast operation scene, live broadcast events (such as audience loss, interaction dip, negative barrage burst and conversion rate dip) have significant influence on live broadcast effects. The occurrence of these events typically occurs in a number of fields of precursor signals, including a main cast side signal (operation state change, expression abnormality), a content side signal (picture quality fluctuation, topic switching abnormality), a viewer side signal (interaction mode change), and a platform side signal (traffic distribution change). Research shows that causal relationships exist among signals in various fields, for example, the abnormal state of the anchor often precedes the abnormal behavior of the audience. Therefore, if the early pre-judgment of the live event can be realized through fusion analysis of the multi-source signals and causal association mining, precious time windows are striven for operation intervention. However, the prior art has the following technical problems in achieving the above object: first, there are significant differences in abnormal signal acquisition frequencies and data formats in different fields. The data of the anchor end is a high-frequency operation sequence and a video frame, the data of the audience end is barrage text and interactive counting, and the data of the platform end is a flow weight parameter. These heterogeneous data are difficult to directly perform a unified comparative analysis. Second, the anomaly signal may be a transient pulse (sporadic fluctuations) or a persistent trend (real anomaly signs), both of which require different treatments. If the detection is not accurate, false alarm of the accidental fluctuation or missing alarm of the real abnormality can be caused. Thirdly, the causal relationship among the anomalies in different fields is not fixed and can be dynamically changed according to factors such as live broadcast type, anchor style, time period and the like. Event prejudging by adopting a fixed causality rule lacks adaptability, and is difficult to cope with diversified live broadcasting scenes. Fourth, existing methods generally only detect anomalies for a single domain, and cannot capture causal chains across domains, resulting in event prejudgement lagging behind the actual occurrence point, losing intervention opportunities. Disclosure of Invention The invention provides an intelligent perception and decision system for distributed multi-mode data fusion, which solves the technical problem that the fixed causal rule in the related technology lacks adaptability and single-field detection leads to prejudgment hysteresis. The invention provides an intelligent perception and decision method for distributed multi-mode data fusion, which comprises the following steps: acquiring multi-domain real-time data streams in a live broadcast process, aligning the multi-domain real-time data streams according to a uniform time dimension, and storing the multi-domain real-time data streams in a sliding window buffer area; extracting features of the data in each field to generate a feature vector sequence in each field; Acquiring historical reference feature distribution of each field, calculating the deviation degree between the feature vector of each field and the corresponding historical reference distribution, and generating a deviation degree time sequence of each field; Detecting abnormal points by applying an accumulation and detection algorithm to the deviation sequences of all the fields, and outputting abnormal alarm information when the accumulated deviation exceeds a dynamic threshold value; Extracting feature change vectors of time periods before and after the abnormality in the field of triggering the abnormality alarm, performing similarity matching with a historical abnormality event library, and outputting the type pre-judgment of the current abnormality; Carrying out time sequence causal analysis on the historical abnormal event sequence to generate a cross-domain abnormal signal causal correlation map; When a trend type abnormal event is detected, inquiring the causal correlation map, acquiring a downstream correlation event list and an expected occurrence time window, calculating the occurrence probability and the urgency degree of each downstream event, and generating early warning information and intervention suggestions; And calculating the deviation by using a Markov distance, and calculating through the difference value between the current feature vector and the historical reference mean value vector and the inverse matrix of the historical reference cov