CN-116910507-B - Abnormality detection method and device

CN116910507BCN 116910507 BCN116910507 BCN 116910507BCN-116910507-B

Abstract

The anomaly detection method is applied to computing equipment provided with a monitoring system, the monitoring system is used for monitoring a plurality of key performance index KPIs in the computing equipment, the method comprises the steps of determining historical feature vectors of KPI data of M dimensions stored at the moment t, wherein the historical feature vectors are used for representing feature vectors of time before the moment t, M is greater than or equal to 1, predicting the KPI vector at the moment t through a prediction model based on the historical feature vectors at the moment t to obtain a prediction feature vector at the moment t, obtaining a true value of the KPI data at the moment t to obtain a vector of the true value, and determining whether the KPI data at the moment t is abnormal or not based on the prediction vector and the vector of the true value. According to the method, abnormal KPIs can be rapidly judged from a large amount of KPI data under the condition of no manual participation.

Inventors

SI HAOTIAN
PEI CHANGHUA
ZHANG HAIMING
LI JIANHUI

Assignees

中国科学院计算机网络信息中心

Dates

Publication Date: 20260512
Application Date: 20230626

Claims (8)

1. An anomaly detection method, applied to a computing device, the computing device being deployed with a monitoring system for monitoring a plurality of key performance indicators KPIs in the computing device, the method comprising: Determining a historical feature vector of the KPI data of M dimensions stored at the moment t, wherein the historical feature vector is used for representing a feature vector of time before the moment t, and M is more than or equal to 1; The method comprises the steps of carrying out feature extraction on a history feature vector at a moment t through a prediction model to obtain the prediction feature vector at the moment t, determining the weight of KPI data of each dimension in the expert feature vector, integrating the plurality of expert feature vectors according to the weight to obtain a special feature vector of KPI data of each dimension, processing the special feature vector through a multi-layer feedforward neural network to obtain a prediction value of KPI data of each dimension at the moment t, integrating the prediction value to determine the prediction feature vector at the moment t, wherein the step of determining the weight of KPI data of each dimension in the expert feature vector comprises the steps of determining M sub-vectors of the history feature vector, wherein the sub-vectors are feature vectors of KPI data in the history feature vector, inputting each sub-vector in the M sub-vectors into a special feature vector of KPI data of each dimension, sharing the weight with the special feature vector in a shared structure, and sharing the special feature vector by the special feature vector; Acquiring a true value of KPI data at a moment t, and obtaining a vector of the true value; and determining whether the KPI data at the moment t is abnormal or not based on the prediction characteristic vector and the vector of the true value.
2. The method according to claim 1, wherein the determining the historical feature vector of the KPI data of M dimensions stored at the time t specifically comprises: Acquiring KPI data of M dimensions stored at a time t; carrying out normalization processing on KPI data of each dimension to obtain a KPI data matrix, wherein each row in the KPI data matrix represents KPI data of one dimension, and each column represents KPI data of different dimensions at the same time; and selecting the KPI data matrix according to a preset sliding time window with a fixed size, and taking the data in the sliding time window as the historical feature vector.
3. The method of claim 2, wherein the normalizing the KPI data for each dimension is performed by the following formula: In the formula, Values recorded in the storage medium for the KPI data under the dimension of interest are characterized, Characterizing the maximum value of KPI data in the dimension of interest, The minimum value of KPI data under the dimension is characterized.
4. The method according to claim 1, wherein the obtaining the predicted value of KPI data for each dimension at time t is calculated according to the following formula: In the formula, Characterizing said historical feature vector of the data at time t, The M-th dimension vector of the historical feature vector is characterized in that F represents a mapping function, G M represents an M-th dimension comprehensive gate structure in a gating network module, h M represents a mapping function corresponding to M-th dimension KPI data in a model, and ⨀ represents a vector inner product operation.
5. The method of claim 1, wherein the inputting each of the M sub-vectors into the exclusive gating structure corresponding to each dimension of KPI data determines an exclusive weight of each sub-vector, specifically according to the following formula: In the formula, A unshared gate structure characterizing KPI data in the k-th dimension, Characterizing the trainable matrix in the Wei-Du-gate, And the kth sub-vector in the historical feature vector of the KPI data at the moment t is characterized.
6. The method of claim 1, wherein the inputting each of the M sub-vectors into the shared gating structure corresponding to each dimension KPI data determines a sharing weight of each sub-vector, specifically according to the following formula: In the formula, The structure of the shared gate is characterized, The trainable matrix in the shared gate is characterized, And the kth sub-vector in the historical feature vector of the KPI data at the moment t is characterized.
7. The method of claim 1, wherein the determining whether an anomaly has occurred at time t based on the vector of predicted feature vectors and the vector of true values comprises: and calculating the distance between the prediction feature vector and the vector of the true value, and determining that the KPI data at the moment t is abnormal under the condition that the distance exceeds a preset threshold value.
8. An anomaly detection apparatus, disposed on a computing device, where the computing device is configured with a monitoring system, the monitoring system configured to monitor a plurality of key performance indicators KPIs in the computing device, the apparatus comprising: The acquisition module is used for determining historical feature vectors of key performance index KPI data of M dimensions stored at the moment t, wherein the historical feature vectors are used for representing feature vectors of time before the moment t, and M is greater than or equal to 1; The processing module is used for predicting the KPI vector at the moment t through a prediction model based on the historical feature vector at the moment t to obtain a predicted feature vector at the moment t, and comprises the steps of extracting features of the historical feature vector through a plurality of different expert networks to obtain a plurality of expert feature vectors, determining the weight of the KPI data at each dimension in the expert feature vector, integrating the expert feature vectors according to the weights to obtain a special feature vector of the KPI data at each dimension, processing the special feature vector through a multi-layer feedforward neural network to obtain a predicted value of the KPI data at the moment t, integrating the predicted value to determine the predicted feature vector at the moment t, determining the weight of the KPI data at each dimension in the expert feature vector, wherein the M sub-vectors of the historical feature vector are determined, the sub-vectors are feature vectors of the KPI data in the historical feature vector, inputting the weight of each sub-vector into the special feature vector corresponding to the KPI data at each dimension, and sharing the weight of the KPI data in each dimension and the special feature vector, and determining the shared feature structure of the KPI and the KPI data in each sub-dimension and the special feature vector; the processing module is also used for obtaining the true value of the KPI data at the moment t and obtaining the vector of the true value; the processing module is further configured to determine whether the KPI data at the time t is abnormal based on the prediction feature vector and the vector of the true value.

Description

Abnormality detection method and device Technical Field The present invention relates to the field of computer technologies, and in particular, to a method and an apparatus for detecting an abnormality. Background With the rapid development of internet applications and the rapid increase of the number of users, it is important to ensure the running stability of the internet application system. In order to ensure smooth running of internet application services, operators may deploy monitoring systems in the system to monitor various key performance indicators (key performance indicator, KPIs) in the system in real time, such as central processing unit (central processing unit, CPU) utilization, page accesses per minute, etc. Through KPI, whether the system is in a normal running state can be known. But in large internet services there may be tens or even hundreds of thousands of KPIs that need to be monitored. Therefore, how to judge the abnormal system service in a large amount of KPI data is a problem to be solved. Disclosure of Invention To solve the problems in the prior art, embodiments of the present application provide an anomaly detection method, apparatus, computing device, computer storage medium, and product containing a computer program, the system service abnormality can be rapidly judged from a large amount of KPI data. In a first aspect, an embodiment of the application provides an anomaly detection method, which comprises the steps of determining historical feature vectors of key performance index KPI data of M dimensions stored at a moment t, wherein the historical feature vectors are used for representing feature vectors of time before the moment t, M is greater than or equal to 1, predicting the KPI vector at the moment t through a prediction model based on the historical feature vectors at the moment t to obtain a prediction feature vector at the moment t, obtaining a true value of the KPI data at the moment t to obtain a vector of the true value, and determining whether the KPI data at the moment t is abnormal or not based on the prediction vector and the vector of the true value. In a possible implementation manner, acquiring historical feature vectors of the KPI data of M dimensions stored at the moment t, specifically comprising the steps of acquiring the KPI data of M dimensions stored at the moment t, carrying out normalization processing on the KPI data of each dimension to obtain a KPI data matrix, wherein each row in the KPI data matrix represents the KPI data of one dimension, each column represents the KPI data of different dimensions at the same moment, selecting the KPI data matrix according to a preset sliding time window with a fixed size, and taking the data in the sliding time window as the historical feature vectors. In one possible implementation, the KPI data for each dimension is normalized by the following formula: Where v 1 characterizes the value recorded in the storage medium for the KPI data in the attributed dimension, v max characterizes the maximum value for the KPI data in the attributed dimension, and v min characterizes the minimum value for the KPI data in the attributed dimension. In a possible implementation manner, a KPI vector at a moment t is predicted through a prediction model based on the historical feature vector at the moment t to obtain a predicted feature vector at the moment t, wherein the method comprises the steps of extracting features of the historical feature vector through a plurality of different expert networks to obtain a plurality of expert feature vectors, determining weights of KPI data at each dimension in the expert feature vectors, integrating the plurality of expert feature vectors according to the weights to obtain a characteristic feature vector of the KPI data at each dimension, processing the characteristic feature vector through a multi-layer feedforward neural network to obtain a predicted value of the KPI data at the moment t, integrating the predicted value, and determining the predicted feature vector at the moment t. In one possible implementation, the predicted value of KPI data of each dimension at time t is obtained, and calculated according to the following formula: Where w t represents the historical feature vector of the data at time t, M-th dimension vector representing historical feature vector, F representing mapping function, G M representing M-th dimension comprehensive gate structure in gate control network module, h M representing mapping function corresponding to M-th dimension KPI data in model, and as such representing vector inner product operation In one possible implementation, determining the weights of KPI data for each dimension in an expert feature vector includes: The method comprises the steps of determining M sub-vectors of a historical feature vector, wherein the sub-vectors are feature vectors of all the dimension KPI data in the historical feature vector, inputting each sub-vector of the M