WO-2026091367-A1 - METHOD AND APPARATUS FOR TRAINING ANOMALOUS TRAFFIC DETECTION MODEL, MONITORING METHOD, AND DEVICE

WO2026091367A1WO 2026091367 A1WO2026091367 A1WO 2026091367A1WO-2026091367-A1

Abstract

The present application relates to the technical field of traffic detection, and discloses a method and apparatus for training an anomalous traffic detection model, a monitoring method, and a device. The method comprises: acquiring a network traffic dataset, the network traffic dataset comprising a plurality of network traffic features; performing feature redundancy elimination on the network traffic dataset on the basis of the importance of each network traffic feature to form an alternative feature dataset; performing balancing processing on the alternative feature dataset to form a target feature dataset; and using the target feature dataset to train a preset network model to obtain an anomalous traffic detection model. In this way, screening network traffic features by means of the importance can retain only important network traffic features, so that the anomalous traffic detection model trained by means of the selected network traffic features is more targeted. Additionally, balancing processing is performed, so that the detection recall rate of small-sample network traffic feature types can be improved without increasing additional model complexity.

Inventors

XIE, CHEN
CHEN, LIANG
ZENG, Yuzhi
XU, Zhonghao

Assignees

上海斗象信息科技有限公司

Dates

Publication Date: 20260507
Application Date: 20250310
Priority Date: 20241031

Claims (9)

A training method for an abnormal traffic detection model, characterized by comprising: Obtain a network traffic dataset; the network traffic dataset includes multiple network traffic features; Based on the importance of each network traffic feature, feature redundancy elimination is performed on the network traffic dataset to form a candidate feature dataset; The candidate feature datasets are balanced to form the target feature dataset; The pre-defined network model is trained using the target feature dataset to obtain an abnormal traffic detection model; The process of balancing the candidate feature datasets to form a target feature dataset includes: for each network traffic feature: determining the distance between the network traffic feature and other network traffic features; identifying network traffic features with a distance less than a preset threshold as nearest neighbor samples; counting the types of each nearest neighbor sample and the number of each type, defining the type with the most numbers as the dominant type; if the dominant type is consistent with the type of the network traffic feature, then retaining the network traffic feature; if the dominant type is inconsistent with the type of the network traffic feature, then deleting the network traffic feature or adjusting the preset weight of the network traffic feature, and the remaining network traffic features forming the target feature dataset.
The method according to claim 1, characterized in that, acquiring the network traffic dataset includes: Obtain the raw network data and the data source for each of the raw network data; Extract key information from each of the original network data sources according to the data sources; The relationships between the various data sources are analyzed, and these relationships are integrated with the key information to obtain several network traffic features; the network traffic features constitute the network traffic dataset.
The method according to claim 1, characterized in that, based on the importance of each of the network traffic features, feature redundancy elimination is performed on the network traffic dataset to form a candidate feature dataset, comprising: Step S11: Input the network traffic dataset into a preset first limit tree model to obtain the importance score of each network traffic feature; Step S12: Remove the network traffic feature with the lowest importance score from the network traffic dataset to obtain a new network traffic dataset; Repeat steps S11 to S12 until a preset condition is met, and the remaining network traffic features form the candidate feature dataset; the preset condition includes: the number of network traffic features in the candidate feature dataset is less than or equal to a preset number.
The method according to claim 1 or 3, characterized in that, based on the importance of each of the network traffic features, feature redundancy elimination is performed on the network traffic dataset to form a candidate feature dataset, further comprising: Generate shadow features that have the same data distribution characteristics as the network traffic features; Several decision trees are constructed using a pre-defined second limit tree model. When splitting a node in each decision tree, the Gini index reduction of each network traffic feature is calculated, and the network traffic feature with the largest Gini index reduction is selected for splitting. The importance score of each network traffic feature is calculated based on all the Gini index reductions of each network traffic feature in the second limit tree model. Several decision trees are constructed using a pre-defined third limit tree model. When splitting a node in each decision tree, the reduction in Gini index of each shadow feature is calculated, and the shadow feature with the largest reduction in Gini index is selected for splitting. The importance score of each shadow feature is calculated based on all the reductions in Gini index of each shadow feature in the third limit tree model. Based on the importance scores of the network traffic features and the shadow features, a subset of the network traffic features are selected to form the candidate feature dataset.
The method according to claim 4, characterized in that, filtering a portion of the network traffic features and forming the candidate feature dataset based on the importance scores of the network traffic features and the shadow features, includes: Obtain the maximum value among the importance scores of all the shadow features; The importance score of each network traffic feature is compared with the maximum value, and the network traffic features with an importance score greater than the maximum value are retained to form the candidate feature dataset.
The method according to claim 1, characterized in that the preset network model is a multilayer perceptron model; training the preset network model using the target feature dataset to obtain an abnormal traffic detection model includes: The target feature dataset is input into the multilayer perceptron model for training, and the features in the target feature dataset are weighted and summed using a multi-head attention mechanism to obtain the output result. The loss value of the output result is calculated according to the preset loss function; The weights and biases of each layer in the multilayer perceptron model are optimized based on the loss value and a preset optimization algorithm until convergence or a set number of iterations is reached, thereby obtaining the abnormal traffic detection model.
An abnormal traffic monitoring method, characterized in that it includes: Obtain the traffic to be detected; The traffic to be detected is input into a preset abnormal traffic detection model to determine whether the traffic to be detected is abnormal; the abnormal traffic detection model is obtained by the method described in any one of claims 1 to 6.
A training device for an abnormal traffic detection model, characterized in that it comprises: The dataset acquisition module is used to acquire network traffic datasets; the network traffic datasets include multiple network traffic features. The first optimization module is used to eliminate feature redundancy in the network traffic dataset according to the importance of each network traffic feature and form a candidate feature dataset. The second optimization module is used to perform balancing processing on the candidate feature dataset and form the target feature dataset. The training module is used to train a preset network model using the target feature dataset to obtain an abnormal traffic detection model. The second optimization module is used to balance the candidate feature dataset and form the target feature dataset in the following way: For each network traffic feature: determine the distance between the network traffic feature and other network traffic features; determine the network traffic features with a distance less than a preset threshold as nearest neighbor samples; count the types of each nearest neighbor sample and the number of each type, and define the type with the most number as the dominant type; if the dominant type is consistent with the type of the network traffic feature, retain the network traffic feature; if the dominant type is inconsistent with the type of the network traffic feature, delete the network traffic feature or adjust the preset weight of the network traffic feature, and the remaining network traffic features form the target feature dataset.
An electronic device, characterized in that it includes a processor and a memory, the memory storing computer-executable instructions that can be executed by the processor, the processor executing the computer-executable instructions to implement the training method of the abnormal traffic detection model according to any one of claims 1 to 6 or the abnormal traffic monitoring method according to claim 7.

Description

Training methods and devices, monitoring methods and equipment for abnormal flow detection models Cross-reference of related applications This application claims priority to Chinese Patent Application No. 202411534575.1, filed on October 31, 2024, entitled “Training Method and Apparatus, Monitoring Method and Equipment for Abnormal Flow Detection Model”, the entire contents of which are incorporated herein by reference. Technical Field This application relates to the field of traffic detection technology, and in particular to a training method and apparatus, monitoring method and equipment for an abnormal traffic detection model. Background Technology Network traffic, as the carrier of information transmission and interaction in cyberspace, contains a large amount of important information. With the continuous development of science and technology, network viruses and attack methods are becoming increasingly complex, resulting in some abnormal network traffic. Abnormal network traffic may threaten network security, so how to automatically and accurately identify abnormal network traffic data is an urgent problem to be solved. It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key/important components or describe the scope of protection of these embodiments, but rather as a prelude to the detailed description that follows. This application provides a training method and apparatus for an abnormal traffic detection model, as well as a monitoring method and device, to automatically and accurately identify abnormal network traffic. This application provides a training method for an abnormal traffic detection model, comprising: acquiring a network traffic dataset; the network traffic dataset including multiple network traffic features; performing feature redundancy elimination on the network traffic dataset according to the importance of each network traffic feature to form a candidate feature dataset; performing balancing processing on the candidate feature dataset to form a target feature dataset; and using the target feature dataset to train a preset network model to obtain an abnormal traffic detection model. In the above embodiments, filtering network traffic features based on importance simplifies the network traffic features, retaining only the most important ones. This makes the anomaly traffic detection model trained using the filtered network traffic features more targeted. By balancing the candidate feature dataset to obtain the target feature dataset, the problem of data type imbalance in the candidate feature dataset can be reduced. This allows the model trained using the optimized candidate feature dataset to improve the detection recall rate for small sample network traffic feature types without increasing additional model complexity. Consequently, the anomaly traffic detection model trained using this method can automatically and accurately identify abnormal network traffic. Furthermore, the anomaly traffic detection model trained using this method can handle large-scale datasets, has a fast processing speed in practical applications, and can respond in real time. Furthermore, acquiring a network traffic dataset includes: acquiring raw network data and data sources for each raw network data source; extracting key information from each raw network data source according to the data source; analyzing the correlation between each data source and fusing the correlation with each key information source to obtain several network traffic features; and the network traffic features constitute the network traffic dataset. In the above implementation, by fusing the correlation and key information to obtain network traffic features, the information contained in the network traffic features can be made richer, so that the abnormal traffic detection model trained using the network traffic features can be more accurate. Furthermore, based on the importance of each of the network traffic features, feature redundancy elimination is performed on the network traffic dataset to form a candidate feature dataset, including: step S11, inputting the network traffic dataset into a preset first limit tree model to obtain the importance score of each of the network traffic features; step S12, removing the network traffic feature with the lowest importance score from the network traffic dataset to obtain a new network traffic dataset; repeating steps S11 to S12 until a preset condition is met, and the remaining network traffic features form the candidate feature dataset; the preset condition in