Search

CN-121711193-B - Encryption traffic self-adaptive updating classification method and system for open network environment

CN121711193BCN 121711193 BCN121711193 BCN 121711193BCN-121711193-B

Abstract

The invention discloses an encryption traffic self-adaptive updating classification method and system for an open network environment. The method comprises the steps of firstly extracting endogenous semantic features and exogenous environmental features based on a causal decoupling mechanism, stripping environmental confusion factors through inverse disturbance and invariance constraint to obtain invariant semantic characterization, constructing macroscopic drift state vectors representing network situations, inputting trained meta-learning super-network intelligent decision self-adaptive control super-parameters, performing cross-modal meta-calibration by using large language model thinking chain reasoning, calculating subspace direction consistency of instantaneous gradient vectors of candidate samples and category optimization track prototypes to screen trusted samples, and introducing gradient orthogonal projection constraint to update low-rank adaptation layer parameters by combining a capacity-limited playback queue. The method solves the problem of concept drift through causal decoupling and meta learning decision, realizes forgetting prevention by using orthogonal projection update, and can improve the online adaptability and robustness of the model in an open environment without additional manual labeling.

Inventors

  • LIU GUANGJIE
  • Tong Zerui
  • ZHAI JIANGTAO
  • SUN JIE
  • DING YIFAN

Assignees

  • 南京信息工程大学

Dates

Publication Date
20260505
Application Date
20260212

Claims (10)

  1. 1. An encryption traffic self-adaptive updating and classifying method facing an open network environment is characterized by comprising the following steps: S1, splitting original encrypted network traffic according to five-tuple and sorting the original encrypted network traffic according to time to obtain independent encrypted traffic, extracting endogenous time sequence features representing service semantics and exogenous time sequence features representing network environment states, applying inverse disturbance to the network environment to obtain exogenous disturbance time sequence features, introducing invariance constraint loss by utilizing an encrypted traffic classification model to obtain invariable semantic feature representation insensitive to network environment noise, and outputting an initial prediction result and a confidence vector; S2, setting a sliding time window, carrying out intra-window statistics on an initial prediction result, a confidence coefficient vector and a constant semantic feature representation, and constructing a macroscopic drift state vector according to the statistics result, wherein the macroscopic drift state vector at least comprises an uncertainty distribution index and a class center overall drift index; S3, screening uncertain target flows based on the self-adaptive confidence coefficient threshold value and/or the self-adaptive energy threshold value, and directly outputting initial prediction results as final classification results when the uncertain target flows are not selected as reliable samples; the method comprises the steps of extracting at least one plaintext metadata from uncertain target streams to construct a text prompt and inputting a large language model, wherein the text prompt comprises preset K target class sets or descriptions, and constraining the large language model to output a cross-mode pseudo tag comprising a deduction path and semantic confidence based on a thinking chain reasoning mechanism; S4, screening candidate pseudo tag samples for online samples according to drift scores, self-adaptive confidence coefficient thresholds, self-adaptive energy thresholds, calibration pseudo tags and semantic confidence coefficients, keeping a backbone network parameter fixed, calculating instantaneous gradient vectors of the candidate pseudo tag samples about the trainable parameters based on the calibration pseudo tags by only taking low-rank adaptation layer or adapter layer parameters as the trainable parameters, calculating subspace direction consistency between the instantaneous gradient vectors and class optimization track prototypes to which the calibration pseudo tags belong; And S5, maintaining a historical playback queue with limited capacity according to the category, constructing a category balance training batch according to the trusted sample and the playback queue, modulating the update strength according to the drift score, introducing gradient orthogonal projection constraint, keeping the main network parameter fixed, only updating the low-rank adaptation layer or adapter layer parameter, keeping the parameter update direction of the new task orthogonal to the historical parameter subspace or minimizing interference, realizing model forgetting prevention self-adaptive update, and outputting the updated model for the online classification prediction of the subsequent encrypted flow.
  2. 2. The method according to claim 1, wherein step S1 specifically comprises: S1-1, session splitting is carried out on the collected original encrypted network traffic according to the five-tuple, messages belonging to the same five-tuple are classified into the same stream, and the messages in each stream are arranged in an ascending order according to the arrival time, so that encrypted stream records arranged in time sequence are obtained; S1-2, extracting message length information and direction mark information which are arranged in time sequence according to each encryption stream, and splicing the message length information and the direction mark information according to time sequence to form an endogenous time sequence characteristic representing service logic; S1-3, calculating a packet interval time difference according to the arrival time of adjacent messages for each encryption stream, counting at least one network state characteristic of packet loss rate, retransmission count, round trip delay estimation, queue load and/or bandwidth utilization rate in a preset time granularity, defining the network state characteristic as an exogenous environment confusion factor fluctuating along with a link state, and splicing a packet interval time sequence with the exogenous environment confusion factor according to time steps to form an exogenous time sequence characteristic representing the state of the network environment; S1-4, randomly rearranging, resampling or denoising network state features, constructing a counterfactual view which is not matched with original semantics, obtaining exogenous disturbance time sequence features, constructing an encrypted flow classification model which is input by sequence pairs (endogenous time sequence features, exogenous time sequence features) and (endogenous time sequence features, exogenous disturbance time sequence features) and comprises a double-branch network and a classifier, and obtaining a corresponding prediction confidence vector And (3) with Constructing a loss function containing invariance constraints , wherein, To be based on cross entropy loss with labeled samples, Is a real label; To measure the distribution difference measure of the prediction result between the original view and the inverse fact view, the distribution difference measure is KL divergence, JS divergence or One of the norms; And as a weight coefficient, the predicted result is kept stable to the environmental disturbance by minimizing the loss, so that causal decoupling of the semantic features and the environmental noise is realized, and the invariant semantic feature representation of the environmental disturbance is obtained.
  3. 3. The method according to claim 1, wherein step S2 specifically comprises: Step S2-1, setting the length as Will be the first The set of traffic samples within a single time window is noted as For each sample Obtaining pairs output by an encrypted traffic classification model Prediction confidence vector for individual categories Wherein Recording the corresponding invariant semantic feature vector obtained based on S1 decoupling as the category number ; Step S2-2, recording the sample Classified logic vector of (2) is The energy fraction is defined as: Defining the prediction entropy as follows: Counting the energy fraction and the prediction entropy in a window to obtain a corresponding mean value, variance or quantile, and defining the mean value, variance or quantile as an uncertainty distribution index; step S2-3, for each category Intra-window prediction as category Is denoted as the sample set of (2) Calculating a category feature center for the invariant semantic feature vector: For adjacent windows And (3) with The amount of drift for category c is defined as: and according to Counting the maximum value, the mean value or the weighted sum to be used as a class center overall drift index for representing semantic distribution change; s2-4, splicing the uncertainty distribution index, the class center overall drift index and the flow load information in the window according to a preset sequence to form a first step Macroscopic drift state vector corresponding to each time window Inputting the macroscopic drift state vector into a meta learning super network obtained by training The meta-learning super-network learns a nonlinear mapping relation from an environment state to a control strategy by learning: Intelligent decision-making self-adaptive confidence threshold value of current window Adaptive energy threshold Drift scoring The method is used for controlling the subsequent meta-calibration and the self-evolution updating strength.
  4. 4. The method of claim 3, wherein the meta-learning super-network is trained by constructing a task set comprising a plurality of meta-tasks based on historical traffic windows during an offline meta-training phase, each meta-task comprising at least a support window for policy generation and a query window for effect verification, a macroscopic drift state vector input to the support window Super network output by meta learning Triggering cross-modal meta-calibration and trusted sample screening according to the method, and carrying out gradient-based meta-updating on the adaptation parameters on the premise of keeping the parameters of the main network fixed Simultaneously based on the purity index of the trusted sample set on the query window Or calibrate consistency indicators Constructing a sample screening quality item Defining a meta optimization objective function of a meta learning super network as follows: ; Wherein, the , As the weight coefficient of the light-emitting diode, The regular term is used for scoring the threshold value and the drift and is used for restraining the smoothness and the stability of output; optimizing objective function pair super network parameters by minimizing the elements And performing double-layer optimization updating to obtain the generalization mapping capability from the macroscopic drift state vector to the optimal self-adaptive control strategy.
  5. 5. A method according to claim 3, wherein step S3 comprises: Step S3-1, record the first The predictive confidence vector for the bar stream is With maximum predictive confidence of The corresponding energy fraction is If (if) And/or If the drift score of the window in which the stream is located is higher than a preset drift threshold value and falls into a specified fuzzy interval, marking the stream as an uncertain target stream as a candidate object for cross-modal element calibration, if And Marking the stream as a reliable sample if the stream does not fall into a fuzzy interval, and directly outputting an initial prediction result as a final classification result, wherein the fuzzy interval surrounds the first Confidence threshold for each time window And an adaptive energy threshold Comprises: Or (b) , And The bandwidth parameter is preset; step S3-2, for each uncertain target stream, extracting unencrypted plaintext metadata from associated transmission control protocol handshakes, transmission layer security sessions, domain name system resolutions and application layer interactions, wherein the plaintext metadata comprises one or more fields of server name indication SNI, digital certificate fields, DNS records, host names, paths or user agents in HTTP/HTTP2 headers, and recording an encrypted traffic classification model as a prediction category of a fast path classification model for the target stream is: Retrieving Top-N samples semantically similar to the metadata of the target stream and real labels thereof from a historical high-confidence sample library as a context example, wherein the metadata, the context example, And Writing a description template together to construct a structured text prompt, wherein a preset category set is explicitly given in the structured text prompt Or the corresponding class name/description to restrict the large language model to output results only in the preset class set; S3-3, inputting the structured text prompt into a pre-trained large language model or a pre-trained transducer model with text reasoning capability, exciting the thinking chain reasoning capability of the model, enabling the model to output a logic deducing path based on metadata, and then collecting the logic deducing path in a preset category Semantic probability distribution of up-output target stream And calculates the recommended category: , Will be Cross-modal pseudo tags as target streams to be Verifying the rationality of the pseudo tag by using the logic deduction path as a calibration basis; Step S3-4, setting a semantic confidence threshold And gain threshold According to Degree of The relationship determination after calibration of the pseudo tag is: ; When (when) And is also provided with Marking the sample as a cross-modal consistent sample and improving the credibility of the pseudo tag thereof when the sample and the sample are inconsistent but meet the requirements When it will Considered as the first-choice label after error correction, and finally the pseudo label after calibration Semantic confidence And the cross-modal consistency mark is used as auxiliary information of a meta-calibration side and is used for guiding subsequent trusted sample screening and self-evolution updating together with the window drift state vector and the self-adaptive control super-parameter.
  6. 6. The method according to claim 5, wherein step S4 specifically comprises: step S4-1, screening samples satisfying the following conditions (1) samples Maximum prediction confidence of (2) Less than or equal to the adaptive confidence threshold And/or energy fraction Greater than or equal to the adaptive energy threshold Or a sample of Maximum prediction confidence of (2) And/or energy fraction The semantic confidence of the sample after bypass calibration is improved and the calibrated pseudo tag Fall within a preset category set An inner part; S4-2, extracting parameters of the network from the trunk features of the encryption traffic classification model Fixing, only updating parameters of an adaptation layer embedded in a preset layer of an encryption traffic classification model For each candidate pseudo tag sample selected To calibrate the pseudo tag Constructing a loss function for supervisory signals In the process of only Obtaining an instantaneous optimized gradient vector of the sample under the constraint of derivation: for each category Selecting a category from the historical labeling samples and the historical high-confidence pseudo tag samples Is a sample set of (1) And (3) counting to obtain a category optimization track prototype representing the category history optimization direction: Category of when Missing results in historic annotated samples and/or historic high confidence pseudo tag samples When the model is empty, performing cold start of the category prototype, wherein the semantic confidence coefficient is selected from the cross-modal calibration result to be not lower than a threshold value And the pseudo tag after calibration is classified into Is a candidate sample set of (1) Maintaining backbone network parameters Fixed and adapted to parameters only Computing instantaneous gradient vectors under derived constraints Initializing a category optimization track prototype by the mean value of the transient gradient vector: then, online updating the category optimization track prototype in a sliding average or exponential sliding average mode based on the newly-entered trusted sample in the self-evolution process; Step S4-3, for candidate pseudo tag samples Record the calibrated pseudo tag category as Definition of samples The directional consistency score in the parameter optimization subspace is: The score indicates whether the update direction of the current sample is consistent with the historical optimization track of the corresponding category Prediction entropy of (2) is Maximum confidence is Semantic confidence is Constructing an uncertainty weight function The function is as follows Increase monotonously and follow Increasing monotonically decreasing, thereby defining samples The comprehensive weight of (2) is as follows: When the sample is Subspace directional consistency scores of (2) Not lower than a preset consistency threshold And a sample is Is the integrated weight of (2) Not lower than a preset weight threshold The sample is recorded as a trusted sample to obtain a trusted sample set, wherein when the sample is classified Initializing category optimization trajectory prototypes by the cold start approach described above Then, according to the subspace direction consistency score And comprehensive weight And judging a trusted sample.
  7. 7. The method according to claim 6, wherein step S5 specifically comprises: step S5-1, for each category Maintenance capacity of Knowledge playback queue of (a) When a newly generated trusted sample Belongs to the category of When the queue is If not, directly enqueuing the data, if the data is queued If the sample in the queue is full, the sample in the queue is comprehensively weighted Or time stamping for new trusted samples, playback teams from each class while performing self-evolving updates Extracting a fixed number of historical samples according to a class balancing strategy, and mixing the historical samples with the credible samples in the current window according to class balancing to construct a training batch considering new and old knowledge distribution; step S5-2, according to the drift score obtained in step S2 And the overall uncertainty level of the current window, and adaptively setting the self-evolution learning rate of the current round And update the step number At each round of self-evolution update In the multiple iterations, the current training batch sample set And weights thereof Based on the original update gradient of the calculation task, introducing gradient orthogonal projection operation, eliminating the components parallel to the subspace of the historical parameters in the original gradient, and obtaining the orthogonalization gradient with the minimum interference of old knowledge Adapting parameters based on the orthogonalization gradient pairs Updating, wherein the updating form is as follows: , wherein, = Representing gradient direction after orthogonal projection on parameter manifold, and always maintaining main network parameters in updating process Is unchanged.
  8. 8. An encrypted traffic adaptive update classification system for an open network environment, comprising: The feature extraction and causal decoupling module is used for shunting the original encrypted network flow according to five-tuple and sorting the original encrypted network flow according to time to obtain independent encrypted flow, extracting endogenous time sequence features representing service semantics and exogenous time sequence features representing network environment states, applying counterfactual disturbance to the network environment to obtain exogenous disturbance time sequence features, introducing invariance constraint loss by utilizing an encrypted flow classification model, obtaining invariable semantic feature representation insensitive to network environment noise, and outputting an initial prediction result and a confidence vector; The macro drift sensing and super-network decision module is used for setting a sliding time window, carrying out intra-window statistics on an initial prediction result, a confidence coefficient vector and a constant semantic feature representation, and constructing a macro drift state vector according to a statistical result, wherein the macro drift state vector at least comprises an uncertainty distribution index and a category center overall drift index; The system comprises a cross-modal thinking chain calibration module, a text prompt and a calibration pseudo tag, wherein the cross-modal thinking chain calibration module is used for screening an uncertain target stream based on a self-adaptive confidence coefficient threshold value and/or a self-adaptive energy threshold value, directly outputting an initial prediction result as a final classification result when the uncertain target stream is not selected as a reliable sample, extracting at least one plaintext metadata from the uncertain target stream to construct the text prompt and inputting a large language model, the text prompt comprises preset K target class sets or descriptions, and the constraint large language model outputs a cross-modal pseudo tag comprising a deduction path and a semantic confidence coefficient based on a thinking chain reasoning mechanism; The system comprises a trusted sample screening module, a main network parameter setting module, a sub-space direction consistency calculation module, a trusted sample analysis module and a trusted sample analysis module, wherein the trusted sample screening module is used for screening candidate pseudo-label samples for online samples according to drift scores, self-adaptive confidence thresholds, self-adaptive energy thresholds, calibration pseudo-labels and semantic confidence, keeping a main network parameter fixed, calculating the instantaneous gradient vector of the candidate pseudo-label samples relative to the trainable parameter based on the calibration pseudo-labels by only taking a low-rank adaptation layer or an adapter layer parameter as a trainable parameter, and calculating the subspace direction consistency between the instantaneous gradient vector and a class optimization track prototype to which the calibration pseudo-label belongs; The anti-forgetting self-evolution updating module maintains a historical playback queue with limited capacity according to the category, builds a category balance training batch according to the trusted sample and the playback queue, modulates the updating strength according to the drift score, introduces gradient orthogonal projection constraint, keeps the parameters of a main network fixed and updates only the parameters of the low-rank adaptation layer or the adapter layer, keeps the parameter updating direction of a new task orthogonal to the subspace of the historical parameters or minimizes interference, realizes model anti-forgetting self-adaption updating, and outputs the updated model for online classified prediction of the follow-up encryption flow.
  9. 9. An electronic device comprising one or more processors, memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs when executed by the processor implementing the steps of the open network environment oriented encryption traffic adaptive update classification method of any of claims 1-7.
  10. 10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the open network environment oriented encryption traffic adaptive update classification method according to any of claims 1-7.

Description

Encryption traffic self-adaptive updating classification method and system for open network environment Technical Field The invention belongs to the technical field of intelligent analysis and deep learning of network traffic, and particularly relates to an encryption traffic self-adaptive updating classification method and system for an open network environment. Background With the large number of applications of TLS, VPN, QUIC encryption protocols, the traffic encryption trend of Web access, mobile applications and cloud services is enhanced, and the traditional detection and identification method relying on plaintext load features is gradually invalid. Therefore, the academia and industry propose an end-to-end encryption traffic classification method by combining statistics and time sequence characteristics such as packet length sequence, direction sequence, packet interval time and the like with a convolutional network, a cyclic network or a transducer. The existing method is mostly used for offline training on a relatively static data set, is easily influenced by long-term changes of link quality, user behaviors and service structures in a real open network environment, and causes inconsistent training distribution and online distribution and performance degradation with time. In order to alleviate the problem, part of the work adopts pseudo tag self-training with a fixed threshold, but the threshold is usually set by experience, and the statistical index can only reflect 'change', so that the concept drift of semantic mode change and covariant offset caused by time delay, packet loss and the like are difficult to distinguish, and the update and pseudo tag noise accumulation are easily triggered by mistake under noise fluctuation. In addition, although meta learning can promote the rapid adaptability of the model to new tasks, multi-aspect offline construction task training lacks a fine granularity drift sensing and self-adaptive adjusting mechanism oriented to online unlabeled flow. On the other hand, the encrypted traffic still keeps partial plaintext metadata at the header of the handshake, certificate, DNS resolution and application layer, research has been attempted for service identification or threat detection, and proposal exploration is performed to resolve text information by using a large language model, but the method is often used as a high-cost main classifier or simple rule supplement, and a cross-modal calibration and pseudo-tag arbitration framework which is cooperated with the underlying encrypted traffic classification model is still lacking. Therefore, the prior art still has the following limitations under an open complex network environment that firstly, semantic related features and network environment fluctuation influence are difficult to effectively distinguish, the covariate offset such as time delay, packet loss and the like are easy to be misjudged as semantic mode change, secondly, pseudo tag screening and online updating often depend on fixed threshold values or experience rules, self-adaptive linkage control is difficult to be realized along with a statistical state, mistriggering updating and noise accumulation are easy to occur, thirdly, plaintext metadata such as handshakes, certificates, DNS (domain name system) and application layer headers and the like and text reasoning results based on the plaintext metadata are not fully utilized, a calibration and pseudo tag arbitration mechanism which is cooperated with a bottom layer classification model is lacked, and pseudo tag noise is still easy to be introduced and catastrophic forgetting is still induced in the long-term online self-adaptive updating process. Disclosure of Invention Aiming at the defects of the prior art, the invention provides an online self-adaptive updating and classifying method and system for the encrypted traffic of an open network environment, which improve the classifying stability through macroscopic drift sensing and cross-modal element calibration and realize efficient self-adaptive updating of parameters. According to the technical scheme, in the first aspect, the encryption traffic self-adaptive updating and classifying method facing the open network environment comprises the following steps: S1, splitting original encrypted network traffic according to five-tuple and sorting the original encrypted network traffic according to time to obtain independent encrypted traffic, extracting endogenous time sequence features representing service semantics and exogenous time sequence features representing network environment states, applying inverse disturbance to the network environment to obtain exogenous disturbance time sequence features, introducing invariance constraint loss by utilizing an encrypted traffic classification model to obtain invariable semantic feature representation insensitive to network environment noise, and outputting an initial prediction result and a confidence vector; S2,