Search

EP-4742613-A1 - SELF-LEARNING MODEL FOR DEDUPLICATING 5G SERVICE-BASED INTERFACE (SBI) PACKETS

EP4742613A1EP 4742613 A1EP4742613 A1EP 4742613A1EP-4742613-A1

Abstract

Systems and methods for deduplicating data packets in a telecommunication network monitoring system (100) include receiving (802) a data packet from a monitoring data source (110) in a telecommunication network; determining (804), for the received data packet, a set of deduplication parameters including at least a consumer Network Function identifier (NFID), a producer NFID, a Network Function (NF) type, and a view associated with the data packet; analyzing (806) the data packet by comparing the deduplication parameters against entries in a deduplication hash table (150); and performing (810) an action based on the analyzing, the action comprising processing the data packet or discarding the data packet.

Inventors

  • ARTOLA CABALLERO, Juan Roberto
  • CISSE, Manuel

Assignees

  • TC France S.A.S.

Dates

Publication Date
20260513
Application Date
20251112

Claims (15)

  1. A method (800) for deduplicating data packets in a telecommunication network monitoring system, the method comprising: receiving (802), from a monitoring data source (110), a data packet associated with a communication between network functions; determining (804), for the data packet, a set of deduplication parameters including at least a consumer Network Function identifier, NFID, a producer NFID, a Network Function, NF, type, and a view; analyzing (806) the data packet by comparing the deduplication parameters with entries in a deduplication hash table (150) that map NFID-pair/view combinations to actions; identifying (808) whether multiple sources report multiple views for packets exchanged between a same consumer NFID and producer NFID; and performing (810), based on the analyzing, an action comprising processing the data packet or discarding the data packet.
  2. The method (800) according to claim 1, wherein determining (804) the deduplication parameters comprises, when the data packet does not include a deduplication header, identifying the consumer NFID, the producer NFID, the NF type, and the view from protocol metadata.
  3. The method (800) according to claim 1, wherein the deduplication hash table (150) is updated by a self-learning model when the deduplication parameters of the data packet are not present in the deduplication hash table (150).
  4. The method (800) according to claim 3, wherein the self-learning model maintains an NFID table that maps NFIDs as Self or Peer with respect to a selected data source (110).
  5. The method (800) according to claim 4, wherein the view is determined using the NFID table, and the view is a consumer view when the consumer NFID is Self and the producer NFID is Peer.
  6. The method (800) according to claim 4, wherein the view is determined using the NFID table, and the view is a producer view when the producer NFID is Self and the consumer NFID is Peer.
  7. The method (800) according to claim 1, wherein performing (810) the action comprises discarding packets associated with a first view for a given NFID pair based on a default prioritization of a second view.
  8. The method (800) according to claim 1, further comprising receiving, in the data packet, a deduplication header comprising metadata specifying at least the consumer NFID, the producer NFID, the NF type, and the view.
  9. The method (800) according to claim 8, wherein the deduplication header further comprises a direction flag, a proxy flag, or a compact NF index, and wherein the method further comprises resolving the compact NF index to a full NFID using a periodically received NF-mapping update.
  10. The method (800) according to claim 1, further comprising configuring a GET_ALL directive for specified NF-type combinations such that all views of the data packet, including duplicates, are retained.
  11. The method (800) according to claim 1, wherein the monitoring data source (110) is a virtual tap, a packet streaming source, or a mirror-based monitoring system.
  12. The method (800) according to claim 1, wherein the deduplication hash table (150) further associates each NFID-pair/view entry with a last-seen timestamp and dynamically ages entries after a timeout period to maintain up-to-date deduplication state.
  13. The method (800) according to claim 1, wherein determining (804) the view further comprises parsing a 3GPP-SBI-NF-Peer-Info field to identify whether the packet corresponds to a request or a response message.
  14. The method (800) according to claim 1, further comprising transmitting deduplication rules from a backend to the monitoring data sources (110) to enable pre-filtering of duplicate packets at a point of capture.
  15. A system (100) comprising one or more processors and a memory storing instructions, the processors being configured to carry out the steps of the method (800) according to any one of the preceding claims.

Description

CROSS-REFERENCE TO RELATED APPLICATION This application claims the benefit of U.S. Provisional Patent Application No. 63/719,308, filed November 12, 2024 (DAS Code 6868), and U.S. Provisional Patent Application No. 63/805,358, filed May 14, 2025 (DAS Code 7162). FIELD OF THE DISCLOSURE The present disclosure relates generally to data packet deduplication. More particularly, the present disclosure relates to systems and methods for deduplicating data packets captured in monitoring telecommunication networks. BACKGROUND OF THE DISCLOSURE In modern telecommunication networks, particularly with the advent of 5G and cloud-native network architectures, the need for robust, real-time monitoring of signaling and traffic flows has become increasingly critical. Network monitoring systems are used to collect, analyze, and interpret signaling data for various purposes including performance analysis, anomaly detection, troubleshooting, security, and subscriber experience optimization. Traditional network monitoring approaches often rely on physical port mirroring or packet brokers to capture traffic. However, with the virtualization of Network Functions (NFs) and the shift toward Service-Based Architectures (SBA) in the 5G Core (5GC), monitoring systems increasingly depend on virtualized taps (vTaps) and packet streaming sources embedded within virtual network elements. These monitoring sources provide different views of network traffic, such as from the perspective of the consumer NF, producer NF, or intermediate proxies. Due to the distributed nature of these monitoring points and the complexity of interactions between NFs, it is common for the same signaling message or data packet to be captured multiple times across different views or sources. This leads to data duplication, which can negatively impact the performance and accuracy of monitoring tools. Duplicate packets consume additional bandwidth, increase processing load, and result in misleading analytical insights if not properly managed. While some conventional deduplication techniques exist, such as simple byte-by-byte comparison or hashing, they are typically insufficient in this new context, particularly when the packets are not exact duplicates in terms of payload or metadata due to transformations or protocol behavior at different capture points. BRIEF SUMMARY OF THE DISCLOSURE The present invention introduces systems and methods for deduplicating data packets captured from various monitoring sources in a virtualized telecommunication network. The invention is particularly suited for use in 5G Core networks and other service-based architectures where multiple virtual monitoring points provide overlapping views of the same network communication. The deduplication system receives data packets from one or more monitoring sources, each of which may include vTaps, packet streaming interfaces, or mirrored feeds. Each packet is analyzed to determine key deduplication parameters including the consumer Network Function ID (NFID), producer NFID, NF type, and the view (e.g., consumer, producer, proxy-ingress, proxy-egress, mirror). These parameters are either extracted from a custom deduplication header included in the packet or inferred through a discovery model that leverages metadata about the monitoring sources and the traffic itself. Once the relevant parameters are identified, the deduplication system consults a deduplication hash table, which maps parameter combinations to actions, either to process the packet or discard it as a duplicate. If a new combination is encountered, a self-learning model updates the hash table and populates an NFID table used to dynamically identify the origin and role of each NF in the traffic flow. The deduplication process is guided by configurable policies, including default view prioritization (e.g., prioritizing consumer views), overrides for specific NF type combinations, and a GET_ALL mode for retaining all packets where necessary. Through this intelligent, adaptive deduplication method, the system reduces processing overhead, preserves monitoring accuracy, and maintains the integrity of network analytics. It enables operators to confidently deploy scalable, efficient, and highly configurable monitoring systems within virtualized 5G environments and beyond. BRIEF DESCRIPTION OF THE DRAWINGS The present disclosure is detailed through various drawings, where like components or steps are indicated by identical reference numbers for clarity and consistency. FIG. 1 is a diagram of a deduplication system.FIG. 2 is a visualization of a deduplication table used in the proposed deduplication process.FIG. 3 is a flowchart outlining the process followed by the deduplication system.FIG. 4 is a visualization of data packet parameters.FIGS. 5A and 5B are flowcharts of a self-learning process.FIG. 6 is a flowchart of a deduplication process.FIG. 7 is a diagram visualizing deduplication across edge components.FIG. 8 is a flowchart of a proce