Search

CN-115623093-B - Message collection method and system

CN115623093BCN 115623093 BCN115623093 BCN 115623093BCN-115623093-B

Abstract

The present application relates to the field of data analysis technologies, and in particular, to a method and a system for collecting messages. The method is applied to a message collection system, the message collection system comprises at least two collectors, each collector comprises a first thread used for collecting messages and a second thread used for processing the messages, the method comprises the steps that if the first collector receives a target message, the first thread of the first collector obtains inner-layer five-tuple information of the target message, the first thread of the first collector judges whether the target collector used for processing the target message is the first collector or not based on the inner-layer five-tuple information of the target message, and if the first thread of the first collector judges that the target collector used for processing the target message is not the first collector, the first thread of the first collector forwards the target message to the second collector, so that the second thread of the second collector processes the target message.

Inventors

  • ZHU LEI

Assignees

  • 新华三技术有限公司

Dates

Publication Date
20260505
Application Date
20221017

Claims (10)

  1. 1. The message collection method is characterized by being applied to a message collection system, wherein the message collection system comprises at least two collectors, each collector comprises a first thread for collecting messages and a second thread for processing the messages, and the method comprises the following steps: If a first collector receives a target message, a first thread of the first collector acquires inner layer quintuple information of the target message; the first thread of the first collector judges whether the target collector for processing the target message is the first collector or not based on the inner five-tuple information of the target message; If the first thread of the first collector determines that the target collector for processing the target message is not the first collector, the first thread of the first collector forwards the target message to the second collector, so that the second thread of the second collector processes the target message.
  2. 2. The method of claim 1, wherein the method further comprises: If the first thread of the first collector determines that the target collector for processing the target message is the first collector, the first thread of the first collector sends the target message to the second thread of the first collector; And the second thread of the first collector processes the target message.
  3. 3. The method of claim 1, wherein each collector included in the message collection system is preset with a mapping relationship between each hash value and a collector, wherein the step of determining, by the first thread of the first collector, whether the target collector for processing the target message is the first collector based on the inner five-tuple information of the target message includes: the first thread of the first collector carries out hash calculation on the inner five-tuple information of the target message to obtain a target hash value; The first thread of the first collector determines a target collector for processing the target message based on the target hash value and the mapping relation between each hash value and the collector; the first thread of the first collector determines whether the target collector is the first collector.
  4. 4. A method according to any one of claims 1-3, wherein the step of the second thread of the first collector performing message processing on the target message comprises: and the second thread of the first collector performs splicing session processing based on the received target message with the same inner-layer quintuple information to obtain a forwarding path of the target message in a forwarding network.
  5. 5. The method of claim 4, wherein the method further comprises: The first collector receives messages forwarded by first threads of other collectors in the message collection system; The first collector sends the received message to a second thread of the first collector; And the second thread of the first collector processes the received message.
  6. 6. A message collection system comprising at least two collectors, each collector comprising a first thread for collecting messages and a second thread for processing messages; If a first collector receives a target message, a first thread of the first collector acquires inner layer quintuple information of the target message; the first thread of the first collector judges whether the target collector for processing the target message is the first collector or not based on the inner five-tuple information of the target message; If the first thread of the first collector determines that the target collector for processing the target message is not the first collector, the first thread of the first collector forwards the target message to the second collector, so that the second thread of the second collector processes the target message.
  7. 7. The system of claim 6, wherein, If the first thread of the first collector determines that the target collector for processing the target message is the first collector, the first thread of the first collector sends the target message to the second thread of the first collector; And the second thread of the first collector processes the target message.
  8. 8. The system of claim 6, wherein each collector included in the message collection system is pre-configured with a mapping relationship between each hash value and a collector, wherein the first thread of the first collector determines, based on inner five-tuple information of the target message, whether the target collector for processing the target message is the first collector, including: the first thread of the first collector carries out hash calculation on the inner five-tuple information of the target message to obtain a target hash value; The first thread of the first collector determines a target collector for processing the target message based on the target hash value and the mapping relation between each hash value and the collector; the first thread of the first collector determines whether the target collector is the first collector.
  9. 9. The system of any of claims 6-8, wherein the step of the second thread of the first collector performing message processing on the target message comprises: and the second thread of the first collector performs splicing session processing based on the received target message with the same inner-layer quintuple information to obtain a forwarding path of the target message in a forwarding network.
  10. 10. The system of claim 9, wherein the system comprises a plurality of sensors, The first collector receives messages forwarded by first threads of other collectors in the message collection system; The first collector sends the received message to a second thread of the first collector; And the second thread of the first collector processes the received message.

Description

Message collection method and system Technical Field The present application relates to the field of data analysis technologies, and in particular, to a method and a system for collecting messages. Background In order to obtain a complete message forwarding path, a technology of encapsulating remote port mirror image (Encapsulated Remote Switch PortAnalyzer, ERSPAN) is generally adopted, all switches in a network send messages to a collector through an IP protocol by configuration, and then the collector correlates the received messages with the same five-tuple to obtain a complete message forwarding path. When the data volume exceeds the processing capacity of a single collector, the number of collectors needs to be laterally expanded to obtain larger collection and processing capacity. However, the original packet is scattered to different collectors, so that the collectors cannot calculate the complete forwarding path. Disclosure of Invention The application provides a message collection method and device. In a first aspect, the present application provides a message collection method, applied to a message collection system, where the message collection system includes at least two collectors, each collector includes a first thread for collecting a message and a second thread for processing a message, and the method includes: If a first collector receives a target message, a first thread of the first collector acquires inner layer quintuple information of the target message; the first thread of the first collector judges whether the target collector for processing the target message is the first collector or not based on the inner five-tuple information of the target message; If the first thread of the first collector determines that the target collector for processing the target message is not the first collector, the first thread of the first collector forwards the target message to the second collector, so that the second thread of the second collector processes the target message. Optionally, the method further comprises: If the first thread of the first collector determines that the target collector for processing the target message is the first collector, the first thread of the first collector sends the target message to the second thread of the first collector; And the second thread of the first collector processes the target message. Optionally, each collector included in the message collection system is preset with a mapping relationship between each hash value and a collector, and the step of determining, by the first thread of the first collector, whether the target collector for processing the target message is the first collector based on the inner five-tuple information of the target message includes: the first thread of the first collector carries out hash calculation on the inner five-tuple information of the target message to obtain a target hash value; The first thread of the first collector determines a target collector for processing the target message based on the target hash value and the mapping relation between each hash value and the collector; the first thread of the first collector determines whether the target collector is the first collector. Optionally, the step of performing message processing on the target message by the second thread of the first collector includes: and the second thread of the first collector performs splicing session processing based on the received target message with the same inner-layer quintuple information to obtain a forwarding path of the target message in a forwarding network. Optionally, the method further comprises: The first collector receives messages forwarded by first threads of other collectors in the message collection system; The first collector sends the received message to a second thread of the first collector; And the second thread of the first collector processes the received message. In a second aspect, the present application provides a message collection system, the message collection system including at least two collectors, each collector including a first thread for collecting a message and a second thread for processing a message; If a first collector receives a target message, a first thread of the first collector acquires inner layer quintuple information of the target message; the first thread of the first collector judges whether the target collector for processing the target message is the first collector or not based on the inner five-tuple information of the target message; If the first thread of the first collector determines that the target collector for processing the target message is not the first collector, the first thread of the first collector forwards the target message to the second collector, so that the second thread of the second collector processes the target message. Optionally, if the first thread of the first collector determines that the target collector for processing the target message is the fir