CN-117331724-B - Log processing method, electronic equipment and computer storage medium

CN117331724BCN 117331724 BCN117331724 BCN 117331724BCN-117331724-B

Abstract

The disclosure provides a log processing method, which comprises the steps of determining a normal log control flow graph based on a history log flow, wherein the history log flow is generated under the condition that a system normally operates, determining a current log control flow graph based on a current log flow, determining initial destination nodes of log printing in the current log control flow graph according to the normal log control flow graph and the current log control flow graph, and filtering the initial destination nodes of the log printing according to a preset filtering algorithm to determine final destination nodes of the log printing. The method has the advantages that the abnormality is automatically found, the key log information which can be used for deducing the root cause of the fault is determined, automatic log enhancement can be realized without manual priori knowledge, the log quality is improved, and the efficiency and the accuracy of fault diagnosis are improved. The disclosure also provides an electronic device and a computer storage medium.

Inventors

HAN JING
ZHANG BAISHENG
Gong Zican
LI YING
JIA TONG
WU YIFAN
Hou Chuanjia

Assignees

中兴通讯股份有限公司
北京大学

Dates

Publication Date: 20260508
Application Date: 20220623

Claims (11)

1. A log processing method, the method comprising: determining a normal log control flow graph based on a historical log flow, wherein the historical log flow is generated under the condition that a system is in normal operation; Determining a current log control flow graph based on the current log flow; Determining an initial destination node of log printing in the current log control flow diagram according to the normal log control flow diagram and the current log control flow diagram; Filtering the initial destination node of the log printing according to a preset filtering algorithm to determine a final destination node of the log printing; The filtering the initial destination node of the log printing according to a preset filtering algorithm to determine a final destination node of the log printing includes: constructing a composite matrix according to the current system fault information and the initial destination node printed by the log, wherein the composite matrix is a matrix which is compounded with the system fault information and the association relationship between the system fault information and the initial destination node printed by the log; and determining a final destination node of the log printing by using the preset filtering algorithm and the composite matrix.
2. The method of claim 1, wherein the step of determining a log control flow graph based on the log flow comprises: Converting each log item in the log stream into a log template; Determining an edge weight between two log templates corresponding to the same request for each pair, and taking each log template as a node, wherein the edge weight is determined according to the generation time difference between the two nodes; And constructing a transfer edge between the two corresponding log templates according to each edge weight so as to obtain the log control flow graph.
3. The method of claim 2, wherein the determining an initial destination node for log printing in the current log control flow graph from the normal log control flow graph and the current log control flow graph comprises: Matching the normal log control flow graph with the current log control flow graph according to nodes; and determining an initial destination node of the log printing according to the node which is abnormal in matching in the current log control flow diagram.
4. The method of claim 3, wherein matching the normal log control flow graph and the current log control flow graph by node comprises: The nodes in the normal log control flow diagram and the nodes in the current log control flow diagram are matched in pairs to determine matching node groups, each matching node group comprises a normal node pair in the normal log control flow diagram and a current node pair in the current log control flow diagram, and the normal node pair in each matching node group is matched with the current node pair; Comparing the edge weights between the normal node pairs in each matched node group with the edge weights between the current node pairs; And when the edge weight between the current node pairs is larger than the edge weight between the normal node pairs, determining the current node pairs in the current matching node group as the nodes with abnormal matching.
5. The method of claim 3, wherein the determining the initial destination node for the log print based on the node in the current log control flow graph that matches an exception comprises: and under the condition that the node in the current log control flow diagram is not matched with any node in the normal log control flow diagram, determining the node in the current log control flow diagram as the initial destination node of the log printing.
6. The method of claim 3, wherein the determining the initial destination node for the log print based on the node in the current log control flow graph that matches an exception comprises: and under the condition that the father node in the current log control flow diagram is matched with the father node in the normal log control flow diagram and the child nodes after the father node in the current log control flow diagram are not matched with the child nodes after the father node in the normal log control flow diagram, determining the father node and the child nodes after the father node in the current log control flow diagram as the initial destination nodes of the log printing.
7. The method of claim 2, wherein converting each log entry in the log stream into a log template comprises: And in response to receiving the conversion instruction, replacing the variable in each log item with a placeholder, and generating the log template according to each replaced log item.
8. The method of claim 2, wherein determining the edge weights between two log templates corresponding to the same request for each pair of log templates comprises: determining each generation time interval of the two log templates; and determining the side weight between the two log templates according to the maximum value in each generation time interval.
9. The method of any of claims 1-8, wherein constructing a composite matrix based on current system failure information and the log printed initial destination node comprises: constructing a label vector f=according to the current system fault information Where m represents the total number of system failures, Representing an mth system failure; Constructing vector according to the initial destination node of the log printing Where n represents the total number of initial destination nodes for the log print, Representing an initial destination node of nth log printing; constructing an association relation matrix M according to the F and the T, wherein elements in the M , ; Constructing the composite matrix D according to the F and the M, wherein D 。
10. An electronic device, comprising: One or more processors; A storage device having one or more programs stored thereon; the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the log processing method of any of claims 1-9.
11. A computer storage medium having stored thereon a computer program, wherein the program when executed implements the log processing method according to any of claims 1-9.

Description

Log processing method, electronic equipment and computer storage medium Technical Field The present invention relates to the field of log technologies, and in particular, to a log processing method, an electronic device, and a computer storage medium. Background With the development of artificial intelligence (ARTIFICIAL INTELLIGENCE, AI), intelligent operation and maintenance (ARTIFICIAL INTELLIGENCE for IT Operations, AIOps) was first proposed by Gartner in 2016, that is, large-scale data from various operation and maintenance tools and devices are analyzed through algorithms such as machine learning (MACHINE LEARNING), problems occurring in a system are automatically found and responded in real time, and then the operation and maintenance capability and automation degree of information technology (Information Technology, IT) are improved. Under AIOps trend, automatic and intelligent fault diagnosis with data analysis of system logs as a core becomes an important component and development trend of a distributed software system fault diagnosis technology. At present, although system logs are widely applied in fault diagnosis technology, logs written by developers are designed for human beings, rather than machines, so that abnormality detection and fault diagnosis can be automatically carried out, so that a plurality of key log information which can be used for deducing the root cause of the fault are lost, and the performance of the automatic and intelligent fault diagnosis technology is seriously affected by the characteristics of the logs. While there are presently log enhancement techniques for improving log quality, such as adding additional variables or adding log print points where errors may occur, the purpose of such log enhancement techniques is to enhance the developer's understanding of the log, rather than for automated, intelligent fault diagnosis, and such log enhancement techniques also require a significant amount of manual a priori knowledge. Therefore, there is a need for an automated log enhancement method that does not require manual a priori knowledge. Disclosure of Invention The present disclosure addresses the above-described deficiencies in the prior art by providing a log processing method, an electronic device, and a computer storage medium. In a first aspect, an embodiment of the present disclosure provides a log processing method, including: determining a normal log control flow graph based on a historical log flow, wherein the historical log flow is generated under the condition that a system is in normal operation; Determining a current log control flow graph based on the current log flow; Determining an initial destination node of log printing in the current log control flow diagram according to the normal log control flow diagram and the current log control flow diagram; and filtering the initial destination node of the log printing according to a preset filtering algorithm to determine the final destination node of the log printing. In some embodiments, the step of determining a log control flow graph based on the log flow comprises: Converting each log item in the log stream into a log template; determining an edge weight between two log templates in each log template pair, wherein the log template pair comprises two log templates corresponding to the same request; And taking each log template as a node, and constructing a transfer edge between the two corresponding log templates according to each edge weight to obtain the log control flow graph. In some embodiments, the determining an initial destination node for log printing in the current log control flow graph according to the normal log control flow graph and the current log control flow graph includes: Matching the normal log control flow graph with the current log control flow graph according to nodes; and determining an initial destination node of the log printing according to the node which is abnormal in matching in the current log control flow diagram. In some embodiments, the matching the normal log control flow graph and the current log control flow graph according to nodes includes: The nodes in the normal log control flow diagram and the nodes in the current log control flow diagram are matched in pairs to determine matching node groups, each matching node group comprises a normal node pair in the normal log control flow diagram and a current node pair in the current log control flow diagram, and the normal node pair in each matching node group is matched with the current node pair; Comparing the edge weights between the normal node pairs in each matched node group with the edge weights between the current node pairs; And when the edge weight between the current node pairs is larger than the edge weight between the normal node pairs, determining the current node pairs in the current matching node group as the nodes with abnormal matching. In some embodiments, the determining the initial destination node of