CN-116414783-B - Log detection method and device, electronic equipment and storage medium

CN116414783BCN 116414783 BCN116414783 BCN 116414783BCN-116414783-B

Abstract

The application provides a log detection method, a device, electronic equipment and a computer storage medium, wherein the method comprises the steps of obtaining a training complete absolute coding model and a relative coding model based on an obtained original log set, wherein the training complete absolute coding model is used for representing specific meanings and time sequence change rules of all logs in the original log set, the training complete absolute coding model is used for representing time sequence change rules of all logs in the original log set, obtaining a log set to be detected, respectively inputting the log set to be detected into the training complete absolute coding model and the relative coding model to obtain respective output results, and determining whether the abnormal log is abnormal or not by utilizing corresponding contents of the abnormal log in the output results of the training complete relative coding model when the abnormal log exists in the log set to be detected according to the output results of the training complete absolute coding model.

Inventors

GUO QIANYING
HUA XIAOLEI
YUAN YE
ZHU LIN
FENG JUNLAN

Assignees

中国移动通信有限公司研究院
中国移动通信集团有限公司

Dates

Publication Date: 20260512
Application Date: 20220105

Claims (9)

1. A log detection method, the method comprising: The method comprises the steps of obtaining a training-completed absolute coding model and a relative coding model based on an obtained original log set, wherein the training-completed absolute coding model is used for representing specific meanings and time sequence change rules of all logs in the original log set; The method comprises the steps of obtaining a log set to be detected, and respectively inputting the log set to be detected into an absolute coding model and a relative coding model which are completed by training to obtain respective output results, wherein the output results represent the classification probability that a log to be predicted is each log in the log set to be detected; When determining that the to-be-detected log set has abnormal logs according to the output result of the absolute coding model after training, determining whether the abnormal logs are abnormal again by utilizing the corresponding content of the abnormal logs in the output result of the relative coding model after training.
2. The method according to claim 1, wherein the inputting the log set to be detected into the trained absolute coding model and the trained relative coding model, respectively, comprises: Determining a log template of each log in the log set to be detected; Mapping a log template of each log in the log set to be detected into an index; The index corresponding to each log in the log set to be detected is determined according to a sliding window mode, and the index in each sliding window is input into the absolute coding model after training is completed; and when the index in each sliding window is determined, encoding the index in each sliding window according to the index size, and inputting the encoded index in each sliding window into the trained relative encoding model.
3. The method of claim 1, wherein the obtaining a trained absolute coding model and a relative coding model based on the obtained raw log set comprises: determining a log template of each log in the original log set; mapping a log template of each log in the original log set into an index; The method comprises the steps of collecting original logs, determining indexes corresponding to each log in an original log set according to a sliding window mode, determining indexes in each sliding window and indexes of corresponding next actual logs, inputting the indexes in each sliding window and the indexes of corresponding next actual logs into an absolute coding model to train to obtain a trained absolute coding model, coding the indexes in each sliding window and the indexes of corresponding next actual logs according to the index size when the indexes in each sliding window are determined, and inputting the indexes in each sliding window and the indexes of corresponding next actual logs after coding into a relative coding model to train to obtain a trained relative coding model.
4. The method according to claim 1, wherein the determining that the log set to be detected has an abnormal log according to the output result of the trained absolute coding model includes: And determining that the log to be predicted is an abnormal log in the log with the k-high log before the classification probability according to the output result of the absolute coding model after the training is completed.
5. The method of claim 1, wherein each log in the original set of logs is a normal log.
6. A log detection device, the device comprising: The system comprises a first obtaining module, a training completion absolute coding model, a training completion relative coding model, a timing sequence change rule, a first obtaining module and a second obtaining module, wherein the first obtaining module is used for obtaining a training completion absolute coding model and a relative coding model based on an obtained original log set; The second obtaining module is used for obtaining a log set to be detected, and respectively inputting the log set to be detected into the absolute coding model and the relative coding model which are completed by training to obtain respective output results, wherein the output results represent that the log to be predicted is the classification probability of each log in the log set to be detected; And the determining module is used for determining whether the abnormal log is abnormal or not by utilizing the corresponding content of the abnormal log in the output result of the trained relative coding model when the abnormal log exists in the log set to be detected according to the output result of the trained absolute coding model.
7. The apparatus of claim 6, wherein the second obtaining module configured to input the log set to be detected into the trained absolute coding model and the trained relative coding model, respectively, comprises: Determining a log template of each log in the log set to be detected; Mapping a log template of each log in the log set to be detected into an index; The index corresponding to each log in the log set to be detected is determined according to a sliding window mode, and the index in each sliding window is input into the absolute coding model after training is completed; and when the index in each sliding window is determined, encoding the index in each sliding window according to the index size, and inputting the encoded index in each sliding window into the trained relative encoding model.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of any one of claims 1 to 5 when the program is executed.
9. A computer storage medium having stored thereon a computer program, which when executed by a processor implements the method of any of claims 1 to 5.

Description

Log detection method and device, electronic equipment and storage medium Technical Field The present application relates to the field of log analysis technologies, and in particular, to a log detection method, a device, an electronic apparatus, and a computer storage medium. Background At present, the system log is used for recording the system execution workflow to help the operation and maintenance personnel to find and locate the problem, and under the normal condition, because the workflow without abnormality usually follows a certain rule, the log data corresponding to the workflow is recorded, the log data under the normal working mode is learned through a model, and when the log data violates the normal working mode, an abnormality is considered to occur. However, in the actual log data, new logs may occur in addition to the original log event, and if the new logs are not processed, the new logs may be all identified as abnormal logs by the model, so as to cause error alarm, and increase time and working cost for the operation and maintenance personnel to check the abnormality. Disclosure of Invention The application provides a log detection method, a log detection device, electronic equipment and a computer storage medium, which can solve the problem of increased operation and maintenance cost caused by the fact that a new log appears in the log detection process and is identified as an abnormal log in the related technology. The technical scheme of the application is realized as follows: the application provides a log detection method, which comprises the following steps: The method comprises the steps of obtaining a training-completed absolute coding model and a relative coding model based on an obtained original log set, wherein the training-completed absolute coding model is used for representing specific meanings and time sequence change rules of all logs in the original log set; The method comprises the steps of obtaining a log set to be detected, and respectively inputting the log set to be detected into an absolute coding model and a relative coding model which are completed by training to obtain respective output results, wherein the output results represent the classification probability that a log to be predicted is each log in the log set to be detected; When determining that the to-be-detected log set has abnormal logs according to the output result of the absolute coding model after training, determining whether the abnormal logs are abnormal again by utilizing the corresponding content of the abnormal logs in the output result of the relative coding model after training. In some embodiments, the inputting the log set to be detected into the trained absolute coding model and the trained relative coding model respectively includes: Determining a log template of each log in the log set to be detected; Mapping a log template of each log in the log set to be detected into an index; The index corresponding to each log in the log set to be detected is determined according to a sliding window mode, and the index in each sliding window is input into the absolute coding model after training is completed; and when the index in each sliding window is determined, encoding the index in each sliding window according to the index size, and inputting the encoded index in each sliding window into the trained relative encoding model. In some embodiments, the obtaining the trained absolute coding model and the relative coding model based on the obtained original log set includes: determining a log template of each log in the original log set; mapping a log template of each log in the original log set into an index; The method comprises the steps of collecting original logs, determining indexes corresponding to each log in an original log set according to a sliding window mode, determining indexes in each sliding window and indexes of corresponding next actual logs, inputting the indexes in each sliding window and the indexes of corresponding next actual logs into an absolute coding model to train to obtain a trained absolute coding model, coding the indexes in each sliding window and the indexes of corresponding next actual logs according to the index size when the indexes in each sliding window are determined, and inputting the indexes in each sliding window and the indexes of corresponding next actual logs after coding into a relative coding model to train to obtain a trained relative coding model. In some embodiments, the determining that the log set to be detected has an abnormal log according to the output result of the absolute coding model after the training is completed includes: And determining that the log to be predicted is an abnormal log in the log with the k-high log before the classification probability according to the output result of the absolute coding model after the training is completed. In some embodiments, each log in the original set of logs is a normal log. The application provi