CN-121996635-A - Log management method and device

CN121996635ACN 121996635 ACN121996635 ACN 121996635ACN-121996635-A

Abstract

The invention provides a log management method and a log management device, which relate to the technical field of log management, and the method comprises the steps of obtaining a target service log and determining whether the target service log is a repeated log; and extracting the log printing time of the target service log, and updating the time variable corresponding to each service server in the application server cluster based on the log printing time. The log management method and the log management device provided by the invention can be used for repeatedly checking the target service log, storing the target service log when the target service log is not the repeated log, and executing the task of collecting the service log of the application server cluster in the next round based on the log printing time, so that the consistency and the integrity of the obtained service log of the application server cluster are ensured, the repeated collection and the missed collection of the service log are avoided, the repeated storage of the log file is avoided, and the log management resource is saved.

Inventors

SHI HAITAO
REN FUJIA

Assignees

杭州老板电器股份有限公司

Dates

Publication Date: 20260508
Application Date: 20241104

Claims (10)

1. The log management method is characterized by being applied to an application server cluster, wherein the application server cluster comprises a plurality of service servers, and the method comprises the following steps: Obtaining a target service log, wherein the target service log is a service log collected based on time variables corresponding to each service server in an application server cluster; Determining whether the target service log is a duplicate log; If not, storing the target service log; And extracting the log printing time of the target service log, and updating the time variable corresponding to each service server in the application server cluster based on the log printing time.
2. The method of claim 1, wherein the application server cluster is configured with a log collection component, a log deduplication service component, and a log storage system; the step of obtaining the target service log comprises the following steps: and collecting target service logs of each business server through the log collecting component, and sending the target service logs to the log deduplication service component.
3. The method of claim 2, wherein the step of determining whether the target service log is a duplicate log comprises: Determining, by the log deduplication service component, whether the target service log is a duplicate log.
4. A method according to claim 3, wherein the step of determining, by the log deduplication service component, whether the target service log is a duplicate log, comprises: Acquiring a history service log of a preset latest history time period through the log deduplication service component; Judging whether the content similarity of the target service log and the historical service log is higher than a preset threshold value or not; if yes, determining the target service log as a repeated log; and if not, determining the target service log as a non-repeated log.
5. The method of claim 2, wherein the step of storing the target service log comprises: and sending the target service log to the log storage system, and storing the target service log through the log storage system.
6. The method of claim 2, wherein the step of extracting the log print time of the target service log and updating the time variable corresponding to each business server in the application server cluster based on the log print time comprises: Responding to successful storage of the log storage system, and extracting log printing time of the target service log; And returning the log printing time to the log collecting component so that the log collecting component updates time variables corresponding to all business servers in the application server cluster according to the log printing time.
7. The method according to claim 2, wherein the method further comprises: and responding to the time variable update corresponding to each service server in the application server cluster, and continuously collecting the target service log of each service server through the log collecting component.
8. The method of claim 2, wherein the application server cluster is further configured with a visualization management component; The method further comprises the steps of: And displaying the target service log stored in the log storage system through the visual management component.
9. A log management device, characterized by being applied to an application server cluster, wherein the application server cluster comprises a plurality of service servers, and the device comprises: the system comprises an acquisition module, a storage module and a storage module, wherein the acquisition module is used for acquiring a target service log, wherein the target service log is acquired based on time variables corresponding to all business servers in an application server cluster; the duplicate removal module is used for determining whether the target service log is a duplicate log or not; The storage module is used for storing the target service log when the deduplication module determines that the target service log is not a duplicate log; And the updating module is used for extracting the log printing time of the target service log and updating the time variable corresponding to each service server in the application server cluster based on the log printing time.
10. A server comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method according to any of the preceding claims 1-8 when the computer program is executed.

Description

Log management method and device Technical Field The present invention relates to the field of log management technologies, and in particular, to a log management method and apparatus. Background In general, log management is important in operation and maintenance management, which is an important reference for problems of a system or an application. The log management is to analyze, store and backup the service records generated by the platform, so that operation and maintenance personnel can be helped to find some potential safety hazards, countermeasures can be timely taken for the potential safety hazards, and the security of the service is ensured. At present, a commonly adopted log system architecture for log management is ELKF (Elasticsearch Logstash Kibana Filebeat) architecture, and ELKF architecture can be used for centralized storage, processing and visualization of log data, but the main focus is collection, indexing and display of the logs, no built-in mechanism is provided for guaranteeing consistency of collected logs and stored logs, and the problem of data delay exists, so that the integrity of the stored logs cannot be guaranteed. Meanwhile, the ELKF architecture is adopted for log management, the log management system needs to be maintained regularly, the operation conditions of all components are monitored, the query performance is optimized, the data is backed up and restored, and the like, so that an operation and maintenance team needs to invest additional time and energy to manage and maintain the whole log scheme, and meanwhile, the ELKF architecture is high in resource consumption and operation cost, and a large amount of computing resources and storage space are needed to support the retrieval and query of log data. In summary, how to ensure consistency and integrity of the log files stored in the log management process, save log management cost, and improve log management efficiency is a problem to be solved by operation and maintenance teams. Disclosure of Invention Accordingly, an objective of the present invention is to provide a log management method and device for alleviating the above technical problems. In a first aspect, an embodiment of the present invention provides a log management method, which is applied to an application server cluster, where the application server cluster includes a plurality of service servers; the method comprises the steps of obtaining a target service log, wherein the target service log is a log collected based on time variables corresponding to all business servers in an application server cluster, determining whether the target service log is a repeated log, storing the target service log if not, extracting log printing time of the target service log, and updating the time variables corresponding to all the business servers in the application server cluster based on the log printing time. With reference to the first aspect, the embodiment of the present invention provides a first possible implementation manner of the first aspect, wherein the application server cluster is configured with a log acquisition component, a log deduplication service component and a log storage system, and the step of acquiring the target service log includes acquiring the target service log of each business server through the log acquisition component, and sending the target service log to the log deduplication service component. With reference to the first possible implementation manner of the first aspect, the embodiment of the present invention provides a second possible implementation manner of the first aspect, where the step of determining whether the target service log is a duplicate log includes determining, by the log deduplication service component, whether the target service log is a duplicate log. With reference to the second possible implementation manner of the first aspect, the embodiment of the present invention provides a third possible implementation manner of the first aspect, where the step of determining, by the log deduplication service component, whether the target service log is a duplicate log includes obtaining, by the log deduplication service component, a historical service log of a preset recent historical time period, determining whether a content similarity between the target service log and the historical service log is higher than a preset threshold, if yes, determining that the target service log is a duplicate log, and if no, determining that the target service log is a non-duplicate log. With reference to the first possible implementation manner of the first aspect, the embodiment of the present invention provides a fourth possible implementation manner of the first aspect, where the step of storing the target service log includes sending the target service log to the log storage system, and storing the target service log by the log storage system. With reference to the first possible implementation manner of the first aspect, th