Search

CN-122019257-A - Log system management method, device, electronic equipment, medium and program product

CN122019257ACN 122019257 ACN122019257 ACN 122019257ACN-122019257-A

Abstract

The disclosure provides a log system management method, a log system management device, an electronic device, a log system management medium and a log system management program product, which can be applied to the technical field of computer data processing. The log system management method comprises the steps of responding to metadata updating events aiming at a log system, determining at least one target component corresponding to the metadata updating events in a plurality of components of the log system, wherein the plurality of components respectively correspond to one of threads, processes and services of the log system, determining at least one target logic storage unit respectively corresponding to the at least one target component in a plurality of logic storage units, wherein the logic storage units are used for storing metadata of the components corresponding to the logic storage units, and writing target metadata into the at least one target logic storage unit in parallel by utilizing the at least one target component.

Inventors

  • SU WEINING
  • CHEN LEI
  • ZHANG XUMING
  • WU RUIQIANG
  • GUO ZHAOBIN

Assignees

  • 曙光信息产业股份有限公司
  • 曙光信息产业(北京)有限公司

Dates

Publication Date
20260512
Application Date
20260127

Claims (10)

  1. 1. A log system management method, the method comprising: in response to a metadata update event for a log system, determining at least one target component corresponding to the metadata update event among a plurality of components of the log system, wherein the plurality of components each correspond to one of a thread, a process, and a service of the log system; Determining at least one target logic storage unit corresponding to each of the at least one target component in a plurality of logic storage units, wherein the logic storage unit is used for storing metadata of the component corresponding to the logic storage unit; Writing target metadata to the at least one target logical storage unit in parallel with the at least one target component.
  2. 2. The method of claim 1, wherein the metadata of the corresponding component includes at least one of resource usage information and data reclamation information of the corresponding component, The resource use information represents storage resources which are pre-allocated to the corresponding component and a log storage area which is occupied in the storage resources at present, and the storage resources of the corresponding component are only used for writing log records of the corresponding component; The data reclamation information represents a recyclable log storage area in the storage resource.
  3. 3. The method of claim 1, wherein the logical storage unit includes a target nonvolatile memory as the first storage medium, the target nonvolatile memory being divided into a plurality of logically independent memory units, each of the memory units for storing metadata of one component; the method further comprises the steps of: and responding to the fault of the log system, and recovering the metadata of the components from the memory units corresponding to the components by the components in parallel.
  4. 4. The method of claim 2, wherein the resource usage information has a data structure in the form of a queue; the data structure includes a start pointer and a stop pointer, and a storage resource between the start pointer and the stop pointer is used for representing an occupied log storage area in the current storage resource.
  5. 5. A method according to claim 3, wherein the components each have a copy of metadata stored on a second storage medium of the logical storage unit; the method further comprises the steps of: for each logical storage unit, in response to the metadata stored by the logical storage unit not being consistent with a corresponding metadata copy, the metadata copy is read from the second storage medium.
  6. 6. A method according to claim 3, characterized in that the method further comprises: And in response to the voltage applied to the target nonvolatile memory being less than a preset value, synchronizing the metadata stored by each memory cell to a third storage medium, wherein the third storage medium comprises a disk.
  7. 7. A log system management apparatus, the apparatus comprising: A first determining module, configured to determine, in response to a metadata update event for a log system, at least one target component corresponding to the metadata update event among a plurality of components of the log system, where the plurality of components each correspond to one of a thread, a process, and a service of the log system; The second determining module is used for determining at least one target logic storage unit corresponding to each of the at least one target component in the plurality of logic storage units, wherein the logic storage unit is used for storing metadata of the component corresponding to the logic storage unit; And a writing module for writing target metadata to the at least one target logical storage unit in parallel with the at least one target component.
  8. 8. An electronic device, comprising: one or more processors; A memory for storing one or more computer programs, Characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1-6.
  9. 9. A computer readable storage medium having stored thereon a computer program or instructions, which when executed by a processor, implement the steps of the method according to any of claims 1 to 6.
  10. 10. A computer program product comprising a computer program or instructions which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 6.

Description

Log system management method, device, electronic equipment, medium and program product Technical Field The present disclosure relates to the field of computer data processing technology, and more particularly, to a log system management method, apparatus, electronic device, medium, and program product. Background In a high performance log system, the root zone records the underlying information of the system. The related root zone management method aiming at the log system has the following problems that the node where the root zone is located fails and can cause the whole system to be inaccessible or service to be interrupted, the root zone needs to process a large number of inquiry requests, and frequent metadata operation can exceed the capability range of a single node. Disclosure of Invention In view of the foregoing, the present disclosure provides a log system management method, apparatus, electronic device, medium, and program product. According to a first aspect of the present disclosure, there is provided a log system management method including determining at least one target component corresponding to a metadata update event among a plurality of components of a log system in response to the metadata update event for the log system, wherein the plurality of components correspond to one of threads, processes, and services of the log system, respectively, determining at least one target logical storage unit corresponding to the at least one target component among a plurality of logical storage units, wherein the logical storage unit is configured to store metadata of the component corresponding to the logical storage unit, and writing the target metadata to the at least one target logical storage unit in parallel with the at least one target component. According to the embodiment of the disclosure, by enabling a plurality of components to respectively correspond to one of threads, processes and services of a log system, at least one target component corresponds to at least one target logic storage unit, the logic storage units are only used for storing metadata of the components corresponding to the logic storage units, and the target metadata are written into the at least one target logic storage unit in parallel by utilizing the at least one target component, even if the logic storage unit corresponding to a certain component fails, only operations such as log access of the component are affected, and other components can still read and write normally through the dedicated logic storage units, so that the whole system is not interrupted. And because the logic storage unit only stores the metadata of the corresponding components, the multi-component parallel writing does not need to compete for shared resources (lock-free design), so that the concurrency conflict can be avoided from the source. According to the embodiment of the disclosure, the metadata of the corresponding component comprises at least one of resource usage information and data recycling information of the corresponding component, wherein the resource usage information represents storage resources which are pre-allocated to the corresponding component and occupied log storage areas in the current storage resources, the storage resources of the corresponding component are only used for writing log records of the corresponding component, and the data recycling information represents the recyclable log storage areas in the storage resources. According to the embodiment of the disclosure, the storage resources pre-allocated to the corresponding components are represented by the resource usage information, the storage resources of the corresponding components are only used for writing the log records of the corresponding components, and the plurality of components can write the log to the exclusive logic storage resources at the same time, so that the performance bottleneck problem caused by a method of writing the related serial or locking into the root zone is solved, and the key for improving the log submitting efficiency is provided. According to an embodiment of the disclosure, the logical storage unit includes a target nonvolatile memory as the first storage medium, the target nonvolatile memory being divided into a plurality of logically independent memory units, each memory unit being for storing metadata of one component, and the log system management method further includes recovering, by the plurality of components, metadata of the component in parallel from memory units corresponding to the plurality of components in response to a failure of the log system. According to the embodiment of the disclosure, by dividing the target nonvolatile memory into a plurality of logically independent memory units, and each memory unit is used for storing metadata of one component, a plurality of system components can be allowed to write data or log records to a root zone in the first storage medium in parallel, and the parallel writing de