CN-122019532-A - Database management method, equipment and storage medium

CN122019532ACN 122019532 ACN122019532 ACN 122019532ACN-122019532-A

Abstract

The application discloses a database management method, equipment and a storage medium, wherein the method comprises the steps of determining the cycle length of a first index based on the data life cycle characteristics corresponding to the first index; determining the number of fragments of the first index in a target period, wherein the target period is a current period or a future period; and according to the number of fragments and the cycle length, creating the first index corresponding to the target cycle in the database. By the method, the storage resource efficiency of the database can be improved.

Inventors

REN ZHIHONG
ZHOU MINGWEI
ZHU LINHAO
YANG FUXING
WANG RULEI
DENG ZHIGUO
YU YI

Assignees

浙江大华技术股份有限公司

Dates

Publication Date: 20260512
Application Date: 20251223

Claims (10)

1. A method of database management, the method comprising: determining the cycle length of a first index based on the data life cycle characteristics corresponding to the first index; Determining the number of fragments of the first index in a target period, wherein the target period is a current period or a future period; and according to the number of fragments and the cycle length, creating the first index corresponding to the target cycle in the database.
2. The method of claim 1, wherein the target period is a future period, and wherein the determining the number of slices of the first index in the target period comprises: Based on the historical period index data amount of the first index, determining the number of fragments of the first index in a future period.
3. The method of claim 2, wherein the historical period index data amount is a maximum index data amount of first index data amounts respectively corresponding to each period of the plurality of periods when the database currently has a first index corresponding to the plurality of periods, and wherein the historical period index data amount is a first index data amount corresponding to a single period when the database currently has only a first index corresponding to the single period.
4. A method according to claim 2 or 3, wherein said determining the number of slices of the first index in a future period based on the historical period index data amount of the first index comprises: determining a reference fragment number based on cluster hardware configuration of the database; Generating a plurality of candidate fragment numbers by utilizing the reference fragment numbers; And selecting the number of fragments of the first index in a future period from the plurality of candidate fragments based on the historical period index data amount of the first index.
5. The method of claim 4, wherein the clustered hardware configuration comprises a clustered physical node and node storage configuration, wherein the determining the reference shard based on the clustered hardware configuration of the database comprises: Determining a product value between the number of the physical nodes of the cluster and the node storage configuration as the reference fragment number; and/or, generating a plurality of candidate fragments by using the reference fragments, including: Taking the product value of the reference fragment number and a plurality of coefficients as a plurality of candidate fragment numbers; and/or, selecting the number of slices of the first index in a future period from the number of candidate slices based on the historical period index data amount of the first index, including: respectively taking the ratio of the historical period index data quantity to each candidate fragment number as a single fragment size corresponding to each candidate fragment number; And selecting the candidate fragment number of which the first corresponding single fragment size is in a preset size range as the fragment number of the first index in a future period according to the sequence from small to large of the plurality of candidate fragments, or selecting the candidate fragment number of which the corresponding single fragment size is closest to an ideal size as the fragment number of the first index in the future period.
6. The method of claim 1, wherein the data lifecycle features include access frequency, time-to-live, data expiration time concentration, and data volume trend of the target data corresponding to the first index; and/or, the determining the period length of the first index based on the data life cycle feature corresponding to the first index includes: Determining, based on a data lifecycle feature corresponding to the first index, that target data corresponding to the first index belongs to a first data type including at least one of a high access frequency data type, a low survival time data type, and a high access frequency and low survival time data type or a second data type including at least one of a low access frequency data type, a high survival time data type, and a low access frequency and high survival time data type; Responding to the target data belonging to the first data type, and selecting the cycle length of the first index from a first cycle length set; and selecting the cycle length of the first index from a second cycle length set in response to the target data belonging to the second data type, wherein each cycle length in the first cycle length set is shorter than each cycle length in the second cycle length set.
7. The method of claim 1, wherein the target period is a future period, wherein the first index corresponding to the future period is created a first number of periods in advance if the period length of the first index belongs to a first set of period lengths, and wherein the first index corresponding to the future period is created a second number of periods in advance if the period length of the first index belongs to a second set of period lengths, wherein each period length in the first set of period lengths is shorter than each period length in the second set of period lengths, and wherein the first number is less than the second number.
8. The method of claim 1, wherein the first index is provided with a time-to-live, and the database further comprises a second index not provided with a time-to-live; the method further comprises the steps of: Re-determining a number of slices for the second index; Creating a new second index for the second index according to the redetermined number of fragments; and after the data of the original second index is migrated to the new second index, deleting the original second index. And/or after the first index corresponding to the target period is created in the database according to the number of fragments and the period length, the method further includes: Determining whether the first index corresponding to the target period is out of date or not based on the survival time and the creation time of the first index corresponding to the target period; In response to expiration of the first index corresponding to the target period, and the first index corresponding to the target period is not currently actively queried, asynchronously cleaning the first index corresponding to the target period; The method comprises the steps of clearing a first index corresponding to a target period, wherein the priority of the thread used for clearing the first index is lower than that of the thread used for writing data of the database, and/or clearing after merging a plurality of out-of-date index execution segments, and/or executing the asynchronous clearing of the first index corresponding to the target period when the cluster load of the database is in a low peak period under the condition that the data volume of the first index corresponding to the target period is larger than a preset data volume.
9. An electronic device comprising a processor, a memory, the processor coupled to the memory, the processor configured to perform one or more steps of the database management method of any of claims 1-8 based on instructions stored in the memory.
10. A computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the steps of the database management method of any one of claims 1 to 8.

Description

Database management method, equipment and storage medium Technical Field The present application relates to the field of database technologies, and in particular, to a database management method, device, and storage medium. Background The data search of existing databases is typically performed in the form of an index. However, the existing database index management method has certain limitations. Taking an elastic search (ES for short) as an example, the ES is constructed based on Apache Lucene (Lucene for short, an open-source high-performance full-text search core library), is specially designed for processing large-scale data, provides near real-time full-text search capability, supports indexing and searching of structured and unstructured data, and is widely applied to the fields of log analysis, search engines, data monitoring, recommendation systems and the like. In these scenes, the data types are rich and various, namely, the data contains logs and monitoring data with strong timeliness, and also have core business data which needs to be preserved for a long time, and the data generally has vivid life cycle characteristics. However, with the explosive growth of various data volumes, the traditional ES index management mode has the problems that the ES is lack of a mechanism for automatically cleaning out expired data, and the like, so that invalid data is accumulated for a long time, index expansion is caused, a large amount of storage resources are wasted, and query response efficiency is remarkably reduced. Disclosure of Invention In order to solve the technical problems, the application provides a database management method, equipment and a storage medium, so as to improve the utilization rate of storage resources. According to an embodiment of the present application, there is provided a database management method including: determining the cycle length of a first index based on the data life cycle characteristics corresponding to the first index; Determining the number of fragments of the first index in a target period, wherein the target period is a current period or a future period; and according to the number of fragments and the cycle length, creating the first index corresponding to the target cycle in the database. In order to solve the technical problems, the technical scheme adopted by the application is that the electronic equipment comprises a memory and a processor, wherein the memory is used for storing a computer program, and the computer program is used for realizing the database management method in the technical scheme when being executed by the processor. In order to solve the technical problems, the technical scheme adopted by the application is to provide a computer readable storage medium which is used for storing a computer program and is used for realizing the database management method in the technical scheme when the computer program is executed by a processor. According to the scheme, the database management method provided by the application is used for determining the cycle length of the first index based on the data life cycle characteristics corresponding to the first index, determining the number of fragments of the first index in the target cycle, and finally creating the first index corresponding to the target cycle in the database according to the number of fragments and the cycle length, so that the index cycle length is attached to the data life cycle characteristics, the invalid data is prevented from occupying storage resources for a long time, and the utilization rate of the storage resources of the database can be improved. Drawings In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. Wherein: FIG. 1 is a schematic diagram of an ES architecture provided by the present application; FIG. 2 is a flowchart of an embodiment of a database management method according to the present application; FIG. 3 is a schematic diagram of an attribute map provided by the present application; FIG. 4 is a flowchart illustrating another embodiment of a database management method according to the present application; FIG. 5 is a schematic diagram of an embodiment of an electronic device according to the present application; Fig. 6 is a schematic structural diagram of an embodiment of a computer readable storage medium according to the present application. Detailed Description The application is described in further detail below with reference to the drawings and examples. It is specifically noted that the following examples are only for illustrating the present application, but do not limit the