CN-122019550-A - Evolution method, system and equipment for database index structure capable of evolution

CN122019550ACN 122019550 ACN122019550 ACN 122019550ACN-122019550-A

Abstract

The application provides an evolution method, an evolution system and evolution equipment for an evolutionary database index structure, and belongs to the technical field of databases. The method comprises the steps of dividing a data table into a plurality of data fragments, determining access characteristics and system resource states of each data fragment, determining access characteristic similarity between adjacent data fragments according to the access characteristics and the system resource states, and executing dynamic splitting or merging on the data fragments according to the access characteristic similarity. And respectively judging whether each data segment meets a preset evolution trigger condition or not based on the access characteristic and a preset index meta model. And under the condition that the preset evolution triggering condition is judged to be met, determining a target index type corresponding to the data fragment, and converting the data fragment from the current index structure to a target index structure corresponding to the target index type through incremental migration. The method realizes the self-adaptive evolution of the database index structure.

Inventors

MIAO JIAN
XING ZHIKANG
YANG GUOQING
ZHANG YUANCHAO
LV XINJIE

Assignees

瀚高基础软件股份有限公司

Dates

Publication Date: 20260512
Application Date: 20260415

Claims (10)

1. A method of evolving an evolutionary database index structure, the method comprising: Dividing a data table into a plurality of data fragments; Determining access characteristics and system resource states of each data segment; Determining access characteristic similarity between adjacent data segments according to the access characteristics and the system resource state, and executing dynamic splitting or merging on the data segments according to the access characteristic similarity; Based on the access characteristics and a preset index meta model, judging whether each data segment meets a preset evolution trigger condition or not respectively; and under the condition that the preset evolution triggering condition is judged to be met, determining a target index type corresponding to the data segment, and converting the data segment from a current index structure to a target index structure corresponding to the target index type through increment migration.
2. The method according to claim 1, wherein the performing dynamic splitting or merging on the data segments according to the access feature similarity specifically comprises: When the access characteristic similarity is continuously smaller than a preset splitting threshold value in N continuous statistical windows, splitting the data segment, wherein N is a natural number larger than 1; And when the access characteristic similarity is continuously larger than a preset merging threshold value in N continuous statistical windows, merging the data fragments, wherein the preset splitting threshold value is smaller than the preset merging threshold value, the splitting and the merging are performed when the system resource utilization rate is smaller than the preset resource threshold value, the minimum data volume of the split data fragments is not smaller than a first preset threshold value, and the maximum data volume of the merged data fragments is not larger than a second preset threshold value.
3. The method according to claim 1, wherein said determining access characteristics and system resource status of each of said data segments comprises: carrying out coarse granularity statistics on all the data fragments in a first preset period, and collecting a first preset index; Screening corresponding hot spot data fragments according to the first preset index, and carrying out fine granularity sampling on the hot spot data fragments in a second preset period to acquire a second preset index, wherein the second preset index is a fine granularity index different from the first preset index; And suspending the fine-grained sampling when the system resource utilization is greater than a preset high-load threshold.
4. The method of claim 1, wherein the index metamodel is constructed by abstracting capability cost information of a plurality of heterogeneous index structures, and the index metamodel at least comprises: the index capability label is used for identifying the applicability of the index structure in the aspects of point inquiry, range scanning, sequential writing optimization or bitmap optimization; An evolution capability label for indicating a target index type that the index structure allows for conversion; And storing the overhead estimation information and the construction time-consuming estimation information.
5. The method according to claim 1, wherein the determining that the preset evolution trigger condition is satisfied specifically comprises: performing linear fitting on the access indexes of the data segments, and calculating a fitting goodness value; When the goodness-of-fit value is larger than a preset trend threshold, determining that the change of the access index is a trend feature, otherwise, determining that the access index temporarily fluctuates and terminating the determination; And after the trend feature is determined, when the heat score of the data segment is larger than a preset heat threshold, the utilization rate of system resources is smaller than a preset safety threshold, and the data segment is not in a preset evolution cooling period, determining that the evolution triggering condition is met.
6. The method of claim 5, wherein the duration of the preset evolution cooling period is dynamically calculated according to the construction cost of the current evolution and the expected performance benefit, and a specific calculation formula is as follows: ; Wherein, the For the duration of the preset evolution cooling period, As a function of the basic cooling time it is, In order to achieve the cost of construction, Setting a preset upper limit and a preset lower limit for the duration of the preset evolution cooling period; the method further comprises the steps of: and when the heat score of the data segment is larger than a preset super-hot point threshold value, forcibly terminating the preset evolution cooling period.
7. The method of claim 5, wherein the determining that the preset evolution trigger condition is satisfied further comprises: Calculating evolution cost according to the construction cost, the storage cost, the expected performance benefit and the historical evolution average net loss of the data segment of the current evolution; The calculation formula of the evolution cost is as follows: ; Wherein, the For the cost of the evolution to be described, In order for the index to be built up in time, The storage space is occupied for the index, For the expected performance benefit of the current evolution, Average net loss for historical evolution of the data segment several times, and , For the cost of the nth historical evolution, As an actual benefit of the nth historical evolution, For the number of historical evolutions, 、、 Preset weight coefficients respectively, and importance weights corresponding to construction time, storage overhead and net benefit respectively; and triggering evolution operation when the evolution cost is smaller than a preset cost threshold value.
8. The method according to claim 1, wherein the converting the data segment from the current index structure to the target index structure corresponding to the target index type by delta migration specifically comprises: Creating a target index structure; Taking an index node as a minimum unit, and migrating the index node from the current index structure increment to the target index structure in batches according to the sequence of determining the node access heat and the continuity of the key value range; in the migration process, double writing is executed on the writing operation corresponding to the index node being migrated, and the current index structure and the target index structure are written at the same time; After all the index nodes are migrated and subjected to consistency verification, switching a current index pointer of the data segment from the current index structure to a target index structure through atomic operation so as to enable the target index structure to be effective; and under the condition that the reference count of the current index structure is detected to be zero, releasing the storage resources occupied by the current index structure.
9. An evolution system of a database index structure capable of evolution, the system comprising: the dividing module is used for dividing the data table into a plurality of data fragments; a first determining module for determining access characteristics and system resource status of each data segment; The determining and executing module is used for determining the similarity of the access characteristics between the adjacent data fragments according to the access characteristics and the system resource state and executing dynamic splitting or merging on the data fragments according to the similarity of the access characteristics; The second determining module is used for respectively judging whether each data segment meets a preset evolution triggering condition or not based on the access characteristic and a preset index meta model; And the evolution module is used for determining a target index type corresponding to the data segment under the condition that the evolution triggering condition is judged to be met, and converting the data segment from the current index structure to the target index structure corresponding to the target index type through increment migration.
10. An evolving device of a database index structure capable of evolution, the device comprising: at least one processor, and A memory communicatively coupled to the at least one processor, wherein, The memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method of evolving a database index structure according to any one of claims 1 to 8.

Description

Evolution method, system and equipment for database index structure capable of evolution Technical Field The present application relates to the field of database technologies, and in particular, to a method, a system, and an apparatus for evolving a database index structure. Background The database index structure plays a central role in query performance, write performance, storage overhead, and adaptation capability. Existing database systems typically provide multiple types of index structures, such as existing B-trees, log Structured Merge-trees (LSM-Tree), bitmap indexes, etc., each having respective performance advantages under specific data distribution and access patterns. However, in an actual business scenario, the data distribution and access modes often change dynamically with time, such as cold and hot data migration, seasonal fluctuations of user behavior, periodic switching of read and write loads, and the like. These index structures are optimized for specific scenarios, and it is difficult for such static designs to adapt to dynamically changing traffic load and data distribution. Moreover, when the access mode is obviously changed, such as sales promotion and sudden access, indexes are repeatedly switched among various structures, and an administrator needs to manually adjust the index structure, so that the labor cost is high, the risk is high and the delay is long. Meanwhile, the improvement schemes such as multi-index coexistence and incremental index construction in the industry bring high maintenance cost, or only optimize the internal structure of a single index, and the problem of index and dynamic load mismatch cannot be solved. Disclosure of Invention The embodiment of the application provides an evolution method, an evolution system and evolution equipment of an evolutionary database index structure, which are used for solving the technical problems that the current database index structure is static, the maintenance cost is high and the index structure cannot be adaptively evolved. In a first aspect, an embodiment of the present application provides a method for evolving a database index structure capable of evolution, where the method includes: Dividing a data table into a plurality of data fragments; Determining access characteristics and system resource states of each data segment; Determining access characteristic similarity between adjacent data segments according to the access characteristics and the system resource state, and executing dynamic splitting or merging on the data segments according to the access characteristic similarity; Based on the access characteristics and a preset index meta model, judging whether each data segment meets a preset evolution trigger condition or not respectively; and under the condition that the preset evolution triggering condition is judged to be met, determining a target index type corresponding to the data segment, and converting the data segment from a current index structure to a target index structure corresponding to the target index type through increment migration. In one implementation manner of the present application, the performing dynamic splitting or merging on the data segment according to the access feature similarity specifically includes: When the access characteristic similarity is continuously smaller than a preset splitting threshold value in N continuous statistical windows, splitting the data segment, wherein N is a natural number larger than 1; And when the access characteristic similarity is continuously larger than a preset merging threshold value in N continuous statistical windows, merging the data fragments, wherein the preset splitting threshold value is smaller than the preset merging threshold value, the splitting and the merging are performed when the system resource utilization rate is smaller than the preset resource threshold value, the minimum data volume of the split data fragments is not smaller than a first preset threshold value, and the maximum data volume of the merged data fragments is not larger than a second preset threshold value. In one implementation manner of the present application, the determining the access characteristic and the system resource status of each data segment specifically includes: carrying out coarse granularity statistics on all the data fragments in a first preset period, and collecting a first preset index; Screening corresponding hot spot data fragments according to the first preset index, and carrying out fine granularity sampling on the hot spot data fragments in a second preset period to acquire a second preset index, wherein the second preset index is a fine granularity index different from the first preset index; And suspending the fine-grained sampling when the system resource utilization is greater than a preset high-load threshold. In one implementation manner of the present application, the index metamodel is constructed by abstracting capability cost infor