CN-121996583-A - APACHE IGNITE-based SLRU page replacement algorithm optimization method

CN121996583ACN 121996583 ACN121996583 ACN 121996583ACN-121996583-A

Abstract

The invention relates to the technical field of distributed cache and memory calculation, and discloses a APACHE IGNITE-based SLRU page replacement algorithm optimization method, which comprises the steps of constructing an improved SLRU data structure comprising a trial section and a protection section and a page heat table based on array indexes; the method comprises the steps of responding to page access requests to update the heat, executing admission judgment based on a heat threshold, promoting to a protection section only when the page heat of the trial section meets the admission condition, monitoring access quantity and time interval, triggering dynamic heat attenuation, calculating attenuation coefficient according to the hit rate of the protection section and updating full heat, monitoring the hit rate of the protection section, calculating expansion and contraction capacity proportion and dynamically adjusting the capacity proportion of the trial section and the protection section. According to the invention, through an independent heat model, a strict admission mechanism and a self-adaptive adjustment strategy, the cache pollution is effectively reduced, the problem of historical heat hysteresis is solved, and the cache hit rate and the resource utilization rate under a high concurrency environment are improved.

Inventors

MA XUANWEI

Assignees

人保信息科技有限公司

Dates

Publication Date: 20260508
Application Date: 20251212

Claims (10)

1. The SLRU page replacement algorithm optimization method based on APACHE IGNITE is characterized by comprising the following steps of: S1, constructing an improved SLRU data structure and a heat degree model, wherein the improved SLRU data structure comprises a trial segment and a protection segment of a logic segment, and the heat degree model comprises a page heat degree table for recording the access frequency of a data page; s2, in response to an access request for a data page, updating a heat value of a target page in the page heat table, and executing page replacement judgment based on the updated heat value; S3, monitoring a heat attenuation triggering condition, and when the condition is met, calculating a dynamic attenuation coefficient and executing full reduction on all page heat values in the page heat table; S4, monitoring a proportion adjustment triggering condition, calculating an adjustment amplitude based on a protection segment hit rate when the condition is met, and dynamically adjusting the capacity proportion of the trial segment and the protection segment; And the page hotlist establishes a mapping relation with the identifier of the data page through the index of the array subscript and is stored independently of the data page object.
2. The method for optimizing the SLRU page replacement algorithm according to claim 1, wherein the step S2 specifically includes: judging the type of the logic segment to which the target page belongs; if the target page is positioned in the trial section and the protection section is full, judging whether the heat value of the target page meets the admission condition or not; If the admission condition is met, executing inter-segment replacement operation, moving the data page at the head of the protection segment out to the trial segment, and moving the target page to the tail of the protection segment; and if the access condition is not met, only moving the target page to the tail part of the trial segment.
3. The method for optimizing the SLRU page replacement algorithm based on APACHE IGNITE of claim 2, wherein the admission condition is that the heat value of the target page is greater than or equal to the heat value of the head page of the protection segment or the heat value of the target page is greater than or equal to the average heat value of the protection segment; the average heat value of the protection segment is calculated by dividing the global protection segment heat statistical variable maintained in real time by the current protection segment page number.
4. The method for optimizing the SLRU page replacement algorithm based on APACHE IGNITE of claim 1, wherein step S2 further comprises: if the target page is positioned in the protection section, acquiring a heat value of a tail page of the protection section, and comparing the heat value of the target page with the heat value of the tail page of the protection section; If the heat value of the target page is greater than or equal to the heat value of the tail page of the protection segment, the target page is moved to the tail of the protection segment; and if the heat value of the target page is smaller than the heat value of the tail page of the protection section, keeping the current position of the target page in the protection section unchanged.
5. The method for optimizing the SLRU page replacement algorithm according to claim 1, wherein the process of monitoring the thermal decay trigger condition in step S3 specifically includes: maintaining a local access quantity counter and a last decay time stamp in real time; judging whether the value of the local access amount counter is larger than or equal to an access amount threshold value or whether the difference value between the current system time and the last decay time stamp is larger than or equal to a time interval threshold value; when any of the above conditions is satisfied, it is determined that the heat decay is triggered.
6. The method for optimizing the SLRU page replacement algorithm based on APACHE IGNITE of claim 1, wherein the process of calculating the dynamic decay factor and performing a full reduction on all page heat values in the page heat table in step S3 specifically includes: Counting the hit rate of the protection segment in the current period; Calculating a proportional coefficient of the reserved heat based on the hit rate of the protection segment and a preset maximum upper limit constant of attenuation, wherein the proportional coefficient and the hit rate of the protection segment are in positive correlation; Traversing the page heat table, multiplying the original heat value of each data page by the proportionality coefficient, and writing the calculation result back to the page heat table as a new heat value after performing integer cutting.
7. The method for optimizing an SLRU page replacement algorithm based on APACHE IGNITE of claim 6, wherein performing a full reduction on all page heat values in the page heat table further comprises: in the process of traversing the page heat table, calculating a new protection segment heat sum by using thread local variable accumulation; and after the traversal is finished, the accumulated thread local variables are used for updating the global protection segment heat statistical variables at one time.
8. The method for optimizing the SLRU page replacement algorithm according to claim 1, wherein the calculating the adjustment range based on the guard band hit rate in step S4 specifically includes: Judging whether the hit rate of a protection segment in the current period is lower than a hit rate down-regulation threshold value and the current protection segment proportion is higher than a protection segment minimum proportion threshold value, if so, judging that the protection segment hit rate is a down-regulation operation, and calculating a target capacity reduction proportion based on the protection segment hit rate; Judging whether the hit rate of the protection segment in the current period is higher than a hit rate up-regulation threshold value and the current protection segment duty ratio is lower than a protection segment maximum proportion threshold value, if so, judging that the protection segment hit rate is up-regulation operation, and calculating a target capacity expansion proportion based on the protection segment hit rate; the target capacity expansion ratio increases with the increase of the hit rate of the protection segment, and the target capacity reduction ratio increases with the decrease of the hit rate of the protection segment.
9. The optimization method of the SLRU page replacement algorithm based on APACHE IGNITE of claim 8, wherein the dynamically adjusting the capacity ratio of the trial segment to the protection segment in step S4 specifically includes: if the operation is judged to be the up operation, calculating the newly increased capacity quota and accumulating the newly increased capacity quota into the current maximum capacity limit of the protection segment; if the operation is judged to be the down-regulating operation, multiplying the total capacity of the cache pages in the memory by the target capacity reduction proportion, and calculating to obtain the number of pages to be reduced; And sequentially taking out the data pages corresponding to the number of the pages to be reduced from the head of the protection segment, moving the data pages to the tail of the trial segment, and resetting the metadata marks of the taken data pages.
10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method according to any of claims 1-9 when executing the computer program.

Description

APACHE IGNITE-based SLRU page replacement algorithm optimization method Technical Field The invention relates to the technical field of distributed cache and memory calculation, in particular to an SLRU page replacement algorithm optimization method based on APACHE IGNITE. Background APACHE IGNITE is used as a distributed database and computing platform with memory as a center, and the performance of the distributed database and computing platform is largely dependent on the management efficiency of memory data. On the premise of limited memory resources, the page replacement algorithm decides which data pages should be kept in the cache and which should be eliminated to free up space. Conventional Least Recently Used (LRU) algorithms, while simple to implement, tend to misinterpret low frequency data as hot spot data and fill up the cache when faced with a full table scan or one-time large data access, resulting in a true high frequency data being squeezed out, a so-called cache pollution problem. To alleviate this problem, a Segmented LRU (SLRU) algorithm is widely used, which logically divides the cache space into a trial segment and a protection segment, and data is accessed for the first time into the trial segment, and can only be promoted to the protection segment after being accessed again, thereby providing a filtering mechanism to a certain extent. However, the existing SLRU algorithm still has significant limitations when faced with complex and variable actual service scenarios. Traditional promotion mechanisms generally rely on simple access hit decisions, and lack quantitative recording and refined management of page access frequency, i.e. "hotness". The mechanism is easy to be interfered by burst access in a short time, and short-term heat and long-term heat cannot be accurately distinguished, so that non-long-term hot spot data can still be mixed in a protection section. Meanwhile, the existing heat maintenance mechanism often lacks an effective time dimension attenuation strategy, so that the historical high heat data still occupies valuable space of a protection section for a long time after access is stopped, and cannot be eliminated in time, and a heat hysteresis phenomenon is generated. In addition, under a high concurrency environment, frequent updating of state information of a data page easily causes lock competition among threads, and the traditional implementation mode often directly binds statistical information on a data object, so that the coupling degree and the cost of a memory structure are increased. On the other hand, existing SLRU implementations typically employ a static configuration on capacity division of the trial and guard segments, e.g., fixed to each account for half or a specific proportion. Such static partitioning cannot accommodate dynamically changing traffic load characteristics. In practical applications, the scale of the hot spot data is not constant, and when the system faces the situation that the hot spot data set suddenly expands, the fixed protection segment space may not be enough to accommodate all hot spots, so that high-value data is frequently replaced, and when the hot spots are sparse, the excessive protection segment quota may cause idle and waste of memory resources. The memory division mode lacking the self-adaptive adjustment capability is difficult to maintain stable cache hit rate in a fluctuating access mode, and restricts the overall throughput performance of the system. Disclosure of Invention Aiming at the defects of the prior art, the invention provides an SLRU page replacement algorithm optimization method based on APACHE IGNITE, which solves the problems of insufficient flexibility, inaccurate hot identification and limited concurrency performance of the existing cache replacement algorithm when facing a complex access mode. In order to achieve the aim, the invention is realized through the following technical scheme that an SLRU page replacement algorithm optimization method based on APACHE IGNITE firstly, an improved SLRU data structure and a heat model are constructed. The data structure is logically divided into a trial segment and a protection segment for distinguishing data pages of different warmth. Meanwhile, a page heat table is established, and the table directly establishes a mapping relation with the identifier of the data page through the index of the array subscript, so that the heat data is stored independently of the data page object, and the direct invasion of the data object and the memory overhead are reduced. In the running process of the system, responding to an access request for a data page, updating the heat value of a target page in the page heat table in real time by the system, and executing page replacement judgment based on the real-time updated heat value. The specific replacement decision logic depends on the type of logical segment to which the target page currently belongs. And when the target page is