CN-122019410-A - Data cache management method, device, storage medium and program product

CN122019410ACN 122019410 ACN122019410 ACN 122019410ACN-122019410-A

Abstract

The application provides a data cache management method, equipment, a storage medium and a program product, wherein the method comprises the steps of determining the cache priority of target data according to the access frequency of the target data in a first preset time interval and the occupied space of the target data; determining a cache level of the target data according to the aging type of the target data, the access frequency of the target data in a first preset time interval and the occupied space of the target data, and adjusting the cache method of the target data according to the cache priority and the cache level of the target data. The cache priority and the cache level of the target data are adjusted in real time according to the change of the access mode of the target data, so that the adaptability of cache management to complex scenes is improved, furthermore, the cache level and the cache priority of the target data are determined from multiple dimensions, the fineness of the cache management is remarkably improved, and the hit rate of the cache is improved.

Inventors

SU DE
Wei Tongbian
WEI JUAN
HUANG QILIANG
WEI TONGMING

Assignees

上汽通用五菱汽车股份有限公司

Dates

Publication Date: 20260512
Application Date: 20251231

Claims (13)

1. The data buffer memory management method is applied to a data buffer memory management system and is characterized by comprising the following steps: Determining the cache priority of target data according to the access frequency of the target data in a first preset time interval and the occupied space of the target data; determining a cache level of the target data according to the aging type of the target data, the access frequency of the target data in a first preset time interval and the occupied space of the target data, wherein the cache level comprises a cache and a low cache; And adjusting the caching method of the target data according to the caching priority and the caching level of the target data.
2. The method of claim 1, wherein determining the cache priority of the target data according to the access frequency of the target data in the first preset time interval and the occupied space of the target data comprises: determining a first cache priority of the target data according to the access frequency of the target data in a first preset time interval; Determining a second cache priority of target data according to a weighted value of the access frequency proportion of the target data in a first preset time interval and the occupied space proportion of the target data; Determining the buffer priority of the target data according to the weighted value of the first buffer priority of the target data and the second buffer priority of the target data; The ratio of the access frequency of the target data in the first preset time interval is the ratio of the access frequency of the target data in the first preset time interval to the access frequency of all data in the first preset time interval, and the ratio of the occupied space of the target data is the ratio of the occupied space of the target data to the available space of the data cache management system.
3. The method of claim 1, wherein determining the cache hierarchy of the target data according to the age type of the target data, the access frequency of the target data in the first preset time interval, and the occupied space of the target data comprises: If the aging type of the target data is real-time data, determining a cache level of the target data as a cache; If the aging type of the target data is non-real-time data, determining a cache level of the target data according to the access frequency of the target data in a first preset time interval and the occupied space of the target data.
4. The method of claim 3, wherein determining the cache hierarchy of the target data according to the access frequency of the target data in the first preset time interval and the occupied space of the target data comprises: Determining a first cache level of the target data according to the access frequency of the target data in a first preset time interval; determining a second cache level of the target data according to the occupied space of the target data; And determining the cache level of the target data according to the weighted values of the first cache level and the second cache level of the target data.
5. The method as recited in claim 1, further comprising: Determining a load index of the data cache management system according to the CPU utilization rate, the memory utilization rate, the network bandwidth and the weighted value of the disk I/O of the data cache management system; and determining a cache capacity adjustment method of the data cache management system according to a comparison result of the load index of the data cache management system and a load index threshold.
6. The method according to claim 5, wherein the determining the buffer capacity adjustment method of the data buffer management system according to the comparison result of the load index of the data buffer management system and the load index threshold value comprises: If the load index of the data cache management system is greater than or equal to a first load index threshold, expanding the cache capacity of the data cache management system; and if the load index of the data cache management system is smaller than or equal to a second load index threshold value, reducing the cache capacity of the data cache management system.
7. The method of claim 6, wherein the step of providing the first layer comprises, The expanding the cache capacity of the data cache management system comprises increasing the number of cache nodes and expanding the cache capacity of the cache nodes; the reducing the cache capacity of the data cache management system comprises reducing the number of cache nodes and reducing the cache capacity of the cache nodes.
8. The method as recited in claim 1, further comprising: Comparing the access frequency of the target data in a first preset time interval with the historical access frequency of the target data; if the ratio of the access frequency of the target data in the first preset time interval to the historical access frequency of the target data is larger than a preset ratio, determining that a cache level of the target data is a cache; and if the ratio of the access frequency of the target data in the first preset time interval to the historical access frequency of the target data is smaller than the preset ratio, determining the cache level of the target data as a low cache.
9. The method as recited in claim 1, further comprising: inputting the historical access record of the data cache management system into a prediction model, and outputting the access frequency of the target data in a second preset time interval and the load index of the data cache management system in the second preset time interval; If the access frequency of the target data in the second preset time interval is greater than the access frequency threshold, determining a cache prefetching strategy of the data cache management system for the target data in the second preset time interval according to a comparison result of the load index of the data cache management system in the second preset time interval and a prefetching load index threshold; the historical access record of the data cache management system comprises access time of the target data, access frequency of the target data in a third preset time interval, occupied space of the target data and load index of the data cache management system in the third preset time interval.
10. The method of claim 9, wherein determining, according to a comparison result of the load index of the data cache management system in the second preset time interval and the prefetch load index threshold, a cache prefetch policy of the data cache management system for the target data in the second preset time interval includes: If the load index of the data cache management system in the second preset time interval is smaller than or equal to a first prefetching load index threshold, cache prefetching is carried out on the target data according to a first preset cache prefetching range; if the load index of the data cache management system in the second preset time interval is greater than or equal to a second prefetching load index threshold, cache prefetching is carried out on the target data according to a second preset cache prefetching range; If the load index of the data cache management system in the second preset time interval is larger than the first prefetching load index threshold and smaller than the second prefetching load index threshold, cache prefetching is carried out on the target data according to a third preset cache prefetching range; The third preset cache prefetching range is larger than the second preset cache prefetching range, and the third preset cache prefetching range is smaller than the first preset cache prefetching range.
11. An electronic device, comprising: A processor; a memory; And a computer program, wherein the computer program is stored in the memory, which when executed by the processor, causes the electronic device to perform the method of any one of claims 1 to 10.
12. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program which, when executed by a processor, implements the method of any one of claims 1 to 10.
13. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1 to 10.

Description

Data cache management method, device, storage medium and program product Technical Field The present application relates to the field of data processing technologies, and in particular, to a method, an apparatus, a storage medium, and a program product for managing a cache of data. Background With the rapid development of information technology, the global data volume is showing explosive growth, and the current society has stepped into a big data age. In the context of mass data, how to perform data query efficiently and with low delay has become a key bottleneck affecting the application performance of various big data. In order to relieve the read-write pressure of the bottom storage system and improve the access speed of data, the caching technology is widely applied to a big data processing architecture. The key point is that target data is stored in storage media with different speeds based on the access mode (such as access frequency) of the data, so that the average inquiry delay of the data is reduced. In the related art, a fixed policy is generally adopted to configure a cache level and a cache priority of target data, namely, the data is configured to be cached based on a preset rule, which has simple logic and low deployment cost, but cannot be adjusted according to real-time data change, so that the method has poor adaptability to complex scenes. Further, the definition of the cache management policy usually only considers the dimension of the access frequency of the data, and the fineness of the cache management is low, so that the optimal cache management policy cannot be made, and the hit rate of the cache is low, so that the user experience is poor. It should be noted that the information disclosed in the background section of the present application is only for enhancement of understanding of the general background of the present application and should not be taken as an admission or any form of suggestion that this information forms the prior art that is well known to a person skilled in the art. Disclosure of Invention The application provides a data cache management method, equipment, a storage medium and a program product, which are beneficial to solving the problems that the cache management in complex scenes is poor in adaptability due to the adoption of fixed cache configuration and the cache hit rate is low due to the fact that only a single dimension is considered in the formulation of a cache management strategy in the related art. In a first aspect, an embodiment of the present application provides a data cache management method, which is applied to a data cache management system, including: Determining the cache priority of target data according to the access frequency of the target data in a first preset time interval and the occupied space of the target data; determining a cache level of the target data according to the aging type of the target data, the access frequency of the target data in a first preset time interval and the occupied space of the target data, wherein the cache level comprises a cache and a low cache; And adjusting the caching method of the target data according to the caching priority and the caching level of the target data. In a possible implementation manner, the determining the buffer priority of the target data according to the access frequency of the target data in the first preset time interval and the occupied space of the target data includes: determining a first cache priority of the target data according to the access frequency of the target data in a first preset time interval; Determining a second cache priority of target data according to a weighted value of the access frequency proportion of the target data in a first preset time interval and the occupied space proportion of the target data; Determining the buffer priority of the target data according to the weighted value of the first buffer priority of the target data and the second buffer priority of the target data; The ratio of the access frequency of the target data in the first preset time interval is the ratio of the access frequency of the target data in the first preset time interval to the access frequency of all data in the first preset time interval, and the ratio of the occupied space of the target data is the ratio of the occupied space of the target data to the available space of the data cache management system. In one possible implementation manner, the determining the cache hierarchy of the target data according to the aging type of the target data, the access frequency of the target data in the first preset time interval, and the occupied space of the target data includes: If the aging type of the target data is real-time data, determining a cache level of the target data as a cache; If the aging type of the target data is non-real-time data, determining a cache level of the target data according to the access frequency of the target data in a first preset time interval