CN-121979461-A - Optimization method and system for direct I/O reading performance under Linux system

CN121979461ACN 121979461 ACN121979461 ACN 121979461ACN-121979461-A

Abstract

The invention discloses a method and a system for optimizing direct I/O reading performance under a Linux system, wherein the method comprises the steps of introducing a cache control mark for controlling cache access, judging whether a cache mechanism is started in a file opening stage, if so, firstly detecting whether a cache hits when a direct I/O reading operation is executed, directly reading and returning data from the cache when the cache hits, calling a standard direct I/O to read and return the data when the cache hits, simultaneously carrying out asynchronous cache on the data and setting a cache mark, and judging whether the cache mechanism is started or not when a direct I/O writing operation is executed, if so, firstly detecting whether the cache hits, setting original cache data as invalid when the cache hits, carrying out asynchronous cache on the data, simultaneously calling the standard direct I/O writing data, and setting the cache data as valid after the asynchronous cache and the data writing operation are completed. The invention can improve the reading performance of direct I/O.

Inventors

HU HAI
Chi Zhiling
LIU ZHENGYUAN

Assignees

麒麟软件有限公司

Dates

Publication Date: 20260505
Application Date: 20260331

Claims (10)

1. The optimization method of the direct I/O reading performance in the Linux system is characterized by comprising the following steps: Introducing a CACHE control mark O_CACHE for controlling CACHE access into the kernel; In the file opening stage, judging whether a caching mechanism is started according to the caching control mark, if so, firstly detecting whether a cache hits when the direct I/O reading operation is executed, directly reading and returning data from the cache when the cache hits, calling a standard direct I/O to read and return data from a memory when the cache misses, and simultaneously asynchronously caching the read data and setting an effective caching mark to construct the caching mechanism for accelerating the subsequent reading of the data; And when the direct I/O write operation is executed, judging whether a cache mechanism is started according to the cache control mark, if so, firstly detecting whether the cache is hit, setting the original cache data as invalid when the cache is hit, carrying out asynchronous cache on the data, calling standard direct I/O write data to a memory, and setting the cache data as valid after the asynchronous cache and the data write operation are finished, so that the cache data is kept in the latest state.
2. The method for optimizing direct I/O read performance in a Linux system according to claim 1, wherein the CACHE control flag o_cache is set in a header file including/uapi/asm-generics/fcntl.h of an opening flag of a Linux kernel definition file, and the CACHE control function is enabled by a unique flag bit, and whether the CACHE control flag is enabled is determined by detecting whether the opening flag set iocb- > ki_ filp- > f_ flags of the file corresponding to the current I/O operation contains o_cache.
3. The method for optimizing direct I/O read performance in a Linux system according to claim 1, wherein the direct I/O read optimization specifically includes: When the direct I/O reading operation is executed, firstly judging whether a cache control mark is started or not, and when the cache control mark is not started, executing the original standard direct I/O reading flow of the system; when the buffer control mark is started, calculating a corresponding buffer page index according to the file offset of the current read data, and searching a corresponding buffer page in the page buffer mapping of the file; If the corresponding cache page is not found, a new cache page is allocated for the current read data, and the cache validity state is initialized to be invalid; If the cache page is found, judging whether the cache data is valid according to the validity state recorded in the cache page, directly reading the data from the cache page and returning the data when the cache data is valid, executing the original standard direct I/O reading flow of the system to complete data reading when the cache data is invalid or not hit, and starting a background asynchronous task to write the read data into the cache page after the reading is completed so as to form the cache data.
4. The method for optimizing direct I/O read performance in a Linux system according to claim 3, wherein whether a corresponding cache PAGE exists or not is searched in a cache mapping of a file according to a PAGE index by a find_get_page function, wherein the cache mapping obtains a PAGE cache mapping structure of the file by obtaining a member f_mapping of a file object file, and the cache PAGE index obtains a PAGE number index where the offset is located by dividing a file offset iocb- > ki_pos by a PAGE size page_shift.
5. The method for optimizing direct I/O read performance in a Linux system according to claim 1, wherein the asynchronous cache filling mechanism when asynchronously caching the data specifically comprises: Constructing a task structure body cache_task for packaging an asynchronous cache task, wherein the task structure body comprises target file information file, a file read-write offset position pos, data description information item to be cached, a cache page for storing cache data and a task control unit work for submitting a task to a kernel work queue; Creating and initializing a global work queue wq for processing cache write tasks in a system initialization phase; when the buffer write is triggered, a task structure body is distributed, corresponding file information objects, buffer page objects, file read-write offset position objects and data description information objects are filled, and the task structure body is submitted to a global work queue; After the global work queue receives the task structure body, copying the data in the data description information to an appointed cache page, establishing an association relation between the appointed cache page and the page cache mapping of the target file, and updating the validity state of the cache page to indicate that the cache writing operation is completed.
6. The method for optimizing direct I/O read performance in a Linux system according to claim 5, wherein an association relationship between a specified cache page and a page cache map of a target file is established through an add_to_page_cache (page_f_mapping, pos) function, wherein add_to_page_cache () is a function implemented by a kernel to establish the association between the page and the cache map, and the file_f_mapping represents a page cache address space mapping structure of a file in the Linux kernel.
7. The method for optimizing direct I/O read performance in a Linux system according to claim 1, wherein the direct I/O write cache specifically includes: when executing direct I/O write operation, firstly detecting whether the open mark of the current file contains a cache control mark, if not, executing the original standard direct I/O write flow of the system; If the cache control mark is included, acquiring a corresponding page cache mapping relation according to a file object of the current writing operation, calculating a corresponding data page index according to a current written file offset position, and searching a corresponding cache page in the page cache mapping; The method comprises the steps of searching a cache page, allocating a new cache page for a current writing operation when the cache page is not searched, setting the validity state of the cache page to be invalid when the cache page is searched, then starting a background asynchronous cache writing task, copying data to be written into the cache page, synchronously executing an original standard direct I/O writing flow of a system, updating the state count of the cache page to indicate that the writing flow is completed after the writing flow is completed, and updating the state count again after the asynchronous cache writing task is completed, so that the state count is set to be a state indicating that the data in the cache page is valid, and realizing the collaborative processing of direct I/O writing and cache updating.
8. The method for optimizing direct I/O read performance in a Linux system according to claim 1, wherein during a file opening stage, validity check is further required for the CACHE control flag o_cache, and if it is detected that the CACHE control flag o_cache is enabled but standard direct I/O is not simultaneously enabled, it is determined that parameter configuration is illegal and an error message is returned, and a subsequent step is terminated.
9. The system for optimizing the direct I/O reading performance in the Linux system comprises a microprocessor and a memory which are connected with each other, and is characterized in that the microprocessor is programmed or configured to execute the steps of the method for optimizing the direct I/O reading performance in the Linux system according to any one of claims 1 to 8.
10. A computer readable storage medium having stored therein a computer program/instructions, characterized in that the computer program/instructions are programmed or configured to execute, by a processor, the steps of the method for optimizing direct I/O read performance in a Linux system according to any of claims 1-8.

Description

Optimization method and system for direct I/O reading performance under Linux system Technical Field The invention relates to the technical field of I/O performance optimization, in particular to a method and a system for optimizing direct I/O reading performance in a Linux system. Background With the rapid development of technology, the performance of Central Processing Units (CPUs), memories and Graphics Processing Units (GPUs) has been significantly enhanced. However, the improvement of the performance of the hard disk is relatively delayed compared with other hardware components, and the improvement of the overall performance of the system is gradually becoming a bottleneck. To solve this problem, the operating system adopts a caching technique by temporarily storing data to be written to the disk in the memory. When the subsequent read-write operation occurs, if the needed data exists in the cache, the operation is directly performed on the data in the cache, so that the performance of the storage system is greatly improved. However, the existing caching technology is not suitable for application scenarios, such as database systems, where data consistency is extremely high. In these particular applications, direct input output (direct I/O) techniques are often employed to ensure a high degree of data consistency. In the direct I/O mode, all read and write operations are directly executed from the disk, and although the method is simple to implement, in a read intensive database application where read operations are far more than write operations, such as a database in an artificial intelligence system, the advantages of memory caching cannot be fully utilized by the direct I/O. In the prior art, chinese patent application No. CN201510236213.9 discloses a self-adaptive Direct I/O acceleration method in a file system facing SSD, which shortens an I/O path by self-adaptively utilizing a Direct I/O mode so as to improve the performance of caching I/O, and Chinese patent application No. CN202510052540.2 discloses a dispatching method and a dispatching system aiming at Direct I/O intensive tasks under a NUMA architecture, which realize efficient dispatching and improve task performance by reducing the overhead brought by accessing a disk across NUMA nodes, and both cannot effectively solve the performance problem of Direct read I/O caused by that the Direct I/O cannot efficiently utilize memory caching under a read intensive application scene. Therefore, a new technology is urgently needed to supplement the existing direct I/O technology so as to better adapt to the application scenario of intensive reading, and the read performance of the direct I/O is further improved through the memory cache while ensuring the data consistency. Disclosure of Invention Aiming at the problems in the prior art, the invention provides a method and a system for optimizing the direct I/O reading performance in a Linux system so as to improve the direct I/O reading performance in a dense reading application scene. In order to solve the technical problems, the invention adopts the following technical scheme: a method for optimizing direct I/O reading performance in a Linux system comprises the following steps: Introducing a CACHE control mark O_CACHE for controlling CACHE access into the kernel; In the file opening stage, judging whether a caching mechanism is started according to the caching control mark, if so, firstly detecting whether a cache hits when the direct I/O reading operation is executed, directly reading and returning data from the cache when the cache hits, calling a standard direct I/O to read and return data from a memory when the cache misses, and simultaneously asynchronously caching the read data and setting an effective caching mark to construct the caching mechanism for accelerating the subsequent reading of the data; And when the direct I/O write operation is executed, judging whether a cache mechanism is started according to the cache control mark, if so, firstly detecting whether the cache is hit, setting the original cache data as invalid when the cache is hit, carrying out asynchronous cache on the data, calling standard direct I/O write data to a memory, and setting the cache data as valid after the asynchronous cache and the data write operation are finished, so that the cache data is kept in the latest state. Further, the buffer control mark O_CACHE is arranged in a header file including a Linux kernel definition file opening mark/uapi/asm-generics/fcntl.h, and the starting of the buffer control function is indicated by a unique mark bit, and whether the buffer control mark is started or not is judged by detecting whether an opening mark set iocb- > ki_ filp- > f_ flags of a file corresponding to the current I/O operation contains O_CACHE or not. Further, the direct I/O read optimization specifically includes: When the direct I/O reading operation is executed, firstly judging whether a cache control