US-12625774-B2 - Servicing file restorations in a deduplication filesystem using a read-ahead cache

US12625774B2US 12625774 B2US12625774 B2US 12625774B2US-12625774-B2

Abstract

An access object (AOB) service receives a restore stream from a client for a file managed by a deduplication filesystem. The file is represented by a segment tree including segments in an upper level referencing chunks of the file. The upper level segments are grouped into similarity groups and the similarity groups are assigned to deduplication object (DOB) services. The upper level segments are iterated over to open multiple internal read-ahead streams from the AOB service to the DOB service to populate a read-ahead cache maintained at the AOB service. The restore stream is serviced using the read-ahead cache.

Inventors

Nitin Madan
Kedar Godbole
Aditi Tejas Gosavi
SRIKANT VISWANATHAN

Assignees

DELL PRODUCTS L.P.

Dates

Publication Date: 20260512
Application Date: 20240724

Claims (9)

1 . A method comprising: receiving, at an access object (AOB) service, a restore stream from a client for a file managed by a deduplication filesystem, the file being represented by a segment tree comprising segments in an upper level referencing chunks of the file, the upper level segments being grouped into similarity groups, and the similarity groups being assigned to deduplication object (DOB) services; iterating over the upper level segments to open multiple internal read-ahead streams from the AOB service to the DOB services to populate a read-ahead cache maintained at the AOB service, the iterating comprising: loading an upper level segment of the file; identifying a similarity group into which the upper level segment has been grouped; identifying a DOB service having responsibility for the similarity group; and issuing a prefetch request from the AOB service to the DOB service for the DOB service to retrieve chunks of the file referenced by the upper level segment and populate the read-ahead cache with the retrieved chunks; and servicing the restore stream using the read-ahead cache, wherein the method further comprises first and second iterations occurring in parallel with each other, the first iteration comprises: loading a first upper level segment of the file; determining that the first upper level segment belongs to a first similarity group; determining that the first similarity group has been assigned to a first DOB service; and issuing a first prefetch request from the AOB service to the first DOB service for the first DOB service to retrieve first chunks of the file referenced by the first upper level segment and populate the read-ahead cache with the first chunks; and the second iteration comprises: loading a second upper level segment of the file; determining that the second upper level segment belongs to a second similarity group; determining that the second similarity group has been assigned to a second DOB service; and issuing a second prefetch request from the AOB service to the second DOB service for the second DOB service to retrieve second chunks of the file referenced by the second upper level segment and populate the read-ahead cache with the second chunks, wherein the AOB service and the first DOB service are hosted on a first node, and the second DOB service is hosted on a second node.
2 . The method of claim 1 wherein the read-ahead cache for the file is a single read-ahead cache maintained on a single AOB service.
3 . The method of claim 1 further comprising: while restoration of the file is in progress, closing one or more multiple internal read-ahead streams based on detecting that system load has exceeded a threshold level; and after the closing the one or more multiple internal read-ahead streams, servicing the restore stream using a reduced number of internal read-ahead streams.
4 . A system comprising: a processor; and memory configured to store one or more sequences of instructions which, when executed by the processor, cause the processor to carry out the steps of: receiving, at an access object (AOB) service, a restore stream from a client for a file managed by a deduplication filesystem, the file being represented by a segment tree comprising segments in an upper level referencing chunks of the file, the upper level segments being grouped into similarity groups, and the similarity groups being assigned to deduplication object (DOB) services; iterating over the upper level segments to open multiple internal read-ahead streams from the AOB service to the DOB services to populate a read-ahead cache maintained at the AOB service, the iterating comprising: loading an upper level segment of the file; identifying a similarity group into which the upper level segment has been grouped; identifying a DOB service having responsibility for the similarity group; and issuing a prefetch request from the AOB service to the DOB service for the DOB service to retrieve chunks of the file referenced by the upper level segment and populate the read-ahead cache with the retrieved chunks; and servicing the restore stream using the read-ahead cache, wherein the system further comprises first and second iterations occurring in parallel with each other, the first iteration comprises: loading a first upper level segment of the file; determining that the first upper level segment belongs to a first similarity group; determining that the first similarity group has been assigned to a first DOB service; and issuing a first prefetch request from the AOB service to the first DOB service for the first DOB service to retrieve first chunks of the file referenced by the first upper level segment and populate the read-ahead cache with the first chunks; and the second iteration comprises: loading a second upper level segment of the file; determining that the second upper level segment belongs to a second similarity group; determining that the second similarity group has been assigned to the first DOB service; and issuing a second prefetch request from the AOB service to the first DOB service for the first DOB service to retrieve second chunks of the file referenced by the second upper level segment and populate the read-ahead cache with the second chunks, wherein the AOB service and the first DOB service are hosted on a first node.
5 . The system of claim 4 wherein the read-ahead cache for the file is a single read-ahead cache maintained on a single AOB service.
6 . The system of claim 4 wherein the processor further carries out the step of: while restoration of the file is in progress, closing one or more multiple internal read-ahead streams based on detecting that system load has exceeded a threshold level; and after the closing the one or more multiple internal read-ahead streams, servicing the restore stream using a reduced number of internal read-ahead streams.
7 . A computer program product, comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein, the computer-readable program code adapted to be executed by one or more processors to implement a method comprising: receiving, at an access object (AOB) service, a restore stream from a client for a file managed by a deduplication filesystem, the file being represented by a segment tree comprising segments in an upper level referencing chunks of the file, the upper level segments being grouped into similarity groups, and the similarity groups being assigned to deduplication object (DOB) services; iterating over the upper level segments to open multiple internal read-ahead streams from the AOB service to the DOB services to populate a read-ahead cache maintained at the AOB service, the iterating comprising: loading an upper level segment of the file; identifying a similarity group into which the upper level segment has been grouped; identifying a DOB service having responsibility for the similarity group; and issuing a prefetch request from the AOB service to the DOB service for the DOB service to retrieve chunks of the file referenced by the upper level segment and populate the read-ahead cache with the retrieved chunks; and servicing the restore stream using the read-ahead cache, wherein the method further comprises first and second iterations occurring in parallel with each other, the first iteration comprises: loading a first upper level segment of the file; determining that the first upper level segment belongs to a first similarity group; determining that the first similarity group has been assigned to a first DOB service; and issuing a first prefetch request from the AOB service to the first DOB service for the first DOB service to retrieve first chunks of the file referenced by the first upper level segment and populate the read-ahead cache with the first chunks; and the second iteration comprises: loading a second upper level segment of the file; determining that the second upper level segment belongs to a second similarity group; determining that the second similarity group has been assigned to the first DOB service; and issuing a second prefetch request from the AOB service to the first DOB service for the first DOB service to retrieve second chunks of the file referenced by the second upper level segment and populate the read-ahead cache with the second chunks, wherein the AOB service and the first DOB service are hosted on a first node.
8 . The computer program product of claim 7 wherein the read-ahead cache for the file is a single read-ahead cache maintained on a single AOB service.
9 . The computer program product of claim 7 wherein the method further comprises: while restoration of the file is in progress, closing one or more multiple internal read-ahead streams based on detecting that system load has exceeded a threshold level; and after the closing the one or more multiple internal read-ahead streams, servicing the restore stream using a reduced number of internal read-ahead streams.

Description

TECHNICAL FIELD The present invention relates generally to information processing systems, and more particularly to large scale filesystems. BACKGROUND A distributed filesystem is a type of filesystem that spans multiple servers, but provides a unified view to clients accessing the filesystem. A deduplicated filesystem is a type of filesystem that seeks to reduce the amount of redundant data that is stored by storing only a single copy of data rather than multiple redundant copies. A deduplicated filesystem can be especially effective in a backup system because of the reduction in the number of duplicate copies of data that are stored. A cluster refers to a group of interconnected servers that work together to run an application as a single system. Clustering can improve performance, enhance availability, and provide scalability. Underlying the filesystem is a storage layer where file and other data is stored. Restoration of a file involves read input/output (IOs) to fetch data from the storage layer and serve the data to the requesting application. Depending on factors such as the size of the file to restore, number of file generations and backups, and other factors, it can require a considerable amount of time to restore a file. There is a need for improved systems and techniques to reduce the time required to restore a file and increase efficiency. The subject matter discussed in the background section should not be assumed to be prior art merely as a result of its mention in the background section. Similarly, a problem mentioned in the background section or associated with the subject matter of the background section should not be assumed to have been previously recognized in the prior art. The subject matter in the background section merely represents different approaches, which in and of themselves may also be inventions. BRIEF SUMMARY An access object (AOB) service receives a restore stream from a client for a file managed by a deduplication filesystem. The file is represented by a segment tree including segments in an upper level referencing chunks of the file. The upper level segments are grouped into similarity groups and the similarity groups are assigned to deduplication object (DOB) services. The upper level segments are iterated over to open multiple internal read-ahead streams from the AOB service to the DOB service to populate a read-ahead cache maintained at the AOB service. The restore stream is serviced using the read-ahead cache. BRIEF DESCRIPTION OF THE FIGURES In the following drawings like reference numerals designate like structural elements. Although the figures depict various examples, the one or more embodiments and implementations described herein are not limited to the examples depicted in the figures. FIG. 1 shows a block diagram of an information processing system having a deduplication filesystem, according to one or more embodiments. FIG. 2 shows an example of a deduplication process, according to one or more embodiments. FIG. 3 shows an example of a tree data structure of the namespace, according to one or more embodiments. FIG. 4 shows layer diagram of the filesystem, according to one or more embodiments. FIG. 5 shows a block diagram of a prefetch operation, according to one or more embodiments. FIG. 6 shows a graph of a read highway, according to one or more embodiments. FIG. 7 shows a file having been divided into chunks, according to one or more embodiments. FIG. 8 shows a table mapping chunks of a file to similarity groups, according to one or more embodiments. FIG. 9 shows a block diagram of stream mapping between an access object service and a deduplication object service, according to one or more embodiments. FIG. 10 shows a block diagram of a read-ahead cache at an access object service, according to one or more embodiments. FIG. 11 shows a flow for populating the read-ahead cache, according to one or more embodiments. FIG. 12 shows a block diagram of a multi-stream restore, according to one or more embodiments. FIG. 13 shows a flow for a multi-stream restore, according to one or more embodiments. FIG. 14 shows a flow for a distributed read-ahead cache, according to one or more embodiments. FIG. 15 shows a block diagram for spreading a read-ahead cache across access object services, according to one or more embodiments. FIG. 16 shows a mapping of offset ranges to memory pages holding read-ahead data, according to one or more embodiments. FIG. 17 shows a block diagram of issuing prefetch requests, according to one or more embodiments. FIG. 18 shows a file having been divided into offset ranges and the offset ranges assigned to access object services, according to one or more embodiments. FIG. 19 shows an operation of issuing prefetch requests based on the offset range assignments, according to one or more embodiments. FIG. 20 shows a flow for concurrent read-aheads across access object services, according to one or more embodiments. FIG. 21 shows a block diagram for