US-12625650-B2 - Metadata compaction

US12625650B2US 12625650 B2US12625650 B2US 12625650B2US-12625650-B2

Abstract

Techniques are provided for compacting indirect blocks. For example, an object is represented as a structure comprising data blocks within which data of the object is stored and indirect blocks comprising block numbers of where the data blocks are located in storage. Block numbers within a set of indirect blocks are compacted into a compacted indirect block comprising a base block number, a count of additional block numbers after the base block number in the compacted indirect block, and a pattern of the block numbers in the compacted indirect block. The compacted indirect block is stored into memory for processing access operations to the object. Storing compacted indirect blocks into memory allows for more block numbers to be stored within memory. In this way, the block numbers are read from memory faster than loading the block numbers from disk.

Inventors

Ananthan Subramanian
Kumaran Nagappan
Sriram Venketaraman
Szu-Wen Kuo
Yong Eun Cho

Assignees

NETAPP, INC.

Dates

Publication Date: 20260512
Application Date: 20240826

Claims (20)

1 . A computing device comprising: a memory comprising machine executable code; and a processor coupled to the memory, the processor configured to execute the machine executable code cause the computing device to: represent a volume as a structure comprising data blocks within which data of the volume is stored, wherein the volume is associated with a set of metadata corresponding to at least one of a size of the volume, a modification date of a file within the volume, a creation time of the file, or a state of a replication relationship of the volume; compact the set of metadata into compacted metadata that includes a base block number, a count of additional block numbers after the base block number in the compacted metadata, and a pattern of the base block number and the additional block numbers in the compacted metadata; and store the compacted metadata into memory for managing the volume.
2 . The computing device of claim 1 , wherein the machine executable code causes the computing device to: create a compaction group of a set of block numbers within the compacted metadata.
3 . The computing device of claim 1 , wherein the pattern specifies that a plurality of block numbers within the compacted metadata comprise repeated block numbers.
4 . The computing device of claim 1 , wherein the pattern specifies that offsets of the additional block numbers after the base block number in the compacted metadata comprise repeated block numbers.
5 . The computing device of claim 1 , wherein the machine executable code causes the computing device to: create the compacted metadata to comprise a compression group indicator to indicate whether a block number within the compacted metadata is part of a compression group of data blocks.
6 . The computing device of claim 1 , wherein the machine executable code causes the computing device to: create a first compaction group of a first set of block numbers within the compacted metadata and a second compaction group of a second set of block numbers within the compacted metadata.
7 . The computing device of claim 6 , wherein the first compaction group comprises a first base block number, a first count of block numbers of the first compaction group, and a first pattern of the block numbers of the first compaction group, and wherein the second compaction group comprises a second base block number, a second count of block numbers of the second compaction group, and a second pattern of the block numbers of the second compaction group.
8 . The computing device of claim 6 , wherein a plurality of compaction groups are created within the compacted metadata, and wherein a number of block numbers of a compaction group is limited to a threshold amount.
9 . The computing device of claim 8 , wherein the machine executable code causes the computing device to: store, within a header of the compacted metadata, offsets of the compaction groups within the compacted metadata.
10 . A method comprising: representing a volume as a structure comprising data blocks within which data of the volume is stored, wherein the volume is associated with a set of metadata corresponding to at least one of a size of the volume, a modification date of a file within the volume, a creation time of the file, or a state of a replication relationship of the volume; creating compacted metadata to comprise a compression group indicator to indicate whether a block number within the compacted metadata is part of a compression group of data blocks; compacting the set of metadata into the compacted metadata; and storing the compacted metadata into memory for managing the volume.
11 . The method of claim 10 , comprising: creating the compacted metadata to comprise a base block number, a count of additional block numbers after the base block number in the compacted metadata, and a pattern of the base block number and the additional block numbers in the compacted metadata.
12 . The method of claim 10 , comprising: creating the compacted metadata to comprise a pattern of the block numbers in the compacted metadata, wherein the pattern specifies that the block numbers within the compacted metadata comprise repeated block numbers.
13 . The method of claim 10 , comprising: creating the compacted metadata to comprise a pattern of the block numbers in the compacted metadata, wherein the pattern specifies that offsets of additional block numbers after a base block number in the compacted metadata that comprise repeated block numbers.
14 . The method of claim 10 , comprising: create a compaction group of a set of block numbers within the compacted metadata.
15 . The method of claim 10 , comprising: creating a first compaction group of a first set of block numbers within the compacted metadata and a second compaction group of a second set of block numbers within the compacted metadata.
16 . A non-transitory machine readable medium comprising instructions for performing a method, which when executed by a machine, causes the machine to perform operations comprising: representing a volume as a structure comprising data blocks within which data of the volume is stored, wherein the volume is associated with a set of metadata corresponding to at least one of a size of the volume, a modification date of a file within the volume, a creation time of the file, or a state of a replication relationship of the volume; compacting the set of metadata into compacted metadata; and creating a first compaction group of a first set of block numbers within the compacted metadata and a second compaction group of a second set of block numbers within the compacted metadata; and storing the compacted metadata into memory for managing the volume.
17 . The non-transitory machine readable medium of claim 16 , wherein the operations comprise: creating the compacted metadata to comprise a pattern of block numbers in the compacted metadata, wherein the pattern specifies that the block numbers within the compacted metadata comprise repeated block numbers.
18 . The non-transitory machine readable medium of claim 16 , wherein the operations comprise: creating the compacted metadata to comprise a pattern of block numbers in the compacted metadata, wherein the pattern specifies that offsets of additional block numbers after a base block number in the compacted metadata comprise repeated block numbers.
19 . The non-transitory machine readable medium of claim 16 , wherein the operations comprise: creating the compacted metadata to comprise a compression group indicator to indicate whether a block number within the compacted metadata is part of a compression group of data blocks.
20 . The non-transitory machine readable medium of claim 16 , wherein the first compaction group comprises a first base block number, a first count of block numbers of the first compaction group, and a first pattern of the block numbers of the first compaction group, and wherein the second compaction group comprises a second base block number, a second count of block numbers of the second compaction group, and a second pattern of the block numbers of the second compaction group.

Description

RELATED APPLICATION This application claims priority to and is a continuation of U.S. Pat. No. 17,963,077, titled “METADATA COMPACTION” and filed on Oct. 10, 2022, which claims priority to and is a continuation of U.S. Pat. No. 11,467,774, titled “METADATA COMPACTION” and filed on Oct. 29, 2020, which claims priority to and is a continuation of U.S. Pat. No. 10,852,994, titled “METADATA COMPACTION” and filed on Mar. 27, 2019, which are incorporated herein by reference. BACKGROUND A volume is a storage area within a file system. The file system may allow users to create and store objects within the volume, such as files, directories, or other storage objects. The file system may store data of objects into data blocks of storage devices. That is, the file system divides the storage capacity of storage devices into data blocks having a particular size, such as 4096 bytes or any other size. The physical location of each data block within a storage device is addressable by a block number, such as a physical block number or virtual block number. An object may be represented as a structure by the file system. For example, a file may be represented by a hierarchical tree structure. The hierarchical tree structure comprises leaf nodes that are the data blocks within which user data is stored. The hierarchical tree structure comprises one or more levels of intermediate nodes that are indirect blocks comprising block numbers (pointers) used to address/locate blocks of a lower level of the hierarchical tree structure. For example, the leaf nodes are at a level (0). Intermediate nodes at a level (1) comprise block numbers pointing to data blocks of the leaf nodes. Intermediate nodes at a level (2) comprise block numbers pointing to the intermediate nodes at the level (1). In this way, the hierarchical tree structure may be traversed to locate and access user data at particular physical locations within storage based upon block numbers within intermediate nodes. DESCRIPTION OF THE DRAWINGS FIG. 1 is a component block diagram illustrating an example clustered network in which an embodiment of the invention may be implemented. FIG. 2 is a component block diagram illustrating an example data storage system in which an embodiment of the invention may be implemented. FIG. 3 is a flow chart illustrating an example method for indirect block compaction. FIG. 4A is a component block diagram illustrating an example system for indirect block compaction. FIG. 4B is a component block diagram illustrating an example system for indirect block compaction, where indirect blocks are compacted and packed into a compacted indirect block. FIG. 5 is an example of a computer readable medium in which an embodiment of the invention may be implemented. FIG. 6 is a component block diagram illustrating an example computing environment in which an embodiment of the invention may be implemented. DETAILED DESCRIPTION Some examples of the claimed subject matter are now described with reference to the drawings, where like reference numerals are generally used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide an understanding of the claimed subject matter. It may be evident, however, that the claimed subject matter may be practiced without these specific details. Nothing in this detailed description is admitted as prior art. A computing device, such as a storage controller, a computing node of a cluster, a computer, a storage virtual machine, a storage service, etc., may implement a file system that allows users to store data within a volume. The file system may represent an object, such as a file, of the volume using a structure comprising data blocks within which data of the object are stored and indirect blocks comprising block numbers of where the data blocks or other indirect blocks are located in storage. When data of the object is requested by an access operation (e.g., a read or write operation), the structure is use to identify the location of corresponding data blocks within storage. The file system may store the data blocks and indirect blocks within storage and/or memory of the computing device. The blocks may be stored according to a block size used by the file system, such as 4096 bytes or any other size. Storing blocks within memory, such as by caching user data, allows the computing device to process access operations targeting the blocks faster than if the data is read from storage (e.g., a hard disk device, cloud storage, a solid state storage device, or other storage device may have higher latency than the memory). The size of memory may be limited compared to the size of a storage device, and thus only so much data can be stored within memory. Because the indirect blocks are used to identify the data blocks for processing access operations, it would be advantageous to store as many indirect blocks containing block numbers as poss