US-12619360-B2 - Techniques for adaptive coalescing of logging operations

US12619360B2US 12619360 B2US12619360 B2US 12619360B2US-12619360-B2

Abstract

Techniques can include: receiving host write I/Os; and performing processing that persistently records the host write I/Os on a log device including a metadata log for storing metadata portions for the host write I/Os. The processing can include: aggregating, in a write aggregation queue, log write requests that write metadata portions for the host write I/Os in a segment of metadata log entries of the metadata log of the log device; and responsive to determining that i) the log write requests corresponding to the segment write an amount of metadata equal to a maximum size, or ii) a maximum amount of time has elapsed during said aggregating, sending an aggregated write request to the metadata log of the log device, wherein the aggregated write request writes the metadata portions to the segment of metadata log entries. The maximum size can denote an integrated unit size of the log device.

Inventors

Michael P. Wahl
Vamsi K. Vankamamidi
James Vega McCoy

Assignees

DELL PRODUCTS L.P.

Dates

Publication Date: 20260505
Application Date: 20241017

Claims (20)

1 . A computer-implemented method comprising: receiving first host write input/output (I/O) requests; and performing first processing that persistently records the first host write I/O requests on a log device, wherein the log device includes a metadata log for storing metadata portions for the first host write I/O requests, wherein the first processing includes: aggregating, in a first write aggregation queue, first log write requests that write first metadata portions for the first host write I/O requests in a first segment of first metadata log entries of the metadata log of the log device; and responsive to determining that i) the first log write requests corresponding to the first segment write a first amount of metadata equal to a maximum size, or ii) a maximum amount of time has elapsed during said aggregating, sending a first aggregated write request to the metadata log of the log device, wherein the first aggregated write request writes the first metadata portions to the first segment of first metadata log entries; wherein said first processing is performed during a time period N when aggregation of log of log write requests that write metadata portions for corresponding host write I/O requests to the metadata log of the log device is enabled thereby resulting in said aggregating and said sending the first aggregated write request.
2 . The computer-implemented method of claim 1 , wherein the maximum size denotes a multiple of an indirection unit (IU) size of the log device, where the IU size denotes the internal block size of the log device.
3 . The computer-implemented method of claim 2 , wherein each write to the log device, that writes an amount of content having a corresponding size that is less than the IU size, results in the log device internally performing a read modify write (RMW) operation to write the content.
4 . The computer-implemented method of claim 3 , wherein each of the first log write requests writes one of the first metadata portions, and wherein each of the first metadata portions has a corresponding size that is less than the IU size.
5 . The computer-implemented method of claim 4 , wherein the first log writes corresponding to the first segment collectively write the first amount of metadata equal to the IU size of the log device.
6 . The computer-implemented method of claim 4 , wherein each of the first metadata portions is associated with a corresponding one of the first host write I/O requests that writes first content to a first target location, and wherein said each metadata portion includes first information describing the first target location.
7 . The computer-implemented method of claim 1 , wherein a first host write I/O operation is included in the first host write I/O requests, wherein the first host write I/O operation is from a first host and writes first content to a first target location, and wherein the processing includes: persistently recording the first content on the log device; and wherein the method includes: responsive to persistently recording the first content and completing the first aggregated write request that writes the first metadata portions to the first segment of first metadata log entries, returning an acknowledgement to the first host regarding completion of the first host write I/O operation.
8 . The computer-implemented method of claim 1 , further comprising: determining, for the time period N, a host write I/O rate and a bandwidth regarding data written to the log device; and responsive to determining, for the time period N, that i) the host write I/O rate exceeds a first threshold or ii) the bandwidth exceeds a second threshold, enabling, for a next time period N+1, aggregation of log write requests that write metadata portions for corresponding host write I/O requests to the metadata log of the log device.
9 . The computer-implemented method of claim 8 , further comprising: responsive to determining, for the time period N, that i) the host write I/O rate does not exceed the first threshold and ii) the bandwidth does not exceed the second threshold, disabling, for the next time period N+1, aggregation of log write requests that write metadata portions for corresponding host write I/O requests to the metadata log of the log device.
10 . The computer-implemented method of claim 1 , wherein the first processing includes: aggregating, in a first allocation aggregation queue, first log allocation requests each requesting allocation of a metadata log entry from the metadata log of the log device for storing one of the first metadata portions for a corresponding one of the first host write I/O requests; and responsive to determining that i) the first log allocation requests write the first amount of metadata equal to the maximum size, or ii) the maximum amount of time has elapsed during said aggregating of the first log allocation requests, sending a first aggregated allocation request to the metadata log of the log device, wherein the first aggregated log request requests allocation of the first segment of the first metadata log entries from the metadata log stored on the log device.
11 . The computer-implemented method of claim 10 , wherein the first metadata log entries of the first segment are contiguous consecutive metadata log entries of the metadata log.
12 . The computer-implemented method of claim 11 , wherein the method is performed in a system with a plurality of processing nodes, and wherein each of the plurality of processing nodes is associated with i) a corresponding one of a plurality of write aggregation queues used to aggregate log write requests for metadata portions written to segments by said each processing node, and ii) a corresponding one of a plurality of allocation aggregation queues used to aggregate log allocation requests for metadata log entries requested by said each processing node.
13 . The computer-implemented method of claim 12 , wherein the plurality of write aggregation queues includes the first write aggregation queue, and wherein the plurality of allocation aggregation queues includes the first allocation aggregation queue.
14 . The computer-implemented method of claim 13 , wherein the plurality of processing nodes includes a first processing node, and wherein the first write aggregation queue is associated with the first processing node whereby the first write aggregation queue aggregates log write requests for data portions written to segments by the first processing node, and wherein the first allocation aggregation queue is associated with the first processing node whereby the first allocation aggregation queue aggregates log allocation requests for metadata log entries requested by the first processing node.
15 . The computer-implemented method of claim 13 , wherein the plurality of processing nodes includes a first processing node and a second processing node, and wherein the first write aggregation queue is associated with the first processing node whereby the first write aggregation queue aggregates log write requests for data portions written to segments by the first processing node, and wherein the first allocation aggregation queue is associated with the second processing node whereby the first allocation aggregation queue aggregates log allocation requests for metadata log entries requested by the second processing node.
16 . The computer-implemented method of claim 13 , further comprising: receiving, by an arbiter, a plurality of aggregated allocation requests requesting allocation of segments of metadata log entries from the metadata log stored on the log device, wherein the plurality of aggregated allocation requests includes the first aggregated allocation request requesting allocation of the first segment of metadata log entries, wherein the first segment has a first size, and wherein the plurality of aggregated allocation requests includes a second aggregated allocation request requesting allocation of a second segment of metadata log entries, wherein the second segment has a second size, wherein the first size is larger than the second size thereby denoting that the first segment has a greater number of metadata log entries than the second segment.
17 . The computer-implemented method of claim 16 , wherein the first aggregated allocation request and the second aggregated allocation request are from the first processing node, and wherein the first aggregated allocation request has a corresponding first time stamp denoting an earliest time at which a log allocation request, that is aggregated into the first aggregated allocation request, was enqueued in the first allocation aggregation queue, and wherein the second aggregated allocation request has a corresponding second time stamp denoting an earliest time at which a log allocation request, that is aggregated into the second aggregated allocation request, was enqueued in the first allocation aggregation queue, wherein the first time stamp denotes an earlier point in time prior to the second time stamp, and wherein the method includes: the arbiter determining, based on the first time stamp and the second time stamp, to service the first aggregated allocation request prior to the second aggregated allocation request thereby resulting in allocation of the first segment prior to the second segment.
18 . A system comprising: one or more processors; and one or more memories comprising code stored thereon that, when executed, performs a method comprising: receiving first host write input/output (I/O) requests; and performing first processing that persistently records the first host write I/O requests on a log device, wherein the log device includes a metadata log for storing metadata portions for the first host write I/O requests, wherein the first processing includes: aggregating, in a first write aggregation queue, first log write requests that write first metadata portions for the first host write I/O requests in a first segment of first metadata log entries of the metadata log of the log device; and responsive to determining that i) the first log write requests corresponding to the first segment write a first amount of metadata equal to a maximum size, or ii) a maximum amount of time has elapsed during said aggregating, sending a first aggregated write request to the metadata log of the log device, wherein the first aggregated write request writes the first metadata portions to the first segment of first metadata log entries; wherein said first processing is performed during a time period N when aggregation of log of log write requests that write metadata portions for corresponding host write I/O requests to the metadata log of the log device is enabled thereby resulting in said aggregating and said sending the first aggregated write request.
19 . One or more non-transitory computer-readable media comprising code stored thereon that, when executed, performs a method comprising: receiving first host write input/output (I/O) requests; and performing first processing that persistently records the first host write I/O requests on a log device, wherein the log device includes a metadata log for storing metadata portions for the first host write I/O requests, wherein the first processing includes: aggregating, in a first write aggregation queue, first log write requests that write first metadata portions for the first host write I/O requests in a first segment of first metadata log entries of the metadata log of the log device; and responsive to determining that i) the first log write requests corresponding to the first segment write a first amount of metadata equal to a maximum size, or ii) a maximum amount of time has elapsed during said aggregating, sending a first aggregated write request to the metadata log of the log device, wherein the first aggregated write request writes the first metadata portions to the first segment of first metadata log entries; wherein the first processing includes: aggregating, in a first allocation aggregation queue, first log allocation requests each requesting allocation of a metadata log entry from the metadata log of the log device for storing one of the first metadata portions for a corresponding one of the first host write I/O requests; and responsive to determining that i) the first log allocation requests write the first amount of metadata equal to the maximum size, or ii) the maximum amount of time has elapsed during said aggregating of the first log allocation requests, sending a first aggregated allocation request to the metadata log of the log device, wherein the first aggregated log request requests allocation of the first segment of the first metadata log entries from the metadata log stored on the log device.
20 . The computer implemented method of claim 19 , wherein the log device used in the first processing is a solid state storage device (SSD), and wherein the SSD has an indirection unit (IU) size defining an internal block size of the log device.

Description

BACKGROUND Systems include different resources used by one or more host processors. The resources and the host processors in the system are interconnected by one or more communication connections, such as network connections. These resources include data storage devices such as those included in data storage systems. The data storage systems are typically coupled to one or more host processors and provide storage services to each host processor. Multiple data storage systems from one or more different vendors can be connected to provide common data storage for the one or more host processors. A host performs a variety of data processing tasks and operations using the data storage system. For example, a host issues I/O operations, such as data read and write operations, that are subsequently received at a data storage system. The host systems store and retrieve data by issuing the I/O operations to the data storage system containing a plurality of host interface units, disk drives (or more generally storage devices), and disk interface units. The host systems access the storage devices through a plurality of channels provided therewith. The host systems provide data and access control information through the channels to a storage device of the data storage system. Data stored on the storage device is provided from the data storage system to the host systems also through the channels. The host systems do not address the storage devices of the data storage system directly, but rather, access what appears to the host systems as a plurality of files, objects, logical units, logical devices or logical volumes. Thus, the I/O operations issued by the host are directed to a particular storage entity, such as a file or logical device. The logical devices generally include physical storage provisioned from portions of one or more physical drives. Allowing multiple host systems to access the single data storage system allows the host systems to share data stored therein. SUMMARY Various embodiments of the techniques herein can include a computer-implemented method, a system and a non-transitory computer readable medium. The system can include one or more processors, and a memory comprising code that, when executed, performs the method. The non-transitory computer readable medium can include code stored thereon that, when executed, performs the method. The method can comprise: receiving first host write I/Os; and performing first processing that persistently records the first host write I/Os on a log device, wherein the log device includes a metadata log for storing metadata portions for the first host write I/Os, wherein the first processing includes: aggregating, in a first write aggregation queue, first log write requests that write first metadata portions for the first host write I/Os in a first segment of first metadata log entries of the metadata log of the log device; and responsive to determining that i) the first log write requests corresponding to the first segment write a first amount of metadata equal to a maximum size, or ii) a maximum amount of time has elapsed during said aggregating, sending a first aggregated write request to the metadata log of the log device, wherein the first aggregated write request writes the first metadata portions to the first segment of first metadata log entries. In at least one embodiment, the maximum size can denote a multiple of an indirection unit (IU) size of the log device, where the IU size can denote the internal block size of the log device. Each write to the log device, that writes an amount of content having a corresponding size that is less than the IU size, results in the log device internally performing a read modify write (RMW) operation to write the content. Each of the first log write requests can write one of the first metadata portions, and wherein each of the first metadata portions can have a corresponding size that is less than the IU size. The first log writes corresponding to the first segment can collectively write the first amount of metadata equal to the IU size of the log device. Each of the first metadata portions can be associated with a corresponding one of the first host write I/Os that writes first content to a first target location, and wherein said each metadata portion can include first information describing the first target location. In at least one embodiment, a first host write I/O operation can be included in the first host write I/O operations, wherein the first host write I/O operation can be from a first host and can write first content to a first target location. The first processing can include persistently recording the first content on the log device; and the method can include, responsive to persistently recording the first content and completing the first aggregated write request that writes the first metadata portions to the first segment of first metadata log entries, returning an acknowledgement to the first host regarding co