Search

US-20260127169-A1 - ARTIFICIAL INTELLIGENCE INFRASTRUCTURE WITH GPU-INITIATED STORAGE OPERATIONS

US20260127169A1US 20260127169 A1US20260127169 A1US 20260127169A1US-20260127169-A1

Abstract

A direct data path between a storage system and memory of one or more processing devices is enabled for a machine learning workload to be executed on the processing devices. Remote direct memory access (RDMA) is configured to transfer data between the storage system and the memory of the one or more processing devices while bypassing host memory. Data transfers for the machine learning workload are executed over the direct data path.

Inventors

  • Brian Gold
  • EMILY WATKINS
  • IVAN JIBAJA
  • Igor Ostrovsky
  • Roy Kim

Assignees

  • PURE STORAGE, INC.

Dates

Publication Date
20260507
Application Date
20251218

Claims (20)

  1. 1 . A method comprising: enabling, within a machine learning infrastructure comprising a storage system and a plurality of processing devices, a direct data path between the storage system and memory of one or more of the plurality of processing devices for a machine learning workload to be executed on the plurality of processing devices; configuring remote direct memory access (RDMA) to transfer data between the storage system and the memory of the one or more processing devices while bypassing host memory; and executing data transfers for the machine learning workload over the direct data path.
  2. 2 . The method of claim 1 , wherein the plurality of processing devices comprises a plurality of graphical processing units (GPUs).
  3. 3 . The method of claim 1 , further comprising: enabling a vendor-specific input/output (I/O) feature for the storage system to support the direct data path.
  4. 4 . The method of claim 1 , further comprising: selecting, for a given dataset, between the direct data path and a network file or object protocol to access the dataset.
  5. 5 . The method of claim 4 , further comprising: selecting among portable operating system interface (POSIX) file, network file system (NFS), server message block (SMB), and simple storage service (S3) object protocols to access the dataset when the direct data path is not selected.
  6. 6 . The method of claim 4 , further comprising: providing multi-protocol data access such that a same stored dataset is available through file protocols and through an object protocol.
  7. 7 . The method of claim 1 , further comprising: detecting a failure condition for the RDMA transfer on the direct data path and retrying the access using a compatibility mode that routes the access via a non-RDMA path.
  8. 8 . The method of claim 1 , further comprising: configuring processing device-accessible endpoints for the direct data path to utilize RDMA on designated network interfaces.
  9. 9 . The method of claim 1 , further comprising: deploying the machine learning infrastructure in a configuration interoperable with a graphical processing unit (GPU) cluster architecture.
  10. 10 . The method of claim 1 , further comprising: operating the direct data path to reduce central processing unit (CPU) utilization associated with data movement for the machine learning workload relative to a host-memory copy path.
  11. 11 . The method of claim 1 , further comprising: performing the data transfers for the machine learning workload over RDMA fabrics selected from InfiniBand and RDMA over Converged Ethernet (RoCE).
  12. 12 . A system comprising: a plurality of processing devices; and a system controller, operatively coupled to the plurality of processing devices, configured to: enable, within a machine learning infrastructure, a direct data path between a storage system and memory of one or more of the plurality of processing devices for a machine learning workload to be executed on the plurality of processing devices; configure remote direct memory access (RDMA) to transfer data between the storage system and the memory of the one or more processing devices while bypassing host memory; and execute data transfers for the machine learning workload over the direct data path.
  13. 13 . The system of claim 12 , wherein the plurality of processing devices comprises a plurality of graphical processing units (GPUs).
  14. 14 . The system of claim 12 , wherein the system controller is further configured to: enable a vendor-specific input/output (I/O) feature for the storage system to support the direct data path.
  15. 15 . The system of claim 12 , wherein the system controller is further configured to: select, for a given dataset, between the direct data path and a network file or object protocol to access the dataset.
  16. 16 . The system of claim 15 , wherein the system controller is further configured to: select among portable operating system interface (POSIX) file, network file system (NFS), server message block (SMB), and simple storage service (S3) object protocols to access the dataset when the direct data path is not selected.
  17. 17 . The system of claim 15 , wherein the system controller is further configured to: provide multi-protocol data access such that a same stored dataset is available through file protocols and through an object protocol.
  18. 18 . The system of claim 12 , wherein the system controller is further configured to: detect a failure condition for the RDMA transfer on the direct data path and retrying the access using a compatibility mode that routes the access via a non-RDMA path.
  19. 19 . The system of claim 12 , wherein the system controller is further configured to: configure processing device-accessible endpoints for the direct data path to utilize RDMA on designated network interfaces.
  20. 20 . A non-transitory computer readable storage medium storing instructions which, when executed, cause a system controller to: enable, within a machine learning infrastructure comprising a storage system and a plurality of processing devices, a direct data path between the storage system and memory of one or more of the plurality of processing devices for a machine learning workload to be executed on the plurality of processing devices; configure remote direct memory access (RDMA) to transfer data between the storage system and the memory of the one or more processing devices while bypassing host memory; and execute data transfers for the machine learning workload over the direct data path.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This is a continuation application for patent entitled to a filing date and claiming the benefit of earlier-filed U.S. patent application Ser. No. 19/257,844, filed Jul. 2, 2025, which is a continuation of U.S. patent application Ser. No. 17/660,735, filed Apr. 26, 2022, issued as U.S. Pat. No. 12,373,428 on Jul. 29, 2025, which is a continuation of U.S. patent application Ser. No. 16/515,698, filed Jul. 18,2019, issued as U.S. Pat. No. 11,403,290 on Aug. 2, 2022, which is a continuation of U.S. patent application Ser. No. 16/045,814, filed Jul. 26, 2018, issued as U.S. Pat. No. 10,360,214 on Jul. 23, 2019, which claims the benefit of earlier-filed: U.S. Provisional Patent Application No. 62/650,736 , filed Mar. 30, 2018; U.S. Provisional Patent Application No. 62/648,368 , filed Mar. 26, 2018; U.S. Provisional Patent Application No. 62/620,286, filed Jan. 22, 2018; U.S. Provisional Patent Application No. 62/579,057 , filed Oct. 30, 2017; and U.S. Provisional Patent Application No. 62/574,534 , filed Oct. 19, 2017; all of the above applications are herein incorporated by reference their entirety. BRIEF DESCRIPTION OF DRAWINGS FIG. 1A illustrates a first example system for data storage in accordance with some implementations. FIG. 1B illustrates a second example system for data storage in accordance with some implementations. FIG. 1C illustrates a third example system for data storage in accordance with some implementations. FIG. 1D illustrates a fourth example system for data storage in accordance with some implementations. FIG. 2A is a perspective view of a storage cluster with multiple storage nodes and internal storage coupled to each storage node to provide network attached storage, in accordance with some embodiments. FIG. 2B is a block diagram showing an interconnect switch coupling multiple storage nodes in accordance with some embodiments. FIG. 2C is a multiple level block diagram, showing contents of a storage node and contents of one of the non-volatile solid state storage units in accordance with some embodiments. FIG. 2D shows a storage server environment, which uses embodiments of the storage nodes and storage units of some previous figures in accordance with some embodiments. FIG. 2E is a blade hardware block diagram, showing a control plane, compute and storage planes, and authorities interacting with underlying physical resources, in accordance with some embodiments. FIG. 2F depicts elasticity software layers in blades of a storage cluster, in accordance with some embodiments. FIG. 2G depicts authorities and storage resources in blades of a storage cluster, in accordance with some embodiments. FIG. 3A sets forth a diagram of a storage system that is coupled for data communications with a cloud services provider in accordance with some embodiments of the present disclosure. FIG. 3B sets forth a diagram of a storage system in accordance with some embodiments of the present disclosure. FIG. 4 sets forth a flow chart illustrating an example method for executing a big data analytics pipeline in a storage system that includes compute resources and shared storage resources according to some embodiments of the present disclosure. FIG. 5 sets forth a flow chart illustrating an additional example method for executing a big data analytics pipeline in a storage system that includes compute resources and shared storage resources according to some embodiments of the present disclosure. FIG. 6 sets forth a flow chart illustrating an additional example method for executing a big data analytics pipeline in a storage system that includes compute resources and shared storage resources according to some embodiments of the present disclosure. FIG. 7 sets forth a flow chart illustrating an additional example method for executing a big data analytics pipeline in a storage system that includes compute resources and shared storage resources according to some embodiments of the present disclosure. FIG. 8A sets forth a diagram illustrating an example computer architecture for implementing an artificial intelligence and machine learning infrastructure that is configured to fit within a single chassis according to some embodiments of the present disclosure. FIG. 8B sets forth a flow chart illustrating an additional example method for executing a big data analytics pipeline in a storage system that includes compute resources and shared storage resources according to some embodiments of the present disclosure. FIG. 9 sets forth a flow chart illustrating an additional example method for executing a big data analytics pipeline in a storage system that includes compute resources and shared storage resources according to some embodiments of the present disclosure. FIG. 10 sets forth a flow chart illustrating an additional example method for executing a big data analytics pipeline in a storage system that includes compute resources and shared storage resources according to some embodimen