EP-4735986-A1 - OBJECT STORAGE SERVICE WITH MANAGED CACHE AND COMMON INTERFACE FOR ACCESSING ACCELERATED AND NON-ACCELERATED ACCESS OBJECTS

EP4735986A1EP 4735986 A1EP4735986 A1EP 4735986A1EP-4735986-A1

Abstract

An object storage system includes mass storage devices that implement general storage for objects stored in the object storage system and additionally includes other storage devices, such as solid-state drives, that provide higher performance storage access. The object storage system implements a common access interface for accessing both accelerated access objects (who are eligible to have cached copies stored on the higher performance storage devices) and non-accelerated access objects stored in the general storage. The cache is fully managed by the service and no changes are required for client applications to receive accelerated access to objects that are classified as accelerated access objects per a customer configurable acceleration policy for the object or for a bucket in which the object is stored.

Inventors

SARTORELLO, Enrico
FELIX, JESSIE E.
MARKLE, SETH W.
WARFIELD, ANDREW KENT
THRANE, LEON
FLUNKERT, VALENTIN
MILADINOVIC, MIROSLAV
BARTENSTEIN, CHRISTOPH
KIRSCHNER, JAMES C.

Assignees

Amazon Technologies, Inc.

Dates

Publication Date: 20260506
Application Date: 20240618

Claims (15)

1. An object storage system, comprising: one or more computing devices configured to: implement an interface for receiving access requests for objects stored in the object storage system, wherein the interface is configured to: receive an access request for an accelerated access object stored in the object storage system and receive an access request for a non-accelerated access object stored in the object storage system; determine, in response to receiving a given access request at the interface, whether an object targeted by the given access request is an accelerated access object or a non-accelerated access object; and in response to determining the object targeted by the given access request is an accelerated access object, attempt to retrieve object data for the targeted object from additional storage devices having a second access performance level; or in response to determining the object targeted by the given access request is a non-accelerated access object, retrieve the object data for the targeted object from the mass storage devices having a first access performance level.
2. The object storage system of claim 1, further comprising: the mass storage devices having the first access performance level, wherein the mass storage devices are configured to store object data and object metadata for objects stored in the object storage system, wherein, for a given object, the object metadata comprises: a version identifier for the given object; and information indicating a storage location at which object data, for the given object having the version identifier, is stored on the mass storage devices; the additional storage devices having the second access performance level, wherein the additional storage devices are configured to store cached copies of a subset of the objects stored in the object storage system, wherein, for a given cached object, the additional storage devices store: a cached copy of the object data for the given cached object; and a version identifier for the cached copy of the object data for the given cached object, and wherein the additional storage devices comprise solid-state drives (SSDs).
3. The object storage system of claim 1 or claim 2, wherein the object storage service is configured to automatically populate the additional storage devices with cached copies of objects, wherein to perform said automatically populating the additional storage devices with the cached copies of the objects, the one or more computing devices are further configured to: for an accelerated access object targeted by an access request, in response to a failed attempt to retrieve the accelerated access object from the additional storage devices: retrieve object data for the targeted accelerated access object from the mass storage devices; return the retrieved object data in a response to the access request targeting the accelerated access object; and asynchronously store in the additional storage devices a cached copy of the object data for the targeted accelerated access object and a cached copy of the version identifier corresponding to the cached copy of the object data for the targeted accelerated access object.
4. The object storage system of claim 1 or claim 2, further comprising: a plurality of availability zones, each comprising: respective ones of the mass storage devices; respective ones of the additional storage devices; and respective ones of the one or more computing devices, wherein the respective ones of the one or more computing devices implement respective access interfaces for the respective ones of the availability zones, wherein: each availability zone comprises a separate cache implemented using the respective set of the additional storage devices of the respective availability zone, and the respective caches of the respective availability zones are separately populated based on access requests received at the respective access interfaces of the respective availability zones.
5. The object storage system of claim 4, further comprising: a request routing service configured to: receive access requests from clients of the object storage system and select respective ones of the access interfaces of the respective availability zones to route the access request to, wherein the access interfaces are selected based on proximity of a requesting client to a given one of the respective availability zones.
6. The object storage system of claim 5, wherein the request routing service is further configured to: determine a given one of the availability zones is unavailable to process an access request from a given client proximate to the given availability zone and select a replacement availability zone to process the access request from the given client, wherein subsequent access requests received from the given client are routed to the replacement availability zone, while the given availability zone remains unavailable.
7. One or more non-transitory, computer-readable, storage media storing program instructions that, when executed on or across one or more processors, cause the one or more processors to: implement an interface for receiving access requests for objects stored in an object storage service, wherein the interface is configured to: receive an access request for an accelerated access object stored in the object storage service and receive an access request for a non-accel erated access object stored in the object storage service; determine, in response to receiving a given access request at the interface, whether an object targeted by the given access request is an accelerated access object or a non-accel erated access object; and in response to determining the object targeted by the given access request is a nonaccelerated access object: retrieve the object data for the targeted object from storage devices of the object storage service, the storage devices having a first access performance level; or in response to determining the object targeted by the given access request is an accelerated access object: attempt to retrieve object data for the targeted object from additional storage devices of the object storage service, the additional storage devices having a second access performance level.
8. The one or more non-transitory, computer-readable, storage media of claim 7, storing additional program instructions that, when executed on or across the one or more processors, cause the one or more processor to implement a request routing service configured to: receive access requests from clients of the object storage service; and select respective ones of a plurality of respective access interfaces of respective availability zones to route the access requests to, wherein the access interfaces are selected based on proximity of a requesting client to a given one of the respective availability zones, and wherein each availability zone comprises a separately populated cache.
9. The one or more non-transitory, computer-readable, storage media of claim 8, wherein the request routing service is further configured to: determine a given one of the availability zones is unavailable; and select a replacement availability zone to process access requests, wherein subsequent access requests received from a same given client are routed to the same replacement availability zone, while the given availability zone remains unavailable.
10. The one or more non-transitory, computer-readable, storage media of claim 7, storing additional program instructions that, when executed on or across the one or more processors, cause the one or more processor to implement a consistency verification application programmatic interface (API) configured to: receive an indication of a version identifier for a cached copy of the object data stored in the storage devices, having the second performance level, for the object targeted by the given access request; determine whether the version identifier for the cached copy matches a version identifier for the object targeted by the request that is stored in the storage devices having the first performance level; and provide, based on the determination, an indication of whether the cached copy of the object data is a valid copy of a current version of the object data for the object targeted by the given access request.
11. The one or more non-transitory, computer-readable, storage media of any of claims 7 through 10, wherein the storage devices having the first performance level store for respective objects stored in the object storage service: a current version identifier for current object data for the respective object stored in the object storage service; and information indicating a storage location on the storage devices having the first performance level at which the current object data for the respective object having the current version identifier is stored.
12. The one or more non-transitory, computer-readable, storage media of claim 11, wherein: the object data stored for the objects stored in the object storage service is immutably stored on the storge devices having the first performance level and, in response to a mutating request, object data for a given object targeted by the mutating request is replaced with an updated instance of object data for the object targeted by the mutating request; and an updated version identifier associated with the updated instance of the object data is provided for the object targeted by the mutating request.
13. The one or more non-transitory, computer-readable, storage media of claim 7, storing additional program instructions that, when executed on or across the one or more processors, cause the one or more processor to implement a control plane for the object storage service, the control plane configured to: receive a request for access acceleration defining a class of objects stored in the object stored service for which accelerated access is to be provided; and perform on or more configuration changes to implement accelerated access for the defined class of objects.
14. The one or more non-transitory, computer-readable, storage media of claim 13, wherein the class of objects for which accelerated access is to be provided are defined using one or more of: a storage location in the object storage service where the objects to be provided accelerated access are stored; a prefix included in respective names of the objects for which accelerated access is to be provided; or an alpha numeric string included in a name, or other metadata, for the objects for which accelerated access is to be provided.
15. The one or more non-transitory, computer-readable, storage media of claim 14, wherein individual objects for which accelerated access is to be provided are defined using a flag or other indicator included with an access request for the individual object.

Description

OBJECT STORAGE SERVICE WITH MANAGED CACHE AND COMMON INTERFACE FOR ACCESSING ACCELERATED AND NON-ACCELERATED ACCESS OBJECTS BACKGROUND [0001] A provider network may offer, to various clients, access to data storage resources, computing resources and/or various other services implemented using data storage resources and/or computing resources. For example, using such resources, a provider network may store data on behalf of customers of the provider network in various types of storage solutions. Also, the provider network may offer various types of storage-related services for managing stored data, such as database services, object-based storage services, cloud computing services, etc. [0002] For various reasons, delays in access to some pieces of data stored in a data storge resource of a provider network for a given customer may have less of an impact on the given customer’s operations than delays in accessing other pieces of data stored for the given customer. Also, storage resources that meet greater access requirements (e.g., shorter time to access) may be more costly than other storage resources with less stringent access requirements. [0003] In some situations, pieces of data that need to be accessed with lower latency may be stored in a separate storage resource that provides lower latency access. However, managing additional separate storage resources and re-configuring client applications to locate pieces of data in the additional separate storage resources may be time-consuming and add additional costs and complexity for customers. BRIEF DESCRIPTION OF THE DRAWINGS [0004] FIG. 1 illustrates a logical block diagram of an object storage service included in a service provider network, wherein the object storage service implements a fully managed cache for accelerated access objects and provides a common interface for accessing both accelerated and non-accel erated access objects, according to some embodiments. [0005] FIG. 2 is a logical block diagram illustrating operations performed by an access server of an object storage service in response to receiving an access request for an object at a common access interface of the object storage service, according to some embodiments. [0006] FIG. 3 is a logical block diagram illustrating an object storage service implemented using storage resources included in multiple data centers in multiple availability zones of a service provider network, according to some embodiments. [0007] FIG. 4 is a logical block diagram illustrating operations performed by a request routing service and other components of an object storage service, wherein the object storage service is implemented in multiple availability zones of a service provider network, according to some embodiments. [0008] FIG. 5 is a logical flow diagram illustrating interactions between various components of an object storage service in response to receiving an access request at a common interface of the object storage service, wherein the access request targets an accelerated access object having a cached copy stored using higher performance storage resources, according to some embodiments. [0009] FIG. 6 is a logical flow diagram illustrating interactions between various components of an object storage service in response to receiving an access request at a common interface of the object storage service, wherein the access request targets an accelerated access object that is not currently stored in cache, according to some embodiments. [0010] FIG. 7 is a logical block diagram illustrating buckets of an object storage service, wherein objects stored in the buckets have object names comprising an alphanumeric string value and may further include tags. FIG. 7 further illustrates example acceleration configurations that may be applied to buckets, or to objects, residing in particular buckets and/or having particular prefixes, tags, or other identifying elements, according to some embodiments. [0011] FIG. 8 is a process flow diagram illustrating a process for providing access to objects stored in an object storage service using a common interface of an object storage service that supports access to both accelerated and non-accel erated access objects, according to some embodiments. [0012] FIG. 9 illustrates an example computing device that may be used in some embodiments. [0013] While embodiments are described herein by way of example for several embodiments and illustrative drawings, those skilled in the art will recognize that embodiments are not limited to the embodiments or drawings described. It should be understood, that the drawings and detailed description thereto are not intended to limit embodiments to the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope as defined by the appended claims. The headings used herein are for organizational purposes only and are not meant to be used to limit the s