Search

EP-4127993-B1 - MANAGED TRAFFIC PROCESSING FOR APPLICATIONS WITH MULTIPLE CONSTITUENT SERVICES

EP4127993B1EP 4127993 B1EP4127993 B1EP 4127993B1EP-4127993-B1

Inventors

  • MAGERRAMOV, JOSEPH ELMAR
  • BHEEMARAO, SHUBHA SHREE
  • MEDURI, KIRAN K

Dates

Publication Date
20260506
Application Date
20210316

Claims (15)

  1. A system, comprising: one or more computing devices; wherein the one or more computing devices include instructions that upon execution on or across the one or more computing devices cause the one or more computing devices to: obtain, via one or more programmatic interfaces, an indication from a client of a traffic management service that (a) a first constituent service of a distributed application is implemented at least in part using a first resource (118A) at a first isolated network (115A), (b) a second constituent service of the distributed application is implemented at least in part using a second resource (118C) at a second isolated network (115B) and (c) the client has opted in to the use of the traffic management service (180) as an intermediary for traffic flowing between at least the first and second constituent services; obtain, via the one or more programmatic interfaces, (a) a connectivity descriptor of the distributed application at the traffic management service, indicating that at least some requests originating at the first constituent service are to be processed at the second constituent service and (b) a traffic processing descriptor of the distributed application, indicating one or more tasks to be performed with respect to the requests, including at least a tracing task; identify, at the traffic management service, at least a first hidden multi-tenant traffic processing agent (120B) of a fleet of agents (120) of the traffic management service to be used as an intermediary for network traffic between the first and second constituent services; cause, by the traffic management service, one or more networking configuration settings associated with the first resource to be set such that a request generated at the first resource and targeted at the second constituent service is directed to the first hidden multi-tenant traffic processing agent, without providing an indication of the first hidden multi-tenant traffic processing agent to the client; cause, by the first hidden multi-tenant traffic processing agent, in response to receiving a particular request generated at the first resource, one or more trace messages to be stored in accordance with the traffic processing descriptor; cause, by the first hidden multi-tenant traffic processing agent, one or more packets corresponding to the particular request to be delivered to the second resource, wherein the one or more packets are delivered without requiring the client to configure a network route between the first and second isolated networks.
  2. The system as recited in claim 1, wherein the one or more computing devices include further instructions that upon execution on or across the one or more computing devices further cause the one or more computing devices to: obtain, at the traffic management service, virtual-to-physical resource mappings of a virtualized computing service; and utilize, by the first hidden multi-tenant traffic processing agent, at least a portion of the virtual-to-physical mappings to cause the one or more packets to be delivered.
  3. The system as recited in claim 1, wherein the first hidden multi-tenant traffic processing agent is implemented using one or more of: (a) a compute instance of a virtualized computing service, (b) a virtualization management offloading card of a virtualization server, (c) a virtualization management software component running on a CPU of a virtualization server, or (d) a non-virtualized server.
  4. The system as recited in claim 1, wherein the one or more computing devices include further instructions that upon execution on or across the one or more computing devices further cause the one or more computing devices to: decrypt, at the first hidden multi-tenant traffic processing agent, at least a portion of contents of the particular request; and encrypt, at the first hidden multi-tenant traffic processing agent, at least a portion of contents of the one or more packets before delivery to the second resource.
  5. The system as recited in claim 1, wherein the one or more computing devices include further instructions that upon execution on or across the one or more computing devices further cause the one or more computing devices to: verify, at the traffic management service, prior to causing the one or more packets to be delivered, that a permission setting associated with the second constituent service allows delivery of packets on behalf of the first constituent service.
  6. A method, comprising: performing, at one or more computing devices: obtaining (1004), via one or more programmatic interfaces, an indication that at least some requests originating at a first constituent service of a first distributed application are to be processed at a second constituent service of the first distributed application, wherein the first constituent service is implemented at least in part at a first resource at a first isolated network, and wherein the second constituent service is implemented at least in part at a second resource at a second isolated network; identifying (1007) at least a first hidden traffic processing agent of a traffic management service to be used as an intermediary for network traffic between the first and second constituent services; storing (1010) one or more networking configuration settings associated with the first resource, such that a request generated at the first resource and targeted at the second constituent service is directed to the first hidden traffic processing agent; and causing (1016), by the first hidden traffic processing agent, one or more packets corresponding to a particular request generated at the first resource to be delivered to the second resource, wherein the one or more packets are delivered without requiring a client to configure a network route between the first and second isolated networks.
  7. The method as recited in claim 6, further comprising performing, at the one or more computing devices: obtaining, via the one or more programmatic interfaces, an indication that one or more tasks are to be performed with respect to requests originating at the first constituent service, including one or more of: (a) a load balancing task, (b) an authorization-related task, (c) an authentication-related task, (d) an affinity-based destination selection task, (e) a logging task, (f) a load shedding-related task, (g) a health evaluation task, or (h) a back pressure-related task; and implementing, at the first hidden traffic processing agent, the one or more tasks with respect to at least one request originating at the first resource.
  8. The method as recited in claim 6, further comprising performing, at the one or more computing devices: storing metadata at the traffic management service, indicating that the first hidden traffic processing agent is to be configured in single-tenant mode, such that requests associated with a second application are not transmitted to the first hidden traffic processing agent.
  9. The method as recited in claim 6, wherein the traffic management service is implemented at least in part at one or more data centers of a provider network, and wherein at least one resource of the first and second resources comprises at least a portion of one or more of: (a) a compute instance of a virtualized computing service of the provider network, (b) a resource of a software-container-based computing service of the provider network, (c) a resource of a dynamically-provisioned event-driven computing service of the provider network, (d) an auto-scaling group of compute resources of the provider network, (e) a destination of a load balancer of the provider network, or (f) a computing device located at a premise of a client of the provider network.
  10. The method as recited in claim 6, further comprising performing, at the one or more computing devices: parsing, at the first hidden traffic processing agent, at least a portion of the particular request in accordance with a request message format indicated to the traffic management service via a programmatic interface.
  11. One or more non-transitory computer-accessible storage media storing program instructions that when executed on or across one or more processors cause the one or more processors to: determine that one or more messages originating at a first constituent service of a first distributed application are to be processed at a second constituent service of the first distributed application, wherein the first constituent service is implemented at least in part at a first resource (118A) at a first isolated network (115A), and wherein the second constituent service is implemented at least in part at a second resource (118C) at a second isolated network (115B); generate one or more networking configuration settings associated with the first resource, such that a message originating at the first resource and targeted at the second constituent service is directed to a traffic processing agent (120B) established by a traffic management service (180); and cause, in response to a receipt of a first message at the traffic processing agent from the first resource, one or more packets corresponding to the first message to be delivered to the second resource, wherein the one or more packets are delivered without requiring a client to configure a network route between the first and second isolated networks.
  12. The one or more non-transitory computer-accessible storage media as recited in claim 11, storing further program instructions that when executed on or across the one or more processors further cause the one or more processors to: generate one or more additional networking configuration settings associated with a third resource of a third constituent service of a second application, such that a request originating at the third resource is directed to the traffic processing agent; and cause, in response to a receipt of a second message at the traffic processing agent, wherein the second message originates at the third resource, one or more packets corresponding to the second message to be delivered to a different constituent service of the second application.
  13. The one or more non-transitory computer-accessible storage media as recited in claim 11, storing further program instructions that when executed on or across the one or more processors further cause the one or more processors to: dynamically instantiate the traffic processing agent subsequent to obtaining the connectivity descriptor.
  14. The one or more non-transitory computer-accessible storage media as recited in claim 11, storing further program instructions that when executed on or across the one or more processors further cause the one or more processors to: transmit, to the traffic processing agent, an indication of conditional routing to be implemented for messages originating at the first resource, wherein in accordance with the conditional routing indication, a first fraction of messages originating at the first constituent service is to be routed to a second fraction of destination resources at the second constituent service; and cause the traffic processing agent to select a destination for the one or more packets based at least in part on the indication of conditional routing.
  15. The one or more non-transitory computer-accessible storage media as recited in claim 11, storing further program instructions that when executed on or across the one or more processors further cause the one or more processors to: drop, by the traffic processing agent, one or more messages originating at the first resource in accordance with a traffic processing request directed to the traffic processing service.

Description

BACKGROUND Many companies and other organizations operate computer networks that interconnect numerous computing systems to support their operations, such as with the computing systems being co-located (e.g., as part of a local network) or instead located in multiple distinct geographical locations (e.g., connected via one or more private or public intermediate networks). For example, data centers housing significant numbers of interconnected computing systems have become commonplace, such as private data centers that are operated by and on behalf of a single organization, and public data centers that are operated by entities as businesses to provide computing resources to customers. Some public data center operators provide network access, power, and secure installation facilities for hardware owned by various customers, while other public data center operators provide "full service" facilities that also include hardware resources made available for use by their customers. The advent of virtualization technologies for commodity hardware has provided benefits with respect to managing large-scale computing resources for many customers with diverse needs, allowing various computing resources to be efficiently and securely shared by multiple customers. For example, virtualization technologies may allow a single physical virtualization host to be shared among multiple users by providing each user with one or more compute instances (such as "guest" virtual machines) hosted by the single virtualization host. Each such compute instance may represent a software simulation acting as a distinct logical computing system that provides users with the illusion that they are the sole operators of a given hardware computing resource, while also providing application isolation and security among the various virtual machines. Instantiating several different compute instances on the same host may also help increase the overall hardware utilization levels at a data center, leading to higher returns on investment. Some complex applications implemented using networked resources are designed as collections of interacting constituent services, e.g., to enable a cleaner separation of functions and responsibilities within the applications. Each constituent service may, for example, be assigned a respective DNS (Domain Name System) name which can be used to direct messages to the constituent service from the other constituent services of the same application via protocols such as HTTP (HyperText Transfer Protocol). When a client of the application submits a request, internal messages may be transmitted among the constituent services to perform the requested work. Document US 2019/392150 discloses providing an offloaded virtualization management component card that is used as an intermediary. Document US 2014/189686 discloses providing a traffic management system that is enabled to offload tasks by deploying component virtual machines. BRIEF DESCRIPTION OF DRAWINGS FIG. 1 illustrates an example system environment in which a fleet of hidden agents may be used to perform traffic processing tasks such as load balancing and request tracing for network traffic flowing between constituent services of a distributed application, according to at least some embodiments.FIG. 2 illustrates example elements of a distributed application implementation specification which may be stored at a traffic management service responsible for assigning traffic processing agents for the distributed application, according to at least some embodiments.FIG. 3 illustrates example supported request/response formats and example types of tasks which may be performed for distributed applications using traffic processing agents, according to at least some embodiments.FIG. 4 illustrates example flows of metadata and messages to and from hidden traffic processing agents, according to at least some embodiments.FIG. 5 illustrates example allocations of pre-configured and dynamically-instantiated traffic processing agents in multi-tenant and single-tenant modes, according to at least some embodiments.FIG. 6 illustrates example programmatic interactions between a client and a traffic management service for distributed applications, according to at least some embodiments.FIG. 7 illustrates an example use of compute instances of a virtualized computing service for traffic processing agents, according to at least some embodiments.FIG. 8 illustrates an example use of a virtualization management offloading card to implement traffic processing agents, according to at least some embodiments.FIG. 9 illustrates an example provider network at which a traffic management service for distributed applications may be implemented, according to at least some embodiments.FIG. 10 is a flow diagram illustrating aspects of operations that may be performed at a traffic management service for processing network traffic flowing between constituent services of a distributed application, according to