US-12619471-B2 - Systems and methods for consistency group based resource throttling

US12619471B2US 12619471 B2US12619471 B2US 12619471B2US-12619471-B2

Abstract

One example method includes, within a microservice architecture: (i) obtaining, via a monitor engine, a resource utilization associated with a set of parameters for each microservice within a consistency group, wherein each microservice within the consistency group is associated with a particular distributed operation within the microservice architecture, and wherein the consistency group defines, for each microservice within the consistency group, one or more threshold values associated with each parameter of the set of parameters, (ii) determining, via an analytical engine, whether resource utilization satisfies a particular threshold of the one or more threshold values associated with a particular parameter of the set of parameters for a particular microservice within the consistency group, and (iii) based upon resource utilization satisfying the particular threshold of the particular parameter for the particular microservice within the consistency group, throttling resource utilization for one or more microservices within the consistency group.

Inventors

Gururaj Kulkarni
Shelesh Chopra

Assignees

DELL PRODUCTS L.P.

Dates

Publication Date: 20260505
Application Date: 20221003

Claims (19)

1 . A method, comprising: within a microservice architecture: grouping microservices operating in the microservice architecture into a plurality of consistency groups, including identifying a particular consistency group associated with a particular distributed operation, wherein the particular consistency group is identified based upon a trace ID associated with that particular distributed operation; obtaining, via a monitor engine, a resource utilization associated with a set of parameters for each microservice within the particular consistency group, wherein each microservice within the particular consistency group is associated with the particular distributed operation within the microservice architecture, and wherein the particular consistency group defines, for each microservice within the particular consistency group, one or more threshold values associated with each parameter of the set of parameters; obtaining, via the monitor engine, a second resource utilization for the particular consistency group as a whole, the second resource utilization associated with one or more second threshold values associated with a second set of parameters for the particular consistency group, wherein the second set of parameters comprises at least one aggregate resource-usage metric representing combined resource usage of the microservices in the consistency group; determining, via an analytical engine, whether the resource utilization of a particular microservice satisfies a particular threshold value of the one or more threshold values associated with a particular parameter of the set of parameters for a particular microservice within the particular consistency group; determining, via the analytical engine, whether the second resource utilization of the particular consistency group satisfies one or more threshold values associated with the second set of parameters; and based upon the resource utilization satisfying the particular threshold value of the particular parameter for the particular microservice within the particular consistency group and based upon the second resource utilization satisfying the one or more second threshold values, throttling resource utilization for one or more microservices within the particular consistency group, wherein such throttling is initiated only when both the particular threshold value satisfied by the resource utilization of the microservice and the second threshold value satisfied by the second resource utilization of the consistency group are satisfied.
2 . The method of claim 1 , wherein each microservice within the consistency group is associated with a service category within the microservice architecture.
3 . The method of claim 1 , wherein at least one microservice within the consistency group is associated with one or more other consistency groups within the microservice architecture.
4 . The method of claim 1 , wherein the set of parameters includes one or more of: processor usage, memory usage, heap usage, swap usage, number of threads, number of file descriptors, number of transmission control protocol connections, jobs distribution, elastic search latency, garbage collection pause, or garbage collection latency.
5 . The method of claim 1 , wherein at least some of the one or more threshold values are determined based upon one or more service level agreements.
6 . The method of claim 1 , wherein the one or more threshold values comprise one or more soft thresholds and one or more hard thresholds for at least some parameters of the set of parameters.
7 . The method of claim 6 , wherein the particular threshold of the one or more threshold values comprises a hard threshold of the one or more hard thresholds.
8 . The method of claim 7 , wherein satisfaction of the one or more soft thresholds triggers a notification.
9 . The method of claim 1 , wherein the monitor engine publishes the resource utilization associated with the set of parameters for each microservice within the consistency group to a message bus.
10 . The method of claim 9 , wherein the analytical engine accesses the resource utilization from the message bus to determine whether the resource utilization satisfies the threshold of the one or more threshold values associated with the particular parameter of the set of parameters for the particular microservice within the consistency group.
11 . The method of claim 10 , wherein the resource utilization is published to the message bus utilizing routing slip techniques.
12 . The method of claim 11 , wherein throttling resource utilization comprises utilizing routing slip techniques to issue throttling for the particular microservice within the consistency group.
13 . The method of claim 1 , further comprising determining throttling parameters for the throttling of resource utilization for the one or more microservices within the consistency group.
14 . The method of claim 1 , wherein throttling resource utilization for the one or more microservices within the consistency group is further based upon a system consistency group comprising all microservices within the microservice architecture, wherein the system consistency group indicates concurrency or total resource usage among all microservices within the microservice architecture.
15 . A system, comprising: one or more processors; and one or more hardware storage devices that store instructions that are executable by the one or more processors to configure the system to: within a microservice architecture: grouping microservices operating in the microservice architecture into a plurality of consistency groups, including identifying a particular consistency group associated with a particular distributed operation, wherein the particular consistency group is identified based upon a trace ID associated with that particular distributed operation; obtaining, via a monitor engine, a resource utilization associated with a set of parameters for each microservice within the particular consistency group, wherein each microservice within the particular consistency group is associated with the particular distributed operation within the microservice architecture, and wherein the particular consistency group defines, for each microservice within the particular consistency group, one or more threshold values associated with each parameter of the set of parameters; obtaining, via the monitor engine, a second resource utilization for the particular consistency group as a whole, the second resource utilization associated with one or more second threshold values associated with a second set of parameters for the particular consistency group, wherein the second set of parameters comprises at least one aggregate resource-usage metric representing combined resource usage of the microservices in the consistency group; determining, via an analytical engine, whether the resource utilization of a particular microservice satisfies a particular threshold value of the one or more threshold values associated with a particular parameter of the set of parameters for a particular microservice within the particular consistency group; determining, via the analytical engine, whether the second resource utilization of the particular consistency group satisfies one or more threshold values associated with the second set of parameters; and based upon the resource utilization satisfying the particular threshold value of the particular parameter for the particular microservice within the particular consistency group and based upon the second resource utilization satisfying the one or more second threshold values, throttling resource utilization for one or more microservices within the particular consistency group, wherein such throttling is initiated only when both the particular threshold value satisfied by the resource utilization of the microservice and the second threshold value satisfied by the second resource utilization of the consistency group are satisfied.
16 . The system of claim 15 , wherein each microservice within the consistency group is associated with a service category within the microservice architecture.
17 . The system of claim 15 , wherein at least one microservice within the consistency group is associated with one or more other consistency groups within the microservice architecture.
18 . The system of claim 15 , wherein the set of parameters includes one or more of: processor usage, memory usage, heap usage, swap usage, number of threads, number of file descriptors, number of transmission control protocol connections, jobs distribution, elastic search latency, garbage collection pause, or garbage collection latency.
19 . One or more hardware non-transitory storage devices that store instructions that are executable by one or more processors of a system to configure the system to: within a microservice architecture: grouping microservices operating in the microservice architecture into a plurality of consistency groups, including identifying a particular consistency group associated with a particular distributed operation, wherein the particular consistency group is identified based upon a trace ID associated with that particular distributed operation; obtaining, via a monitor engine, a resource utilization associated with a set of parameters for each microservice within the particular consistency group, wherein each microservice within the particular consistency group is associated with the particular distributed operation within the microservice architecture, and wherein the particular consistency group defines, for each microservice within the particular consistency group, one or more threshold values associated with each parameter of the set of parameters; obtaining, via the monitor engine, a second resource utilization for the particular consistency group as a whole, the second resource utilization associated with one or more second threshold values associated with a second set of parameters for the particular consistency group, wherein the second set of parameters comprises at least one aggregate resource-usage metric representing combined resource usage of the microservices in the consistency group; determining, via an analytical engine, whether the resource utilization of a particular microservice satisfies a particular threshold value of the one or more threshold values associated with a particular parameter of the set of parameters for a particular microservice within the particular consistency group; determining, via the analytical engine, whether the second resource utilization of the particular consistency group satisfies one or more threshold values associated with the second set of parameters; and based upon the resource utilization satisfying the particular threshold value of the particular parameter for the particular microservice within the particular consistency group and based upon the second resource utilization satisfying the one or more second threshold values, throttling resource utilization for one or more microservices within the particular consistency group, wherein such throttling is initiated only when both the particular threshold value satisfied by the resource utilization of the microservice and the second threshold value satisfied by the second resource utilization of the consistency group are satisfied.

Description

FIELD OF THE INVENTION Embodiments of the present invention generally relate to microservice architecture systems. More particularly, at least some embodiments of the present disclosure relate to systems, hardware, software, computer-readable media, and methods for controlling resource allocation (e.g., by throttling, self-healing, etc.) within a microservice architecture based upon consistency groups of microservices within the microservice architecture. BACKGROUND In a monolithic server architecture, components invoke one another via function calls (which may utilize a single programming language). Resource requirements (such as memory, file descriptor usage, CPU usage, TCP connections) for such monolithic architectures are statically sized (e.g., since there is usually a relatively small number of services that perform unique sets of sub-operations for different use cases). Such services in monolithic architectures may run on a single machine or highly available clustered machines or containers. In a microservice server architecture, services (i.e., “microservices) interact with other services running on different machines (e.g., as agents running on different hosts). Each microservice instance can be configured to perform unique sets of tasks, often in a manner that is independent of task performance by other microservices. Microservices can communicate with other microservices using various communication schemes, such as REST API or message bus architectures. Therefore, resource requirements for different microservices within a microservice architecture can vary from one microservice to the next. Microservice based architectures continue to grow as an architecture of choice for developing complex application stacks. Microservice architectures can reduce multiple levels of dependencies in agile methodologies and DevOps cycle and can improve go-to-market strategies. BRIEF DESCRIPTION OF THE DRAWINGS In order to describe the manner in which at least some of the advantages and features of the invention may be obtained, a more particular description of embodiments of the invention will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings. Understanding that these drawings depict only typical embodiments of the invention and are not therefore to be considered to be limiting of its scope, embodiments of the invention will be described and explained with additional specificity and detail through the use of the accompanying drawings. FIG. 1A illustrates an example representation of a microservice architecture according to some example embodiments. FIG. 1B illustrates example consistency groups of microservices within the microservice architecture according to some example embodiments. FIG. 1C illustrates example representations of monitor and analytical engines for determining and/or modifying resource utilization of microservices within the microservice architecture according to some example embodiments. FIGS. 1D, 1E, and 1F depicts example information flow among the microservices, the monitor engine, and the analytical engine of the microservice architecture according to some example embodiments. FIG. 2 illustrates an example data structure depicting various aspects of consistency groups of a microservice architecture according to some example embodiments. FIG. 3 illustrates an example data structure depicting aspects of microservice performance within a microservice architecture according to some example embodiments. FIGS. 4, 5, and 6 illustrate example methods according to some example embodiments. FIG. 7 illustrates aspects of an example computing entity that may be operable to perform any of the disclosed methods, processes, and/or operations. DETAILED DESCRIPTION OF SOME EXAMPLE EMBODIMENTS Embodiments of the present invention generally relate to microservice architecture systems. More particularly, at least some embodiments of the present disclosure relate to systems, hardware, software, computer-readable media, and methods for controlling resource allocation (e.g., by throttling, self-healing, etc.) within a microservice architecture based upon consistency groups of microservices within the microservice architecture. As noted above, microservice architectures are a prevalent choice when constructing server architectures for supporting applications. Significant investment has been allocated toward implementing dynamic adjustment of resources within microservice architectures to meet system demand (which can vary). For example, a system may experience a spike in resource demand at unexpected intervals or time periods (which may occur infrequently), and failure to effectively to respond to such spikes can result in microservice failure (which may cascade to the entire system). Example factors that can affect the overall performance of microservice systems (e.g., when a demand surge occurs in any of these factors) may include: (i) memory usage surging cont