EP-3848805-B1 - WARM START TECHNIQUE FOR CLOUD-HOSTED FUNCTIONS

EP3848805B1EP 3848805 B1EP3848805 B1EP 3848805B1EP-3848805-B1

Inventors

ZHANG, HONG
FOURIE, HENRY LOUIS
SUNAVALA, FARHAD P.

Dates

Publication Date: 20260513
Application Date: 20180831

Claims (9)

A system comprising: • a non-transitory memory storage comprising instructions; and • one or more processors in communication with the non-transitory memory storage, wherein the one or more processors are configured to execute the instructions to: ∘ identify (1002) a cloud-hosted function configured to be executed within a cloud computing environment; ∘ determine (1004) a hold time to maintain an instantiation of the cloud-hosted function within the cloud computing environment based on a plurality of metrics calculated for the cloud-hosted function, wherein the plurality of metrics includes an execution duration for the cloud-hosted function and at least one of the following metrics: ∘ an image size of the cloud-hosted function, ∘ a memory allocation required for the cloud-hosted function, and ∘ an interval between sequential events that trigger the cloud-hosted function, wherein the metrics are calculated using historical information collected from prior event arrivals that trigger the cloud-hosted function within the cloud computing environment; ∘ instantiate the cloud-hosted function within the cloud computing environment; ∘ after instantiating the cloud-hosted function within the cloud computing environment, maintain (1006) the instantiation of the cloud-hosted function within the cloud computing environment for the determined hold time.
The system of claim 1, wherein the hold time to maintain the instantiation of the cloud-hosted function within the cloud computing environment is determined utilizing a predefined function that: ∘ receives, as an input, the plurality of metrics calculated for the cloud-hosted function, ∘ processes the plurality of metrics to determine the hold time, and ∘ outputs the determined hold time.
The system of claim 1, wherein instantiating the cloud-hosted function within the cloud computing environment includes creating an instance of the cloud-hosted function within the cloud computing environment.
The system of claim 1, wherein instantiating the cloud-hosted function within the cloud computing environment includes: ∘ creating a container for the cloud-hosted function within the cloud computing environment, the container being an execution environment for cloud-hosted function, ∘ loading the image of the cloud-hosted function within the container, and ∘ initializing a runtime for the cloud-hosted function.
The system of claim 1, wherein the one or more processors are further configured to execute the instructions to: ∘ determine that the hold time has elapsed without the cloud-hosted function being executed within the cloud computing environment, and ∘ in response to determining that the hold time has elapsed without the cloud-hosted function being executed within the cloud computing environment, release the instantiation of the cloud-hosted function from within the cloud computing environment.
A warm start method, the method comprising the steps of: ∘ identifying (1002), by one or more processors, a cloud-hosted function configured to be executed within a cloud computing environment; ∘ determining (1004), by the one or more processors, a hold time to maintain an instantiation of the cloud-hosted function within the cloud computing environment based on a plurality of metrics calculated for the cloud-hosted function, wherein the plurality of metrics includes an execution duration for the cloud-hosted function and at least one of the following metrics: o an image size of the cloud-hosted function, ∘ a memory allocation required for the cloud-hosted function, and ∘ an interval between sequential events that trigger the cloud-hosted function, wherein the metrics are calculated using historical information collected from prior event arrivals that trigger the cloud-hosted function within the cloud computing environment, ∘ instantiating, by the one or more processors, the cloud-hosted function within the cloud computing environment; ∘ after instantiating the cloud-hosted function within the cloud computing environment, maintaining (1006), by the one or more processors, the instantiation of the cloud-hosted function within the cloud computing environment for the determined hold time.
The warm start method of claim 6, wherein the hold time to maintain the instantiation of the cloud-hosted function within the cloud computing environment is determined utilizing a predefined function that: ∘ receives, as an input, the plurality of metrics calculated for the cloud-hosted function, ∘ processes the plurality of metrics to determine the hold time, and ∘ outputs the determined hold time.
The warm start method of claim 6, wherein instantiating the cloud-hosted function within the cloud computing environment includes creating an instance of the cloud-hosted function within the cloud computing environment.
The warm start method of claim 6, wherein instantiating the cloud-hosted function within the cloud computing environment includes: ∘ creating a container for the cloud-hosted function within the cloud computing environment, the container being an execution environment for the cloud-hosted function, ∘ loading the image of the cloud-hosted function within the container, and ∘ initializing a runtime for the cloud-hosted function.

Description

FIELD OF THE INVENTION The present disclosure relates to cloud-hosted functions, and more particularly to reducing latency related to the invocation of cloud-hosted functions. BACKGROUND The "cloud" is an abstraction that relates to resource management over a network and, more specifically, to a data center architecture that provides a platform for delivering services via a network. For example, the cloud may refer to various services delivered over the Internet such as network-based storage services or compute services. Typical cloud architecture deployments include a layered hierarchy that includes a physical layer of network hardware, and one or more software layers that enable users to access the network hardware. For example, one common type of cloud architecture deployment includes a physical layer of network resources (e.g., servers, storage device arrays, network switches, etc.) accompanied by a multi-layered hierarchical software framework that includes a first layer that implements Infrastructure as a Service (IaaS), a second layer that implements Platform as a Service (PaaS), and a third layer that implements Software as a Service (SaaS). In general, although there may be exceptions, resources in the third layer are dependent on resources in the second layer, resources in the second layer are dependent on resources in the first layer, and resources in the first layer are dependent on resources in the physical layer. More recently, a serverless cloud architecture has been developed that enables users to execute functions in the cloud without provisioning resources in the traditional hierarchical structure described above. For example, Amazon Web Services (AWS) has developed a service referred to as Amazon® AWS Lambda that enables users to run code without provisioning or managing servers to run the code, like in a traditional compute service. Consequently, a web application can be developed that calls functions uploaded to the AWS Lambda service, where compute resources to execute the function are managed and provisioned by Amazon rather than the web application. In some environments, in order to invoke a cloud-hosted function (also referred to herein as a cloud function), a request is sent from a server application to a function manager. The function manager processes the request by provisioning a container on a server or virtual machine that includes all of the resources (e.g. compute resource, storage resources, etc.) necessary to execute the cloud function. An image of the cloud function is then loaded into the container for execution. However, creating the container and loading the image of the cloud function into the container takes computation cycles and may introduce to the execution of the cloud function a latency of hundreds of milliseconds. The latency increases the delay between receiving the request and returning a result from the cloud function to the server application There is a need to reduce the latency associated with the invocation of cloud-hosted functions. Further, US 2017/0090961 A1 refers to a system for monitoring incoming code execution requests and scheduling the corresponding code executions. Further, US 2017/0177860 A1 refers to that a software image associated with a first customer of a computing resource service provider and criteria for identifying an event is received, the software image comprising a set of layers. Further, US 2016/0188377 A1 refers to systems, apparatuses, and methods for classification based automated instance management. The document AT & T GNS Belgium SPRL: "TST004: Proposals for Metrics and Methods (See 6)", XP014283577 shows an early standardization document draft, in which metrics for evaluating the performance of instances in virtual computing are shown. SUMMARY The above mentioned problem is solved by the subject matter of the independent claims. Further implementation forms are provided in the dependent claims The system of claim 1 and the method of claim 6 provide a warm start technique for cloud-hosted functions. This warm start technique may reduce latency associated with a time to execution of a cloud-hosted function, such as by maintaining an already instantiated cloud-hosted function for hold time that is dynamically determined for the cloud-hosted function (in anticipation of a later requested execution of the cloud-hosted function being made during that hold time). It should be noted that the aforementioned potential advantages are set forth for illustrative purposes only and should not be construed as limiting in any manner. BRIEF DESCRIPTION OF THE DRAWINGS Figures 1A and 1B illustrate the infrastructure for implementing a cloud, in accordance with the prior art;Figure 2 is a conceptual illustration of a cloud architecture, in accordance with the prior art;Figure 3 is a conceptual illustration of a serverless cloud architecture, in accordance with the prior art;Figure 4 is a conceptual illustration of a serverless cloud architec