US-12627632-B2 - Multiple connectivity modes for containerized workloads in a multi-tenant network

Abstract

The disclosure provides a method for isolated environments for containerized workloads within a virtual private cloud in a networking environment. The method generally includes defining, by a user, a subnet custom resource object for creating a subnet in the virtual private cloud, wherein defining the subnet custom resource object comprises defining a connectivity mode for the subnet; deploying the subnet custom resource object such that the subnet is created in the virtual private cloud with the connectivity mode specified for the subnet; defining, by the user, a subnet port custom resource object for assigning a node to the subnet, wherein one or more containerized workloads are running on the node; and deploying the subnet port custom resource object such that the node is assigned to the subnet.

Inventors

• Xiaopei Liu
• Danting LIU
• Jianjun SHEN
• Qian Sun
• Wenfeng Liu
• Donghai HAN

Assignees

• VMware LLC

Dates

Publication Date

20260512

Application Date

20230314

Claims (20)

1. 1 . A method comprising: managing, by an orchestration control plane implemented by a container orchestrator in a virtual private cloud (VPC) networking environment, a container-based cluster comprising a plurality of containers, the plurality of containers comprising a first container on a first virtual machine (VM) executing a first containerized workload, wherein the first container is managed by a first container engine running on the first VM on a first host; providing, within the VPC networking environment, a plurality of available connectivity modes for the first workload on the first container, the plurality of available connectivity modes comprising: a first connectivity mode in which internet protocol (IP) addresses are directly addressable only within the VPC; a second connectivity mode in which IP addresses are directly addressable only within a first level isolation construct that includes the VPC; and a third connectivity mode in which IP addresses are directly addressable only within a second level isolation construct that includes the VPC and the first level isolation construct; identifying a subnet custom resource definition (CRD) object specifying a selected connectivity mode to create an isolated environment for the first containerized workload; providing an isolated environment for the first containerized workload by creating a subnet based on a subnet custom resource definition (CRD) object, wherein addressability of IP addresses within the subnet is based on the connectivity mode specified by the subnet CRD object; and assigning the first containerized workload to the subnet by deploying a subnet port CRD object that associates the subnet with the first VM.
2. 2 . The method of claim 1 , wherein: the VPC networking environment is divided into one or more second level isolation constructs, including the second level isolation construct; each of the one or more second level isolation constructs is divided into one or more first level isolation constructs, including the first level isolation construct; and each of the one or more first level isolation constructs is divided into one or more VPCs, including the VPC.
3. 3 . The method of claim 1 , wherein: the connectivity mode specified for the subnet is the first connectivity mode; creating the subnet in the VPC comprises assigning one or more private IP addresses in a private IP classless inter-domain routing (CIDR) block to the subnet, the one or more private IP addresses addressable within the VPC; assigning the first VM to the subnet comprises assigning a private IP address of the one or more private IP addresses assigned to the subnet to the first VM; and the first VM is directly addressable by the private IP address within the VPC and not outside the VPC.
4. 4 . The method of claim 1 , wherein: the connectivity mode specified for the subnet is the second connectivity mode; creating the subnet in the VPC comprises assigning one or more project IP addresses in a project IP CIDR block to the subnet, the one or more project IP addresses addressable within the first level isolation construct; assigning the first VM to the subnet comprises assigning a project IP address of the one or more project IP addresses assigned to the subnet to the first VM; and the first VM is directly addressable by the project IP address within the first level isolation construct and not outside the first level isolation construct.
5. 5 . The method of claim 1 , wherein: the connectivity mode specified for the subnet is the third connectivity mode; creating the subnet in the VPC with the connectivity mode specified for the subnet comprises assigning one or more public IP addresses in a public IP CIDR block to the subnet, the one or more public IP addresses addressable within the second level isolation construct; assigning the first VM to the subnet comprises assigning a public IP address of the one or more public IP addresses assigned to the subnet to the first VM; and the first VM is directly addressable by the public IP address within the second level isolation construct and not outside the second level isolation construct.
6. 6 . The method of claim 1 , further comprising performing network address translation at a first gateway associated with the VPC to translate IP addresses addressable within the VPC.
7. 7 . The method of claim 6 , further comprising performing network address translation at a second gateway associated with the first level isolation construct to translate IP addresses addressable within the first level isolation construct.
8. 8 . The method of claim 7 , further comprising performing network address translation at a third gateway associated with the second level isolation construct to translate IP addresses addressable within the second level isolation construct.
9. 9 . A system comprising: one or more processors; and at least one computer-readable medium storing instructions executable by the one or more processors to perform operations comprising: managing, by an orchestration control plane implemented by a container orchestrator in a virtual private cloud (VPC) networking environment, a container-based cluster comprising a plurality of containers, the plurality of containers comprising a first container on a first virtual machine (VM) executing a first containerized workload, wherein the first container is managed by a first container engine running on the first VM on a first host; providing, within the VPC networking environment, a plurality of available connectivity modes for the first workload on the first container, the plurality of available connectivity modes comprising: a first connectivity mode in which internet protocol (IP) addresses are directly addressable only within the VPC; a second connectivity mode in which IP addresses are directly addressable only within a first level isolation construct that includes the VPC; and a third connectivity mode in which IP addresses are directly addressable only within a second level isolation construct that includes the VPC and the first level isolation construct; identifying a subnet custom resource definition (CRD) object specifying a selected connectivity mode to create an isolated environment for the first containerized workload; providing an isolated environment for the first containerized workload by creating a subnet based on a subnet custom resource definition (CRD) object, wherein addressability of IP addresses within the subnet is based on the connectivity mode specified by the subnet CRD object; and assigning the first containerized workload to the subnet by deploying a subnet port CRD object that associates the subnet with the first VM.
10. 10 . The system of claim 9 , wherein: the VPC networking environment is divided into one or more second level isolation constructs, including the second level isolation construct; each of the one or more second level isolation constructs is divided into one or more first level isolation constructs, including the first level isolation construct; and each of the one or more first level isolation constructs is divided into one or more VPCs, including the VPC.
11. 11 . The system of claim 9 , wherein: the connectivity mode specified for the subnet is the first connectivity mode; creating the subnet in the VPC comprises assigning one or more private IP addresses in a private IP classless inter-domain routing (CIDR) block to the subnet, the one or more private IP addresses addressable within the VPC; assigning the first VM to the subnet comprises assigning a private IP address of the one or more private IP addresses assigned to the subnet to the first VM; and the first VM is directly addressable by the private IP address within the VPC and not outside the VPC.
12. 12 . The system of claim 9 , wherein: the connectivity mode specified for the subnet is the second connectivity mode; creating the subnet in the VPC comprises assigning one or more project IP addresses in a project IP CIDR block to the subnet, the one or more project IP addresses addressable within the first level isolation construct; assigning the first VM to the subnet comprises assigning a project IP address of the one or more project IP addresses assigned to the subnet to the first VM; and the first VM is directly addressable by the project IP address within the first level isolation construct and not outside the first level isolation construct.
13. 13 . The system of claim 9 , wherein: the connectivity mode specified for the subnet is the third connectivity mode; creating the subnet in the VPC with the connectivity mode specified for the subnet comprises assigning one or more public IP addresses in a public IP CIDR block to the subnet, the one or more public IP addresses addressable within the second level isolation construct; assigning the first VM to the subnet comprises assigning a public IP address of the one or more public IP addresses assigned to the subnet to the first VM; and the first VM is directly addressable by the public IP address within the second level isolation construct and not outside the second level isolation construct.
14. 14 . The system of claim 9 , wherein the operations further comprise performing network address translation at a first gateway associated with the VPC to translate IP addresses addressable within the VPC.
15. 15 . The system of claim 14 , wherein the operations further comprise performing network address translation at a second gateway associated with the first level isolation construct to translate IP addresses addressable within the first level isolation construct.
16. 16 . The system of claim 15 , wherein the operations further comprise performing network address translation at a third gateway associated with the second level isolation construct to translate IP addresses addressable within the second level isolation construct.
17. 17 . A non-transitory computer-readable medium comprising instructions executable by one or more processors to perform operations comprising: managing, by an orchestration control plane implemented by a container orchestrator in a virtual private cloud (VPC) networking environment, a container-based cluster comprising a plurality of containers, the plurality of containers comprising a first container on a first virtual machine (VM) executing a first containerized workload, wherein the first container is managed by a first container engine running on the first VM on a first host; providing, within the VPC networking environment, a plurality of available connectivity modes for the first workload on the first container, the plurality of available connectivity modes comprising: a first connectivity mode in which internet protocol (IP) addresses are directly addressable only within the VPC; a second connectivity mode in which IP addresses are directly addressable only within a first level isolation construct that includes the VPC; and a third connectivity mode in which IP addresses are directly addressable only within a second level isolation construct that includes the VPC and the first level isolation construct; identifying a subnet custom resource definition (CRD) object specifying a selected connectivity mode to create an isolated environment for the first containerized workload; providing an isolated environment for the first containerized workload by creating a subnet based on a subnet custom resource definition (CRD) object, wherein addressability of IP addresses within the subnet is based on the connectivity mode specified by the subnet CRD object; and assigning the first containerized workload to the subnet by deploying a subnet port-custom resource CRD object that associates the subnet with the first VM.
18. 18 . The non-transitory computer-readable medium of claim 17 , wherein: the VPC networking environment is divided into one or more second level isolation constructs, including the second level isolation construct; each of the one or more second level isolation constructs is divided into one or more first level isolation constructs, including the first level isolation construct; and each of the one or more first level isolation constructs is divided into one or more VPCs, including the VPC.
19. 19 . The non-transitory computer-readable medium of claim 17 , wherein: the connectivity mode specified for the subnet is the first connectivity mode; creating the subnet in the VPC comprises assigning one or more private IP addresses in a private IP classless inter-domain routing (CIDR) block to the subnet, the one or more private IP addresses addressable within the VPC; assigning the first VM to the subnet comprises assigning a private IP address of the one or more private IP addresses assigned to the subnet to the first VM; and the first VM is directly addressable by the private IP address within the VPC and not outside the VPC.
20. 20 . The non-transitory computer-readable medium of claim 17 , wherein: the connectivity mode specified for the subnet is the second connectivity mode; creating the subnet in the VPC comprises assigning one or more project IP addresses in a project IP CIDR block to the subnet, the one or more project IP addresses addressable within the first level isolation construct; assigning the first VM to the subnet comprises assigning a project IP address of the one or more project IP addresses assigned to the subnet to the first VM; and the first VM is directly addressable by the project IP address within the first level isolation construct and not outside the first level isolation construct.

Description

Modern applications are applications designed to take advantage of the benefits of modern computing platforms and infrastructure. For example, modern applications can be deployed in a multi-cloud or hybrid cloud fashion. A multi-cloud application may be deployed across multiple clouds, which may be multiple public clouds provided by different cloud providers or the same cloud provider or a mix of public and private clouds. The term, “private cloud” refers to one or more on-premises data centers that might have pooled resources allocated in a cloud-like manner. Hybrid cloud refers specifically a combination of public cloud and private clouds. Thus, an application deployed across a hybrid cloud environment consumes both cloud services executing in a public cloud and local services executing in a private data center (e.g., a private cloud). Within the public cloud or private data center, modern applications can be deployed onto one or more virtual machines (VMs), containers, application services, and/or the like. A container is a package that relies on virtual isolation to deploy and run applications that depend on a shared operating system (OS) kernel. Containerized applications, also referred to as containerized workloads, can include a collection of one or more related applications packaged into one or more containers. In some orchestration platforms, a set of one or more related containers sharing storage and network resources, referred to as a pod, may be deployed as a unit of computing software. Container orchestration platforms automate the lifecycle of containers, including such operations as provisioning, deployment, monitoring, scaling (up and down), networking, and load balancing. Kubernetes® (K8S®) software is an example open-source container orchestration platform that automates the deployment and operation of such containerized workloads. In particular, Kubernetes may be used to create a cluster of interconnected nodes, including (1) one or more worker nodes that run the containerized workloads (e.g., in a worker plane) and (2) one or more control plane nodes (e.g., in a control plane) having control plane components running thereon that control the cluster. Control plane components make global decisions about the cluster (e.g., scheduling), and can detect and respond to cluster events (e.g., starting up a new pod when a workload deployment's intended replication is unsatisfied). As used herein, a node may be a physical machine, or a VM configured to run on a physical machine running a hypervisor. In some cases, multiple tenants (e.g., users or customers) run containerized workloads in the same networking environment, such as in a public cloud. For security purposes, the containerized workloads of different tenants may need to be network isolated from one another within the networking environment. Further, a tenant may be divided into a number of sub-tenants, such that certain containerized workloads of different sub-tenants of a given tenant may also need to be network isolated from one another within the networking environment. SUMMARY One or more embodiments provide a method for isolated environments for containerized workloads within a virtual private cloud in a networking environment. The method generally includes defining, by a user, a subnet custom resource object for creating a subnet in the virtual private cloud. Defining the subnet custom resource object includes defining a connectivity mode for the subnet. The connectivity mode determines whether internet protocol (IP) addresses associated with the subnet are addressable within the virtual private cloud, within a first level isolation construct that includes the virtual private cloud, or within a second level isolation construct that includes the virtual private cloud and the first level isolation construct. The method further includes deploying the subnet custom resource object such that the subnet is created in the virtual private cloud with the connectivity mode specified for the subnet. The method further includes defining, by the user, a subnet port custom resource object for assigning a node to the subnet. One or more containerized workloads are running on the node. The method further includes deploying the subnet port custom resource object such that the node is assigned to the subnet. Further embodiments include a non-transitory computer-readable storage medium comprising instructions that cause a computer system to carry out the above methods, as well as a computer system configured to carry out the above methods. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1A depicts example physical and virtual network components in a networking environment in which embodiments of the present disclosure may be implemented FIG. 1B illustrates an example cluster for running containerized workloads in the network environment of FIG. 1A, according to an example embodiment of the present disclosure. FIG. 2A illustrates example operations for creating