In two previous posts (here and here) I described how to enable vSphere with Kubernetes in vSphere 7. Although VMware Cloud Foundation (VCF) is required from a licensing/support perspective to run vSphere with Kubernetes, it’s technically possible to enable vSphere with Kubernetes without VCF.

After vSphere with Kubernetes is enabled, you have two options to run containers:

1. Using vSphere Pods, this is the equivalent of a Kubernetes Pod, has a small footprint to run one or more containers. vSphere Pods are objects in your vCenter Server. vSphere Pods run directly on ESXi leveraging the Container Runtime for ESXi (CRX). Read more about vSphere Pods in this blogpost.
2. Using Tanzu Kubernetes Grid Service, the TKG Service provides you the capability to deploy Tanzu Kubernetes clusters onto vSphere. A Tanzu Kubernetes cluster is a full distribution of open-source Kubernetes that is built, signed and supported by VMware. It’s an opinionated, production ready and a fully supported installation of Kubernetes that is integrated with vSphere and managed by Kubernetes. Read more about Tanzu Kubernetes clusters here.

Of course the question is when to use vSphere Pods or Tanzu Kubernetes clusters? There’s an explanation in the official documentation what I would recommend to read. If you want to learn more about the different Tanzu Kuberentes Grid offerings, I would recommend to read my article on this topic.

Enable Workload Management and create a Namespace

Before you can deploy a Tanzu Kubernetes cluster, you first have to enable Workload Management in the vSphere WebClient. Workload Management is the term used for the vSphere with Kubernetes capability. The process to enable Workload Management is explained here and here or if you’re using VMware Cloud Foundation follow this workflow.

After Workload Management is enabled, you have to create a new Namespace in the vSphere WebClient. Select Create Namespace under workload management and enter a name for the new namespace.

After the namespace has been created, don’t forget to set the permissions for team members that should be able to manage this namespace.

Users/groups that you include here, are allowed to connect to the Namespace through kubectl and/or the vSphere with K8S API.

You also want to configure a storage policy for the newly created namespace. Storage policies control how persistent volumes and Tanzu Kubernetes cluster nodes are places on the available vSphere datastores. Add a storage policy to your namespace, this storage policy will also be included in the YAML file that is required to deploy a Tanzu Kubernetes cluster.

Note: I recommend to include only lowercase characters in the policy name, otherwise your policy might not be recognised correctly in a later stage of the configuration process.

Create a Subscribed Content Library

You will a Subscribed Content Library before you can deploy Tanzu Kubernetes (TKG) clusters. The virtual machine image that is used for your TKG clusters is pulled from a Subscribed Content Library.

Create a new Content Library and create a subscription on https://wp-content.vmware.com/v2/latest/lib.json.

Add a shared storage volume for the content library contents. After you created the library, the initial synchronisation will start:

Now it’s time to link the Content Library to your environment: Select Cluster, Configure, General (under Namespaces). Click Edit next to Content Library and select the newly created content library.

It may take a while before the synchronisation to complete.

Deploy a Tanzu Kubernetes cluster

Now connect to this new namespace using kubectl.

kubectl vsphere login --server 172.16.200.33 -u devops@viktorious.local --insecure-skip-tls-verify
kubectl config use-context viktorious01
kubectl get virtualmachineimages

The last command will provide some information on the available VM image for deployment of your TKG cluster.

NAME                                                        AGE
ob-15957779-photon-3-k8s-v1.16.8---vmware.1-tkg.3.60d2ffd   3d18h

A TKG cluster is deployed through a YAML file that contains the configuration parameters for the TKG cluster. A very basic configuration of the YAML configuration is:

apiVersion: run.tanzu.vmware.com/v1alpha1
kind: TanzuKubernetesCluster                   
metadata:
  name: tkg-cluster-1                          
  namespace: viktorious01                       
spec:
  distribution:
    version: v1.16                             
  topology:
    controlPlane:
      count: 1                                 
      class: best-effort-small                 
      storageClass: vsphere-with-kubernetes          
    workers:
      count: 3                                 
      class: best-effort-small                 
      storageClass: vsphere-with-kubernetes

You have to include a name for your TKG cluster, include the namespace and storage class to be used for the deployment. The vsphere-with-kubernetes storage class was linked in one of the previous steps.

Apply this YAML file using

kubectl apply -f tkg.yaml

and your TKG cluster will be deployed:

tanzukubernetescluster.run.tanzu.vmware.com/tkg-cluster-1 created

With these commands you can monitor the status of your TKG cluster:

kubectl get tanzukubernetesclusters
kubectl describe tanzukubernetescluster tkg-cluster-1

Here you can find additional operational commands for TKG.

I hope this was helpful.