Deploying Tanzu Application Platform on AKS

In this article I will share my experiences installing Tanzu Application Platform on Azure Kubernetes Service.

A first important question to answer is, what is Tanzu Application Platform?

VMware Tanzu Application Platform is a modular, application-aware platform that provides a rich set of developer tooling and a prepaved path to production to build and deploy software quickly and securely on any compliant public cloud or on-premises Kubernetes cluster (link).

Tanzu Application Platform (TAP) is a packaged set of components that helps developers to more easily build, deploy, and manage on a Kubernetes platform. TAP itself also runs on Kubernetes, supported are Azure Kubernetes Service (AKS), Amazon Elastic Kubernetes Service (EKS), Google Kubernetes Engine (GKE), Minikube and of course VMware Tanzu Kubernetes Grid (TKG). Both TKG multi-cloud and vSphere with Tanzu (sometimes referred to as TKG service) are supported. For vSphere with Tanzu, vSphere 7.0u3a and with TAP version higher than 1.0.0 are required.

The real power of TAP is of course in using the platform; but before you can do that you first have to install the solution ;). In this article I will just focus on the deployment of TAP 1.2.1 on Azure Kubernetes Service and share my experiences plus provide some tips & tricks.

Prerequisites

Before we get started we need a few things:

• A Tanzu Network account.
• A v1.22 or v.1.23 Kubernetes cluster is required for TAP, in this example we’re using Azure Kubernetes Service, exact resource requirements are described below.
• Both the Kubernetes CLI (kubectl) and Tanzu CLI are required.
• A container registry is required, in this example we will use Azure Container Registry.
• We need a (sub)domain that we can use for some DNS records. In this example I will use tap.viktorious.nl.
• In this example I will use GitHub to authenticate the TAP installation, so a GitHub account is required. A Git repository is required in a next stage to build TAP software catalogs (GitHub, GitLab and Azure DevOps are all supported).
• Azure command line tools are also required, installation details here.

We’re going to do an on-line deployment, which means that the registry at registry.tanzu.vmware.com is used for the installation. This is fine for a test/demo environment, but not recommended for a production environment. The TAP deployment requires access to the registry, no SLAs are provided for registry.tanzu.vmware.com so I private registry (with an SLA) is preferred for a production environment.

VMware recommends relocating the images from VMware Tanzu Network registry to your own container image registry before attempting installation. If you don’t relocate the images, Tanzu Application Platform will depend on VMware Tanzu Network for continued operation, and VMware Tanzu Network offers no uptime guarantees. The option to skip relocation is documented for evaluation and proof-of-concept only.

The setup is a simple single cluster architecture.

Accept EULA, install Tanzu CLI and kubectl

Before you can download TAP from Tanzu Network, you will need to accept the EULA following the procedure as described here, select release 1.2.1. Also accept the EULA for Cluster Essentials for VMware Tanzu. For now, skip the step about the setting the Kubernetes context because we first need to deploy a Kubernetes cluster.

Next step is to install Tanzu CLI follow this procedure (select TAP release 1.2.1 on ), you will download v0.11.6 of the framework bundle.

Of course you will also need kubectl, depending on your OS/distro there are various options available including native package management, Chocolately and/or brew. Notice that your kubectl version is within one minor version of the Kubernetes cluster you’re running.

For easy management I would at least also recommend to install kubectx and kubens, for more details check out this article.

Create and configure an AKS cluster + container registry

Now we’ve the prerequisites in place, it’s time to deploy a Kubernetes cluster. For this we will use Azure Kubernetes Service (AKS).

Logon to your Azure subscription and select Kubernetes services. Select “Create a Kubernetes cluster” and choose the following configuration options:

• Create a new resource group that will contain you Kubernetes cluster and registry (name the resource group AKS for example).
• Select Dev/Test as the cluster preset configuration.
• Think of Kubernetes cluster name (e.g. AKS01).
• Leave the default Kubernetes version (it’s 1.22.11 at the moment of writing).
• API server availability is 99.5%.
• Leave the node size at B4ms (4 vCPUs, 16 GiB memory). A smaller node size will make your deployment to fail.
• Choose scale method manual, with the number of nodes 3 (at least) or 4 (recommended). 3 is enough for the default installation, however if you’re also deploying apps to this cluster you might run short on resources. You can of course also choose auto-scaling, but make sure you have at least 3 nodes available.

Click next after you’ve verified all the settings.

At the Node Pools, Access and Networking tab, leave everything at the default settings. As part of the wizard, create a new container registry with the following settings:

Leave the advanced & tags tab at their defaults. Review your settings and create the Kubernetes cluster and registry. You can also use my template.json and parameters.json for your reference and automated deployment through the “deploy a custom template” option in Azure. The template.json parameters.json are included in the azure-tap-template.zip.

Deployment of the cluster + registry will take around 5-10 minutes.

After creating the registry we’re going to enable the admin user on the registry to provide easy access (not necessarily a best practice, but for the test setup this is fine). Open the registry, access keys, enable the admin user and write down one of the two passwords, you will need them later.

Now open you Kubernetes cluster in the Azure portal and click connect to retrieve the details on how to connect to the cluster. You will need to execute two commands:

• az account set –subscription <your-subscription-id>
• az aks get-credentials –resource group <resource-group-name> –name <kubernetes-cluster-name>

This is will inject the credentials in your local kubectl configuration.

Now switch to the context of your AKS cluster using kubectl config set-context <kubernetes-cluster-name> or kubectx <kubernetes-cluster-name>. Now run kubectl get nodes to verify the connection and get some details on your Kubernetes cluster nodes.

You’re all set, let’s continue with the tap installation.

Setup GitHub as an authentication provider

In this example we’re going to use GitHub as an authentication provider for Tanzu Application Platform. This can be achieved by adding TAP as an OAuth app to GitHub through github.com/settings/developers (first logon to your GitHub account to make this work). Select New OAuthApp and provide the required details:

The homepage URL is the URL where the application lives, in my scenario this is http://tap-gui.tap.viktorious.nl. The callback URL is also http://tap-gui.tap.viktorious.nl.

After you’ve configured the authentication you will get a Client ID + Client Secret that you will need at a later stage.

Set environment variables

To simplify deployment please set the following environment variables:

export TANZU_NET_USERNAME=<YOUR-TANZUNET-USERNAME>
export TANZU_NET_PASSWORD=<YOUR-TANZUNET-PASSWORD>

Install Cluster Essentials for Tanzu on your AKS cluster

Before we can install TAP, we need to deploy Cluster Essentials for Tanzu to your AKS cluster. You need to deploy Cluster Essentials as part of the TAP installation if you’re running AKS, EKS, GKE or Minikube. If you’re running Tanzu Kubernetes Grid you don’t need to install Cluster Essentials because the components are already installed on your TKG cluster.

To be more specific: VMware Tanzu Kubernetes Grid clusters, Tanzu Community Edition clusters, and clusters provisioned by Tanzu Mission Control already have the contents of Cluster Essentials installed. Clusters deployed directly in vSphere with Tanzu still requires you to install Cluster Essentials.

You can follow this procedure to install Cluster Essentials. So download the cluster essentials, export them to $HOME/tanzu-cluster-essentials and then run the following script:

export INSTALL_BUNDLE=registry.tanzu.vmware.com/tanzu-cluster-essentials/cluster-essentials-bundle@sha256:e00f33b92d418f49b1af79f42cb13d6765f1c8c731f4528dfff8343af042dc3e
export INSTALL_REGISTRY_HOSTNAME=registry.tanzu.vmware.com
export INSTALL_REGISTRY_USERNAME=$TANZU_NET_USERNAME
export INSTALL_REGISTRY_PASSWORD=$TANZU_NET_PASSWORD
cd $HOME/tanzu-cluster-essentials
./install.sh --yes

Installation will take about 1-2 minutes.

Installing Tanzu Application Platform

The further TAP installation flow, I’m using a based on the official documentation and the stuff Adib Saikali has published on his GitHub repository. My colleague Robert Jensen also provided some support on writing the tap-values.yaml (more on that later).

Let’s first create a namespace for the TAP installation:

kubectl create ns tap-install

Let’s now create a secret for the Tanzu registry, based on your Tanzu net credentials (I’m referring to the environment variables we’ve defined earlier):

export INSTALL_REGISTRY_HOSTNAME=registry.tanzu.vmware.com
export INSTALL_REGISTRY_USERNAME=$TANZU_NET_USERNAME
export INSTALL_REGISTRY_PASSWORD=$TANZU_NET_PASSWORD

tanzu secret registry add tap-registry \
--username ${INSTALL_REGISTRY_USERNAME} --password ${INSTALL_REGISTRY_PASSWORD} \
--server ${INSTALL_REGISTRY_HOSTNAME} \
--export-to-all-namespaces --yes --namespace tap-install

Now add the TAP repository to your cluster:

export INSTALL_REGISTRY_HOSTNAME=registry.tanzu.vmware.com
export INSTALL_REGISTRY_USERNAME=$TANZU_NET_USERNAME
export INSTALL_REGISTRY_PASSWORD=$TANZU_NET_PASSWORD
export TAP_VERSION=1.2.1

tanzu package repository add tanzu-tap-repository \
--url registry.tanzu.vmware.com/tanzu-application-platform/tap-packages:${TAP_VERSION} \
--namespace tap-install

The TAP_VERSION variable is the actual version of TAP you’re installing, in this example 1.2.1.

You can use

tanzu package repository list -n tap-install

to verify the repository.

Use

tanzu package available list --namespace tap-install

to see all the available packages.

Before we can continue with the actual installation of TAP, we first need to define a tap-values.yaml file that contains all the settings for TAP. The YAML I’m using is originally coming from Asaikali’s GitHub site. I’ve updated the file to make it work for my environment:

# indicates that all packages that are part of tap.tanzu.vmware.com should be installed
profile: full

# indicates that the customer acknwoldges that the CEIP data collection
# program has been explained to them.
ceip_policy_disclosed: true # Installation fails if this is set to 'false'

# Configure contour / envoy with an external loadbalancer whos
# ip you can add to DNS so that you can access the TAP domains
# You will need a wild card subdoman for example *.tap.example.com
#
# TAP has a gui that the rest of this file will configure to deplay
# on tap-gui.tap.example.com
#
# TAP has learning center that run on demand workshops. We will
# configure Tanzu Learning center to deploy to
# *.tlc.tap.example.com
#
# The Cloud Native Runtime is going to need a domain name to
# deploy knative services onto. We will configure this domain
# to be *.cnr.tap.example.com
#
# To be able to configure all these domain you need to configure
# contour deployment to use an externa LoadBalancer
contour:
  envoy:
    service:
      type: LoadBalancer

# Setup the Cloud Native Runtime Settings
# Deploy and app called foo into namespace bar
#
# foo-bar.cnr.tap.example.com
#
# demoapp in namespace test
#
# demoapp-test.cnr.tap.example.com
cnrs:
  domain_name: cnr.tap.viktorious.nl #change it to your URL
  domain_template: "{{.Name}}-{{.Namespace}}.{{.Domain}}"

# learning center
learningcenter:
  ingressDomain: "tlc.tap.viktorious.nl" #change it to your URL

# tap gui
tap_gui:
  service_type: ClusterIP
  ingressEnabled: "true"
  ingressDomain: "tap.viktorious.nl" #change it to your URL
  app_config:
    auth:
      environment: development
      providers:
        github:
          development:
            clientId: "<Put the GitHub ClientID here>"  
            clientSecret: "<Put the GitHub ClientSecret here>"
    app:
      baseUrl: http://tap-gui.tap.viktorious.nl #change it to your URL
    catalog:
      locations:
        - type: url
          target: https://github.com/asaikali/tap-gui-sample-catalog/blob/main/catalog-info.yaml
    backend:
      baseUrl: http://tap-gui.tap.viktorious.nl #change it to your URL
      cors:
        origin: http://tap-gui.tap.viktorious.nl #change it to your URL

# setup the build service
buildservice:
  kp_default_repository: "viktorious01.azurecr.io/build-service"  #Change to the URL of your registry
  kp_default_repository_username: "<registry admin user here>"
  kp_default_repository_password: "<registry admin password here>"
  tanzunet_username: "<tanzunet username here>" # creds for tanzu net so that installer can pull images from tanzu net and put them in the defalut repo
  tanzunet_password: "<tanzunet password here>"
  descriptor_name: "full"
  enable_automatic_dependency_updates: true


# setup supply chain
supply_chain: basic
# supply_chain: testing
#supply_chain: testing_scanning

# ootb_supply_chain_testing:

#ootb_supply_chain_testing_scanning:
ootb_supply_chain_basic:
  registry:
    server: "viktorious01.azurecr.io" #Change to the URL of your registry
    repository: "apps"  # containerized app images will go into this repo
  gitops:
    ssh_secret: ""

grype:
  namespace: "default" # (optional) Defaults to default namespace.
  targetImagePullSecret: "tap-registry"

Now install TAP:

export TAP_VERSION=1.2.1
tanzu package install tap -p tap.tanzu.vmware.com -v $TAP_VERSION --values-file tap-values.yml -n tap-install

Refer to your tap-values file. Installation will take 10-15 minutes. You can use

kubectl get packageinstalls -n tap-install

to monitor the progress. A succesful installation will look like this:

Now we have to configure DNS so we can access TAP using a FQDN. Use

k get service -n tanzu-system-ingress

to learn the IP of your TAP installation.

Configure your DNS so it’s pointing to this IP address:

I have configured a low TTL, so if you redeploy a TAP test environment you can quickly update your DNS.

If everything went fine, Tanzu Application Platform is running now:

That’s it! I hope this was helpful.