Wednesday, September 9, 2020

Deploying Tanzu Application Service to Kubernetes (TKG)

 Introduction

Tanzu Application Service (TAS) is a powerful Platform as a Service abstraction  which can be deployed on many different cloud infrastructure providers.  It allows developers to write their application in any of the commonly used programming language and "push" their code to a runtime which takes care of all of the provisioning required to allow the program to run and does the plumbing to make the application accessible.

TAS has been available for many years now, based on the Open Source Cloud Foundry Application Service.   At time of writing VMWare have been moving TAS from running on Virtual Machines to run as containers on a kubernetes environment.  Currently in Beta the first GA release should be available in the fall of 2020.  (Or as I would say Autumn.)

This blog post runs through the steps involved to get it up and running, essentially summarising the information in the documentation. (Note - this is a link to beta documentation and the URL may well change soon.)

Installation

Start by downloading the TAS system which can be downloaded as a single .tar file.  When extracted will create a directory called tanzu-application-service with a number of subdirectories.  For this blog posting using the 0.4.0 Beta version of TAS, the installation process may significantly change over coming releases as the product goes GA.

Kubernetes

To get started we need to create a kubernetes cluster to deploy TAS4k8s onto, in this example I am using TKG running in an AWS environment.  (See posting http://ablether.blogspot.com/2020/08/first-experiences-with-tanzu-kubernetes.html about setting up TKG on AWS)
TAS will create quite a few containers and the current default setup is a minimal install with only an instance of most components.  According to the documentation you need at least 5 worker nodes with a minimum of 2CPU/7.5Gb RAM, this equates to worker nodes of t3.large on AWS.  I found that I could get away with 5 t3.medium sized ones but in order to deploy a few of your own containers I recommend 7 workers at t3.large size.

% tkg create cluster clust-tas --plan dev -w 7 --worker-size t3.large
Logs of the command execution can also be found at: /var/folders/nn/p0h624x937l6dt3bdd00lkmm0000gq/T/tkg-20200827T154322852129303.log
Validating configuration...
Creating workload cluster 'clust-tas'...
Waiting for cluster to be initialized...
Waiting for cluster nodes to be available...

Workload cluster 'clust-tas' created

Once our cluster is up and running we need to get the context to it for kubectl and then we can deploy a couple of additional capabilities in preparation for TAS.



% tkg get credentials clust-tas
Credentials of workload cluster 'clust-tas' have been saved
You can now access the cluster by running 'kubectl config use-context clust-tas-admin@clust-tas'
% kubectl config get-contexts
CURRENT   NAME                                      CLUSTER            AUTHINFO                 NAMESPACE
          aws-mgmt-cluster-admin@aws-mgmt-cluster   aws-mgmt-cluster   aws-mgmt-cluster-admin
          clust-tas-admin@clust-tas                 clust-tas          clust-tas-admin
          minikube                                  minikube           minikube
*         test-cluster-admin@test-cluster           test-cluster       test-cluster-admin
% kubectl config use-context clust-tas-admin@clust-tas
Switched to context "clust-tas-admin@clust-tas".


There are two things we need to do to the newly created cluster, firstly we need to install the metrics server which goes into the kube-system namespace.

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.3.6/components.yaml
clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created
clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created
rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created
apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created
serviceaccount/metrics-server created
deployment.apps/metrics-server created
service/metrics-server created
clusterrole.rbac.authorization.k8s.io/system:metrics-server created
clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created
Installing the default storage class for aws gp2 based ext4 storage.

Then for AWS we want to set a default storage class.  Obviously the choice here may depend on what you feel you need for your environment, for the purposes of the test environment we will be using a standard gp2 storage from AWS.

Inside the download directory there is a config-optional directory that contains a yaml file called aws-default-storage-class.yml.  Use this to upload to your Kubernetes cluster.  This introduces another tool which is part of the suite of tools known as k14s or Carvel.  They can be downloaded from k14s.io (or get-kapp.io or for TAS specifically VMWare package a version).    kapp -  Kubernetes Application Management Tool is a CLI designed to manage resources in bulk and is used to package up and deploy/manage TAS4k8s.


$ kapp deploy -a default-storage-class -f <download dir>/tanzu-application-service/config-optional/aws-default-storage-class.yml
Target cluster 'https://clust-tas-apiserver-352766885.eu-west-2.elb.amazonaws.com:6443' (nodes: ip-10-0-0-56.eu-west-2.compute.internal, 7+)

Changes

Namespace  Name  Kind          Conds.  Age  Op      Wait to    Rs  Ri
(cluster)  gp2   StorageClass  -       -    create  reconcile  -   -

Op:      1 create, 0 delete, 0 update, 0 noop
Wait to: 1 reconcile, 0 delete, 0 noop

Continue? [yN]: y

9:43:18AM: ---- applying 1 changes [0/1 done] ----
9:43:18AM: create storageclass/gp2 (storage.k8s.io/v1) cluster
9:43:18AM: ---- waiting on 1 changes [0/1 done] ----
9:43:18AM: ok: reconcile storageclass/gp2 (storage.k8s.io/v1) cluster
9:43:18AM: ---- applying complete [1/1 done] ----
9:43:18AM: ---- waiting complete [1/1 done] ----

Succeeded

Tanzu Application Service

The first step towards configuring TAS is to decide on what domain name to use.  I have a domain donaldforbes.com which I use for dev/testing and so decided to use a system domain of sys.tas.donaldforbes.com for my deployment.  This domain points to a load balancer which I will run in AWS to handle all the ingress.  (wildcard DNS used)

Next we need to create a set of configuration values which are specific to this install.  Create a configuration-values directory and then run the tanzu-application-service/bin/generate-values.sh script to populate configuration parameters for your environment.   The script generates a file with a set of config values - it uses self signed certificates by default and the various passwords for TAS components.

$ ./bin/generate-values.sh -d sys.tas.donaldforbes.com > configuration-values/deployment-values.yml

Using AWS we will also use a load balancer to manage ingress to the platform.  This requires the presence of another file in the configuration-values directory.  Called load-balancer-values.yml

$ cat load-balancer-values.yml
#@data/values
---
enable_load_balancer: True

We also need to be able to access two different registries (they could be the same). Firstly a system registry which is used by the TAS installer to reach out for the images used to run the various components/containers of TAS itself.  This is configured in a file called system-registry-values.yml

$ cat system-registry-values.yml
#@data/values
---
system_registry:
  hostname: "registry.pivotal.io"
  username: "dforbes@pivotal.io"
  password: "<RedactedPassword>"

To gain access simply sign up to https://network.pivotal.io.

The second registry is one that TAS will use to store the containers it creates for every application that is deployed.  Most registries will be possible to used for this.  Simply create a file called app-registry-values.yml and populate it with values to enable the login to the registry and ensure that the user has read/write access.  In my example below I decided to use the docker registry on docker hub.

$ cat app-registry-values.yml
#@library/ref "@github.com/cloudfoundry/cf-for-k8s"
#@data/values
---
app_registry:
  hostname: https://index.docker.io/v1/
  repository_prefix: "donaldf"
  username: "donaldf"
  password: "<Redacted Password>"


Once done we are ready to install TAS.  Simply run the install-tas.sh script in the bin directory of tanzu-application-service passing in to it a parameter which points to your custom configuration directory.

$ ./bin/install-tas.sh ./configuration-values

This script will do all the kubernetes configuration, upload the images from the system repository and create the necessary deployments.  Typically this process will take several minutes to complete.

Once done there is a load balancer created for the ingress.  You must point your DNS to this load balancer to access TAS.  (I actually used route 53 to handle the wildcard DNS need to point to the load balancer and used a CNAME from my domain registration.)