Zone Support in vSphere with Kublr

Introduction

In vSphere infrastructure, default dynamic volume provisioner use the shared datastores wich is connected to all Kubernetes nodes. But in some cases, you don’t want to use these datastores, or perhaps you don’t have one, so using the zones can help you create a Kubernetes cluster on that infrastructure.

With enabling feature as zones support in Kubernetes, the cloud provider reads the topology of the cloud infrastructure and zones from vCenter. Kubernetes nodes and volumes will be labeled with zone information. And pod scheduler will use zone labeling to place pods on nodes.

Purpose

Quick overview on how to create VMware vSphere Kubernetes clusters with zone support.

Prerequisites

  1. Look at the prerequisites that is required for vSphere.
  2. Create and associate Tag Zones and Regions in vCenter

By default the Kublr uses two tag categories: “k8s-zone” and “k8s-region”. Use them to designate a zone and region in vCenter.

Before You Get Started

First download and install Kublr-in-a-box from here.

Creating VMware vSphere Cluster with Zone Support

Creating a cluster with Zone support is not much different from normal. We guide you through these differences:

  1. On the CLUSTER tab, expand the Advanced Options section.

  2. Select the Enable Zone Support option.

    Add Zone support

  3. If your cluster does not have a shared Datastore which is connected to all virtual machines specify one of the parameters:

    • Default Storage Class Zones
    • Default Storage Class Datastore

    NOTES

    • You can also define these fields if you want to use a specific Datastore for default StorageClass.
    • Do not specify both sections at the same time.
    • See each option details and example in the sections below.
  4. You don’t need to create “StorageClass” - Kublr will create it automatically.

  5. On the INSTANCES tab, describe at least two instance groups for worker nodes in different zones.

Default Storage Class Zones

This is a zone in which the volume will be dynamically created. This zone will be included into the “allowedTopologies” section to the “StorageClass” and it will look like:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: kublr-system
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: kubernetes.io/vsphere-volume
parameters:
  diskformat: thin
  fstype: ext4
allowedTopologies:
- matchLabelExpressions:
  - key: failure-domain.beta.kubernetes.io/zone
    values:
    - zone-a
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer

Default Storage Class Datastore

This is the datastore where PersistentVolume will be dynamically created. This datastore will be included into default “StorageClass” and it will look like:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: kublr-system
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: kubernetes.io/vsphere-volume
parameters:
  datastore: fast-datastore
  diskformat: thin
  fstype: ext4
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer