Deploy Kubernetes in Air-Gapped Environments with Kublr

First things first

To deploy Kubernetes in air-gapped environment (environment with no connectivity) using Kublr Demo/Installer, please use your Kublr license number or request the Kublr evaluation license via email at contact@kublr.com or the contact us form.

Additionally, you need to download the BASH scripts from https://repo.kublr.com

You also need to download Helm package archives and Docker images:

Supported Kubernetes versions

Download Kublr agent gobinaries and Docker images for Kubernetes components:

v1.18

v1.17

v1.16 (Deprecated in 1.20.0)

v1.15 (End of support in 1.20.0)

Important notice!

All the provided scripts are checked for use with the Sonatype OSS Nexus. If you are using a different repository, you might need to modify the given scripts.

System Requirements for Cluster Nodes

  1. It should have x86 64-bit hardware
  2. Hardware recommendation can be found here Hardware recommendation
  3. Minimal supported OS on nodes should be: RedHat Enterprise Linux 7.5+ or Ubuntu 16.04 LTS
  4. It should have root access to each node
  5. Existing RAW repository for uploading Helm and Kublr agent archives and Docker registry should be available from each node
  6. With nodes connected to your network, Kublr Demo/Installer configured IP should be accessible from these nodes (ping)
  7. From your nodes, firewall rules should allow traffic to Kublr Demo/Installer on port 9080
  8. Kublr Demo/Installer should be correctly configured to be accessible in your local network. Note: It is critical that you provide the correct IP address during the virtual machine startup. If you skip this step, please re-run provisioning and configure your firewall rules to deliver traffic to your computer.

Persistence Data Storage for Kublr KCP on Hosts

Component NameNode TypeDefault Storage PathOwner UIDMinimum Disk Size
ETCDmaster/mnt/master-pd04G
Elasticsearch data nodenode/var/lib/kublr/elasticserach/data1000128G
Elasticsearch master nodenode/var/lib/kublr/elasticserach/master10004G
Grafananode/var/lib/kublr/grafana 01G
MongoDBnode/var/lib/kublr/mongodb10018G
MySQL DBnode/var/lib/kublr/mysql9998G
Prometheusnode/var/lib/kublr/prometheus025G
RabbitMQnode/var/lib/kublr/rabbitmq9993G

Repository Requirements

  1. It should have RAW repository (e.g. Sonatype OSS Nexus) for store go binary and Helm packages. At least 50Mb free space is required.
  2. It should have Docker repository (e.g. Sonatype OSS Nexus or Docker Registry) for Docker image management. At least 6.5Gb free space is required.

Repository Preparation

Use the downloaded files on external media or download SHELL scripts and run them as is. All necessary archives will be downloaded automatically. Internet access is required.

  1. Upload archives with Kublr agent and Helm packages

    $ bash kublr-agent-load-gobins-1.17.7-4.sh https://192.168.3.8/repository/raw/
    Upload kublr-agent-1.17.7-4.tar.gz to local repo:
    ######################################################################## 100.0%
    
    $ bash kublr-load-helm-1.19.3.sh https://192.168.3.8/repository/raw/
    Processing kublr-helm-1.19.3.tar.gz:
    ######################################################################## 100.0%
    cleaning...
    
  2. Push Kublr Control Plane docker images into your Docker registry

    $ docker login --username admin --password admin123 192.168.3.8:5000
    $ bash kublr-controlplane-load-images-1.19.3.sh 192.168.3.8:5000
    
  3. Push non-default Kublr agent and Kuberntes Docker images

In case you need to deploy non-default Kuberntes cluster (e.g. v1.16.4), download the required Docker images and artifacts from the list above.

For example, for Kubernets v1.16.4

$ bash kublr-agent-load-gobins-1.16.4-5.sh https://192.168.3.8/repository/raw/
Upload kublr-agent-1.16.4-5.tar.gz to local repo:
######################################################################## 100.0%
$ docker login --username admin --password admin123 192.168.3.8:5000
$ bash kublr-agent-load-images-1.16.4-5.sh 192.168.3.8:5000

In this example, 192.168.3.8 is the IP address of the local RAW and docker repositories. You will need to change this to your repository IP or DNS name.

Running the Kublr Demo/Installer in Air-Gap Mode

Specify the addresses of repositories to store artifacts. Please also determine the address of the Docker repository when you run Kublr Demo/Installer.

export HELM_REPOSITORY=https://192.168.3.8/repository/raw
export KUBLR_AGENT_REPOSITORY=https://192.168.3.8/repository/raw
export KUBLR_VERSION=1.19.3
export KUBLR_LICENSE=<your-kublr-license-number>

docker run -p 9080:9080 -d --restart=unless-stopped --name kublr \
        -e SKIP_TLS_VERIFY=true \
        -e HELM_REPOSITORY=${HELM_REPOSITORY} \
        -e KUBLR_AGENT_REPOSITORY=${KUBLR_AGENT_REPOSITORY} \
        -e KUBLR_LICENSE=${KUBLR_LICENSE} \
        ${DOCKER_REGISTRY}/kublr/kublr:${KUBLR_VERSION}

Determine Your Own IP Address

Creating a cluster in your machine network is simple. Use your machine’s IP address for this network. If you don’t know how to get the IP address, contact your system administrator or read your OS manual.

To install Kublr clusters in a different network on complex network topologies, provide the IP address of your machine on that network.

Creating an Air-Gapped cluster in Kublr

For more detail see On-Premises Installation

  1. Open KCP UI on https://Kublr-Demo-Installer-IP:9080/
  2. Create Docker registry credential, add username, password if needed and add CA cert file, or set insecure Docker Registry
  3. Click create cluster or platform
  4. In KCP create mode, expand Advanced options and override docker repositrory Docker Override

Advanced Settings for an Air-Gapped Installation

By default Kublr takes images from a number of public docker image registries: Docker Hub, Google GCR, Quay.io etc

To enable the creation of clusters in a fully network-isolated environment, Kublr allows specifying substitution Docker registries and Docker image substitution in the cluster spec.

More info: Docker images customization

metadata:
  name: cluster-name
spec:
  dockerRegistry:
    auth:
      - secretRef: docker-repo
      - secretRef: quayio-repo
    override:
      default: '192.168.3.8:5000'
      quay_io: '192.168.3.7:5000'

Air gap