Air-Gapped Kubernetes with Kublr

First things first

To deploy Kubernetes in air gapped environments (environments with no connectivity) with the Kublr Demo/Installer, please request the launch command via email at contact@kublr.com or the contact us form.

Additionally, you’ll need to download the BASH scripts from https://repo.kublr.com

…as well as Helm package archives, the Kublr agent and Docker images:

Important notice!

All provided scripts are checked for use with the Sonatype OSS Nexus. If you are using a different repository, you may need to modify the provided scripts.

System Requirements for Cluster Nodes

  1. x86 64-bit hardware
  2. Hardware recommendation Hardware recommendation
  3. Minimal supported OS on nodes: RedHat Enterprise Linux 7.5+ or Ubuntu 16.04 LTS
  4. Root access to each node
  5. Existing RAW repository for upload Helm and Kublr agent archives and Docker registry, available from each node
  6. Nodes are connected in your network, and the Kublr Demo/Installer configured IP is accessible from these nodes (ping)
  7. From your nodes, firewall rules should allow traffic to Kublr Demo/Installer on port 9080
  8. Kublr Demo/Installer should be correctly configured to be accessible in your local network. Note: It is critical that you provide the correct IP address during the virtual machine startup. If you skipped this step, please re-run provisioning and configure your firewall rules to allow traffic to your computer.

Persistence data storage for Kublr KCP on hosts

Component Name Node Type Default Storage Path Owner UID Minimum Disk Size
ETCD master /mnt/master-pd 0 4G
Elasticsearch data node node /var/lib/kublr/elasticserach/data 1000 128G
Elasticsearch master node node /var/lib/kublr/elasticserach/master 1000 4G
Grafana node /var/lib/kublr/grafana  0 1G
MongoDB node /var/lib/kublr/mongodb 1001 8G
MySQL DB node /var/lib/kublr/mysql 999 8G
Prometheus node /var/lib/kublr/prometheus 0 25G
RabbitMQ node /var/lib/kublr/rabbitmq 999 3G

Repository Requirements

  1. RAW repository (e.g. Sonatype OSS Nexus) for store go binary and Helm packages. At least 50Mb free space required.
  2. Docker repository (e.g. Sonatype OSS Nexus or Docker Registry) for Docker image management. At least 6.5Gb free space required.

Repository Preparation

Use the downloaded files on external media or download SHELL scripts and run them as-is. All necessary archives will be downloaded automatically. Internet access is required.

  1. Upload archives with Kublr agent and Helm packages

    $ bash kublr-load-agent.sh https://192.168.3.8/repository/raw/
    Upload kublr-1.13.0-linux.tar.gz to local repo:
    ######################################################################## 100.0%
    $ bash kublr-load-helm.sh https://192.168.3.8/repository/raw/
    Processing kublr-helm-1.13.0.tar.gz:
    ######################################################################## 100.0%
    cleaning...
  2. Push all needed Docker images into your Docker registry

    $ docker login --username admin --password admin123 192.168.3.8:5000
    $ bash kublr-load-images.sh 192.168.3.8:5000

In these examples, 192.168.3.8 is the IP address of local RAW and docker repository. You will need to change this to your IP or DNS name.

Running Kublr Demo/Installer in AirGap mode

Specify the addresses of repositories to store artifacts, as well as the address of the Docker repository when you run Kublr Demo/Installer. To run in AirGap mode, specify ISOLATED_ENV=true and KUBLR_HOST=HOST_IP:HOST_PORT for the on-premise installation.

export HELM_REPOSITORY=https://192.168.3.8/repository/raw
export KUBLR_AGENT_REPOSITORY=https://192.168.3.8/repository/raw
export KUBLR_VERSION=1.13.0

docker run -p 9080:9080 -d --restart=unless-stopped --name kublr \
       -e HELM_REPOSITORY=${HELM_REPOSITORY} \
       -e KUBLR_AGENT_REPOSITORY=${KUBLR_AGENT_REPOSITORY} \
       ${DOCKER_REGISTRY}/kublr/kublr:${KUBLR_VERSION}

Determine Your Own IP Address

Creating a cluster in your machine’s network is simple. Use your machine’s IP address for this network. If you don’t know how to get the IP address, contact your system administrator or read your OS manual.

To install Kublr clusters in a different network on complex network topologies, provide the IP address of your machine from that network.

Creating AirGap cluster in Kublr

For more details see On-Premise Installation

  1. Open KCP UI on http://Kublr-Demo-Installer-IP:9080/
  2. Create Docker registry credential, add username, password if needed and add CA cert file, or set insecure Docker Registry
  3. Click create cluster or platform
  4. In KCP create mode select Advanced Options in Full Kublr Platform Credentials section and override docker repositrory Docker Override Docker Override

Advanced settings for AirGap installation

By default Kublr takes images from a number of public docker image registries: Docker Hub, Google GCR, Quay.io etc

To enable the creation of clusters in a fully network-isolated environment, Kublr allows specifying substitution Docker registries and Docker image substitution in the cluster spec.

More info: Docker images customization

```yaml
metadata:
  name: cluster-name
spec:
  dockerRegistry:
    auth:
      - secretRef: docker-repo
      - secretRef: quayio-repo
    override:
      default: '192.168.3.8:5000'
      quay_io: '192.168.3.7:5000'
...
```

Air gap