Using Cilium CNI in Kublr

Kublr natively supports Cilium as a CNI provider. Cilium is an eBPF-based open source solution for securing and observing network connectivity between application services. Because eBPF runs inside the Linux kernel, Cilium security policies can be applied and updated without any changes to application code or container configuration.

When installed into a Kublr-managed cluster, the following Cilium features are available:

• kube-proxy replacement — Cilium fully replaces kube-proxy using an eBPF-based dataplane, reducing CPU utilization and latency while improving throughput and scale. See Kubernetes Without kube-proxy.

• Inter-node traffic encryption with WireGuard (kernel ≥ 5.6 with WireGuard module, or kernel ≥ 5.15 built-in) or IPsec.

  NOTE WireGuard is the recommended option. IPsec requires additional key and CA configuration.

• Hubble — eBPF-powered networking, service, and security observability. Provides service dependency maps, network monitoring, application monitoring, and security visibility with minimal overhead.

• ClusterMesh — multi-cluster networking and service discovery.

Installation

Setting the CNI provider

To create a new Kublr cluster with Cilium, select cni-cilium in the new cluster creation form, Cluster tab, Advanced Options section, CNI Provider field or, if you are using the cluster specification directly, set spec.network.provider to cni-cilium as shown below:

spec:
  network:
    provider: cni-cilium
    podCIDR: 10.0.0.0/16
    serviceCIDR: 10.96.0.0/12
    # ... other network parameters

When this provider is set, the Kublr agent automatically:

• Removes the kube-proxy static pod manifest from all nodes (kubelet deletes the pod automatically).
• Writes the cni-cilium-chart.yaml addon, which causes the helm-manager to install the Cilium Helm chart into the cluster.

Configuring Cilium

All Cilium configuration is passed as Helm values via kublrAgentConfig in the cluster specification. This is the only supported configuration path - fields directly under spec.network.cilium (such as clusterId or hubbleEnabled) are not reliably processed and should not be used.

The configuration is set at the cluster level (applies to all nodes):

spec:
  kublrAgentConfig:
    cluster:
      network:
        cilium:
          helm_values:
            # Cilium Helm values go here
            prop1: overrideValue

Note that if it is necessary to unset some values by specifying null, the helm_values field must be set as a multi-line string to protect from Kublr API discarding null-valued fields.

This format can also be useful when implementing advanced templates in the configuration (available since Kublr 1.28):

spec:
  parameters:
    foo: bar
  kublrAgentConfig:
    cluster:
      network:
        cilium:
          helm_values: |
            # Cilium Helm values go here

            # null values are treated by helm as a signal to remove the property defined in the default values.yaml
            prop1: null

            prop2: {{.spec.parameters.foo}}

Example additional configuration

spec:
  network:
    provider: cni-cilium
    podCIDR: 10.0.0.0/16
    serviceCIDR: 10.96.0.0/12
  kublrAgentConfig:
    cluster:
      network:
        cilium:
          helm_values: |
            cluster:
              id: 0
              name: my-cluster

Full configuration example (WireGuard encryption + Hubble)

spec:
  network:
    provider: cni-cilium
    podCIDR: 10.0.0.0/16
    serviceCIDR: 10.96.0.0/12
  kublrAgentConfig:
    cluster:
      network:
        cilium:
          helm_values: |
            cluster:
              id: 1
              name: my-cluster
            encryption:
              enabled: true
              nodeEncryption: true
              type: wireguard
            hubble:
              enabled: true
              peerService:
                clusterDomain: cluster.local
              ui:
                enabled: true
              relay:
                enabled: true
              dashboards:
                enabled: false

NOTE cluster.id must be a unique integer in the range 1-255 across all clusters if ClusterMesh is planned. Use 0 to disable ClusterMesh.

NOTE If the cluster DNS domain is not the default cluster.local (i.e. spec.network.dnsDomain is set to a custom value), you must set hubble.peerService.clusterDomain to match. Otherwise the Hubble relay will fail to resolve the peer service and enter a crash loop.

Configuration parameters

Refer to the Cilium Helm chart values reference for full documentation of specific values that can be set.

kube-proxy replacement

Kublr enables kube-proxy replacement automatically when provider: cni-cilium is set. No additional configuration is required.

Cilium provides an eBPF-based alternative to the iptables/IPVS mechanisms implemented by kube-proxy, with lower CPU utilization, reduced latency, improved throughput, and better scalability.

WireGuard node-to-node encryption

encryption:
  enabled: true
  type: wireguard
  nodeEncryption: true

Encrypts all node-to-node traffic transparently, with no application changes or additional proxies required. Cilium handles automatic key rotation with overlapping keys. All traffic is encrypted including non-standard protocols such as UDP.

Requirements: Linux kernel ≥ 5.6 with the wireguard module, or kernel ≥ 5.15 where WireGuard is built in.

Hubble observability

hubble:
  enabled: true
  peerService:
    clusterDomain: cluster.local  # set to your cluster DNS domain
  ui:
    enabled: true
  relay:
    enabled: true
  dashboards:
    enabled: false  # set to true only if Grafana is configured with sidecar dashboard discovery

Hubble provides deep visibility into service communication and networking behavior. When enabled, the Hubble UI is accessible via port-forward:

kubectl port-forward -n kube-system svc/hubble-ui 12000:80

Then open http://localhost:12000 in a browser.

Hubble Grafana dashboards (IDs: 16611–16614) can be imported manually from grafana.com.

Migration from Calico or Canal

This section describes how to migrate a running Kublr cluster from Calico (or Canal) CNI to Cilium. The migration replaces the CNI without requiring cluster recreation. Some brief downtime is expected while traffic routing transitions from Calico to Cilium.

Prerequisites

Before starting, verify:

• All nodes run Linux kernel ≥ 5.6 (for WireGuard; ≥ 5.15 recommended for built-in support).

• No Calico-specific resources are in use that have no Cilium equivalent:

  kubectl get globalnetworkpolicies.crd.projectcalico.org
  kubectl get hostendpoints.crd.projectcalico.org
  kubectl get bgppeers.crd.projectcalico.org
  

  Standard NetworkPolicy objects are fully supported by Cilium without changes.

• Take note of your current pod CIDR and service CIDR — these are preserved during migration.

Step 1: Update the cluster specification in KCP

In the Kublr Control Platform, navigate to the cluster and edit its specification. Change spec.network.provider from cni-canal (or cni-calico) to cni-cilium, and add the kublrAgentConfig section with your desired Cilium Helm values:

spec:
  network:
    provider: cni-cilium   # was: cni-canal or cni-calico
    # all other network parameters remain unchanged
  kublrAgentConfig:
    cluster:
      network:
        cilium:
          helm_values: |
            cluster:
              id: 1
              name: my-cluster
            encryption:
              enabled: true
              nodeEncryption: true
              type: wireguard
            hubble:
              enabled: true
              peerService:
                clusterDomain: cluster.local
              ui:
                enabled: true
              relay:
                enabled: true
              dashboards:
                enabled: false

After saving, the Kublr agent propagates the change to all nodes. It will:

• Remove the kube-proxy static pod manifest from each node.
• Write the cni-cilium-chart.yaml addon file on master nodes.
• The helm-manager will install the Cilium Helm chart (this may take a few minutes).

NOTE The Calico Helm release is not automatically removed when the CNI provider changes. Calico and Cilium will run simultaneously until Calico is explicitly removed in a later step.

Step 2: Monitor Cilium deployment

Wait for all Cilium pods to reach Running state:

kubectl get pods -n kube-system -l k8s-app=cilium -w

Verify Cilium is healthy using the Cilium CLI:

kubectl -n kube-system exec ds/cilium -- cilium status

Expected output shows all components OK, encryption enabled, and all nodes reachable:

    /¯¯\
 /¯¯\__/¯¯\    Cilium:             OK
 \__/¯¯\__/    Operator:           OK
 /¯¯\__/¯¯\    Envoy DaemonSet:    OK
 \__/¯¯\__/    Hubble Relay:       OK
    \__/       ClusterMesh:        disabled

Cluster health:    9/9 reachable

Also verify encryption is active:

kubectl -n kube-system exec ds/cilium -- cilium status --verbose | grep -A4 Encryption

Step 3: Remove Calico

Once Cilium is healthy, remove Calico completely.

3a. Uninstall the Calico Helm release:

If the tigera-operator Helm release exists (check with helm list -n tigera-operator):

helm uninstall tigera-operator -n tigera-operator --no-hooks

The --no-hooks flag is required if any pre/post-delete Jobs from a previous operation are still present in a completed state.

3b. Delete all Calico and Tigera CRDs:

kubectl get crd | grep -E 'projectcalico\.org|tigera\.io' | awk '{print $1}' | xargs kubectl delete crd

3c. Delete the Calico namespaces:

kubectl delete namespace calico-system tigera-operator calico-apiserver --ignore-not-found

If any namespace gets stuck in Terminating, it is likely due to tigera.io/cni-protector finalizers on service accounts inside the namespace. Remove them:

# Example for calico-system namespace
for sa in calico-cni-plugin calico-node; do
  kubectl patch serviceaccount $sa -n calico-system \
    -p '{"metadata":{"finalizers":[]}}' --type=merge
done

Similarly, if the installations.operator.tigera.io CRD itself is stuck deleting:

kubectl patch crd installations.operator.tigera.io \
  -p '{"metadata":{"finalizers":[]}}' --type=merge

Step 4: Restart pods with stale CNI addresses

After Calico is removed, some pods (particularly DaemonSets that were not restarted during the migration) may still hold IP addresses from the old CNI. Check the count of Cilium-managed pods:

cilium status | grep "Cluster Pods"

If the managed count is less than the total running pod count, identify the unmanaged pods and restart their controllers:

# Restart all DaemonSets in all namespaces to pick up new Cilium networking
kubectl get ds -A --no-headers | awk '{print $1, $2}' | while read ns name; do
  kubectl rollout restart daemonset/$name -n $ns
done

Wait for the rollout to complete and re-verify cilium status.

Step 5: Validate

Run a full connectivity test using the Cilium CLI:

cilium connectivity test

Verify node-to-node encryption (WireGuard peers should equal the number of other nodes):

kubectl -n kube-system exec ds/cilium -- cilium status --verbose | grep -A10 Encryption

Check Hubble is receiving flows:

cilium hubble observe --last 20

Usage

Working with Cilium

Use the Cilium CLI to inspect cluster state:

# Overall status
cilium status

# Verbose status including encryption peers and IPAM
kubectl -n kube-system exec ds/cilium -- cilium status --verbose

# List all Cilium-managed services (eBPF load balancer)
kubectl -n kube-system exec ds/cilium -- cilium service list

Network performance test

The Cilium CLI includes a built-in network performance test:

cilium connectivity test --perf

Hubble observability

Hubble enables deep visibility into service communication, network flows, and security events. It operates at the node level and, when ClusterMesh is configured, across clusters.

Access the Hubble UI via port-forward:

kubectl port-forward -n kube-system svc/hubble-ui 12000:80

Observe live network flows from the CLI:

cilium hubble observe --follow
cilium hubble observe -n my-namespace --last 50

See the Introduction to Cilium & Hubble for full documentation.

See Also

• This documentation:

  • CNI Provider

• Cilium documentation:

  • Introduction to Cilium & Hubble
  • Kubernetes Without kube-proxy
  • Cilium Helm chart values reference
  • Multi-Cluster (ClusterMesh)
  • Migrating a cluster to Cilium

• MS Azure documentation: Bring your own Container Network Interface (CNI) plugin with Azure Kubernetes Service (AKS)