Deploying Calico CNI

In this guide you will learn how to set up Calico CNI on Talos in two mode eBPF and NFtables.

This documentation is designed to get you up and running with Talos and Calico CNI. Since both Calico and Talos support multiple networking technologies, you will learn how to run your environment with both the Calico eBPF dataplane and NFTables. Optionally, you can also enable Calico’s network observability stack to gain insights into your cluster networking and policy behavior.

Configuring Talos

To install Calico, you first need to disable the default CNI. This can be done by applying a patch file during cluster creation. The store the following YAML template in a file (patch.yaml).

yaml
cluster:
  network:
    cni:
      name: none

After generating the patch file add the --config-patch argument to your talosctl gen config.

bash
talosctl gen config \
    my-cluster https://calico-talos.local:6443 \
    --config-patch @patch.yaml

Installing Tigera Operator

Recommended way to install Calico is via Tigera-operator manifest. The operator will make sure that all Calico components are always up and running.

Note If you like to install Calico using Helm checkout this document.

Use the following command to install the latest Tigera operator.

bash
kubectl create -f https://docs.tigera.io/calico/latest/manifests/tigera-operator.yaml

Configuring Calico Networking

Calico has a pluggable dataplane architecture that lets you choose the networking technology based on your use case. You can configure the dataplane by setting the linuxDataplane key in the installation manifest.

Note If you like to learn more about the available Calico configurations checkout this document.

By default, Calico uses the /var directory to mount cgroups. However, since this path is not writable in Talos, you need to change it to /sys/fs/cgroup.

Use the following command to update the cgroup mount path:

bash
kubectl create -f -<<EOF
apiVersion: crd.projectcalico.org/v1
kind: FelixConfiguration
metadata:
  name: default
spec:
  cgroupV2Path: "/sys/fs/cgroup"
EOF

Note If you’d like to learn more about the available Calico configurations, checkout this document.

In eBPF mode, Calico completely replaces the need for kube-proxy by programming all networking logic via eBPF programs. Before disabling kube-proxy, however, you need to ensure that Calico components can reach the API server. This can be done by creating a kubernetes-services-endpoint ConfigMap.

Store the following YAML template in a file (e.g., endpoint.yaml), and replace and with your Kubernetes API server host and port. If KubePrism is enabled (which is the default), use localhost as the API server host and 7445 as the port.

yaml
kind: ConfigMap
apiVersion: v1
metadata:
  name: kubernetes-services-endpoint
  namespace: tigera-operator
data:
  KUBERNETES_SERVICE_HOST: '<API server host>'
  KUBERNETES_SERVICE_PORT: '<API server port>'

After editing the file, apply it using:

bash
kubectl create -f endpoint.yaml

You can now safely disable kube-proxy by using the following command:

bash
kubectl patch ds -n kube-system kube-proxy -p '{"spec":{"template":{"spec":{"nodeSelector":{"non-calico": "true"}}}}}'

Next, you have to configure Calico:

bash
kubectl create -f -<<EOF
# This section includes base Calico installation configuration.
apiVersion: operator.tigera.io/v1
kind: Installation
metadata:
  name: default
spec:
  calicoNetwork:
    bgp: Disabled
    linuxDataplane: BPF
  cni:
    ipam:
      type: HostLocal
    type: Calico
  kubeletVolumePluginPath: None
---
# Kubectl integration for Calico unique resources.
apiVersion: operator.tigera.io/v1
kind: APIServer
metadata:
  name: default
spec: {}
EOF

Use the following command to run Calico with NFTables backend.

bash
kubectl create -f -<<EOF
# This section includes base Calico installation configuration.
apiVersion: operator.tigera.io/v1
kind: Installation
metadata:
  name: default
spec:
  calicoNetwork:
    bgp: Disabled
    linuxDataplane: Nftables
  cni:
    ipam:
      type: HostLocal
    type: Calico
  kubeletVolumePluginPath: None
---
# Kubectl integration for Calico unique resources.
apiVersion: operator.tigera.io/v1
kind: APIServer
metadata:
  name: default
spec: {}
EOF

Deploy Calico Whisker Network Observability Stack

Use the following command to enable Calico observability stack:

bash
kubectl create -f -<<EOF
# Configures the Calico Goldmane flow aggregator.
apiVersion: operator.tigera.io/v1
kind: Goldmane
metadata:
  name: default
---
# Configures the Calico Whisker observability UI.
apiVersion: operator.tigera.io/v1
kind: Whisker
metadata:
  name: default
EOF

Use the following command to access Calico Whisker:

bash
kubectl port-forward -n calico-system service/whisker 8081:8081

Fire up a browser and point it to localhost:8081 to observe your policies and network flows.

Next steps

  • Enable Calico Prometheus and Grafana integrations, click here to learn more.

Considerations

In eBPF mode, if you cannot disable kube-proxy for any reason please make sure to adjust BPFKubeProxyIptablesCleanupEnabled to false. This can be done with kubectl as follows:

bash
kubectl patch felixconfiguration default --patch='{"spec": {"bpfKubeProxyIptablesCleanupEnabled": false}}'