Edit

Purpose: For platform engineers, provides the full specification of the win-kubeadm role — variables, task sequence, created filesystem layout, and network configuration.

Overview

The opencenter-cloud.opencenter_windows_workers.win-kubeadm role downloads Kubernetes binaries, configures kubelet as a Windows service via NSSM, installs BGP routing features for Calico compatibility, and joins the node to an existing cluster.

Variables

| Variable | Default | Required | Description | |---|---|---|---| | kube_version | (none) | Yes | Kubernetes version to install (e.g., 1.29.0). The role prepends v if missing. | | k8s_internal_ip | (none) | Yes | Control plane internal IP or load balancer VIP. Written to the Windows hosts file as lb-apiserver.kubernetes.local. | | hostname_override | {{ ansible_hostname }} | No | Value passed to kubelet’s --hostname-override flag. | | kubernetes_path | C:\k | No | Directory for kubelet, kubeadm, and the StartKubelet.ps1 script. | | kubelet_log_path | C:\var\log\kubelet | No | Directory for kubelet stdout/stderr logs managed by NSSM. | | nssm_install_directory | C:\Program Files\nssm | No | Installation directory for NSSM (Non-Sucking Service Manager). | | kubernetes_version | {{ kube_version }} | No | Resolved internally. Normalized to include v prefix. |

Source: roles/win-kubeadm/defaults/main.yaml

Inventory requirements

The role delegates to the first host in the oc_controlplane_nodes group to: - Generate a kubeadm join token (kubeadm token create --print-join-command) - Create a kube-dns Service in kube-system namespace (required for Windows DNS resolution)

Your inventory must define this group:

[oc_controlplane_nodes]
control-plane-01 ansible_host=10.0.0.10

Task sequence

  1. Validate that the containerd service is running. Fails immediately if not.

  2. Normalize kubernetes_version to include v prefix.

  3. Create directory tree:

    • {{ kubernetes_path }} (default C:\k)

    • {{ kubelet_log_path }} (default C:\var\log\kubelet)

    • C:\var\lib\kubelet\etc\kubernetes

    • C:\etc\kubernetes\pki

    • C:\etc\kubernetes\manifests

  4. Create symbolic links:

    • C:\var\lib\kubelet\etc\kubernetes\pki → C:\etc\kubernetes\pki\

    • C:\etc\kubernetes\ssl → C:\etc\kubernetes\pki

  5. Download kubelet.exe and kubeadm.exe from https://dl.k8s.io/.

  6. Add kubernetes_path to system PATH.

  7. Download and install NSSM 2.24 from Microsoft’s mirror.

  8. Create StartKubelet.ps1 — reads kubeadm-flags.env and starts kubelet with Windows-specific flags.

  9. Register kubelet as a Windows service via NSSM (if not already registered):

    • stdout log: {{ kubelet_log_path }}\kubelet.out.log

    • stderr log: {{ kubelet_log_path }}\kubelet.err.log

    • Log rotation: enabled, daily, 10 MB max per file

    • Dependency: containerd service

  10. Start kubelet service (start_mode: auto).

  11. Open firewall port 10250/TCP (kubelet API).

  12. Add hosts file entry: {{ k8s_internal_ip }} lb-apiserver.kubernetes.local.

  13. Clean up NSSM temp files.

  14. Install Windows features for BGP routing: RemoteAccess, RSAT-RemoteAccess-PowerShell, Routing. Reboot if required.

  15. Configure RemoteAccess with Install-RemoteAccess -VpnType RoutingOnly.

  16. Start RemoteAccess service.

  17. (tag: join) Delegate to control plane: generate join token.

  18. (tag: join) Verify kubelet and kubeadm are in PATH. Reboot if not found.

  19. (tag: join) Delegate to control plane: create kube-dns Service.

  20. (tag: join) Execute kubeadm join with --cri-socket "npipe:////./pipe/containerd-containerd".

Tags

| Tag | Scope | |---|---| | join | Steps 17–20 only (token generation, PATH check, kube-dns creation, kubeadm join). Use to rejoin a node without reinstalling binaries. |

Kubelet flags

The StartKubelet.ps1 script starts kubelet with these flags:

| Flag | Value | |---|---| | --cert-dir | C:\var\lib\kubelet\pki | | --config | /var/lib/kubelet/config.yaml | | --bootstrap-kubeconfig | /etc/kubernetes/bootstrap-kubelet.conf | | --kubeconfig | /etc/kubernetes/kubelet.conf | | --hostname-override | {{ hostname_override }} | | --node-ip | {{ ansible_host }} | | --pod-infra-container-image | mcr.microsoft.com/oss/kubernetes/pause:3.6 | | --enable-debugging-handlers | (flag present) | | --cgroups-per-qos | false | | --enforce-node-allocatable | "" (empty) | | --resolv-conf | "" (empty) |

Additional flags are read from /var/lib/kubelet/kubeadm-flags.env (written by kubeadm join).

Firewall rules

| Rule name | Port | Protocol | Direction | |---|---|---|---| | kubelet | 10250 | TCP | Inbound |

Filesystem layout created

C:\k\                          kubelet.exe, kubeadm.exe, StartKubelet.ps1
C:\Program Files\nssm\         nssm.exe
C:\var\log\kubelet\            kubelet.out.log, kubelet.err.log
C:\var\lib\kubelet\            config.yaml, kubeadm-flags.env
C:\var\lib\kubelet\pki\        kubelet certificates
C:\var\lib\kubelet\etc\kubernetes\pki  → symlink to C:\etc\kubernetes\pki
C:\etc\kubernetes\pki\         cluster PKI certificates
C:\etc\kubernetes\ssl          → symlink to C:\etc\kubernetes\pki
C:\etc\kubernetes\manifests\   static pod manifests (empty)
C:\var\run\calico\endpoint-status\  Calico endpoint tracking directory

Download URLs

kubelet:  https://dl.k8s.io/{version}/bin/windows/amd64/kubelet.exe
kubeadm:  https://dl.k8s.io/{version}/bin/windows/amd64/kubeadm.exe
NSSM:     https://k8stestinfrabinaries.blob.core.windows.net/nssm-mirror/nssm-2.24.zip

Reboots

This role may trigger up to 2 reboots: . After RemoteAccess/Routing feature installation. . If kubelet/kubeadm binaries are not found in PATH after installation (PATH update requires reboot).

Reboots use a 600-second timeout and wait for the SSH service.