Skip to main content

Networking Issues

Purpose: For operators, shows how to resolve CNI problems, MetalLB IP allocation failures, and DNS resolution issues.

Prerequisites​

  • kubectl access to the cluster
  • calicoctl installed (for Calico issues)

Calico CNI Issues​

Pods Stuck in ContainerCreating​

Symptom: New pods do not get an IP address.

  1. Check Calico pods are running:
kubectl get pods -n calico-system
  1. Check Calico node status:
kubectl get nodes -o wide
calicoctl node status
  1. If Calico pods are crash-looping, check logs:
kubectl logs -n calico-system -l k8s-app=calico-node --tail=50

Common cause: BGP peering failure. Verify the pod CIDR in the Calico IPPool matches opencenter.kubernetes.pod_cidr in the cluster config.

Cross-Node Pod Communication Fails​

Pods on the same node communicate, but pods on different nodes cannot reach each other.

  1. Verify BGP sessions:
calicoctl node status

  1. Check that the underlying network allows BGP (TCP port 179) or IP-in-IP (protocol 4) between nodes.

  2. On cloud providers, verify security groups allow the required protocols.

MetalLB Issues​

No External IP Assigned​

Symptom: LoadBalancer services show <pending> for external IP.

  1. Check MetalLB speaker pods:
kubectl get pods -n metallb-system
  1. Verify the IP address pool is configured and has available addresses:
kubectl get ipaddresspool -n metallb-system
  1. Check MetalLB logs for allocation errors:
kubectl logs -n metallb-system -l app=metallb,component=speaker --tail=50
  1. Verify the IP range in the IPAddressPool does not conflict with DHCP or other allocations on the network.

ARP Not Resolving​

Symptom: External IP is assigned but unreachable from outside the cluster.

MetalLB in L2 mode requires ARP to work between the cluster nodes and the upstream network.

  1. Verify MetalLB is in L2 mode and the L2Advertisement resource exists:
kubectl get l2advertisement -n metallb-system
  1. Check that the nodes' network interfaces are on the same L2 segment as the clients.

DNS Resolution Failures​

Cluster DNS Not Resolving​

Symptom: Pods cannot resolve service names or external domains.

  1. Check CoreDNS pods:
kubectl get pods -n kube-system -l k8s-app=kube-dns
  1. Test resolution from a debug pod:
kubectl run dns-test --image=busybox:1.36 --rm -it -- nslookup kubernetes.default
  1. If CoreDNS pods are running but resolution fails, check CoreDNS logs:
kubectl logs -n kube-system -l k8s-app=kube-dns --tail=50

External DNS Not Resolving​

Pods can resolve cluster services but not external domains.

Check the upstream DNS servers in the CoreDNS ConfigMap:

kubectl get configmap coredns -n kube-system -o yaml

Verify the forward directive points to reachable DNS servers.