Network Performance
Purpose: For platform engineers and network engineers, documents CNI performance characteristics, tuning options, and throughput benchmarks.
CNI Performance Comparison
| CNI | Mode | Pod-to-Pod Throughput | Latency (p99) | NetworkPolicy Performance |
|---|---|---|---|---|
| Calico | VXLAN overlay | 8.5 Gbps | 0.15ms | Good (iptables) |
| Calico | Direct routing (BGP) | 9.8 Gbps | 0.08ms | Good (iptables) |
| Calico | eBPF dataplane | 9.9 Gbps | 0.06ms | Excellent (kernel bypass) |
| Kube-OVN | Geneve overlay | 8.0 Gbps | 0.18ms | Good (OVS flows) |
| Kube-OVN | Underlay | 9.7 Gbps | 0.09ms | Good (OVS flows) |
Tested on 25 Gbps NICs, MTU 9000, iperf3 TCP streams.
MTU Tuning
| Network Type | Recommended MTU | Calculation |
|---|---|---|
| Physical NIC | 9000 (jumbo) | If switch supports; else 1500 |
| VXLAN overlay | 8950 | Physical MTU − 50 (VXLAN header) |
| Geneve overlay | 8942 | Physical MTU − 58 (Geneve header) |
| Direct routing | 9000 | Same as physical |
Set MTU in the cluster configuration YAML:
networking:
mtu: 8950
eBPF Acceleration
Calico eBPF dataplane provides:
- Kernel-level packet processing (bypasses iptables)
- Native host-routing without encapsulation overhead
- Direct Server Return (DSR) for LoadBalancer services
- ~30% reduction in CPU usage for networking at scale
Enable via Kustomize overlay:
calico:
bpfEnabled: true
bpfExternalServiceMode: DSR
Service Mesh Overhead
| Configuration | Throughput Impact | Latency Added (p99) |
|---|---|---|
| No mesh | Baseline | — |
| Istio sidecar (mTLS) | −15% | +2ms |
| Istio ambient (ztunnel) | −5% | +0.5ms |
Recommendations
- Use jumbo frames (MTU 9000) whenever switch infrastructure supports it
- Prefer direct routing over overlay for bare-metal deployments
- Enable eBPF dataplane for clusters above 50 nodes
- Disable connection tracking for high-throughput stateless services