Service Mesh & APIs

Purpose: For developers, shows Istio service mesh patterns including mTLS, traffic management, and when to use service mesh vs. Gateway API.

Task Summary

Istio service mesh is an optional component in openCenter clusters. This guide covers when to enable it, how to configure mTLS between services, and how it interacts with Gateway API. Most applications do not need Istio — Gateway API with NetworkPolicies covers the common case.

Prerequisites

• An openCenter cluster with Istio deployed (optional platform service)
• istioctl CLI installed for debugging
• Application namespace labeled for sidecar injection

When to Use Service Mesh vs. Gateway API

Requirement	Gateway API + NetworkPolicy	Istio Service Mesh
External HTTPS routing	Yes	Yes (via Istio Gateway)
Path/header-based routing	Yes	Yes
mTLS between services	No	Yes — automatic with sidecar
Traffic shifting (canary)	No	Yes — VirtualService weights
Circuit breaking	No	Yes — DestinationRule
Distributed tracing (auto)	No (manual instrumentation)	Yes — sidecar injects trace headers
Multi-tenant isolation	NetworkPolicy only	NetworkPolicy + mTLS identity
Overhead	None	~50MB memory per sidecar, ~2ms latency

Decision rule: Use Gateway API alone unless you need mTLS between services, traffic shifting, or circuit breaking. Istio adds operational complexity and resource overhead.

Steps

1. Enable Sidecar Injection

Label your namespace to enable automatic Istio sidecar injection:

kubectl label namespace my-app istio-injection=enabled

Restart existing pods to pick up the sidecar:

kubectl rollout restart deployment -n my-app

Verify sidecars are injected:

kubectl get pods -n my-app -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{range .spec.containers[*]}{.name}{" "}{end}{"\n"}{end}'
# Each pod should show: my-app istio-proxy

2. Configure Strict mTLS

By default, Istio uses permissive mTLS (accepts both plain and mTLS traffic). For zero-trust, enforce strict mTLS:

apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata:
  name: strict-mtls
  namespace: my-app
spec:
  mtls:
    mode: STRICT

With strict mTLS, only services with valid Istio identity certificates can communicate. Non-mesh services (without sidecars) are blocked.

3. Traffic Shifting (Canary Deployments)

Route a percentage of traffic to a new version:

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: my-app
  namespace: my-app
spec:
  hosts:
    - my-app
  http:
    - route:
        - destination:
            host: my-app
            subset: stable
          weight: 90
        - destination:
            host: my-app
            subset: canary
          weight: 10
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: my-app
  namespace: my-app
spec:
  host: my-app
  subsets:
    - name: stable
      labels:
        version: v1
    - name: canary
      labels:
        version: v2

Gradually increase the canary weight as confidence grows. Roll back by setting stable weight to 100.

4. Circuit Breaking

Protect downstream services from cascading failures:

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: backend-api
  namespace: my-app
spec:
  host: backend-api
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 100
      http:
        h2UpgradePolicy: DEFAULT
        http1MaxPendingRequests: 50
        http2MaxRequests: 100
    outlierDetection:
      consecutive5xxErrors: 5
      interval: 30s
      baseEjectionTime: 60s
      maxEjectionPercent: 50

This ejects endpoints that return 5 consecutive 5xx errors, removing them from the load balancing pool for 60 seconds.

Verification

Check mTLS status between services:

istioctl x describe pod <pod-name> -n my-app
# Shows: mTLS mode, destination rules, virtual services

Verify traffic routing:

istioctl proxy-config routes <pod-name> -n my-app

Troubleshooting

Symptom	Cause	Fix
Connection refused between services after enabling strict mTLS	Destination service has no sidecar	Enable injection in the destination namespace and restart pods
503 errors during canary rollout	Canary pods not matching subset labels	Verify pod labels match DestinationRule subset selectors
High latency after mesh enablement	Sidecar resource limits too low	Increase sidecar proxy CPU/memory via sidecar.istio.io/proxy* annotations

Further Reading

• Gateway API & TLS — HTTPS routing without service mesh
• Application Patterns — Deployment patterns for mesh and non-mesh apps
• GitOps Model — How Istio resources fit into the GitOps workflow