Skip to main content

Customizing Services

Purpose: For platform engineers, shows how to use the Kustomize overlay pattern to customize base services with cluster-specific values.

How Service Customization Works​

Every platform service in openCenter follows a base+overlay pattern. The openCenter-gitops-base repository provides hardened base manifests (HelmRelease + default values). Customer clusters reference these bases via FluxCD GitRepository sources and apply cluster-specific overrides using Kustomize overlays. This keeps base security defaults intact while allowing per-cluster tuning.

The Base+Overlay Pattern​

Base (in openCenter-gitops-base)​

Each service in the base repository contains:

openCenter-gitops-base/applications/base/services/<service>/
├── namespace.yaml                          # Namespace definition
├── source.yaml                             # HelmRepository source
├── helmrelease.yaml                        # HelmRelease with valuesFrom
└── helm-values/
    └── hardened-values-<version>.yaml      # Security-hardened defaults

The base HelmRelease references its hardened values via a ConfigMap or inline values. These defaults enforce security contexts, resource limits, and other compliance settings.

Overlay (in customer cluster repo)​

Each cluster's overlay directory contains service-specific overrides:

applications/overlays/<cluster>/services/<service>/
├── kustomization.yaml       # References the base path
└── override-values.yaml     # Cluster-specific Helm value overrides

FluxCD Kustomization (ties them together)​

A FluxCD Kustomization resource in the cluster's services/fluxcd/ directory connects the overlay to the base:

# applications/overlays/<cluster>/services/fluxcd/<service>.yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: <service>-base
  namespace: flux-system
spec:
  interval: 10m
  sourceRef:
    kind: GitRepository
    name: opencenter-<service>
  path: applications/base/services/<service>
  prune: true
  targetNamespace: <service-namespace>
  postBuild:
    substituteFrom:
      - kind: ConfigMap
        name: cluster-vars

Writing Override Values​

The override-values.yaml file contains Helm values that merge on top of the base hardened values. Only include values you need to change:

# applications/overlays/<cluster>/services/cert-manager/override-values.yaml
replicaCount: 3
resources:
  requests:
    cpu: 100m
    memory: 256Mi
  limits:
    memory: 512Mi

The base HelmRelease uses valuesFrom to load both the hardened defaults and the override file. Override values take precedence.

Using postBuild Variable Substitution​

FluxCD's postBuild.substituteFrom replaces ${VARIABLE} placeholders in manifests with values from ConfigMaps or Secrets. openCenter uses a cluster-vars ConfigMap for cluster-wide variables:

# applications/overlays/<cluster>/cluster-vars.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: cluster-vars
  namespace: flux-system
data:
  CLUSTER_NAME: "k8s-sandbox"
  CLUSTER_DOMAIN: "sandbox.example.com"
  STORAGE_CLASS: "longhorn"
  INGRESS_IP: "192.168.1.100"

Manifests in the base can reference these variables:

# In a base HelmRelease or overlay
spec:
  values:
    hostname: dashboard.${CLUSTER_DOMAIN}
    storageClass: ${STORAGE_CLASS}

FluxCD substitutes the variables during reconciliation. This avoids duplicating cluster-specific values across multiple service overrides.

Step-by-Step: Customizing a Service​

  1. Identify the service you want to customize (e.g., metallb).

  2. Check the base HelmRelease to understand available Helm values:

    openCenter-gitops-base/applications/base/services/metallb/helm-values/

  3. Create or edit the override file in your cluster overlay:

    # applications/overlays/<cluster>/services/metallb/override-values.yaml
    controller:
      resources:
        limits:
          memory: 256Mi

  4. Ensure the Kustomization in services/fluxcd/metallb.yaml references the correct GitRepository source and base path.

  5. Commit and push. FluxCD reconciles the change automatically.

Verification​

# Check that FluxCD picked up the change
flux get kustomizations -n flux-system | grep <service>

# Verify the HelmRelease has the merged values
kubectl get helmrelease <service> -n flux-system -o yaml | grep -A 20 "valuesFrom"

# Check the deployed resources reflect your overrides
kubectl get pods -n <service-namespace> -o yaml | grep -A 5 "resources"

# Force reconciliation if needed
flux reconcile kustomization <service>-base -n flux-system

Common Customization Patterns​

  • Resource limits: Adjust CPU/memory in override-values.yaml for your cluster's capacity.
  • Replica count: Increase replicas for HA in production; reduce for dev/test clusters.
  • Storage class: Set storageClass via cluster-vars ConfigMap so all services use the correct provider (Longhorn, Cinder, vSphere).
  • Domain names: Use ${CLUSTER_DOMAIN} substitution for service hostnames.
  • Feature toggles: Enable or disable optional features (e.g., Harbor vulnerability scanning, Kyverno audit mode).

What Not to Do​

  • Do not modify files in openCenter-gitops-base directly. All customization goes in the overlay.
  • Do not remove security-related values from the base (e.g., securityContext, runAsNonRoot). Override only what you need.
  • Do not commit unencrypted secrets in override files. Use SOPS encryption for any sensitive values.