Managed Kafka: Overview & Architecture
Purpose: For platform engineers, explains what openCenter Managed Kafka provides and how Strimzi maps to the blueprint model.
What openCenter Managed Kafka Is
openCenter Managed Kafka is a Strimzi-based Apache Kafka service delivered as a versioned blueprint on openCenter-managed Kubernetes. It provides an event backbone for application teams that need durable asynchronous messaging, decoupled integration, or streaming ingestion — without assembling the operator, security, and observability stack themselves.
The service follows ADR-004's blueprint model: a narrow, opinionated topology with GitOps lifecycle management, security hardening, air-gap packaging, and defined support boundaries.
Architecture
Strimzi and the CRD Model
Strimzi manages Kafka through Kubernetes Custom Resource Definitions (CRDs). The Strimzi Cluster Operator watches these CRDs and reconciles the actual cluster state to match the declared spec.
| CRD | Purpose |
|---|---|
Kafka | Declares the broker cluster: replicas, storage, listeners, TLS, auth |
KafkaTopic | Declares a topic: partitions, replication factor, retention config |
KafkaUser | Declares a client identity: authentication type, ACL rules |
KafkaConnect | Declares a Connect cluster (not included in v1 blueprint) |
All CRDs are managed declaratively through GitOps. Changes go through the FluxCD pipeline — no imperative kubectl apply against production.
Production Topology
The v1 blueprint defines a 3-broker HA topology:
- 3 Kafka brokers spread across failure domains (where infrastructure supports it)
- Persistent storage on approved storage classes with minimum IOPS qualification
- Internal listeners with TLS by default
- External exposure only through approved ingress/LB patterns
- JMX exporter sidecars for Prometheus metrics collection
Non-production environments can use a smaller topology (single broker) for cost efficiency, but this is not supported for production workloads.
Platform Dependencies
openCenter Managed Kafka relies on services already present in the platform:
| Dependency | Role |
|---|---|
| cert-manager | Issues and rotates TLS certificates for broker and client communication |
| Keycloak | Provides OIDC-based authentication; integrates with mTLS for client auth |
| Prometheus + Grafana | Collects JMX metrics from brokers; pre-built dashboards for broker health |
| Loki | Aggregates broker and operator logs |
| Harbor | Mirrors Strimzi and Kafka container images for air-gap deployments |
| Kyverno | Enforces pod security policies on Kafka workloads |
| FluxCD | Reconciles all Kafka CRDs from the GitOps repository |
Lifecycle Management
The Kafka blueprint follows the same GitOps lifecycle as all openCenter services:
- Blueprint values are defined in
openCenter-gitops-base - Customer overlays customize namespace, storage class, and sizing
- FluxCD reconciles the Strimzi operator and Kafka CR
- Upgrades follow a tested path: non-prod first, then production with a maintenance window
- Rollback is supported by reverting the Git commit that introduced the change
Shared Responsibility
openCenter owns the Kafka platform: deployment, broker health, monitoring, certificate rotation, patching, and upgrades. Customers own application use of the service: topic design, producer/consumer code, retention policies, and data semantics. Capacity planning is a shared responsibility.
This boundary is defined in ADR-004 and enforced through support tier agreements.
Further Reading
- Deploying Kafka — provision a cluster step by step
- Kafka Security — TLS, authentication, and ACLs
- Kafka Monitoring — metrics, dashboards, and alerts
- Kafka Topics — manage topics via KafkaTopic CRDs
- Portfolio Strategy — why Kafka is the first data service