Message Bus
Overview
The CIVITAS/CORE platform uses Apache Kafka as its central message bus for asynchronous, event-driven communication between platform components. The message bus is a foundational infrastructure element that enables loose coupling, independent scalability, and resilient integration across the distributed service landscape.
Rationale
CIVITAS/CORE is a modular, cloud-native Smart City platform composed of multiple independently deployable services. As the number of services and external integrations grows, direct synchronous communication becomes a bottleneck for resilience and scalability.
The architecture follows these principles:
- Asynchronous by default -- State changes and cross-component integration flows are communicated over the message bus. Synchronous calls are reserved for use cases that require immediate request-response behavior (e.g., user-facing queries).
- Loose coupling -- Components publish and consume events without direct knowledge of each other. This allows independent evolution, deployment, and scaling.
- Least privilege -- Each component receives exactly the permissions it needs to fulfill its purpose, enforced through fine-grained topic-level authorization.
Why Kafka
Apache Kafka was selected based on the following criteria (see ADR 026):
| Criterion | Assessment |
|---|---|
| Scalability | Horizontal scaling for high message volumes (configuration and payload data) |
| Topic management | Fine-grained topic structure with per-topic access control |
| Authorization | Read/write permissions enforceable at topic level |
| Ecosystem | Broad SDK support across programming languages and frameworks |
| Community | Large open-source community providing long-term stability |
| Payload flexibility | Suitable for both small configuration events and larger data payloads |
Message Categories
The message bus handles two distinct categories of messages:
Configuration Messages
Configuration messages carry platform management operations such as user provisioning, route configuration, or dataset setup. They are distributed via dedicated Configuration Adapters and follow the CloudEvents standard as message envelope (ADR 013).
Key characteristics:
- Standardized CloudEvents envelope enabling routing based on event type
- Structured JSON payloads containing only the data required for the target operation
- Dedicated topics per tenant, domain, entity, and operation
- Asynchronous processing with explicit synchronization state tracking
Payload Data
Payload data messages carry domain-specific content (e.g., sensor data, geodata) distributed through pipeline bundles with context-specific topic structures. These are distinct from configuration messages in both purpose and processing model.
Event Envelope: CloudEvents
All configuration events use the CloudEvents specification as the message envelope (ADR 013). This provides:
- A standardized metadata structure across all event types
- Transport-independent event semantics
- When used with Kafka, the envelope can be mapped to message headers, enabling routing by event type without deserializing the payload
- Flexible support for different payload schemas
The platform defines its own event types on top of the CloudEvents standard. The concrete event definitions are specified in ADR 021.
Architecture Patterns
The message bus implementation uses two established distributed systems patterns to ensure reliability and consistency:
Transactional Outbox Pattern
The Outbox pattern decouples local persistence from event publication. Entity changes and their corresponding events are committed in a single database transaction. Events are then published to Kafka asynchronously, ensuring reliable delivery without blocking API responses.
Orchestrated Saga Pattern
For multi-step provisioning workflows that span multiple Config Adapters (e.g., creating a dataset requires FROST project setup, API gateway route, and pipeline deployment in sequence), the platform uses an orchestrated Saga. A dedicated orchestrator module manages workflow execution, state persistence, and compensating actions on failure.
Topic Naming and Configuration
Topics follow a structured naming convention that encodes tenant, domain, entity, and operation context. This enables fine-grained access control and clear semantic routing.
--> Topic Configuration -- Details
Authorization
An authorization concept is being developed to enforce fine-grained, role-based access control on the Kafka bus. The model introduces distinct principal types for config producers/consumers, data pipeline bundles, outbox relays, and saga coordinators -- each with explicit, least-privilege topic permissions.
--> Authorization Concept (Draft)
Technology Stack
| Component | Technology | Purpose |
|---|---|---|
| Message Bus | Apache Kafka | Central event transport |
| Event Envelope | CloudEvents | Standardized metadata and routing |
| Config Adapter SDK | Plain Java (minimal dependencies) | Adapter implementation library |
| Serialization | Jackson / JSON | Event payload encoding |
| Saga State Store | PostgreSQL | Workflow state persistence |
The CA SDK is implemented in plain Java with a minimal dependency set (ADR 016), avoiding heavyweight frameworks. Required integrations are provided through focused libraries: Kafka clients, CloudEvents SDK, Jackson, JAX-RS/Jersey.
Design Constraints
- Event versioning and schema governance are mandatory design concerns as the number of event types grows.
- Idempotent consumers must be implemented by all event handlers to support safe retries.
- Eventually consistent behavior must be reflected in UI design and operational procedures.
- Observability (event tracing, dead-letter handling, sync state monitoring) is required for production operation.
Related Architecture Decisions
| ADR | Title | Status |
|---|---|---|
| ADR 013 | Use CloudEvents Standard for Bus-based Configuration Communication | Accepted |
| ADR 016 | Configuration Adapter (CA) SDK Language | Reviewed |
| ADR 021 | Definition of Configuration Events | Reviewed |
| ADR 026 | Select Message Bus | Reviewed |
| ADR 030 | Asynchronous Outbox for Config Adapter Synchronization | Reviewed |
| ADR 031 | Orchestrated Saga for Multi-Adapter Provisioning | Reviewed |
| ADR 036 | Event-driven Communication and Loose Coupling | Reviewed |