Version: 2.0-rc2

ADR 031: Orchestrated Saga for Multi-Adapter Provisioning

Date: 2026-02-11

Status: Reviewed

Decision Makers: Architecture Board

Context

The Config-Adapter Framework currently handles independent, stateless operations: each adapter receives a command event, configures its target system, and publishes a result event. This works well for isolated tasks like creating a Keycloak realm or an APISIX route.

However, Datasset / Dataspace provisioning introduces sequential, cross-adapter dependencies:

Step	System	Action	Output	Depends On
1	FROST Server	Create project	`projectId: "proj-123"` → internal URL	None
2	APISIX Gateway	Create read-only route	External URL → internal FROST URL	`projectId` from step 1
3	Redpanda Connect	Deploy pipelines	Pipeline writing to FROST project	URL from step 1, route from step 2

If step 3 fails after steps 1 and 2 succeeded, all previous changes must be rolled back to maintain consistency.

Key Constraints

Adapters must remain simple command handlers without workflow knowledge
Rollback must guarantee eventual consistency across all involved systems
The solution must scale to additional backends (NGSI-LD/Stellio, MinIO, ...) without modifying existing adapters
Saga state must survive process restarts
The existing resultTopic mechanism in ConfigEvent must be preserved

Example Scenario

Dataset "Zählerdaten Stadtwerke" (Dataspace-ID: ds-stadtwerke-zaehler)

Contains two pipelines:

DB-Pipeline: Reads meter master data from a database, transforms to OGC STA entities, writes to FROST Server

MQTT-Pipeline: Receives meter readings via MQTT, transforms to OGC STA Observations, writes to FROST Server

Provisioning requires FROST project creation → APISIX route creation → Redpanda pipeline deployment, in strict order.

Why Not Choreography?

An initial Choreography-Based Saga proposal was evaluated. With ≥3 sequential steps, choreography leads to:

Implicit workflow coupling: Each adapter must know its predecessor and successor in the chain
Cascading compensation: Rollback propagates backwards through the entire chain; if any compensation fails, subsequent compensations never trigger
Distributed routing logic: The APISIX adapter must distinguish FROST vs. Stellio predecessors and route compensations accordingly — violating Single Responsibility
Bloated event payloads: The initial event carries configuration for all adapters, violating Information Hiding

A detailed analysis (last comment) documents these problems with sequence diagrams and code examples.

Checked Architecture Principles

[full] Model-centric data flow
[full] Distributed architecture with unified user experience
[full] Modular design — Orchestrator is a separate module; adapters remain independent
[full] Integration capability through defined interfaces — Uses existing ConfigEvent/ConfigResultEvent contracts
[full] Open source as the default — Built on Apache Kafka
[full] Cloud-native architecture — Stateless adapters, persistent saga state
[full] Prefer standard solutions over custom development — Saga Orchestration is a well-established pattern
[partial] Self-contained deployment — Adds one additional deployment unit (orchestrator module)
[full] Technological consistency to ensure maintainability — Same Java 21, Maven, Kafka stack
[full] Multi-tenancy — Saga context carries dataspace/tenant identifiers
[full] Security by design — No new attack surface; uses existing Kafka ACLs

Decision

Introduce a dedicated config-adapter-orchestrator module that centrally manages multi-step provisioning workflows using the Orchestrated Saga pattern.

Architecture

                    Portal Backend
                         │
                         │ dataspace.create.requested
                         ▼
              ┌───────────────────────┐
              │      Orchestrator     │
              │                       │
              │    Saga State (DB)    │
              │    Workflow Engine    │
              │    Compensation Mgr   │
              └──┬───────┬────────┬───┘
                 │       │        │
   frost.project.│   apisix.route.│  redpanda.pipeline.
   create/delete │   create/delete│  deploy/delete
                 │       │        │
                 ▼       ▼        ▼
              ┌──────┐┌──────┐┌────────┐
              │FROST ││APISIX││Redpanda│
              │Adapt.││Adapt.││Adapt.  │
              └──────┘└──────┘└────────┘
                 │        │        │
                 │  result events  │
                 └────────┴────────┘
                          │
                          ▼
               ┌─────────────────────┐
               │    Orchestrator     │
               │  (updates saga)     │
               └─────────────────────┘
                          │
                          │ dataspace.create.completed / .failed
                          ▼
                     Portal Backend

Orchestrator Responsibilities

The orchestrator is the only component with workflow knowledge:

Responsibility	Description
Workflow execution	Executes steps sequentially, passing outputs from one step as inputs to the next
Saga state management	Persists step results (resource IDs, URLs) to survive restarts
Data mapping	Builds adapter-specific commands from previous step outputs (e.g., `projectId` → APISIX upstream URL)
Compensation	On failure, triggers DELETE commands for all successfully completed steps — in parallel
Status reporting	Reports final success/failure to the Portal Backend via a single result event

Adapter Responsibilities

Adapters remain pure command handlers. They do not know whether a command is part of a saga, a manual operation, or a compensation:

Adapter receives: frost.project.create {name: "my-dataset"}
Adapter returns:  frost.project.result  {status: SUCCESS, resourceId: "proj-123",
                                         resultData: {"projectId": "proj-123",
                                                      "baseUrl": "http://frost/v1.1/projects/proj-123"}}

Adapter receives: frost.project.delete {projectId: "proj-123"}
Adapter returns:  frost.project.result  {status: SUCCESS}

The adapter does not know the delete is a rollback. It is a regular delete operation.

Success Flow

Backend → Kafka:  dataspace.create.requested {backendType: "FROST", ...}
Orchestrator creates saga, persists state
Orchestrator → FROST Adapter:   frost.project.create
FROST Adapter → Orchestrator:   frost.project.result {projectId, baseUrl}
Orchestrator persists projectId, builds APISIX command
Orchestrator → APISIX Adapter:  apisix.route.create {upstream: baseUrl, uri: ...}
APISIX Adapter → Orchestrator:  apisix.route.result {routeId}
Orchestrator persists routeId, builds Redpanda command
Orchestrator → Redpanda Adapter: redpanda.pipeline.deploy {targetUrl, pipelineJson}
Redpanda Adapter → Orchestrator: redpanda.pipeline.result {pipelineId}
Orchestrator → Backend:         dataspace.create.completed

Failure Flow (Redpanda Fails)

Saga State after step 7:
  FROST:    SUCCESS  (proj-123)
  APISIX:   SUCCESS  (route-456)
  Redpanda: FAILED   (connection refused)

Orchestrator triggers parallel compensation:
  → apisix.route.delete  {routeId: "route-456"}
  → frost.project.delete {projectId: "proj-123"}

Both compensations complete independently.
Orchestrator → Backend: dataspace.create.failed {reason, compensated: true}

Workflow Definition (Conceptual)

workflows:
  dataspace-create-frost:
    trigger: "dataspace.create.requested"
    condition: "payload.backendType == 'FROST'"
    steps:
      - name: create-frost-project
        command: "frost.project.create"
        input:
          projectName: "{{payload.dataspaceName}}"
        output:
          projectId: "{{result.resultData.projectId}}"
          baseUrl: "{{result.resultData.baseUrl}}"
        compensate:
          command: "frost.project.delete"
          input:
            projectId: "{{steps.create-frost-project.output.projectId}}"

      - name: create-apisix-route
        command: "apisix.route.create"
        input:
          uri: "/api/dataspace/{{payload.dataspaceId}}/*"
          upstreamUrl: "{{steps.create-frost-project.output.baseUrl}}"
          methods: ["GET"]
        output:
          routeId: "{{result.resultData.routeId}}"
        compensate:
          command: "apisix.route.delete"
          input:
            routeId: "{{steps.create-apisix-route.output.routeId}}"

      - name: deploy-pipelines
        command: "redpanda.pipeline.deploy"
        input:
          pipelineJson: "{{payload.pipelineJson}}"
          targetUrl: "{{steps.create-frost-project.output.baseUrl}}"
        compensate:
          command: "redpanda.pipeline.delete"
          input:
            pipelineId: "{{steps.deploy-pipelines.output.pipelineId}}"

Adding a new backend (e.g., Stellio) means adding a new workflow definition — no existing adapter code changes.

Required API Extension

The ConfigResultEvent needs one additional field to carry structured result data:

// New field (backwards-compatible, nullable)
private Map<String, String> resultData;

Existing adapters that do not set resultData continue to work via the existing resourceId field.

Topic Structure

Topic	Publisher	Subscriber	Purpose
`core.civitas.dataspace.create.requested`	Portal Backend	Orchestrator	Trigger provisioning workflow
`core.civitas.frost.project.create`	Orchestrator	FROST Adapter	Create FROST project
`core.civitas.frost.project.delete`	Orchestrator	FROST Adapter	Delete FROST project (or compensate)
`core.civitas.frost.project.result`	FROST Adapter	Orchestrator	Step result
`core.civitas.apisix.route.create`	Orchestrator	APISIX Adapter	Create APISIX route
`core.civitas.apisix.route.delete`	Orchestrator	APISIX Adapter	Delete APISIX route (or compensate)
`core.civitas.apisix.route.result`	APISIX Adapter	Orchestrator	Step result
`core.civitas.redpanda.pipeline.deploy`	Orchestrator	Redpanda Adapter	Deploy pipeline
`core.civitas.redpanda.pipeline.delete`	Orchestrator	Redpanda Adapter	Delete pipeline (or compensate)
`core.civitas.redpanda.pipeline.result`	Redpanda Adapter	Orchestrator	Step result
`core.civitas.dataspace.create.completed`	Orchestrator	Portal Backend	Saga succeeded
`core.civitas.dataspace.create.failed`	Orchestrator	Portal Backend	Saga failed (with compensation status)

Saga State Machine

           ┌──────────┐
           │ PENDING  │
           └────┬─────┘
                │ first step started
                ▼
           ┌──────────┐
     ┌─────│EXECUTING │─────┐
     │     └──────────┘     │
     │ all steps done       │ step failed
     ▼                      ▼
┌─────────┐          ┌──────────────┐
│COMPLETED│          │ COMPENSATING │
└─────────┘          └──────┬───────┘
                            │
                ┌───────────┴───────────┐
                │                       │
                ▼                       ▼
         ┌────────────┐      ┌─────────────────────┐
         │COMPENSATED │      │ COMPENSATION_FAILED │
         │  (clean)   │      │(manual intervention)│
         └────────────┘      └─────────────────────┘

Limitation: Saga State Persistence

The orchestrator requires persistent storage for saga state. Options:

Storage	Pros	Cons
Kafka Streams state store	No external DB needed	Harder to query, operational complexity
PostgreSQL (shared with backend)	Proven, transactional, queryable	Shared DB dependency
Dedicated DB	Full isolation	Additional infrastructure

Recommendation: Use PostgreSQL (shared or dedicated), as the stack already includes it.

Consequences

Benefits

Adapters stay simple: No workflow knowledge, no compensation chains, no predecessor/successor awareness
Central workflow visibility: Saga state is queryable in one place — enables dashboards, debugging, and manual intervention
Parallel compensation: On failure, all successful steps are compensated simultaneously (not cascading)
Scalable to new backends: Adding Stellio/NGSI-LD = new workflow definition, zero adapter code changes
Existing adapters unchanged: APISIX and Keycloak adapters continue to work as-is
Idempotent by design: Orchestrator can safely retry failed steps or compensations using existing adapter idempotency

Trade-offs

Additional deployment unit: One new module (config-adapter-orchestrator) to build, deploy, and monitor
Saga state storage: Requires a database for persistence (PostgreSQL recommended)
Development effort: Orchestrator framework must be built (workflow engine, state management, compensation logic)
Single point of coordination: Orchestrator must be highly available; if it is down, no new sagas start (in-flight sagas resume on restart)

Affected Components

Component	Change	Impact
`config-adapter-api`	Add `resultData: Map<String, String>` to `ConfigResultEvent`	Backwards-compatible, minimal
`config-adapter-orchestrator`	New module	New development
`config-adapter-apisix`	None	Unchanged
`config-adapter-keycloak`	None	Unchanged
`config-adapter-application`	Wire orchestrator module	Minimal
Portal Backend	Send `dataspace.create.requested`, receive `completed`/`failed`	Minor
FROST Adapter (new)	Implement as simple command handler	New development
Redpanda Adapter (new)	Implement as simple command handler	New development

Alternatives Considered

Alternative	Decision	Rationale
Choreography-Based Saga	Rejected	Adapters accumulate workflow routing and compensation logic; cascading rollback breaks on partial failure; every new backend requires changes to existing adapters. See detailed analysis (TODO: add link).
Portal Backend as Orchestrator (Option A)	Deferred	Viable long-term, but requires significant backend changes for async Kafka processing and saga state management. The dedicated module can be migrated into the backend later. A standalone orchestrator has the benefit of decoupled extensibility and maintenance
Process engine (Redpanda)		Introduces heavyweight infrastructure dependency; overkill for the current number of workflows. Can be reconsidered if workflow complexity grows significantly.

Migration Path

Phase 1: Build config-adapter-orchestrator with saga state management and the FROST dataspace workflow
Phase 2: Refactor FROST adapters as simple command handlers. Create Redpanda adapter as simple command handler
Phase 3: Connect Portal Backend to orchestrator topics
Phase 4 (optional): Add Stellio/NGSI-LD workflow definition — existing adapters remain unchanged
Long-term (optional): Migrate orchestrator logic into the Portal Backend if architecturally desired

Context​

Key Constraints​

Example Scenario​

Why Not Choreography?​

Checked Architecture Principles​

Decision​

Architecture​

Orchestrator Responsibilities​

Adapter Responsibilities​

Success Flow​

Failure Flow (Redpanda Fails)​

Workflow Definition (Conceptual)​

Required API Extension​

Topic Structure​

Saga State Machine​

Limitation: Saga State Persistence​

Consequences​

Benefits​

Trade-offs​

Affected Components​

Alternatives Considered​

Migration Path​

References​