Project Structure
CIVITAS/CORE V2 is spread across different repositories. This page shows which repository holds what, how the civitas-core-platform repository is organized, and where to look when you want to contribute. For further questions, see the Contribution Guide.
Repositories
| Repository | Content | Use it to |
|---|---|---|
civitas-core-platform | Source code of the platform's own components: Portal Backend, Portal Frontend, Config Adapter, authorization, Model Forge | Change or extend platform behavior |
civitas-core-deployment | Helmfile-based deployment, component charts, local Kubernetes development environment | Deploy the platform, add or update an integrated component |
apache-nifi-helm | Helm chart for Apache NiFiApache NiFiA stream processing and connector framework. In CIVITAS/CORE it is the pipeline engine for data integration and transformation (see ADR 047) and implements dataset-defined data flows., a fork of sakkiii/apache-nifi-helm | Change how NiFiApache NiFiA stream processing and connector framework. In CIVITAS/CORE it is the pipeline engine for data integration and transformation (see ADR 047) and implements dataset-defined data flows. is deployed |
docker-images | Build contexts and CI for custom container images. Images used by V2: superset, cicd, etcdctl, opa | Change or add a custom container image |
documentation | Source of this documentation (Docusaurus, versioned) | Change the documentation |
note
This project structure applies to CIVITAS/CORE V2 only. For version 1, see the V1 documentation.
For setting up a local environment, see Local Development Setup.
Structure of civitas-core-platform
| Directory | Purpose | Technology | Read more |
|---|---|---|---|
portal-backend/ | Management REST API (datasetsDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions., sources, structures, pipelines, IAM objects), KafkaApache KafkaA distributed event streaming platform. In CIVITAS/CORE it is used as the message bus to transport events, models and data in data flows. integration, host of Model Forge | Java 25, Spring Boot, Maven | Backend Architecture |
portal-model/ | Shared JPA entities, enums, and saga payloads, used by backend and Config Adapter | Java 25, Maven | Backend Architecture |
portal-frontend/ | Web UI of the platform | TypeScript, Next.js, React, pnpm | Frontend Architecture |
config-adapter/ | Configures the integrated components (KeycloakKeycloakAn open-source Identity and Access Management (IAM) solution providing SSO and OAuth2/OpenID Connect flows. In CIVITAS/CORE it is used to authenticate users and issue JWTs., APISIXApache APISIXAn open-source API gateway for traffic management, security and observability. In CIVITAS/CORE it is used as the centralized entrypoint to route and protect externally exposed APIs., FROST, GeoServer, PostGIS, NiFiApache NiFiA stream processing and connector framework. In CIVITAS/CORE it is the pipeline engine for data integration and transformation (see ADR 047) and implements dataset-defined data flows.) from KafkaApache KafkaA distributed event streaming platform. In CIVITAS/CORE it is used as the message bus to transport events, models and data in data flows. events | Java 25, Maven | Saga Pattern |
authz/ | OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. RegoRegoOPA's declarative policy language, used to express authorization rules such as checking role assignments at the most specific scope first. policies (authz/rego), AuthZ Repository (authz/repository), end-to-end tests | RegoRegoOPA's declarative policy language, used to express authorization rules such as checking role assignments at the most specific scope first., Java 25 | Open Policy Agent, Authorization Data Model |
model-forge/ | Embedded model and schema registry: contract, runtime, Spring Boot starter, admin UI | Java 25, Spring Boot | Model Management |
nifi-extensions/ | Custom Apache NiFiApache NiFiA stream processing and connector framework. In CIVITAS/CORE it is the pipeline engine for data integration and transformation (see ADR 047) and implements dataset-defined data flows. components, built as NAR (for example the FROST processor) | Java 21, Maven | Pipeline Engine |
api/ | Bruno collections for the Portal Backend API (requests and tests) | Bruno | |
dev-environment/ | Docker Compose environments and start scripts for working on the platform without a Kubernetes cluster | Docker Compose | |
shared-context/ | Shared guidelines, vendored from the agent-context repository | Markdown | |
.gitlab/ | CI configuration, one include per component | GitLab CI |
note
nifi-extensions builds with Java 21, because it follows the Java version of the Apache NiFiApache NiFiA stream processing and connector framework. In CIVITAS/CORE it is the pipeline engine for data integration and transformation (see ADR 047) and implements dataset-defined data flows. API. All other Maven modules use Java 25.
warning
Do not edit shared-context/ or the generated block between BEGIN shared-context and END shared-context in AGENTS.md. Both are copies. Change them in the agent-context repository.
How the modules depend on each other
In the CI pipeline, portal-model and config-adapter are built first and published to the Maven registry. Backend and authorization build afterwards.
Where do I change what?
| I want to … | Start in |
|---|---|
| Add or change a REST endpoint | portal-backend/src/main/java/de/civitascore/portal/controller and service |
| Change a shared entity or a saga payload | portal-model/ (affects backend and Config Adapter) |
| Change how a component such as KeycloakKeycloakAn open-source Identity and Access Management (IAM) solution providing SSO and OAuth2/OpenID Connect flows. In CIVITAS/CORE it is used to authenticate users and issue JWTs. or FROST is configured by the platform | config-adapter/config-adapter-<component> |
| Change an access rule | authz/rego/policy and its tests in authz/rego/test |
| Add or change a page in the UI | portal-frontend/src/app and src/components |
| Change the model registry | model-forge/ |
| Try the platform locally without Kubernetes | dev-environment/start-portal-dev.sh |
| Change the Helm deployment or add a component | civitas-core-deployment (components/) |
| Change the NiFiApache NiFiA stream processing and connector framework. In CIVITAS/CORE it is the pipeline engine for data integration and transformation (see ADR 047) and implements dataset-defined data flows. Helm chart | apache-nifi-helm repository |
| Change a custom container image such as Superset or OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. | docker-images repository, <image>/Dockerfile |
| Change this documentation | documentation repository, docs_v2/ |
Shared conventions
- Default branch:
develop. Changes go in through merge requests, see Contribute. - Code style: Java is formatted with Spotless (Google Java Format), see the Java Style Guide. Frontend rules are in the Frontend Style Guide.
- Tests: see the Testing Concept.
- Architecture: the Platform Architecture shows the components at runtime. This page shows where their code lives.