Configuration

This page is intended to help you to find the configuration options for further customisation from the default installation. You should already have a local copy of deployment repository from following the installation page.

The Deployment Directory

Your configuration is intended to fully live inside the deployment/ directory. Its structure can be copied from defaults/deployment/. It contains three helmfiles used for installation and a folder for each predefined helmfile environment. Environments separate different configurations (installations, customers, stages). You can choose which one to use with helmfile -e environment-name.

deployment
├── environments
│  ├── local
│  │  └── global.yaml.gotmpl
│  └── production
│      ├── global.yaml.gotmpl
│      └── component-override.yaml.gotmpl
├── helmfile-instance.yaml.gotmpl  # multi instance installation
├── helmfile-operators.yaml        # multi instance installation
└── helmfile.yaml                  # single instance installation

tip

The deployment directory is in .gitignore of the deployment repository. It can be safely used to version control your configuration independently.

warning

You should only modify files in the deployment/ directory. The defaults/ and components/ directories contain the upstream configuration and should not be changed.

To create a new environment, create a new folder with a global configuration file in it and add it to the helmfile. Each environment needs to be listed in the helmfile environments key and in the values for ../helmfile-root.yaml.gotmpl so that default values can be injected.

mkdir -v deployment/environments/my-custom-environment
touch deployment/environments/my-custom-environment/global.yaml.gotmpl

deployment/helmfile.yaml

---
environments:
  testing:
    values: []
  local:
    values: []
  production:
    values: []
  my-custom-environment:
    values: []

---
helmfiles:
  - path: "../helmfile-root.yaml.gotmpl"
    values:
      - environments:
          - testing
          - local
          - production
          - my-custom-environment

Global Configuration

Each environment should set its global values in global.yaml.gotmpl. You will probably only ever need to edit this file for your environment. If your use-case is not covered, then take a look at the component configuration section for more advanced configuration options.

deployment/environments/example/global.yaml.gotmpl

global:
    # domain with the following subdomains resolving to
    # the ingress controller configured below.
    # - api.domain.example
    # - idm.domain.example
    # - portal.domain.example
    # - dashboard.domain.example
    domain: civitas.test

    # unique identifier for this deployment used to derive namespace names
    instanceSlug: dev

    # if you create a smtp secret for keycloak before the initial deployment,
    # this user will receive his password via mail.
    # otherwise, you will need to reset the password via the admin user.
    initialUserEmail: "admin@civitas.test"

    # switch to production for:
    # - increased resource requests/limits
    # - increased replicaCount for high availability
    # - autoscaling
    # - pod disruption budgets
    # - info/warn level logging
    # - prometheus metrics
    # - stricter security and network policies
    profile: development

    # deploy each component in the same/its own namespace
    # if false, each component will get its own namespace prefixed with the instanceSlug
    singleNamespace: true
    createNamespaces: true

    ingress:
        # any ingress class that receives traffic for
        # the subdomains configured earlier.
        # no ingress controller specific annotations are used.
        ingressClass: "nginx"

        # `cert-manager.io/cluster-issuer` annotation for the ingress.
        # creates certificates for the subdomains configured earlier.
        # https://cert-manager.io/docs/usage/ingress/#how-it-works
        clusterIssuer: "selfsigned-ca"

    # service mesh integration, linkerd must be installed separately.
    # set `proxy.nativeSidecar` in linkerd to prevent init jobs from lingering.
    # https://linkerd.io/docs/features/native-sidecars/
    serviceMesh:
        enable: true
        # currently only linkerd is supported
        type: "linkerd"
        # add `linkerd.io/inject=enabled` annotation to namespaces
        patchNamespaces: true
        # allow mesh-external traffic to apisix.
        # enable this if your ingress controller is NOT meshed.
        allowUnauthenticatedIngress: false
        # alternatives: "cluster-authenticated" "all-authenticated" "all-unauthenticated" "audit"
        # https://linkerd.io/docs/reference/authorization-policy/#default-policies
        defaultInboundPolicy: "cluster-authenticated"

    # kyverno policies to further restrict access to operator-managed
    # resources and protect against label spoofing. if these cause issues,
    # you can either disable or set failureAction to "Audit".
    # see https://kyverno.io/docs/policy-types/cluster-policy/validate/#failure-action
    runtimePolicies:
      enabled: true
      failureAction: "Enforce"

    # control operator installation for multi-instance use cases.
    # only relevant if you use the operator/instance helmfiles.
    operators:
      # should be different for multi-instance to avoid instance deinstallation removing them
      # single-instance installations may opt to have all components in the same namespace
      namespace: "civitas-operators"
      watchAllNamespaces: true

    # expose metrics or install service/pod monitor CRDs where applicable
    metrics:
        enabled: false

# install only operator/instance components. empty means both.
# required for multiple instances to share the same cluster.
deployLayer: ""  # "operators" or "instance"

# name and order (matters for dependencies) of components to install.
# DO NOT SPECIFY this unless you need to remove components from this list.
# see defaults/environment/global.yaml for the defaults of your version.
components:
    - prepare
    - secrets
    - networkpolicies
    - runtime-policies
    - postgres
    - etcd
    - kafka
    - keycloak
    - authz
    - apisix
    - frost
    - nifi
    - config-adapters
    - portal
    - geoserver
    - valkey
    - superset

Component Configuration

In addition to the global configuration, more advanced configuration for each component can be made via its own root key. If you use a different filename different from global.yaml.gotmpl, you can access the values set in it.

Refer to components/<component>/default-environment.yaml.gotmpl for component-level default values. You override helm chart values for each part in component.part.rawValues.

warning

Avoid using rawValues, it bypasses the configuration hierarchy and can break components.

deployment/environments/example/postgres.yaml.gotmpl

# override replica count of app part from the keycloak component
keycloak:
  app:
    rawValues:
      replicaCount: 3

# override helm values directly for the cluster part of the postgres component
postgres:
  cluster:
    rawValues:
      cluster:
        postgresql:
          parameters:
            max_connections: "1000"

Runtime Policies

The runtime-policies deploys Kyverno ClusterPolicy resources that enforce runtime invariants, such as linkerd sidecar presence and protection against label spoofing. Disable them if you dont have kyverno and its CRDs installed or set their failure mode to Audit to only collect reports on violations without blocking anything.

ClusterPolicy objects are cluster scoped, with failureAction: "Enforce" (the default) they will block every matching resource (even independently installed ones) across all namespaces. See Security Hardening → Runtime Kyverno Policies for what this means on a shared cluster.

External Postgres Databases

You can use externally provisioned postgres databases instead of the bundles CloudNativePG operator. To do this, you can remove the postgres component from the components list and add the connection details of your external database in the environment config file. For this create a file inside the environment folder named databases.yaml.gotmpl and for every databases.yaml file inside the components/<component>/ folders copy everything, set embedded: false, set secret.generate: false and add the connection details of your external database. The secrets must be created manually in the cluster before deploying the platform.

Example for the portal component:

# databases.yaml.gotmpl
portal:
  name: 'portal'
  embedded: false
  user: 'portal'
  secret:
    name: 'db-portal'
    key: 'password'
    generate: false
  host: 'external-db-host.example.com'
  port: 5432

Be sure to create a authz-readonly-user user with read-only access to the portal database.

The following databases with extensions are expected to be provided by an external database:

Database Name	Required Extensions
flowable
frost	postgis, postgis_topology, fuzzystrmatch, postgis_tiger_geocoder
geoserver
keycloak
payload_data	postgis
portal
superset

External Kafka Clusters

note

To be documented...

Repository Structure

civitas-core-deployment/
├── deployment/                     # user configuration, not tracked by git
│   ├── helmfile.yaml               # single instance helmfile
│   ├── helmfile-instance.yaml
│   ├── helmfile-operators.yaml
│   └── environments/               # environment-specific values
│       └── local/
├── helmfile-root.yaml.gotmpl       # root helmfile global settings
├── helmfile-components.yaml.gotmpl # root helmfile component settings
├── defaults/                       # default configuration (do not modify)
│   ├── environment/                # templates applied to all environments
│   │   ├── global.yaml.gotmpl      # global settings
│   │   └── *.yaml.gotmpl           # cross-component value aggregation
│   └── deployment/                 # default deployment scaffolding (copy from here)
├── dev-deployment/                 # local cluster setup scripts
└── components/
    ├── prepare/                    # preparation hooks
    ├── secrets/                    # generates secrets for other components
    ├── networkpolicies/            # generates networkpolicies for other components
    ├── postgres/                   # CloudNativePG - manages postgres databases
    ├── etcd/                       # etcd key-value store
    ├── kafka/                      # Apache Kafka (Strimzi)
    ├── keycloak/                   # Identity & Access Management
    ├── authz/                      # Authorization (Open Policy Agent)
    ├── apisix/                     # API gateway
    ├── frost/                      # FROST-Server (OGC SensorThings API)
    ├── nifi/                       # Apache NiFi data pipelines
    ├── config-adapters/            # Configuration management
    ├── portal/                     # Web UI and backend API
    ├── geoserver/                  # Geospatial data server
    ├── valkey/                     # Valkey (Redis-compatible) cache
    └── superset/                   # Apache Superset BI dashboards

Helmfile processing order

CIVITAS/CORE v2 is divided into components (components/) that consist of individual helm-releases/parts. Your helmfile deployment/helmfile.yaml will parse the following/files directories in order to prepare the final values for all components. This two-stage process is required for values such as the component list to be available during cross-component value aggregation where a template parses configuration files from directories of other components.

• helmfile-root.yaml.gotmpl:
  • defaults/environment/global.yaml: untemplated default values
  • deployment/environments/<env>/global.yaml.gotmpl: user overrides
  • helmfile-components.yaml.gotmpl: aggregate values for component helmfiles
    • defaults/environment/*.yaml.gotmpl: cross-component value aggregation
    • deployment/environments/<env>/*.yaml.gotmpl: user overrides
    • components/*/helmfile.yaml.gotmpl: template each component with final values

Configuration Precedence

The following configuration files determine the final values passed to components and parts (precedence lowest to highest). Two stages are required because the default/aggregated component values can depend on the global settings from the first stage. Except for user overrides, the configurations should be adding keys and not remove them.

Component

1. defaults/environment/global.yaml: global defaults (domain, profile, components list)
2. deployment/environments/<env>/global.yaml.gotmpl: user overrides
3. defaults/environment/*.yaml.gotmpl aggregates/derives across components:
   • components/<components>/default-environment.yaml.gotmpl: component values
   • components/<components>/charts.yaml: charts used by parts
   • components/<components>/images.yaml: images
   • other component specific aggregations (dbs, routes, users, ...)
4. deployment/environments/<env>/*.yaml.gotmpl: user overrides

Part

1. components/<name>/values/<part>/base-values.yaml.gotmpl: part defaults
2. components/<name>/values/<part>/<profile>-values.yaml.gotmpl: part profile overrides
3. $.Values.<component>.<part>.rawValues: user overrides

Component Directory Layout

In addition to helmfile-components.yaml.gotmpl, each component directory contains:

• civitas-component.yaml: meta-information about this component and its parts
• default-environment.yaml.gotmpl: value export for cross-component value aggregation

The component entrypoint helmfile is responsible for rendering the parts of the component from:

• charts.yaml: chart versions and reference (either remote or in-tree under charts/)
• images.yaml: image versions for the charts
• values/<part>/: default helm values for each part

Other files for cross-component values aggregation.

Cross-Component Value Aggregation

The component directory can optionally contain various configuration files for e.g. keycloak accounts or database names that will be aggregated by the templates in deployment/environments/*.yaml.gotmpl and finally be passed to all components.

For example, each components images.yaml file specifies the container images and tags used by each part. However, images.yaml is not parsed by the component itself. Images are aggregated across components in helmfile-components.yaml.gotmpl which, after applying user overrides, passes the image versions back to all components.

You can verify this by running the following command and finding the images for e.g. parts of the apisix component.

helmfile -f deployment/helmfile.yaml print-env |
    sed -n '\_components/apisix/helmfile.yaml.gotmpl_,/---/p'

filePath: ./helmfile-components.yaml.gotmpl
name: default
values:
  # [...]
  images:
    apisix:  # component
      apisix:  # part
        gateway:
          repository: apache/apisix
          tag: 3.17.0-ubuntu
      configuration:  # part
        repository: alpine/curl
        tag: 8.20.0
  # [...]

These values will later be used by the component helmfile via the parts helm values, adapting it to the chart-specific value structure:

components/apisix/values/apisix/base-values.yaml.gotmpl

image:
  repository: {{ .Values.images.apisix.apisix.gateway.repository }}
  tag: {{ .Values.images.apisix.apisix.gateway.tag }}
  pullPolicy: IfNotPresent