Version: Next

Inventory Customization Guide

This guide is organized from the top of the inventory to the bottom and has the goal to give a complete description for all fields. Many of the options can be left as defined by default. Important values will be specifically highlighted.

Configuration

We start with the explanation right at the beginning of the inventory file.

General settings

In the first section, three very important settings are defined.

    # (Sub-)Domain for the Installation
    # DOMAIN: "staging.example.com"
    DOMAIN: "ccp.urban-data.cloud"

    # Name of the envrionment where to deploy to
    #   - 'prd' (for production environment)
    #   - 'stg' (for staging environment)
    #   - 'dev' (for development environment)
    ENVIRONMENT: 'ccp'

    # Name of the kubeconfig file
    kubeconfig_file: urban-data-cloud.yaml

The DOMAIN setting defines the (sub-)domain, which points to the cluster's external IP-Address, which in turn must be reachable from the internet. This IP MUST be reachable for this version of the platform, because we use the services of Let's Encrypt to deploy the SSL certificates for the services automatically.

The domain should be configured as a Wildcard A-Record to the IP. If no wildcard is configured, the domain and the respective server names be configured one by one.

The ENVIRONMENT setting defines an identifying prefix for the current deployment. As stated in the comment, it can be used to separate different stages of the same installation. Another usage can be to separate different customer installations in the same cluster.

The kubeconfig_file setting defines the name of the kubeconfig file which is used to connect to the cluster. The file must be located in the ~/.kube directory.

Local execution setup

This part is configured to match the machine, the ansible-playbook is executed on. If you run it from your local machine, you only need to verify the location of your python3 binary.

    controller:
      hosts:
        localhost:
          ansible_host: 127.0.0.1
          ansible_connection: local
          ansible_python_interpreter: /usr/bin/python3

For example on a UNIX machine the which command will give you this information:

$ which python3
/usr/bin/python3

Email Server

The email server settings define the email server connection for outgoing emails from the platform. These are configured in several platform components.

IMPORTANT: If you need to open firewall settings to allow email sending, you must open port 587.

          server: "<email_server>"
          user: "<user>"
          password: "<password>"
          email_from: "<email-from>"

Kubernetes environment settings

The next section defines the settings for the kubernetes environment.

        ## Kubernetes general settings
        inv_k8s:
          config:
            context: ""
          ingress_class: nginx
          ### Storage class for k8s PersistentVolumes 
          storage_class:
            rwo: ""
            rwx: ""
            loc: ""
          gitlab_access:
            user_email: ""
            user: ""
            token: ""
          cert_manager:
            le_email: ""
            staging_issuer_name: ""
            prod_issuer_name: ""

The setting context defines which configured context of the kubeconfig file should be used.

The setting ingress_class defines the ingress class which should be used.

IMPORTANT: The ingress controller must be installed on the cluster and use an nginx controller.

The storage_class settings define the storage classes which should be used for the specified access modes. rwx is currently not used in the playbook. loc is currently assigned to the database volumes. If your cluster supports local storage provisioning, then you can use this to specify the matching class here, which could provide more performance.

The gitlab_access defines the user and token for the gitlab deployment. The data is used to authenticate the image pull process of some deployments.

The cert_manager defines the settings for the cert-manager. The email is used, when Let's Encrypt needs to contact you for the certificate renewal information.

Common settings

For the following sections a few fields are reoccurring, like the ones in the following example.

          enable: true
          ns_create: true
          ns_name: "{{ ENVIRONMENT }}-access-stack"
          ns_kubeconfig: "{{ kubeconfig_file }}"

These fields have always the same meaning:

enable: This is general flag to enable or disable a section of the deployment/configuration.
ns_create: This flag tells the script if the k8s-namespace shall be created (set it to false, if you already have namespaces set up).
ns_name: The desired k8s-namespace for related components to be deployed. This field point as default to {{ ENVIRONMENT }}-access-stack and can be kept as is in most caes.
ns_kubeconfig: The kubeconfig for the specific namespace. This setting is only needed, if you have to separate this in you installation. If you have a general kubeconfig for the cluster, you can use {{ kubeconfig_file }}, which refers to the previously defined variable from the cluster-setup.

Managed databases

This section describes the databases that are expected to be existing on the cluster when you want the hoster to provide managed databases. If you do enable the managed databases, the in-cluster-database-deployments (see below) should be deactivated.

The following example shows all required attributes. The structure is the same for all required databases:

keycloak_postgres
frost_postgres
geodata_postgres
superset_postgres
superset_upload_postgres
stellio_postgres
stellio_subscription_postgres
quantumleap_postgres

        inv_mngd_db:
          enable: "true"
          ns_kubeconfig: "{{ kubeconfig_file }}"
          keycloak_postgres:
            enable: false
            ns_name: "dev-databases"
            db_address: "dev-psql.dev-databases.svc.cluster.local"
            db_name: "keycloak"
            user_k8s_secret: "keycloak.dev-psql.credentials.postgresql.acid.zalan.do"
            admin_k8s_secret: "postgres.dev-psql.credentials.postgresql.acid.zalan.do"

ns_name: The name of the k8s-namespace the secrets are stored for accessing the databases.
db_address: The "path" of the k8s-service pointing to the database.
db_name: The given name of the database inside the Postgres cluster.
user_k8s_secret: The name of the k8s-secret containing the user credentials for database usage.
admin_k8s_secret: The name of the k8s-secret containing the database-admin credentials.

Operation Stack

The operation stack will be the place, where all management components will live. You can disable components which are provided by your hoster.

       inv_op_stack:
          ns_create: true
          ns_name: "{{ ENVIRONMENT }}-operation-stack"
          ns_kubeconfig: "{{ kubeconfig_file }}"
          prometheus:
            enable: false
          postgres_operator:
            enable: false
            ns_name: "{{ ENVIRONMENT }}-operation-stack"
            logical_backup:
              enabled: false
              access_key: ""
              bucket: ""
              endpoint: ""
              region: ""
              secret: ""
              schedule: ""
              retention_time: "" # S3 retention time for stored backups for example "2 week" or "7 days"   
          keel_operator:
            enable: false
            ns_name: "{{ ENVIRONMENT }}-operation-stack"
            admin: "admin@{{ DOMAIN }}"
            password: ""
          velero:
            enable: false
            ns_name: "{{ ENVIRONMENT }}-operation-stack"
            velero_namespace_suffix: "backup"
            backup:
              location_name: ""
              access_key: ""
              bucket: ""
              region: ""
              endpoint: ""
              secret: ""
          pgadmin:
            enable: true
            default_email: "admin@{{ DOMAIN }}"
            default_password: ""
            config_enhanced_cookie_protection: true
          monitoring:
            enable: true
            ns_name: "{{ ENVIRONMENT }}-operation-stack"
            prometheus:
              enable: true
              replicas: 1
              storage_size: 20Gi
              retention_days: 14d
            grafana:  
              enable: true
              deploy_default_dashboards: true
            alertmanager:
              enable: true
              replicas: 1
              storage_size: 5Gi
              ingress:
                enable: false
            loki:
              enable: true
              deploy_default_dashboards: true
              replicas: 1
              storage_size: 20Gi
              activate_log_retention: true # If set to false, logs will be stored indefinitely
              logs_retention_period: 24h # Should be specified in hours. Minimum is 24h. Is ignored if activate_log_retention is set to false 
              gateway_replicas: 1
            promtail:
              enable: true

The prometheus section is used to indicate, if prometheus is active in the cluster and the other components should activate metrics endpoints.

The postgres_operator is used to activate the operator itself. If you are on a cluster on your own, the enable setting should be true. On a shared cluster, the operator can only be deployed once. So you have to check if it should be installed or not. If it is installed, you have to provide S3 Connection parameter in order to use the logical backup feature of the operator. Later in this file you find a parameter create_logical_backup, which lets you decided if you specifically want to dis-/enable a backup for a database. Note: The schedule field follows the cronjob notation. In order to create/check your schedule you can use this website: crontab.guru

The keel_operator is used to activate the operator itself. If you are on a cluster on your own, the enable setting should be true. On a shared cluster, the operator can only be deployed once. So you have to check if it should be installed or not. Additionally you need to set username (which is defaulted to "admin@{{ DOMAIN }})" and a password, which can be used to access the user interface if needed.

The section velero enables a backup-restore tool which backups complete namespaces. Same as above, you may link it to any S3 storage by adding your information, which will be used to store these backups. The official documentation of velero can guide you through processes, like restoring a namespace.

The pgadmin section is used to activate pgAdmin in the cluster. Additionally you need to set username (which is defaulted to "admin@{{ DOMAIN }}") and a password, which can be used to access the user interface if needed.

The monitoring section is used to activate the monitoring stack. If you dont have a monitoring system already in your cluster, you should set the enable option to true, furthermore, you can enable and disable single components of the monitoring stack, depending on what is needed. Grafana is used to display statistics collected form the cluster, if you wish to use them, you should set grafana.enable to true. Under loki.logs_retention_period you can adjust how long logs for each pod in your cluster will be stored.

Central Database

The central database is a postgres database for all components using postgres as db backend.

        inv_central_db:
          enable: true
          ns_create: true
          ns_name: "{{ ENVIRONMENT }}-database-stack"
          pg_version: "14"
          replicas: 2
          storage_size: 10Gi
          enable_logical_backup: false

Beside the activation and namespace settings, you can define the database version and number of replicas. Also the storage size for each replica can be set.

With the enable_logical_backup you can decide, if you want to enable logical backups.

Access Management Stack

The access management section configures the identity and api management.

IMPORTANT: You need to carefully set your master_password in the "IDM Users" section here , as well as admin_role and viewer_role for apisix, and jwt_secret and password for dashboard.

Attention: The master_password requires at least one special character

        ## Access management stack
        inv_access:
          enable: true
          # When enabled, the base_domain should be "https://idm.{{ DOMAIN }}"
          # Else specify your external IDM Domain
          base_domain: "https://idm.{{ DOMAIN }}"

          ns_create: true
          ns_name: "{{ ENVIRONMENT }}-access-stack"
          # If this works, the code can be simplified
          db_ns_name: "{{ ENVIRONMENT }}-access-stack"
          ns_kubeconfig: "{{ kubeconfig_file }}"
          open_data_dataspace: ds_open_data

          ## IDM Users
          platform:
            admin_first_name: "Admin"
            admin_surname: "Admin"
            admin_email: "admin@{{ DOMAIN }}"
            master_username: "admin@{{ DOMAIN }}"
            master_password: ""
          ## IDM General Values
          keycloak:
            enable: true
            realm_name: "{{ ENVIRONMENT }}"
            k8s_secret_name: "{{ ENVIRONMENT }}-keycloak-admin"
            scope: "openid"
            enable_events: true
            enable_adminEvents: true
            log_level: "INFO"
            replicas: 1
            theme: "keycloak" # Use "ocp-custom" or, "keycloak" for basic theme
            #theme_url: "<theme-url>" # Use this parameter for the theme extension
            enable_logical_backup: false
          group:
            default_tenant_name: "default_dataspace"
          database:
            enable: false
            central: true
            replicas: 1
          apisix:
            enable: true
            ns_create: true
            replicas: 1
            ns_name: "{{ ENVIRONMENT }}-access-stack"
            ns_kubeconfig: "{{ kubeconfig_file }}"
            api_credentials:
              admin_role: ""
              viewer_role: ""
            dashboard:
              jwt_secret: "" #openssl rand -base64 12
              admin:  
                username: "admin@{{ DOMAIN }}"
                password: ""
            etcd:
              replicas: 1
            stream: # enable for MQTT Integration
              enable: true
              mqtt_service_type: "LoadBalancer" # "ClusterIP", "NodePort" or "LoadBalancer"
              mqtt_service_port: 1883
              mqtt_service_sni: "mqtt.{{ DOMAIN }}"
              enable_tls_for_mqtt: true # See Inventory Guide for additional instructions         
              dns01_wait_minutes: 5        
          apis: 
            import: true

The attribute open_data_dataspace configures the name of the open data dataspace of the platform. This is used to enable public access to data.

The platform section defines the general admin user for the platform, as well as the configuration of Keycloak itself.

The keycloak section describes the realm configuration. The field idm-secret-name defines the k8s-secret which is used to store the admin credentials. Additionally, it is possible to enable basic and admin events, which can for instance log certain login events or show an audit of the resource creation. More can be found in the Auditing section of the official documentation

The field default_tenant_name is used for the Stellio and acts as fiware-service. It is used to define dataspaces on the platform. Give it any name suitable for your first dataspace.

The database section allows to switch between using the central database or an own instance of a postgres cluster, indiviudally for keycloak. The replica attribute defines the number of replicas for the own database, if used.

The apisix section allows to enable or disable the API Management. The two replicas ttributes allow to control how many gateway or etcd instances are created.

Additionally the section of api_credetials defines tokes for the apisix management APIs.

The section of api_dashboard defines the credentials and the jwt secret for the dashboard.

The stream section is used to enable or disable the MQTT TCP Integration. The configuration of TLS is a half automated process. This is described in detail in the document MQTT Integration.

In the inventory the stream section allows the following parameters to be configured. mqtt_service_type determines the type how the service is connected to the internet. Options are the standard service type options of kubernetes services: ClusterIP, NodePort, LoadBalancer. The default is LoadBalancer. mqtt_service_type sets the public port of the service. The default is 1883 (Standard MQTT). For using TLS with MQTT the next two settings have to be set. mqtt_service_sni sets the host name that must be used in the Certificate for working with TLS. The default is "mqtt.{{ DOMAIN }}". enable_tls_for_mqtt enables TLS for MQTT. The default is true. dns01_wait_minutes sets the amount of time the script waits for the DNS Challenge. The default is 5.

With the last setting apis the initial configuration of potential apis can be activated. Each component of the platform, which can provide an external api, registers itself for configuration during playbook execution. This setting here defines, if these self-registrations are considered during playbook execution.

Context Management

For the context management section we split the documentation a bit. We provide the configuration of three different context management components. All three can be used on their own, so we describe the configuration separately.

SensorThings - FROST Server

          frost:
            enable: true
            ns_create: true
            ns_name: "{{ ENVIRONMENT }}-context-stack"
            ns_kubeconfig: "{{ kubeconfig_file }}"
            db:
              enable: false
              central: true
              enable_logical_backup: false
              pg_version: "14.7"
              replicas: 1
              storage_size: 10Gi
            http:
              replicas: 1
            mqtt:
              enable: true
              replicas: 1  

Beside the activation and namespace settings, you can define the database version and number of replicas. Also the storage size for each replica can be set.

With the enable_logical_backup you can decide, if you want to enable logical backups.

Additionally, you can decide if the mqtt broker of the FROST Server should be deployed and how many replicas the mqtt broker and http service should have.

NGSI-LD - Stellio Broker

          stellio:
            enable: true
            ns_create: true
            ns_name: "{{ ENVIRONMENT }}-context-stack"
            ns_kubeconfig: "{{ kubeconfig_file }}"
            helm_credentials:
              username: "kernblick-deploy-helmchart" # Gitlab username
              password: "gldt-GugWg8t4baf38LnY1WWv" # Create an access token in Gitlab with read_api
            replicas: 1
            db:
              enable: false
              central: true
              enable_logical_backup: false              
              pg_version: "14"
              replicas: 1
              storage_size: 10Gi
          quantumleap: 
            enable: true
            ns_create: true
            ns_name: "{{ ENVIRONMENT }}-context-stack"
            ns_kubeconfig: "{{ kubeconfig_file }}"
            replicas: 1          
            db:
              enable: false
              central: true
              enable_logical_backup: false              
              pg_version: "14"
              replicas: 1
              storage_size: 10Gi                 

Beside the activation and namespace settings, you can define the database version and number of replicas. Also the storage size for each replica can be set.

With the enable_logical_backup you can decide, if you want to enable logical backups.

Additionally you can decide how many replicas the Stellio broker should have.

If you use a private helm chart repository, you must specify here the login username and password for the helm chart repository.

For quantumleap - a component for persting the timeseries data in and additional database, the activation and namespace settings can be defined, you can define the database version and number of replicas. Also the storage size for each replica can be set.

With the enable_logical_backup you can decide, if you want to enable logical backups.

Additionally you can decide how many replicas Quantumleap should have.

Mimir

          mimir:
            enable: true
            ns_create: true
            ns_name: "{{ ENVIRONMENT }}-context-stack"
            ns_kubeconfig: "{{ kubeconfig_file }}"

The Mimir integration can be activated. More details in the configuration could come up later.

Dashboard Stack

We provide two alternative integrations of Grafana and Apache Superset as dashboard tools. These are described separately below.

Grafana

          grafana:
            enable: true
            admin: "admin"
            password: "inW92eOvnEOTE1gGHZTY"
            storage_size: 10Gi

Grafana can be activated with the enable option.

Set username and password (admin, password) for Grafanas internal admin. The credentials are used for api calls within the playbook. The end-user login is done using OIDC.

Additionally, the local storage size for Grafana can be set.

Apache Superset

          superset:
            enable: true
            mapbox_api_token: ""
            # Generate once per System with > openssl rand -base64 42
            db_secret: "Gc3bbMK9mtEpQBvki43Ezs0IDLXEMDGA98P1c8cnRcI1QqEN33NQ7x3j"
            admin_user_password: "a94pyJkZLtd2Djm!TTRe"
            admin_user_name: admin
            redis_auth_pasword: "E*FD4rVJqJ8Mc-ChzWWB"
            replicas: 1
            db:
              enable: false
              central: true 
              replicas: 1
              pg_version: "14"
              storage_size: 10Gi
              enable_logical_backup: false

Apache Superset can be activated with the respective enable option.

Optionally, set the mapbox_api_token to activate the mapbox integration.

Set username and password (admin_user_name, admin_user_password) for Supersets internal admin. The credentials are used for api calls within the playbook. The enduser login is done using OIDC.

Generate an redis_auth_pasword password, this will be used internally.

In the db section, you can define if the central database should be used. Beside the activation and namespace settings, you can define the database version and number of replicas for an own database instance. Also the storage size for each replica can be set.

With the enable_logical_backup you can decide, if you want to enable logical backups.

Additionally, you can decide how many replicas the Superset should have.

Geostack

The Geostack provides different features, which can be used independently. The feature blocks are describe in the following sections.

Beside the common activation settings, the general settings contain two further aspects:

          domain: "{{ DOMAIN }}"
          enable_apis: true

With the domain attribute the geostack domain domain names can be adjusted individually. The enable_apis attribute determines if the geostack API should be activated.

External IDM

If an external IDM should be used, the following section has to be configured.

          idm_integration:
            use_internal_idm: true
            configure_external_idm: false
            realm_name: ""
            host: ""
            client: ""
            secret: ""
            public_client: ""
            dataspace_claim: ""

The external IDM integration can be activated with the use_internal_idm option. To activate the external idm, set this false to false.

The next parameter configure_external_idm can be used to decide, if the external idm should be configured by the playbook. If you set it to false, you have to create the needed configuration manually.

The host attribute must be set to the idm host name. The realm_name has to be set to the idm realm, that should be used. The attributes client and secret have to be set to the idm client id and secret for a protected client in the idm. This is used for all OIDC configurations except to masterportal login.

The get the masterportal login working, you have to specify the public client id in the public_client attribute. This client must not have a client secret configured.

The dataspace_claim attribute has to be set to the matching dataspace claim in the generated tokes. By default the name "dataspaces" is used.

Masterportal configuration

          masterportal_1:
            enable: true
            image_project: "registry.gitlab.com/civitas-connect/civitas-core/geoportal-components/geoportal"
            image_tag: "main"
          masterportal_2:
            enable: true
            image_project: "registry.gitlab.com/civitas-connect/civitas-core/geoportal-components/betaportal"
            image_tag: "main"

By default the geostack can deploy two masterportal instances. One is the current 2.xx Version of the masterportal. The other one is the beta version of the 3.00 Version.

For both cases the image_project and image_tag have to be set. The image project is the url to the container image containing the masterportal. In the example the link points to the gitlab container registry. The image_tag is used to set the specific version of the image.

Own Database

          gisdb:
            enable: false
            central: true
            replicas: 1
            storageSize: 10Gi
            enable_logical_backup: false

In the gisdb section, you can define if the central database should be used. Beside the activation and namespace settings, you can define the database version and number of replicas for an own database instance. Also the storage size for each replica can be set.

With the enable_logical_backup you can decide, if you want to enable logical backups.

Mapfish configuration

          mapfish:
            enable: true
            image_project: "registry.gitlab.com/civitas-connect/civitas-core/geoportal-components/mapfish_print"
            image_tag: "staging"

Mapfish is used to generate pdf versions of the configured content in the masterportal. It can be enabled or disabled.

For mapfish the image_project and image_tag have to be set. The image project is the url to the container image containing the mapfish software and templates. In the example the link points to the gitlab container registry. The image_tag is used to set the specific version of the image.

Geoserver

          geoserver:
            enable: true
            prefix: ""
            geoserverUser: admin
            geoserverPassword: ""
            dataDirSize: 10Gi
            cacheDirSize: 10Gi
            s3access:
              region: ""
              user: ""
              password: ""
              endpoint: ""   

The geoserver deployment can be activated or deactivated.

The prefix is used to support multiple helm releases in parallel, if needed.

The geoserverUser and geoserverPassword have to be set to the admin user and password of the geoserver.

The dataDirSize and cacheDirSize have to be set to the size of the geoserver data directory and cache directory. The default values are 10GiB.

The s3access section has to be set enable authenticated s3 access for large raster files for the geoserver. The generated alias in the geoserver is s3://bucket/path.

Portal Backend configuration

          portal_backend:
            enable: true
            image_tag: "staging"
            image_project: "registry.gitlab.com/civitas-connect/civitas-core/geoportal-components/geoportal_backend"

The Portal Backend is used to general role-dependant masterportal configurations.

For Portal Backend the image_project and image_tag have to be set. The image project is the url to the container image containing the Portal Backend software and configuration. In the example the link points to the gitlab container registry. The image_tag is used to set the specific version of the image.

Addons

        inv_addons:
          import: true
          addons: []

To activate addons, clone the addon repository into the addons folder. Alternativly, you may copy or link the files into the addons folder. For an addon example, you may copy the files from the addons folder of the civitas_core deployment repository.

After that, activate the addon in your inventory by adding the following snippet to your inventory. Of course, you must adapt the addon name/path.

        inv_addons:
          import: true
          addons:
          - 'addons/mimir_addon/tasks.yml'

Status

The inventory will change with each additional feature of the platform. Currently we plan to make it easier to configure the platform, be defaulting and removing options from the inventory. So it is important to check this documentation for updates and changes.

Inventory Customization Guide

Configuration​

General settings​

Local execution setup​

Email Server​

Kubernetes environment settings​

Common settings​

Managed databases​

Operation Stack​

Central Database​

Access Management Stack​

Context Management​

SensorThings - FROST Server​

NGSI-LD - Stellio Broker​

Mimir​

Dashboard Stack​

Grafana​

Apache Superset​

Geostack​

External IDM​

Masterportal configuration​

Own Database​

Mapfish configuration​

Geoserver​

Portal Backend configuration​

Addons​

Status​

Configuration

General settings

Local execution setup

Email Server

Kubernetes environment settings

Common settings

Managed databases

Operation Stack

Central Database

Access Management Stack

Context Management

SensorThings - FROST Server

NGSI-LD - Stellio Broker

Mimir

Dashboard Stack

Grafana

Apache Superset

Geostack

External IDM

Masterportal configuration

Own Database

Mapfish configuration

Geoserver

Portal Backend configuration

Addons

Status