PostgreSQL Artifact Registry

Overview

Model Forge persists all model artifacts in a PostgreSQL-backed artifact registry that it owns. The schema is created and migrated automatically by Flyway on startup; the application requires a datasource to run.

The registry stores:

• the logical identity of each artifact (format-independent)
• immutable artifact versions, each with an authored primary format
• per-version, per-format representations — JSON Schema, raw XSD, core-json
• the XSD namespace lookup
• the reference graph between artifacts

Reads are projections built at the boundary from the stored artifact document, not ORM entities — the artifact document remains canonical and no second representation can drift from it.

Design principles

• CORE URNs are the stable public identity of every artifact — the format is never part of the identity. The same Element may be authored as JSON Schema in one version and as XSD in another.
• Artifact payloads are canonical JSON documents, stored as jsonb.
• XSD payloads are stored as raw text; a JSON Schema representation is generated from them on read (and cached).
• Each version records its authored primary format; the format-specific content lives in artifact_representation (one row per format).
• Versioning is explicit, backend-owned and modelled in the database.
• References are stored relationally for fast dependency/impact queries, and the complete graph is kept — cycles included.
• Persistence access uses a thin JdbcClient repository layer; large dynamic JSON trees are stored as jsonb/text rather than decomposed into entity fields (no JPA/Hibernate).

Tables

The schema lives in the Flyway migrations under db/model-forge/migration; the DDL below reflects the combined result of all migrations.

artifact — logical identity

create table artifact (
  id              uuid primary key,
  logical_urn     text not null unique,
  artifact_type   text not null,   -- element | dataset | mapping | pipeline | datasource | datasink | datastructure
  name            text not null,
  title           text,
  description     text,
  current_version text,            -- the default read version
  created_at      timestamptz not null,
  updated_at      timestamptz not null
);

artifact_type is the functional CORE category. Identity is format-agnostic, so the format is not stored here. Both JSON Schema and XSD Elements have artifact_type = 'element'; the format is recorded per version (artifact_version.primary_format) and per representation (below). The logical_urn always uses the element segment; a :xsd: artifact-type URN is rejected.

Indexes: artifact_type, name, updated_at.

artifact_version — immutable version metadata

create table artifact_version (
  id               uuid primary key,
  artifact_id      uuid not null references artifact(id) on delete cascade,
  version          text not null,
  primary_format   text not null,   -- jsonschema | core-json | xsd
  title            text,            -- versioned metadata (set on rename)
  description      text,
  created_at       timestamptz not null,
  created_by       text,
  unique (artifact_id, version)
);

create index idx_artifact_version_artifact on artifact_version (artifact_id);

• A version records its authored (primary) format and points at its representations; the content lives in artifact_representation.
• primary_format selects the default representation a schema read returns when no explicit format is requested.
• title/description are versioned metadata set by a rename: a rename creates a new patch version with the new title, format-independently (so XSD versions are renamable too); older versions keep their own title. A read serves the version's title when set, else the URN name segment.
• artifact.current_version points at the latest version.

artifact_representation — per-version, per-format content

create table artifact_representation (
  id            uuid primary key,
  version_id    uuid not null references artifact_version(id) on delete cascade,
  format        text not null,   -- jsonschema | core-json | xsd
  content_type  text,
  content_jsonb jsonb,
  content_text  text,
  content_hash  text,
  generation    text not null default 'stored',  -- stored | generated
  created_at    timestamptz not null,
  unique (version_id, format)
);

• One row per format of a version. JSON content goes into content_jsonb; raw XSD goes into content_text.
• Each version has exactly one authored representation (generation = 'stored'), in its primary_format. A JSON Schema derived from an XSD may later be persisted with generation = 'generated'; a generated representation never counts as authored content.
• content_hash makes repeated imports of identical authored content idempotent (no spurious new version).
• unique (version_id, format) permits at most one representation per format per version; a different format in a new version is always allowed (v1 XSD, v2 JSON-only is valid).
• A GIN index on content_jsonb supports payload-level search.

artifact_reference — dependency edges

create table artifact_reference (
  id                 uuid primary key,
  from_version_id    uuid not null references artifact_version(id) on delete cascade,
  target_urn         text not null,
  target_artifact_id uuid references artifact(id) on delete set null,
  target_version_id  uuid references artifact_version(id) on delete set null,
  reference_type     text not null,  -- schema-ref | dataset-ref | pipeline-node | xsd-import | association-ref | mapping-source | mapping-target
                                     -- | datasource-element | datasink-element | datastructure-ref
  reference_name     text,
  sort_order         int,
  created_at         timestamptz not null
);

• The complete reference graph is stored, including cycles.
• target_urn is stored verbatim as authored — a pinned …:1.0.0 or the …:latest token — never normalised to the logical form.
• target_artifact_id resolves the reference to the target artifact's logical identity; target_version_id resolves a pinned reference to its concrete version (null for latest, logical, or a not-yet-imported pin). Both are nullable / on delete set null, and are back-filled when a previously-missing target artifact or target version is later created.

xsd_namespace — namespace lookup

create table xsd_namespace (
  namespace   text primary key,
  artifact_id uuid not null references artifact(id) on delete cascade,
  version_id  uuid references artifact_version(id) on delete set null,
  updated_at  timestamptz not null
);

A durable namespace→artifact index for XSD-backed Elements, so xs:import resolution does not depend on a startup scan completing.

Read projections

Reads are built from artifact plus the selected artifact_version: the authored content comes from artifact_representation (content_jsonb/content_text), the versioned title falls back to the URN name segment, and an XSD version is additionally readable as derived JSON Schema. The facade serves these as plain JSON documents (getArtifact/getBundledView/getInlinedView).

Reference extraction

Edges are derived from the artifact content when it is written and persisted as artifact_reference rows:

reference_type	Source
schema-ref	$ref values (CORE URNs) in a JSON Schema
xsd-import	xs:import/@namespace: a CORE-URN namespace links that Element directly (any format); a classic XML namespace is resolved via xsd_namespace
association-ref	an Element's concrete x-core-ref association targets (the by-reference counterpart to schema-ref's by-value $ref edges)
mapping-source / mapping-target	top-level source / target of a Mapping
pipeline-node	the sourceRef / sinkRef / mappingRef — and an enrich node's lookupSourceRef — of a Pipeline's top-level nodes[]
dataset-ref	every member of the DataSet manifest's *Refs arrays; reference_name carries the member kind
datasource-element / datasink-element	the element field of a DataSource / DataSink — its own payload Element binding
datastructure-ref	each $defs.*.$ref of a DataStructure — its member Elements

The DataSet manifest stays the canonical document; the relational rows are a derived index, written in the same transaction.

Versioning

Versioning is owned by Model Forge — clients never choose the next version directly, they declare how it should be bumped.

• The logical URN identifies the artifact independent of version; the versioned URN identifies one immutable version.

• The first version of a new artifact is 1.0.0.

• An update creates a new artifact_version; the number is computed from current_version and the requested bump:

  1.2.3 + patch -> 1.2.4
  1.2.3 + minor -> 1.3.0
  1.2.3 + major -> 2.0.0

• Older versions are never overwritten; current_version is the default read.

• The bump is request metadata, not part of the payload — a versionBump option (patch | minor | major) on the facade's update commands.

• A request targeting a versioned URN resolves the logical identity and bumps from current_version; the URN's version segment is ignored as the new number.

• A rename is a patch-version bump that records the new title/description as versioned metadata, format-independently — so XSD Elements are renamable too (the XSD content is unchanged); older versions keep their title.

• A read id (and a reference) may use the :latest token, resolving to the current (highest-SemVer) version. :latest is never a writable identity — a client-supplied :latest (like a legacy :xsd:) identity is rejected.

Model Forge performs no schema-compatibility checks and exposes no compatibility metadata; there is no compatibility surface on the facade.

Write flow

Every artifact write runs in one transaction:

1. Normalise the incoming ID to a logical CORE URN.
2. Validate the payload with the existing validation services.
3. Reject a :xsd: artifact-type URN (before any write).
4. Upsert artifact (insert on first sight; first version 1.0.0).
5. If the current version's authored representation in the same format has an identical content hash → no-op (idempotent). A different format is always a new version.
6. Otherwise compute the next SemVer version and insert a new artifact_version with primary_format = the written format.
7. Insert the version's authored artifact_representation (generation = 'stored').
8. Replace the version's artifact_reference rows from the extracted references.
9. Advance artifact.current_version.
10. For XSDs, upsert the xsd_namespace row (scoped to the version).

A storage failure rolls the whole write back and surfaces as an error to the calling host application.

Read flow

• Versioned URN → load that artifact_version.
• Logical URN → load artifact.current_version's version.
• Pick the representation: a schema read without a requested format returns the authored (primary) representation; requesting json-schema returns the JSON Schema (an XSD version is converted on read); requesting xsd returns the raw XSD (an error when the version has no XSD representation). A version's readable formats are its stored representations plus the derivable jsonschema for XSD versions.

Conditional requests (ETags)

The same content_hash that makes writes idempotent is exposed to callers: The registry keeps the content hash of the stored representation a read would serve (the requested format, or the authored format when omitted); a derived representation has no stored hash.

Model Forge itself has no HTTP surface, but a host application that exposes one can use this hash as a strong ETag ("sha256:<hex>") to implement HTTP conditional requests (If-None-Match → 304, If-Match → 412) on top of the facade in its own transport layer. Because the hash is over the serialised authored content it is byte-stable but not canonical: two semantically equal JSON documents with different key order can hash differently (acceptable for caching and lost-update detection).

Artifact lifecycle

The artifact lifecycle (draft/review/release status and similar) is entirely the host application's concern. Model Forge stores immutable versions and keeps no lifecycle state of its own; current_version is always the default read.

Dependency graph & cache

The durable source of dependency edges is artifact_reference. Two in-memory indexes serve runtime queries and are rebuilt from PostgreSQL on startup (on ApplicationReadyEvent, after Flyway has migrated):

• DependencyGraphService — a version-precise graph whose nodes are versioned URNs and whose edges are kept verbatim (pinned or :latest); rebuilt from each artifact's current version on startup. It serves dependencies and dependents queries — reporting concrete versioned IDs (a logical or :latest query resolves to the current version) — and tolerates cycles. It survives a restart because the data lives in PostgreSQL.

Service boundary

The single seam between the domain services and storage is the ArtifactRegistry port; PostgresArtifactRegistryClient is its only implementation. Model Forge always runs against a real Postgres registry — without a datasource the starter wires no registry and no facade.

Domain behaviour (validation, URN creation, reference extraction, graph rebuild, inlined/bundled views) stays in the services.

Implementation notes

• Access: a thin JdbcClient repository layer; jsonb is written with cast(:json as jsonb) and read back as text. No JPA/Hibernate.
• Migrations: plain-SQL Flyway migrations under db/model-forge/migration.
• Search: scalar metadata (URN, name, title, description, type) first; a GIN index on content_jsonb is available for payload-level queries.
• Tests: the adapter's outage translation and transaction wrapping are unit-tested with a mocked JdbcClient (PostgresArtifactRegistryClientTest); the admin-ui's AdminUiApplicationTest boots the full stack against a Testcontainers PostgreSQL (skipped when no Docker daemon is available).