Skip to main content
Version: V2-Next

PostgreSQL Artifact Registry

Overview​

Model Forge persists all model artifactsModel ArtifactUmbrella term for any persistable CORE unit: Element, DataStructure, Mapping, DataSource, DataSink, Pipeline and DataSet. Every artifact has a stable CORE URN identity and is versioned. in a PostgreSQL-backed artifact registry that it owns. The schema is created and migrated automatically by Flyway on startup; the application requires a datasource to run.

The registry stores:

  • the logical identity of each artifact (format-independent)
  • immutable artifact versions, each with an authored primary format
  • per-version, per-format representations — JSON Schema, raw XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD., core-json
  • the XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. namespace lookup
  • the reference graph between artifacts

Reads are projections built at the boundary from the stored artifact document, not ORM entities — the artifact document remains canonical and no second representation can drift from it.

Design principles​

  • CORE URNsCORE URNThe global identity Model Management assigns to every model artifact: a unique, stable and versionable URN that exists independently of storage location, file names or registry technology. are the stable public identity of every artifact — the format is never part of the identity. The same Element may be authored as JSON Schema in one version and as XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. in another.
  • Artifact payloads are canonical JSON documents, stored as jsonb.
  • XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. payloads are stored as raw text; a JSON Schema representation is generated from them on read (and cached).
  • Each version records its authored primary format; the format-specific content lives in artifact_representation (one row per format).
  • Versioning is explicit, backend-owned and modelled in the database.
  • References are stored relationally for fast dependency/impact queries, and the complete graph is kept — cycles included.
  • Persistence access uses a thin JdbcClient repository layer; large dynamic JSON trees are stored as jsonb/text rather than decomposed into entity fields (no JPA/Hibernate).

Tables​

The schema lives in the Flyway migrations under db/model-forge/migration; the DDL below reflects the combined result of all migrations.

artifact — logical identity​

create table artifact (
id uuid primary key,
logical_urn text not null unique,
artifact_type text not null, -- element | dataset | mapping | pipeline | datasource | datasink | datastructure
name text not null,
title text,
description text,
current_version text, -- the default read version
created_at timestamptz not null,
updated_at timestamptz not null
);

artifact_type is the functional CORE category. Identity is format-agnostic, so the format is not stored here. Both JSON Schema and XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. Elements have artifact_type = 'element'; the format is recorded per version (artifact_version.primary_format) and per representation (below). The logical_urn always uses the element segment; a :xsd: artifact-type URN is rejected.

Indexes: artifact_type, name, updated_at.

artifact_version — immutable version metadata​

create table artifact_version (
id uuid primary key,
artifact_id uuid not null references artifact(id) on delete cascade,
version text not null,
primary_format text not null, -- jsonschema | core-json | xsd
title text, -- versioned metadata (set on rename)
description text,
created_at timestamptz not null,
created_by text,
unique (artifact_id, version)
);

create index idx_artifact_version_artifact on artifact_version (artifact_id);
  • A version records its authored (primary) format and points at its representations; the content lives in artifact_representation.
  • primary_format selects the default representation a schema read returns when no explicit format is requested.
  • title/description are versioned metadata set by a rename: a rename creates a new patch version with the new title, format-independently (so XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. versions are renamable too); older versions keep their own title. A read serves the version's title when set, else the URN name segment.
  • artifact.current_version points at the latest version.

artifact_representation — per-version, per-format content​

create table artifact_representation (
id uuid primary key,
version_id uuid not null references artifact_version(id) on delete cascade,
format text not null, -- jsonschema | core-json | xsd
content_type text,
content_jsonb jsonb,
content_text text,
content_hash text,
generation text not null default 'stored', -- stored | generated
created_at timestamptz not null,
unique (version_id, format)
);
  • One row per format of a version. JSON content goes into content_jsonb; raw XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. goes into content_text.
  • Each version has exactly one authored representation (generation = 'stored'), in its primary_format. A JSON Schema derived from an XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. may later be persisted with generation = 'generated'; a generated representation never counts as authored content.
  • content_hash makes repeated imports of identical authored content idempotent (no spurious new version).
  • unique (version_id, format) permits at most one representation per format per version; a different format in a new version is always allowed (v1 XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD., v2 JSON-only is valid).
  • A GIN index on content_jsonb supports payload-level search.

artifact_reference — dependency edges​

create table artifact_reference (
id uuid primary key,
from_version_id uuid not null references artifact_version(id) on delete cascade,
target_urn text not null,
target_artifact_id uuid references artifact(id) on delete set null,
target_version_id uuid references artifact_version(id) on delete set null,
reference_type text not null, -- schema-ref | dataset-ref | pipeline-node | xsd-import | association-ref | mapping-source | mapping-target
-- | datasource-element | datasink-element | datastructure-ref
reference_name text,
sort_order int,
created_at timestamptz not null
);
  • The complete reference graph is stored, including cycles.
  • target_urn is stored verbatim as authored — a pinned …:1.0.0 or the …:latest token — never normalised to the logical form.
  • target_artifact_id resolves the reference to the target artifact's logical identity; target_version_id resolves a pinned reference to its concrete version (null for latest, logical, or a not-yet-imported pin). Both are nullable / on delete set null, and are back-filled when a previously-missing target artifact or target version is later created.

xsd_namespace — namespace lookup​

create table xsd_namespace (
namespace text primary key,
artifact_id uuid not null references artifact(id) on delete cascade,
version_id uuid references artifact_version(id) on delete set null,
updated_at timestamptz not null
);

A durable namespace→artifact index for XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD.-backed Elements, so xs:import resolution does not depend on a startup scan completing.

Read projections​

Reads are built from artifact plus the selected artifact_version: the authored content comes from artifact_representation (content_jsonb/content_text), the versioned title falls back to the URN name segment, and an XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. version is additionally readable as derived JSON Schema. The facade serves these as plain JSON documents (getArtifact/getBundledView/getInlinedView).

Reference extraction​

Edges are derived from the artifact content when it is written and persisted as artifact_reference rows:

reference_typeSource
schema-ref$ref values (CORE URNsCORE URNThe global identity Model Management assigns to every model artifact: a unique, stable and versionable URN that exists independently of storage location, file names or registry technology.) in a JSON Schema
xsd-importxs:import/@namespace: a CORE-URN namespace links that Element directly (any format); a classic XML namespace is resolved via xsd_namespace
association-refan Element's concrete x-core-ref association targets (the by-reference counterpart to schema-ref's by-value $ref edges)
mapping-source / mapping-targettop-level source / target of a Mapping
pipeline-nodethe sourceRef / sinkRef / mappingRef — and an enrich node's lookupSourceRef — of a Pipeline's top-level nodes[]
dataset-refevery member of the DataSet manifest's *Refs arrays; reference_name carries the member kind
datasource-element / datasink-elementthe element field of a DataSource / DataSink — its own payload Element binding
datastructure-refeach $defs.*.$ref of a DataStructure — its member Elements

The DataSet manifest stays the canonical document; the relational rows are a derived index, written in the same transaction.

Versioning​

Versioning is owned by Model Forge — clients never choose the next version directly, they declare how it should be bumped.

  • The logical URN identifies the artifact independent of version; the versioned URN identifies one immutable version.

  • The first version of a new artifact is 1.0.0.

  • An update creates a new artifact_version; the number is computed from current_version and the requested bump:

    1.2.3 + patch -> 1.2.4
    1.2.3 + minor -> 1.3.0
    1.2.3 + major -> 2.0.0
  • Older versions are never overwritten; current_version is the default read.

  • The bump is request metadata, not part of the payload — a versionBump option (patch | minor | major) on the facade's update commands.

  • A request targeting a versioned URN resolves the logical identity and bumps from current_version; the URN's version segment is ignored as the new number.

  • A rename is a patch-version bump that records the new title/description as versioned metadata, format-independently — so XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. Elements are renamable too (the XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. content is unchanged); older versions keep their title.

  • A read id (and a reference) may use the :latest token, resolving to the current (highest-SemVer) version. :latest is never a writable identity — a client-supplied :latest (like a legacy :xsd:) identity is rejected.

Model Forge performs no schema-compatibility checks and exposes no compatibility metadata; there is no compatibility surface on the facade.

Write flow​

Every artifact write runs in one transaction:

  1. Normalise the incoming ID to a logical CORE URNCORE URNThe global identity Model Management assigns to every model artifact: a unique, stable and versionable URN that exists independently of storage location, file names or registry technology..
  2. Validate the payload with the existing validation services.
  3. Reject a :xsd: artifact-type URN (before any write).
  4. Upsert artifact (insert on first sight; first version 1.0.0).
  5. If the current version's authored representation in the same format has an identical content hash → no-op (idempotent). A different format is always a new version.
  6. Otherwise compute the next SemVer version and insert a new artifact_version with primary_format = the written format.
  7. Insert the version's authored artifact_representation (generation = 'stored').
  8. Replace the version's artifact_reference rows from the extracted references.
  9. Advance artifact.current_version.
  10. For XSDsXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD., upsert the xsd_namespace row (scoped to the version).

A storage failure rolls the whole write back and surfaces as an error to the calling host application.

Read flow​

  • Versioned URN → load that artifact_version.
  • Logical URN → load artifact.current_version's version.
  • Pick the representation: a schema read without a requested format returns the authored (primary) representation; requesting json-schema returns the JSON Schema (an XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. version is converted on read); requesting xsd returns the raw XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. (an error when the version has no XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. representation). A version's readable formats are its stored representations plus the derivable jsonschema for XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. versions.

Conditional requests (ETags)​

The same content_hash that makes writes idempotent is exposed to callers: The registry keeps the content hash of the stored representation a read would serve (the requested format, or the authored format when omitted); a derived representation has no stored hash.

Model Forge itself has no HTTP surface, but a host application that exposes one can use this hash as a strong ETag ("sha256:<hex>") to implement HTTP conditional requests (If-None-Match → 304, If-Match → 412) on top of the facade in its own transport layer. Because the hash is over the serialised authored content it is byte-stable but not canonical: two semantically equal JSON documents with different key order can hash differently (acceptable for caching and lost-update detection).

Artifact lifecycle​

The artifact lifecycle (draft/review/release status and similar) is entirely the host application's concern. Model Forge stores immutable versions and keeps no lifecycle state of its own; current_version is always the default read.

Dependency graph & cache​

The durable source of dependency edges is artifact_reference. Two in-memory indexes serve runtime queries and are rebuilt from PostgreSQL on startup (on ApplicationReadyEvent, after Flyway has migrated):

  • DependencyGraphService — a version-precise graph whose nodes are versioned URNs and whose edges are kept verbatim (pinned or :latest); rebuilt from each artifact's current version on startup. It serves dependencies and dependents queries — reporting concrete versioned IDs (a logical or :latest query resolves to the current version) — and tolerates cycles. It survives a restart because the data lives in PostgreSQL.

Service boundary​

The single seam between the domain services and storage is the ArtifactRegistry port; PostgresArtifactRegistryClient is its only implementation. Model Forge always runs against a real Postgres registry — without a datasource the starter wires no registry and no facade.

Domain behaviour (validation, URN creation, reference extraction, graph rebuild, inlined/bundled views) stays in the services.

Implementation notes​

  • Access: a thin JdbcClient repository layer; jsonb is written with cast(:json as jsonb) and read back as text. No JPA/Hibernate.
  • Migrations: plain-SQL Flyway migrations under db/model-forge/migration.
  • Search: scalar metadata (URN, name, title, description, type) first; a GIN index on content_jsonb is available for payload-level queries.
  • Tests: the adapter's outage translation and transaction wrapping are unit-tested with a mocked JdbcClient (PostgresArtifactRegistryClientTest); the admin-ui's AdminUiApplicationTest boots the full stack against a Testcontainers PostgreSQL (skipped when no Docker daemon is available).