PostgreSQL Artifact Registry
Overview
Model Forge persists all model artifactsModel ArtifactUmbrella term for any persistable CORE unit: Element, DataStructure, Mapping, DataSource, DataSink, Pipeline and DataSet. Every artifact has a stable CORE URN identity and is versioned. in a PostgreSQL-backed artifact registry that it owns. The schema is created and migrated automatically by Flyway on startup; the application requires a datasource to run.
The registry stores:
- the logical identity of each artifact (format-independent)
- immutable artifact versions, each with an authored primary format
- per-version, per-format representations — JSON Schema, raw XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD., core-json
- the XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. namespace lookup
- the reference graph between artifacts
Reads are projections built at the boundary from the stored artifact document, not ORM entities — the artifact document remains canonical and no second representation can drift from it.
Design principles
- CORE URNsCORE URNThe global identity Model Management assigns to every model artifact: a unique, stable and versionable URN that exists independently of storage location, file names or registry technology. are the stable public identity of every artifact — the format is never part of the identity. The same Element may be authored as JSON Schema in one version and as XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. in another.
- Artifact payloads are canonical JSON documents, stored as
jsonb. - XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. payloads are stored as raw text; a JSON Schema representation is generated from them on read (and cached).
- Each version records its authored primary format; the format-specific content
lives in
artifact_representation(one row per format). - Versioning is explicit, backend-owned and modelled in the database.
- References are stored relationally for fast dependency/impact queries, and the complete graph is kept — cycles included.
- Persistence access uses a thin
JdbcClientrepository layer; large dynamic JSON trees are stored asjsonb/text rather than decomposed into entity fields (no JPA/Hibernate).
Tables
The schema lives in the Flyway migrations under db/model-forge/migration; the
DDL below reflects the combined result of all migrations.
artifact — logical identity
create table artifact (
id uuid primary key,
logical_urn text not null unique,
artifact_type text not null, -- element | dataset | mapping | pipeline | datasource | datasink | datastructure
name text not null,
title text,
description text,
current_version text, -- the default read version
created_at timestamptz not null,
updated_at timestamptz not null
);
artifact_type is the functional CORE category. Identity is format-agnostic, so
the format is not stored here. Both JSON Schema and XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. Elements have
artifact_type = 'element'; the format is recorded per version
(artifact_version.primary_format) and per representation (below). The
logical_urn always uses the element segment; a :xsd: artifact-type
URN is rejected.
Indexes: artifact_type, name, updated_at.
artifact_version — immutable version metadata
create table artifact_version (
id uuid primary key,
artifact_id uuid not null references artifact(id) on delete cascade,
version text not null,
primary_format text not null, -- jsonschema | core-json | xsd
title text, -- versioned metadata (set on rename)
description text,
created_at timestamptz not null,
created_by text,
unique (artifact_id, version)
);
create index idx_artifact_version_artifact on artifact_version (artifact_id);
- A version records its authored (primary) format and points at its
representations; the content lives in
artifact_representation. primary_formatselects the default representation a schema read returns when no explicit format is requested.title/descriptionare versioned metadata set by a rename: a rename creates a new patch version with the new title, format-independently (so XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. versions are renamable too); older versions keep their own title. A read serves the version's title when set, else the URN name segment.artifact.current_versionpoints at the latest version.
artifact_representation — per-version, per-format content
create table artifact_representation (
id uuid primary key,
version_id uuid not null references artifact_version(id) on delete cascade,
format text not null, -- jsonschema | core-json | xsd
content_type text,
content_jsonb jsonb,
content_text text,
content_hash text,
generation text not null default 'stored', -- stored | generated
created_at timestamptz not null,
unique (version_id, format)
);
- One row per format of a version. JSON content goes into
content_jsonb; raw XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. goes intocontent_text. - Each version has exactly one authored representation (
generation = 'stored'), in itsprimary_format. A JSON Schema derived from an XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. may later be persisted withgeneration = 'generated'; a generated representation never counts as authored content. content_hashmakes repeated imports of identical authored content idempotent (no spurious new version).unique (version_id, format)permits at most one representation per format per version; a different format in a new version is always allowed (v1 XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD., v2 JSON-only is valid).- A GIN index on
content_jsonbsupports payload-level search.
artifact_reference — dependency edges
create table artifact_reference (
id uuid primary key,
from_version_id uuid not null references artifact_version(id) on delete cascade,
target_urn text not null,
target_artifact_id uuid references artifact(id) on delete set null,
target_version_id uuid references artifact_version(id) on delete set null,
reference_type text not null, -- schema-ref | dataset-ref | pipeline-node | xsd-import | association-ref | mapping-source | mapping-target
-- | datasource-element | datasink-element | datastructure-ref
reference_name text,
sort_order int,
created_at timestamptz not null
);
- The complete reference graph is stored, including cycles.
target_urnis stored verbatim as authored — a pinned…:1.0.0or the…:latesttoken — never normalised to the logical form.target_artifact_idresolves the reference to the target artifact's logical identity;target_version_idresolves a pinned reference to its concrete version (null forlatest, logical, or a not-yet-imported pin). Both are nullable /on delete set null, and are back-filled when a previously-missing target artifact or target version is later created.
xsd_namespace — namespace lookup
create table xsd_namespace (
namespace text primary key,
artifact_id uuid not null references artifact(id) on delete cascade,
version_id uuid references artifact_version(id) on delete set null,
updated_at timestamptz not null
);
A durable namespace→artifact index for XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD.-backed Elements, so
xs:import resolution does not depend on a startup scan completing.
Read projections
Reads are built from artifact plus the selected artifact_version: the
authored content comes from artifact_representation
(content_jsonb/content_text), the versioned title falls back to the URN
name segment, and an XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. version is additionally readable as derived JSON
Schema. The facade serves these as plain JSON documents
(getArtifact/getBundledView/getInlinedView).
Reference extraction
Edges are derived from the artifact content when it is written and persisted as
artifact_reference rows:
reference_type | Source |
|---|---|
schema-ref | $ref values (CORE URNsCORE URNThe global identity Model Management assigns to every model artifact: a unique, stable and versionable URN that exists independently of storage location, file names or registry technology.) in a JSON Schema |
xsd-import | xs:import/@namespace: a CORE-URN namespace links that Element directly (any format); a classic XML namespace is resolved via xsd_namespace |
association-ref | an Element's concrete x-core-ref association targets (the by-reference counterpart to schema-ref's by-value $ref edges) |
mapping-source / mapping-target | top-level source / target of a Mapping |
pipeline-node | the sourceRef / sinkRef / mappingRef — and an enrich node's lookupSourceRef — of a Pipeline's top-level nodes[] |
dataset-ref | every member of the DataSet manifest's *Refs arrays; reference_name carries the member kind |
datasource-element / datasink-element | the element field of a DataSource / DataSink — its own payload Element binding |
datastructure-ref | each $defs.*.$ref of a DataStructure — its member Elements |
The DataSet manifest stays the canonical document; the relational rows are a derived index, written in the same transaction.
Versioning
Versioning is owned by Model Forge — clients never choose the next version directly, they declare how it should be bumped.
-
The logical URN identifies the artifact independent of version; the versioned URN identifies one immutable version.
-
The first version of a new artifact is
1.0.0. -
An update creates a new
artifact_version; the number is computed fromcurrent_versionand the requested bump:1.2.3 + patch -> 1.2.41.2.3 + minor -> 1.3.01.2.3 + major -> 2.0.0 -
Older versions are never overwritten;
current_versionis the default read. -
The bump is request metadata, not part of the payload — a
versionBumpoption (patch|minor|major) on the facade's update commands. -
A request targeting a versioned URN resolves the logical identity and bumps from
current_version; the URN's version segment is ignored as the new number. -
A rename is a patch-version bump that records the new
title/descriptionas versioned metadata, format-independently — so XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. Elements are renamable too (the XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. content is unchanged); older versions keep their title. -
A read id (and a reference) may use the
:latesttoken, resolving to the current (highest-SemVer) version.:latestis never a writable identity — a client-supplied:latest(like a legacy:xsd:) identity is rejected.
Model Forge performs no schema-compatibility checks and exposes no compatibility metadata; there is no compatibility surface on the facade.
Write flow
Every artifact write runs in one transaction:
- Normalise the incoming ID to a logical CORE URNCORE URNThe global identity Model Management assigns to every model artifact: a unique, stable and versionable URN that exists independently of storage location, file names or registry technology..
- Validate the payload with the existing validation services.
- Reject a
:xsd:artifact-type URN (before any write). - Upsert
artifact(insert on first sight; first version1.0.0). - If the current version's authored representation in the same format has an identical content hash → no-op (idempotent). A different format is always a new version.
- Otherwise compute the next SemVer version and insert a new
artifact_versionwithprimary_format= the written format. - Insert the version's authored
artifact_representation(generation = 'stored'). - Replace the version's
artifact_referencerows from the extracted references. - Advance
artifact.current_version. - For XSDsXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD., upsert the
xsd_namespacerow (scoped to the version).
A storage failure rolls the whole write back and surfaces as an error to the calling host application.
Read flow
- Versioned URN → load that
artifact_version. - Logical URN → load
artifact.current_version's version. - Pick the representation: a schema read without a requested format returns the
authored (primary) representation; requesting
json-schemareturns the JSON Schema (an XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. version is converted on read); requestingxsdreturns the raw XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. (an error when the version has no XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. representation). A version's readable formats are its stored representations plus the derivablejsonschemafor XSDXML Schema DefinitionA W3C language for describing the structure of XML documents. Model Management imports XSD into JSON Schema and can serialize an Element as XSD. versions.
Conditional requests (ETags)
The same content_hash that makes writes idempotent is exposed to callers:
The registry keeps the content hash of the stored
representation a read would serve (the requested format, or the authored format
when omitted); a derived representation has no stored hash.
Model Forge itself has no HTTP surface, but a host application that exposes one
can use this hash as a strong ETag ("sha256:<hex>") to implement HTTP
conditional requests (If-None-Match → 304, If-Match → 412) on top of the
facade in its own transport layer.
Because the hash is over the serialised authored content it is byte-stable but not
canonical: two semantically equal JSON documents with different key order can hash
differently (acceptable for caching and lost-update detection).
Artifact lifecycle
The artifact lifecycle (draft/review/release status and similar) is entirely the
host application's concern. Model Forge stores immutable versions and keeps no
lifecycle state of its own; current_version is always the default read.
Dependency graph & cache
The durable source of dependency edges is artifact_reference. Two in-memory
indexes serve runtime queries and are rebuilt from PostgreSQL on startup
(on ApplicationReadyEvent, after Flyway has migrated):
DependencyGraphService— a version-precise graph whose nodes are versioned URNs and whose edges are kept verbatim (pinned or:latest); rebuilt from each artifact's current version on startup. It serves dependencies and dependents queries — reporting concrete versioned IDs (a logical or:latestquery resolves to the current version) — and tolerates cycles. It survives a restart because the data lives in PostgreSQL.
Service boundary
The single seam between the domain services and storage is the
ArtifactRegistry port; PostgresArtifactRegistryClient is its only
implementation. Model Forge always runs against a real Postgres registry —
without a datasource the starter wires no registry and no facade.
Domain behaviour (validation, URN creation, reference extraction, graph rebuild, inlined/bundled views) stays in the services.
Implementation notes
- Access: a thin
JdbcClientrepository layer;jsonbis written withcast(:json as jsonb)and read back as text. No JPA/Hibernate. - Migrations: plain-SQL Flyway migrations under
db/model-forge/migration. - Search: scalar metadata (URN, name, title, description, type) first; a GIN
index on
content_jsonbis available for payload-level queries. - Tests: the adapter's outage translation and transaction wrapping are
unit-tested with a mocked
JdbcClient(PostgresArtifactRegistryClientTest); the admin-ui'sAdminUiApplicationTestboots the full stack against a Testcontainers PostgreSQL (skipped when no Docker daemon is available).