Deletion & DataSet Membership Policy

Status: implemented (Model Forge + portal-backend). See "Implementation" below.

This document defines when a CORE artifact may be deleted, how DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. membership affects deletion, and how to find artifacts that belong to no DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions.. It is the authoritative reference for the delete behaviour of ModelForge.deleteArtifact(…) and the portal-backend delete endpoints.

Artifact kinds and references

Every stored artifact (DataStructure, Element, Mapping, Pipeline, DataSource, DataSink, DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions.) can reference other artifacts. Model Forge records each reference as a typed edge in artifact_reference (reference_type), for example:

• datastructure-ref — a DataStructure grouping one of its member Elements.
• mapping-source / mapping-target — a Mapping binding its DataStructures.
• pipeline-node — a Pipeline node referencing a source / sink / mapping.
• dataset-ref — a DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. manifest referencing one of its members (introduced with the DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. artifact; see below).

Referential integrity is the baseline: an artifact that another artifact still points at must not silently disappear, or the reference would dangle.

DataSet membership

A DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. is a first-class artifact — a manifest that references its members by versioned CORE URNCORE URNThe global identity Model Management assigns to every model artifact: a unique, stable and versionable URN that exists independently of storage location, file names or registry technology. (pipelineRefs, dataSinkRefs, plus derived dataSourceRefs / mappingRefs / datastructureRefs). Membership is expressed as dataset-ref edges from the DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. manifest to each member.

• Pipeline — the only exclusive member: a Pipeline carries a @NotNull dataSet FK, so it belongs to exactly one DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions.. It is linked by writing it with the optional dataSet argument (dataset-ref edge manifest → pipeline).
• Reusable artifacts — DataSources, DataSinks, Mappings and DataStructures are reusable and may belong to several DataSetsDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. at once.

Membership of a reusable artifact in a DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. arises two ways (kept consistent):

1. Auto-link through Pipelines — when a Pipeline is written with dataSet = D, Model Forge links the Pipeline and every artifact it references (its sources/sinks/mappings and, transitively, their DataStructures) as members of D.
2. Explicit assignment — a reusable artifact can be added to (or removed from) a DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. directly, independent of any Pipeline using it.

Both produce dataset-ref edges from the DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. manifest to the member; an artifact can therefore be a member of zero, one, or many DataSetsDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions..

Delete rules

Let D(x) = the set of DataSetsDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. whose manifest references artifact x (dataset-ref in-edges), and N(x) = every non-DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. artifact that references x (e.g. a Pipeline referencing a Mapping).

Deleting a member artifact x

1. Non-DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. references block, always. If N(x) is non-empty, the delete is rejected with ArtifactInUseException listing the blockers. You must remove/update those referencing artifacts first. (A Mapping used by a Pipeline cannot be deleted while that Pipeline exists.)
2. DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. membership is count-based:
   • |D(x)| = 0 (orphan) → deletable.
   • |D(x)| = 1 → deletable, and Model Forge removes x from that one DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions.'s manifest as part of the delete (auto-unlink).
   • |D(x)| ≥ 2 → blocked. The member is shared across DataSetsDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions.; remove it from the DataSetsDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. first (all but at most one), then delete.

So a member is deletable exactly when it has no non-DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. referrers and is in at most one DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions..

Deleting a DataSet S

Deleting a DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. deletes only its manifest — its members are not deleted (a DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. is a grouping, not an owner). The manifest's outgoing dataset-ref edges disappear with it, so each former member's D(x) shrinks by one (a member that was in only S becomes an orphan, still present and reusable).

Nothing references a DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. manifest, so a DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. is never blocked by member references.

cascade and force

Two opt-in modifiers exist; both require deliberate intent and are dangerous.

• cascade — delete the target and its members, descending the reference graph. A member is cascade-deleted only if it becomes an orphan once the target is gone (no other non-DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. referrer, in no other DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions.); shared members are kept. Cascade recurses and always stops at anything still in use. Deleting a DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. with cascade therefore removes the manifest plus every member that was exclusive to it, keeping members shared with other DataSetsDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. or used by surviving Pipelines.

• force — override the blocking checks and delete the target regardless of referrers. This can leave dangling references and must be reserved for administrative repair; it is never the default and should be gated + audited. Prefer removing the blockers, or cascade, over force.

  The platform does not expose force. It is a parameter of the Model Forge facade, and ModelRegistryGateway always passes false; the gateway's own deleteArtifact has no force parameter at all. A force delete therefore cannot be triggered through the portal backend API. The paragraph above describes what the library can do, not what an operator can call.

The safe default is deleteArtifact(x) = no cascade, no force: it deletes only when the rules above permit, and never touches anything else (beyond the single-DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. auto-unlink).

Finding orphans (artifacts in no DataSet), by type

It must be possible to list artifacts of a given type that belong to no DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. — e.g. "all Mappings / Pipelines / DataSources not in any DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions." — so they can be reviewed, assigned, or cleaned up. This is a generic membership query, expressed as: artifacts of kind T with no dataset-ref in-edge.

Model Forge exposes this generically (kind + membership filter) rather than one method per type, e.g. an orphans query taking an artifact kind and returning the matching ArtifactSummary list. The portal-backend surfaces it as a read-only maintenance endpoint (filterable by type).

Facade note: today the facade offers search (by text/type/format) and dependents (reverse edges) — the building blocks — but not a single orphan-by-membership query. This policy adds that generic query rather than composing it client-side per type.

Summary table

Target	Referenced by non-DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. artifact?	In N DataSetsDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions.	Default delete
Member	yes	any	blocked (remove referrers first)
Member	no	0	deleted
Member	no	1	deleted + unlinked from that DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions.
Member	no	≥2	blocked (remove from DataSetsDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. first)
DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions.	(n/a — nothing references it)	—	deleted, members kept

cascade extends a delete downward into exclusively-owned members; force overrides the blocks and may dangle references — use with great care.

Implementation

• Model Forge — saveArtifact/createArtifact take an optional dataSet URN; when set, the artifact is linked into that DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions.'s manifest, and for a Pipeline its whole reference closure (sources/sinks/mappings and their DataStructures) is linked too. Membership is stored as uniform dataset-ref edges. deleteArtifact(id, cascade, force) applies the policy above; orphans(kind) returns artifacts of a kind with no dataset-ref in-edge; linkToDataSet/unlinkFromDataSet are the explicit-assignment operations.
• portal-backend — a DataSetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions.'s manifest is created when the datasetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. is created (DataSet.manifestLogicalUrn) and deleted with the datasetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. (members kept). PipelineService passes its datasetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions.'s URN on save (auto-link). Endpoints: GET /v1/orphans?type=…, POST /v1/datasets/{id}/members ({artifactUrn}), DELETE /v1/datasets/{id}/members?artifactUrn=….
• Verified — the roundtrip E2E asserts a saved pipeline links its entire closure into the datasetDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. manifest as dataset-ref members, and the full suite stays green (the policy does not break existing deletes).