ADR 051: DataSet as an artifact manifest
Date: 2026-07-15
Status: Accepted
Decision Makers: @derlinne, @luckey
Context
A DataSet composes several sub-resources that also need to exist and evolve independently.
Decision
The DataSet is a lightweight manifest in the registry that references its sub-resources by URN (elementRefs, mappingRefs, pipelineRefs, dataSourceRefs, dataSinkRefs). Sub-resources are independent artifacts.
Rationale
- Each resource has its own versioning history.
- Compositions (one schema in multiple DataSets) are possible without data duplication.
- Clear separation: DataSet = composition container, sub-resources = reusable building blocks.
Consequences
A DataSet read returns the stored manifest; consumers fetch referenced artifacts individually by their URNs.