ADR 029: Collection Endpoint Authorization Filtering
Date: 2026-02-05
Status: Proposed
Decision Makers: Architecture Board
Context
Resource endpoints (e.g., GET /v2/datasets/{id}) can be authorized by verifying the user's permission scope matches the specific resource ID. However, collection endpoints (e.g., GET /v2/datasets) present a challenge: how do we filter results to only include resources the user is authorized to see?
Key Constraints
- OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. must remain the sole Policy Decision PointPolicy Decision PointThe component that decides whether a request is authorized. CIVITAS/CORE uses Open Policy Agent (OPA) as its PDP. (PDPPolicy Decision PointThe component that decides whether a request is authorized. CIVITAS/CORE uses Open Policy Agent (OPA) as its PDP.)
- Backend services should not contain authorization logic
- Solution must work with existing APISIXApache APISIXAn open-source API gateway for traffic management, security and observability. In CIVITAS/CORE it is used as the centralized entrypoint to route and protect externally exposed APIs. + OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. architecture
- Performance: avoid duplicate HTTP calls where possible
Example Scenario
- User has
READ_DATASETpermission scoped todataspace-Aanddataspace-B GET /v2/datasetsshould only return datasetsDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. in those two dataspaces- User should NOT see datasetsDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. in
dataspace-C
Checked Architecture Principles
- [full] Model-centric data flow
- [full] Distributed architecture with unified user experience
- [full] Modular design
- [partial] Integration capability through defined interfaces — Requires header contract between OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. and backend
- [full] Open source as the default — Uses APISIXApache APISIXAn open-source API gateway for traffic management, security and observability. In CIVITAS/CORE it is used as the centralized entrypoint to route and protect externally exposed APIs., OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization.
- [full] Cloud-native architecture
- [full] Prefer standard solutions over custom development — Uses existing APISIXApache APISIXAn open-source API gateway for traffic management, security and observability. In CIVITAS/CORE it is used as the centralized entrypoint to route and protect externally exposed APIs. OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. plugin feature
- [full] Self-contained deployment
- [full] Technological consistency to ensure maintainability — Extends existing OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization./RegoRegoOPA's declarative policy language, used to express authorization rules such as checking role assignments at the most specific scope first. patterns
- [full] Multi-tenancy — Scope filtering is essential for multi-tenantTenantAn isolated organizational partition that owns Data pools, Datasets, Users, Groups, and Roles. All access rules exist within their Tenant, and the Tenant is the widest Scope of a Role. Currently, one Tenant corresponds to the Platform. isolation
- [full] Security by design — Centralized authorization in OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization., no authz logic in backend
Decision
Use APISIXApache APISIXAn open-source API gateway for traffic management, security and observability. In CIVITAS/CORE it is used as the centralized entrypoint to route and protect externally exposed APIs. OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. plugin's send_headers_upstream feature to pass allowed scope IDs to the backend via HTTP headers.
Key Insight: The APISIXApache APISIXAn open-source API gateway for traffic management, security and observability. In CIVITAS/CORE it is used as the centralized entrypoint to route and protect externally exposed APIs. OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. plugin supports send_headers_upstream (PR #9710, merged July 2023), allowing a single OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. call to handle both authorization AND scope header passing — no additional plugins required.
Architecture
ANY /v2/* request
│
▼
APISIX
│ 1. proxy-rewrite: Set backend identifier header
│ 2. openid-connect: JWT validation
│ 3. opa: Authorization + scope extraction
▼
OPA
│ Evaluates policy
│ Returns: {"allow": true, "headers": {"X-Allowed-Scope-Ids": "scope-1,scope-2"}}
▼
APISIX
│ allow=true → Forward request + pass X-Allowed-Scope-Ids header
│ allow=false → Return 403 Forbidden
▼
Backend Service
│ Parse header, apply query filter
▼
Filtered results
Header Contract
The X-Allowed-Scope-Ids header communicates OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization.'s authorization decision to the backend:
| Header Value | Meaning | Backend Action |
|---|---|---|
* | TENANTTenantAn isolated organizational partition that owns Data pools, Datasets, Users, Groups, and Roles. All access rules exist within their Tenant, and the Tenant is the widest Scope of a Role. Currently, one Tenant corresponds to the Platform. scope (wildcard) | Skip filtering, return all results |
id1,id2,id3 | Specific scope IDs (UUIDs) | Filter results to these scopes |
| Empty/missing | No scopes or not applicable | Return empty results (fail-secure) |
Component Responsibilities
OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. (Policy Decision PointPolicy Decision PointThe component that decides whether a request is authorized. CIVITAS/CORE uses Open Policy Agent (OPA) as its PDP.)
- Evaluates user permissions for the requested resource type
- Determines allowed scope IDs based on user's group assignments
- Returns scope IDs in the decision response headers
- TENANTTenantAn isolated organizational partition that owns Data pools, Datasets, Users, Groups, and Roles. All access rules exist within their Tenant, and the Tenant is the widest Scope of a Role. Currently, one Tenant corresponds to the Platform.-scoped permissions return wildcard (
*) to avoid header bloat
APISIXApache APISIXAn open-source API gateway for traffic management, security and observability. In CIVITAS/CORE it is used as the centralized entrypoint to route and protect externally exposed APIs. (Policy Enforcement PointPolicy Enforcement PointThe component that enforces the authorization decision made by the Policy Decision Point. CIVITAS/CORE uses Apache APISIX as its PEP, adopting a centralized, gateway-enforced authorization architecture.)
- Forwards OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization.'s headers to the backend via
send_headers_upstream - No modification to header values — pure passthrough
- Blocks requests if OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. denies authorization
Backend Service (Query Execution)
- Parses
X-Allowed-Scope-Idsheader early in request lifecycle - Stores scope IDs in request-scoped context
- Applies scope filter to collection queries via JPA Specification pattern
- Does NOT make authorization decisions — only executes the filter OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. specified
Header Size Considerations
HTTP headers have practical size limits (~32KB). Since scope IDs are UUIDs (36 chars each):
| Scope Count | Approximate Header Size |
|---|---|
| 100 UUIDs | ~3.7 KB |
| 500 UUIDs | ~18.5 KB |
| 850 UUIDs | ~32 KB |
Mitigations:
- Wildcard for TENANTTenantAn isolated organizational partition that owns Data pools, Datasets, Users, Groups, and Roles. All access rules exist within their Tenant, and the Tenant is the widest Scope of a Role. Currently, one Tenant corresponds to the Platform. scope: Users with tenantTenantAn isolated organizational partition that owns Data pools, Datasets, Users, Groups, and Roles. All access rules exist within their Tenant, and the Tenant is the widest Scope of a Role. Currently, one Tenant corresponds to the Platform.-level access receive
*instead of enumerated IDs - Realistic limits: Users typically have 1-20 scope assignments; 200 scopes is an extreme edge case
- Configurable buffer sizes: APISIXApache APISIXAn open-source API gateway for traffic management, security and observability. In CIVITAS/CORE it is used as the centralized entrypoint to route and protect externally exposed APIs. and backend can be configured for 32KB headers if needed
Pattern Name
This approach is known as "Authorization Scope Filtering" or "Policy-Enforced Query Filtering". It's a variant of Row-Level Security (RLS) where:
- The Policy Decision PointPolicy Decision PointThe component that decides whether a request is authorized. CIVITAS/CORE uses Open Policy Agent (OPA) as its PDP. (OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization.) determines the allowed scope IDs
- The Policy Enforcement PointPolicy Enforcement PointThe component that enforces the authorization decision made by the Policy Decision Point. CIVITAS/CORE uses Apache APISIX as its PEP, adopting a centralized, gateway-enforced authorization architecture. (APISIXApache APISIXAn open-source API gateway for traffic management, security and observability. In CIVITAS/CORE it is used as the centralized entrypoint to route and protect externally exposed APIs.) propagates this as a header
- The Backend applies the filter mechanically without making authorization decisions
Related patterns:
- Row-Level Security (RLS): Database-native filtering based on user context
- Attribute-Based Access Control (ABAC): Policy decisions based on attributes (scopes are an attribute)
- Authorization Context Propagation: OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization.'s decision context flows through the request chain
This is NOT "query plan pushing" — that term refers to distributed database query optimizations.
Limitation: Requires Backend Control
IMPORTANT: This solution only works when we control the backend implementation.
Why Backend Control is Required
The filtering mechanism requires the backend to:
- Parse the
X-Allowed-Scope-Idsheader - Inject scope constraints into database queries
- Ensure all collection endpoints respect the filter
This is only possible when we can modify the backend code.
External Backends
For external/third-party backends (e.g., FROST Server, Stellio), we cannot apply this pattern because:
- No code modification: We can't add header parsing or query filtering logic
- Different query languages: External backends use OData, NGSI-LDNGSI-LDAn Open API and data model specification for context management, published by ETSI. It defines how context information (entities, relationships, and properties) is represented and exchanged., or proprietary query syntax
- No injection hook: These backends don't expose a mechanism to inject scope filters
Recommendations for External Backends
| Approach | Viability | Notes |
|---|---|---|
| Block collection endpoints | Recommended | OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. denies collection requests; only resource endpoints allowed |
| Filtering proxy | Future option | Build a proxy that translates scope IDs to backend-native query syntax |
| Accept data leakage | Not acceptable | Violates security requirements |
Current recommendation: For external backends, do not expose collection endpoints. Configure OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. to deny requests to collection patterns. Users must query by specific resource ID, where standard scope validation applies.
Consequences
Benefits
- OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. remains sole PDPPolicy Decision PointThe component that decides whether a request is authorized. CIVITAS/CORE uses Open Policy Agent (OPA) as its PDP.: Authorization decisions stay centralized
- Clean separation: OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. decides, backend filters
- Single OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. call: No additional latency from separate scope queries
- Debuggable: Header value visible in logs and traces
- Extensible: Same pattern works for any scoped entity type
Trade-offs
- Backend code changes required: Each filtered entity needs a query specification
- Header size limits: Very large scope counts may approach limits (mitigated by wildcard)
- External backends excluded: Pattern doesn't work without backend control
Affected Components
- OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. policy: Scope header generation rules
- APISIXApache APISIXAn open-source API gateway for traffic management, security and observability. In CIVITAS/CORE it is used as the centralized entrypoint to route and protect externally exposed APIs. configuration:
send_headers_upstreamdirective - Backend: Request filter, scope bean, JPA specifications
- Each service needing filtering: Query preprocessing logic
Alternatives Considered
| Alternative | Decision | Rationale |
|---|---|---|
| Backend calls AuthZ Repository directly | Rejected | Moves authorization logic into backend, violating "OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. as sole PDPPolicy Decision PointThe component that decides whether a request is authorized. CIVITAS/CORE uses Open Policy Agent (OPA) as its PDP." |
| OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. Partial Evaluation / Compile API | Rejected | Overly complex; requires parsing OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. AST and converting to SQL |
| Backend calls OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. directly | Rejected | Bypasses APISIXApache APISIXAn open-source API gateway for traffic management, security and observability. In CIVITAS/CORE it is used as the centralized entrypoint to route and protect externally exposed APIs., loses gateway observability |
| Shared cache (Redis) | Deferred | Added infrastructure complexity; OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. already fetches user context |
| Return all data, filter in application | Rejected | Security risk (data leakage) and performance impact |