Skip to main content
Version: 2.0-rc2

ADR 029: Collection Endpoint Authorization Filtering

Date: 2026-02-05

Status: Proposed

Decision Makers: Architecture Board

Context​

Resource endpoints (e.g., GET /v2/datasets/{id}) can be authorized by verifying the user's permission scope matches the specific resource ID. However, collection endpoints (e.g., GET /v2/datasets) present a challenge: how do we filter results to only include resources the user is authorized to see?

Key Constraints​

  • OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. must remain the sole Policy Decision PointPolicy Decision PointThe component that decides whether a request is authorized. CIVITAS/CORE uses Open Policy Agent (OPA) as its PDP. (PDPPolicy Decision PointThe component that decides whether a request is authorized. CIVITAS/CORE uses Open Policy Agent (OPA) as its PDP.)
  • Backend services should not contain authorization logic
  • Solution must work with existing APISIXApache APISIXAn open-source API gateway for traffic management, security and observability. In CIVITAS/CORE it is used as the centralized entrypoint to route and protect externally exposed APIs. + OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. architecture
  • Performance: avoid duplicate HTTP calls where possible

Example Scenario​

  • User has READ_DATASET permission scoped to dataspace-A and dataspace-B
  • GET /v2/datasets should only return datasetsDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. in those two dataspaces
  • User should NOT see datasetsDatasetA data-related element that contains processed data and makes it available for consumption. A Dataset is populated via Pipelines and carries Metadata and access permissions. in dataspace-C

Checked Architecture Principles​

  • [full] Model-centric data flow
  • [full] Distributed architecture with unified user experience
  • [full] Modular design
  • [partial] Integration capability through defined interfaces — Requires header contract between OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. and backend
  • [full] Open source as the default — Uses APISIXApache APISIXAn open-source API gateway for traffic management, security and observability. In CIVITAS/CORE it is used as the centralized entrypoint to route and protect externally exposed APIs., OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization.
  • [full] Cloud-native architecture
  • [full] Prefer standard solutions over custom development — Uses existing APISIXApache APISIXAn open-source API gateway for traffic management, security and observability. In CIVITAS/CORE it is used as the centralized entrypoint to route and protect externally exposed APIs. OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. plugin feature
  • [full] Self-contained deployment
  • [full] Technological consistency to ensure maintainability — Extends existing OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization./RegoRegoOPA's declarative policy language, used to express authorization rules such as checking role assignments at the most specific scope first. patterns
  • [full] Multi-tenancy — Scope filtering is essential for multi-tenantTenantAn isolated organizational partition that owns Data pools, Datasets, Users, Groups, and Roles. All access rules exist within their Tenant, and the Tenant is the widest Scope of a Role. Currently, one Tenant corresponds to the Platform. isolation
  • [full] Security by design — Centralized authorization in OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization., no authz logic in backend

Decision​

Use APISIXApache APISIXAn open-source API gateway for traffic management, security and observability. In CIVITAS/CORE it is used as the centralized entrypoint to route and protect externally exposed APIs. OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. plugin's send_headers_upstream feature to pass allowed scope IDs to the backend via HTTP headers.

Key Insight: The APISIXApache APISIXAn open-source API gateway for traffic management, security and observability. In CIVITAS/CORE it is used as the centralized entrypoint to route and protect externally exposed APIs. OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. plugin supports send_headers_upstream (PR #9710, merged July 2023), allowing a single OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. call to handle both authorization AND scope header passing — no additional plugins required.

Architecture​

ANY /v2/* request
│
▼
APISIX
│ 1. proxy-rewrite: Set backend identifier header
│ 2. openid-connect: JWT validation
│ 3. opa: Authorization + scope extraction
▼
OPA
│ Evaluates policy
│ Returns: {"allow": true, "headers": {"X-Allowed-Scope-Ids": "scope-1,scope-2"}}
▼
APISIX
│ allow=true → Forward request + pass X-Allowed-Scope-Ids header
│ allow=false → Return 403 Forbidden
▼
Backend Service
│ Parse header, apply query filter
▼
Filtered results

Header Contract​

The X-Allowed-Scope-Ids header communicates OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization.'s authorization decision to the backend:

Header ValueMeaningBackend Action
*TENANTTenantAn isolated organizational partition that owns Data pools, Datasets, Users, Groups, and Roles. All access rules exist within their Tenant, and the Tenant is the widest Scope of a Role. Currently, one Tenant corresponds to the Platform. scope (wildcard)Skip filtering, return all results
id1,id2,id3Specific scope IDs (UUIDs)Filter results to these scopes
Empty/missingNo scopes or not applicableReturn empty results (fail-secure)

Component Responsibilities​

OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. (Policy Decision PointPolicy Decision PointThe component that decides whether a request is authorized. CIVITAS/CORE uses Open Policy Agent (OPA) as its PDP.)

  • Evaluates user permissions for the requested resource type
  • Determines allowed scope IDs based on user's group assignments
  • Returns scope IDs in the decision response headers
  • TENANTTenantAn isolated organizational partition that owns Data pools, Datasets, Users, Groups, and Roles. All access rules exist within their Tenant, and the Tenant is the widest Scope of a Role. Currently, one Tenant corresponds to the Platform.-scoped permissions return wildcard (*) to avoid header bloat

APISIXApache APISIXAn open-source API gateway for traffic management, security and observability. In CIVITAS/CORE it is used as the centralized entrypoint to route and protect externally exposed APIs. (Policy Enforcement PointPolicy Enforcement PointThe component that enforces the authorization decision made by the Policy Decision Point. CIVITAS/CORE uses Apache APISIX as its PEP, adopting a centralized, gateway-enforced authorization architecture.)

  • Forwards OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization.'s headers to the backend via send_headers_upstream
  • No modification to header values — pure passthrough
  • Blocks requests if OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. denies authorization

Backend Service (Query Execution)

  • Parses X-Allowed-Scope-Ids header early in request lifecycle
  • Stores scope IDs in request-scoped context
  • Applies scope filter to collection queries via JPA Specification pattern
  • Does NOT make authorization decisions — only executes the filter OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. specified

Header Size Considerations​

HTTP headers have practical size limits (~32KB). Since scope IDs are UUIDs (36 chars each):

Scope CountApproximate Header Size
100 UUIDs~3.7 KB
500 UUIDs~18.5 KB
850 UUIDs~32 KB

Mitigations:

  1. Wildcard for TENANTTenantAn isolated organizational partition that owns Data pools, Datasets, Users, Groups, and Roles. All access rules exist within their Tenant, and the Tenant is the widest Scope of a Role. Currently, one Tenant corresponds to the Platform. scope: Users with tenantTenantAn isolated organizational partition that owns Data pools, Datasets, Users, Groups, and Roles. All access rules exist within their Tenant, and the Tenant is the widest Scope of a Role. Currently, one Tenant corresponds to the Platform.-level access receive * instead of enumerated IDs
  2. Realistic limits: Users typically have 1-20 scope assignments; 200 scopes is an extreme edge case
  3. Configurable buffer sizes: APISIXApache APISIXAn open-source API gateway for traffic management, security and observability. In CIVITAS/CORE it is used as the centralized entrypoint to route and protect externally exposed APIs. and backend can be configured for 32KB headers if needed

Pattern Name​

This approach is known as "Authorization Scope Filtering" or "Policy-Enforced Query Filtering". It's a variant of Row-Level Security (RLS) where:

  1. The Policy Decision PointPolicy Decision PointThe component that decides whether a request is authorized. CIVITAS/CORE uses Open Policy Agent (OPA) as its PDP. (OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization.) determines the allowed scope IDs
  2. The Policy Enforcement PointPolicy Enforcement PointThe component that enforces the authorization decision made by the Policy Decision Point. CIVITAS/CORE uses Apache APISIX as its PEP, adopting a centralized, gateway-enforced authorization architecture. (APISIXApache APISIXAn open-source API gateway for traffic management, security and observability. In CIVITAS/CORE it is used as the centralized entrypoint to route and protect externally exposed APIs.) propagates this as a header
  3. The Backend applies the filter mechanically without making authorization decisions

Related patterns:

  • Row-Level Security (RLS): Database-native filtering based on user context
  • Attribute-Based Access Control (ABAC): Policy decisions based on attributes (scopes are an attribute)
  • Authorization Context Propagation: OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization.'s decision context flows through the request chain

This is NOT "query plan pushing" — that term refers to distributed database query optimizations.

Limitation: Requires Backend Control​

IMPORTANT: This solution only works when we control the backend implementation.

Why Backend Control is Required​

The filtering mechanism requires the backend to:

  1. Parse the X-Allowed-Scope-Ids header
  2. Inject scope constraints into database queries
  3. Ensure all collection endpoints respect the filter

This is only possible when we can modify the backend code.

External Backends​

For external/third-party backends (e.g., FROST Server, Stellio), we cannot apply this pattern because:

  1. No code modification: We can't add header parsing or query filtering logic
  2. Different query languages: External backends use OData, NGSI-LDNGSI-LDAn Open API and data model specification for context management, published by ETSI. It defines how context information (entities, relationships, and properties) is represented and exchanged., or proprietary query syntax
  3. No injection hook: These backends don't expose a mechanism to inject scope filters

Recommendations for External Backends​

ApproachViabilityNotes
Block collection endpointsRecommendedOPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. denies collection requests; only resource endpoints allowed
Filtering proxyFuture optionBuild a proxy that translates scope IDs to backend-native query syntax
Accept data leakageNot acceptableViolates security requirements

Current recommendation: For external backends, do not expose collection endpoints. Configure OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. to deny requests to collection patterns. Users must query by specific resource ID, where standard scope validation applies.

Consequences​

Benefits​

  • OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. remains sole PDPPolicy Decision PointThe component that decides whether a request is authorized. CIVITAS/CORE uses Open Policy Agent (OPA) as its PDP.: Authorization decisions stay centralized
  • Clean separation: OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. decides, backend filters
  • Single OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. call: No additional latency from separate scope queries
  • Debuggable: Header value visible in logs and traces
  • Extensible: Same pattern works for any scoped entity type

Trade-offs​

  • Backend code changes required: Each filtered entity needs a query specification
  • Header size limits: Very large scope counts may approach limits (mitigated by wildcard)
  • External backends excluded: Pattern doesn't work without backend control

Affected Components​

  • OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. policy: Scope header generation rules
  • APISIXApache APISIXAn open-source API gateway for traffic management, security and observability. In CIVITAS/CORE it is used as the centralized entrypoint to route and protect externally exposed APIs. configuration: send_headers_upstream directive
  • Backend: Request filter, scope bean, JPA specifications
  • Each service needing filtering: Query preprocessing logic

Alternatives Considered​

AlternativeDecisionRationale
Backend calls AuthZ Repository directlyRejectedMoves authorization logic into backend, violating "OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. as sole PDPPolicy Decision PointThe component that decides whether a request is authorized. CIVITAS/CORE uses Open Policy Agent (OPA) as its PDP."
OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. Partial Evaluation / Compile APIRejectedOverly complex; requires parsing OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. AST and converting to SQL
Backend calls OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. directlyRejectedBypasses APISIXApache APISIXAn open-source API gateway for traffic management, security and observability. In CIVITAS/CORE it is used as the centralized entrypoint to route and protect externally exposed APIs., loses gateway observability
Shared cache (Redis)DeferredAdded infrastructure complexity; OPAOpen Policy AgentThe platform's authorization service: it evaluates 'who may do what' against the CIVITAS/CORE authorization model. OPA is the platform's central Policy Decision Point (PDP) for API authorization. already fetches user context
Return all data, filter in applicationRejectedSecurity risk (data leakage) and performance impact

References​