Skip to content

Commit 3d1f865

Browse files
committed
feat(relay): surface untracked privacy budget on sensitive aggregates
A dataset classified personal, confidential, or secret that declares a dataset-level access.aggregate_only_execution aggregate now raises the relay.aggregates.privacy_budget_untracked deployment finding, bound finding_warn under hosted_lab, production, and evidence_grade. Only dataset-level declarations count: the aggregate query routes resolve aggregates through dataset.aggregates alone, so table-level and entity-level declarations have no runtime route and raise no finding. Aggregate routes apply per-result k-anonymity suppression but track no longitudinal query budget; this is the documented accepted limitation from the known-limitations page, so the gate warns everywhere rather than blocking startup or readiness, and a deployment waiver naming the finding records the operator's acknowledgement. Boot is loud: an active finding emits a dedicated deployment.privacy_budget_untracked warn line pointing at the waiver remediation; a waived one reports through the existing deployment.gate_waived line. The finding reaches the posture endpoint and registryctl doctor through the existing catalog plumbing. Security note (deployment defaults): adds a deployment gate finding; no runtime behavior or API surface changes. Closes #585 Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
1 parent 2ff00d9 commit 3d1f865

6 files changed

Lines changed: 524 additions & 2 deletions

File tree

crates/registry-relay/CHANGELOG.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,12 @@
88
that shape denies the release and an optional one is omitted. The OpenAPI
99
contract documents the scalar-only claim bundle and top-level-only claim
1010
selection.
11+
- A dataset classified `personal`, `confidential`, or `secret` with a
12+
dataset-level `access.aggregate_only_execution` aggregate now raises the
13+
`relay.aggregates.privacy_budget_untracked` deployment finding (warn at
14+
every bound profile) and a dedicated boot-log warning, surfacing that
15+
aggregate routes track no longitudinal privacy budget. A deployment waiver
16+
naming the finding records the operator's acknowledgement.
1117

1218
## 0.16.2 - 2026-08-01
1319

crates/registry-relay/docs/configuration.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1403,16 +1403,19 @@ Waiver references and summaries are visible only in the restricted posture tier;
14031403
| `relay.audit.retention_local_only` | (not bound) | warn | startup_fail |
14041404
| `relay.audit.shipping_unverified` | (not bound) | warn | startup_fail |
14051405
| `relay.audit.shipping_stale` | (not bound) | error | readiness_fail |
1406+
| `relay.aggregates.privacy_budget_untracked` | warn | warn | warn |
14061407

14071408
`relay.audit.retention_local_only` fires when the audit sink is a local rotating `file` sink and `evidence.audit_offhost_shipping` is not declared: a local rotating file caps retention, and an attacker with host access can destroy the audit trail. `stdout` sinks are exempt (retention is the orchestrator's log pipeline's concern) and `syslog` sinks are exempt (forwarding is the syslog daemon's own surface).
14081409

14091410
`relay.audit.shipping_unverified` and `relay.audit.shipping_stale` read the ack cursor's observed health. `shipping_unverified` fires when any shipping target (`stdout`, `syslog`, or an attested local `file` sink) lacks `evidence.audit_ack_cursor_path`. It warns under `production` and refuses startup under `evidence_grade`, because a missing observation capability cannot heal at runtime. `shipping_stale` fires when a cursor is configured but is missing, unreadable, malformed, too old, too slow to read, or names a `last_acked_hash` other than the live keyed audit-chain tail. It fails readiness under `evidence_grade` and recovers when the trusted shipper advances a fresh cursor to the current tail. Neither hard gate is waivable. Runtime tail equality establishes that the claimed watermark belongs to this chain and the local backlog is zero; the unsigned local cursor is not cryptographic proof of remote receipt. Offline `doctor` cannot bind to a live chain and therefore reports a fresh cursor as `unverified`, never `ok`; an evidence-grade offline check consequently reports the hard shipping gate. The signed-bundle acceptance audit advances the tail before Relay serves requests, so the shipper must run independently of application readiness and acknowledge that boot record before `/ready` can return 200. Remediation: configure the cursor maintained by the off-host shipper, restore shipping, adjust `evidence.audit_ack_max_age_secs` if the cadence is legitimately slower, or repair a path or watermark mismatch. Removing the cursor does not satisfy `evidence_grade`.
14101411

1412+
`relay.aggregates.privacy_budget_untracked` fires when a dataset classified `personal`, `confidential`, or `secret` has a dataset-level aggregate with `access.aggregate_only_execution: true`. Only dataset-level declarations count: they are the only ones the aggregate query routes serve, so table-level and entity-level declarations raise no finding. Aggregate routes apply per-result k-anonymity suppression (`disclosure_control.min_cell_size`), but track no longitudinal query budget: `query_budget.tracked` is always false. This is a documented, accepted limitation, not a fixable misconfiguration, so it warns at every bound profile, including `evidence_grade`, rather than blocking startup or readiness. A deployment waiver naming this finding records the operator's acknowledgement of the limitation. See "Aggregates are not privacy-budgeted" in [Known limitations and non-guarantees](https://docs.registrystack.org/explanation/known-limitations/).
1413+
14111414
The current deployment profile, its findings, and active waivers are reported under `deployment` in the operations posture (`GET /admin/v1/posture`).
14121415

14131416
### Boot-time visibility
14141417

1415-
Reduced posture is loud at boot, not only visible on the posture surface. Every config load warns once per waiver-suppressed finding (`deployment.gate_waived`, with the finding id, reference, optional summary, and expiry), once per expired waiver (`deployment.waiver_expired`), and once when the profile is undeclared (`deployment.profile_undeclared`). The serve path additionally writes one operational audit record per waived gate at boot, once the audit pipeline exists: event `deployment.gate_waived` at audit path `/__events/deployment.gate_waived`, with `error_code` set to the gate id. That minimized audit record does not copy waiver metadata.
1418+
Reduced posture is loud at boot, not only visible on the posture surface. Every config load warns once per waiver-suppressed finding (`deployment.gate_waived`, with the finding id, reference, optional summary, and expiry), once per expired waiver (`deployment.waiver_expired`), and once when the profile is undeclared (`deployment.profile_undeclared`). A sensitive dataset's aggregate-only-execution aggregate additionally warns once per config load with `deployment.privacy_budget_untracked`, naming the finding id; a waived occurrence is reported through the generic `deployment.gate_waived` line instead, not a duplicate line. The serve path additionally writes one operational audit record per waived gate at boot, once the audit pipeline exists: event `deployment.gate_waived` at audit path `/__events/deployment.gate_waived`, with `error_code` set to the gate id. That minimized audit record does not copy waiver metadata.
14161419

14171420
This boot-time audit write inherits `audit.write_policy` (see below). Under `fail_closed` (the default), a failed write aborts startup. Under `availability_first`, the failure is logged (`audit.operational_event_write_failed`) and startup continues, so the durable record is best-effort; the per-gate boot log warnings above remain the guaranteed floor.
14181421

@@ -1730,6 +1733,8 @@ aggregates:
17301733

17311734
Supported aggregate functions include the configured V1 set used by tests and examples, such as `count`, `sum`, and `avg`. The runtime config key remains `indicators` for compatibility; public aggregate APIs expose these configured series as measures. `temporal_field` is optional; when present, native aggregate `temporal.from` and `temporal.to` are translated into the declared range-capable allowed filter for that source-entity field. Dataset measure and dimension discovery is derived from these aggregate declarations, so keep ids stable and labels consumer-friendly. Keep disclosure thresholds explicit and reviewable.
17321735

1736+
`disclosure_control` and `access.aggregate_only_execution` bound per-result k-anonymity suppression on a single query; they do not track a longitudinal query budget across repeated aggregate queries. On a `personal`, `confidential`, or `secret` dataset, a dataset-level `aggregate_only_execution` aggregate raises the `relay.aggregates.privacy_budget_untracked` deployment finding described in the findings catalog under "Deployment profile" above; see "Aggregates are not privacy-budgeted" in [Known limitations and non-guarantees](https://docs.registrystack.org/explanation/known-limitations/) for the underlying limitation.
1737+
17331738
### Spatial EDR aggregates
17341739

17351740
Spatial EDR exposure is opt-in. Requires `--features ogcapi-edr`.

crates/registry-relay/src/api/admin.rs

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1153,6 +1153,62 @@ datasets: []
11531153
assert!(ids.contains(&"relay.ingress.rate_limit_missing"));
11541154
}
11551155

1156+
/// A sensitive dataset with an aggregate-only-execution aggregate surfaces
1157+
/// `relay.aggregates.privacy_budget_untracked` on the posture surface at
1158+
/// `finding_warn`, the same generic path every other catalog gate takes.
1159+
#[test]
1160+
fn deployment_summary_reports_untracked_privacy_budget_finding() {
1161+
let yaml = r#"
1162+
server:
1163+
bind: "127.0.0.1:8080"
1164+
catalog:
1165+
title: "Test Registry"
1166+
base_url: "https://data.example.test"
1167+
publisher: "Test Ministry"
1168+
auth:
1169+
mode: api_key
1170+
api_keys: []
1171+
audit:
1172+
sink: stdout
1173+
datasets:
1174+
- id: sensitive_ds
1175+
title: "Sensitive Dataset"
1176+
description: "desc"
1177+
owner: "owner"
1178+
sensitivity: personal
1179+
access_rights: restricted
1180+
update_frequency: daily
1181+
tables:
1182+
- id: t1
1183+
source:
1184+
type: file
1185+
path: "data/t1.csv"
1186+
refresh:
1187+
mode: manual
1188+
schema:
1189+
fields: []
1190+
aggregates:
1191+
- id: agg1
1192+
description: "test aggregate"
1193+
source_entity: record
1194+
disclosure_control:
1195+
min_group_size: 2
1196+
access:
1197+
aggregate_only_execution: true
1198+
"#;
1199+
let mut config = parse_minimal_config(yaml);
1200+
config.deployment.profile = Some(DeploymentProfile::HostedLab);
1201+
let summary = deployment_summary(&config, ConfigSource::LocalFile);
1202+
let finding = summary["findings"]
1203+
.as_array()
1204+
.expect("findings array")
1205+
.iter()
1206+
.find(|finding| finding["id"] == "relay.aggregates.privacy_budget_untracked")
1207+
.expect("privacy-budget finding is reported");
1208+
assert_eq!(finding["severity"], "finding_warn");
1209+
assert_eq!(finding["status"], "active");
1210+
}
1211+
11561212
/// The full posture document is schema-valid for every declared profile and
11571213
/// for the undeclared default. `evidence_grade` from a local file would
11581214
/// trip a startup gate at load time, so its posture is exercised with a

crates/registry-relay/src/config/validate.rs

Lines changed: 157 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -584,6 +584,13 @@ fn log_deployment_boot_findings(evaluation: &crate::deployment::GateEvaluation)
584584
code = "deployment.profile_undeclared",
585585
"deployment profile is undeclared; no profile gates bind"
586586
);
587+
} else if finding.id == "relay.aggregates.privacy_budget_untracked" {
588+
tracing::warn!(
589+
code = "deployment.privacy_budget_untracked",
590+
finding = %finding.id,
591+
action = "record acknowledgement with a deployment waiver naming this finding; see explanation/known-limitations, \"Aggregates are not privacy-budgeted\"",
592+
"aggregate routes on a sensitive dataset apply per-result k-anonymity suppression but track no longitudinal privacy budget"
593+
);
587594
}
588595
}
589596
}
@@ -4726,6 +4733,96 @@ datasets: []
47264733
serde_saphyr::from_str(&deployment_config_yaml(extra)).expect("config parses")
47274734
}
47284735

4736+
/// A dataset with `sensitivity` classification carrying one dataset-level
4737+
/// aggregate with `access.aggregate_only_execution: true` (the only
4738+
/// aggregate level the query routes serve), plus `extra` (typically a
4739+
/// `deployment:` stanza) appended at the top level.
4740+
fn sensitive_aggregate_dataset_config_yaml(sensitivity: &str, extra: &str) -> String {
4741+
format!(
4742+
r#"
4743+
server:
4744+
bind: "127.0.0.1:8080"
4745+
catalog:
4746+
title: "Test Registry"
4747+
base_url: "https://data.example.test"
4748+
publisher: "Test Ministry"
4749+
auth:
4750+
mode: api_key
4751+
api_keys: []
4752+
audit:
4753+
sink: stdout
4754+
datasets:
4755+
- id: sensitive_ds
4756+
title: "Sensitive Dataset"
4757+
description: "desc"
4758+
owner: "owner"
4759+
sensitivity: {sensitivity}
4760+
access_rights: restricted
4761+
update_frequency: daily
4762+
tables:
4763+
- id: t1
4764+
source:
4765+
type: file
4766+
path: "data/t1.csv"
4767+
refresh:
4768+
mode: manual
4769+
primary_key: record_id
4770+
schema:
4771+
strict: true
4772+
fields:
4773+
- name: record_id
4774+
type: string
4775+
nullable: false
4776+
- name: region_code
4777+
type: string
4778+
nullable: true
4779+
aggregates:
4780+
- id: agg1
4781+
title: "Records by region"
4782+
description: "test aggregate"
4783+
source_entity: record
4784+
default_group_by:
4785+
- region_code
4786+
dimensions:
4787+
- id: region_code
4788+
label: Region
4789+
field: region_code
4790+
indicators:
4791+
- id: record_count
4792+
label: Records
4793+
function: count
4794+
column: id
4795+
unit_measure: records
4796+
disclosure_control:
4797+
min_group_size: 2
4798+
suppression: omit
4799+
access:
4800+
aggregate_only_execution: true
4801+
entities:
4802+
- name: record
4803+
table: t1
4804+
fields:
4805+
- name: id
4806+
from: record_id
4807+
- name: region_code
4808+
access:
4809+
metadata_scope: "sensitive_ds:metadata"
4810+
aggregate_scope: "sensitive_ds:aggregate"
4811+
read_scope: "sensitive_ds:rows"
4812+
api:
4813+
default_limit: 10
4814+
max_limit: 100
4815+
require_purpose_header: true
4816+
{extra}
4817+
"#
4818+
)
4819+
}
4820+
4821+
fn parse_sensitive_aggregate_dataset_config(sensitivity: &str, extra: &str) -> Config {
4822+
serde_saphyr::from_str(&sensitive_aggregate_dataset_config_yaml(sensitivity, extra))
4823+
.expect("config parses")
4824+
}
4825+
47294826
fn consultation_deployment_config_yaml(extra: &str) -> String {
47304827
format!(
47314828
r#"
@@ -5334,6 +5431,66 @@ deployment:
53345431
);
53355432
}
53365433

5434+
#[test]
5435+
fn sensitive_aggregate_only_execution_is_loud_in_the_boot_log() {
5436+
let config = parse_sensitive_aggregate_dataset_config(
5437+
"personal",
5438+
"deployment:\n profile: production",
5439+
);
5440+
let (result, rendered) = run_with_captured_logs(&config);
5441+
result.expect(
5442+
"an aggregate-only-execution finding on a sensitive dataset must not block startup",
5443+
);
5444+
assert!(
5445+
rendered.contains("deployment.privacy_budget_untracked"),
5446+
"expected deployment.privacy_budget_untracked in boot log: {rendered}"
5447+
);
5448+
assert!(
5449+
rendered.contains("relay.aggregates.privacy_budget_untracked"),
5450+
"expected the finding id in boot log: {rendered}"
5451+
);
5452+
assert!(
5453+
rendered.contains("known-limitations"),
5454+
"expected a known-limitations reference in boot log: {rendered}"
5455+
);
5456+
}
5457+
5458+
#[test]
5459+
fn public_sensitivity_keeps_the_privacy_budget_boot_log_quiet() {
5460+
let config = parse_sensitive_aggregate_dataset_config(
5461+
"public",
5462+
"deployment:\n profile: production",
5463+
);
5464+
let (result, rendered) = run_with_captured_logs(&config);
5465+
result.expect("a public-sensitivity dataset must not block startup");
5466+
assert!(
5467+
!rendered.contains("deployment.privacy_budget_untracked"),
5468+
"no privacy-budget line expected for a public-sensitivity dataset: {rendered}"
5469+
);
5470+
}
5471+
5472+
#[test]
5473+
fn waived_privacy_budget_finding_shows_the_generic_gate_waived_line_only() {
5474+
let config = parse_sensitive_aggregate_dataset_config(
5475+
"personal",
5476+
"deployment:\n profile: production\n waivers:\n - finding: relay.aggregates.privacy_budget_untracked\n reference: OPS-TEST-PRIVACY-BUDGET\n expires: \"2999-01-01\"",
5477+
);
5478+
let (result, rendered) = run_with_captured_logs(&config);
5479+
result.expect("a waived privacy-budget finding must not block startup");
5480+
assert!(
5481+
rendered.contains("deployment.gate_waived"),
5482+
"expected deployment.gate_waived in boot log: {rendered}"
5483+
);
5484+
assert!(
5485+
rendered.contains("relay.aggregates.privacy_budget_untracked"),
5486+
"expected the waived finding id in boot log: {rendered}"
5487+
);
5488+
assert!(
5489+
!rendered.contains("deployment.privacy_budget_untracked"),
5490+
"the dedicated privacy-budget line must not appear once waived: {rendered}"
5491+
);
5492+
}
5493+
53375494
#[test]
53385495
fn evidence_grade_via_signed_bundle_source_boots() {
53395496
// The same evidence_grade config that a local file rejects must validate

0 commit comments

Comments
 (0)