Skip to content

Expose buildDiff in /eval/:id?compare=X JSON response - #1839

Open
luochen1990 wants to merge 1 commit into
NixOS:masterfrom
luochen1990:feat-eval-diff-json-api
Open

Expose buildDiff in /eval/:id?compare=X JSON response#1839
luochen1990 wants to merge 1 commit into
NixOS:masterfrom
luochen1990:feat-eval-diff-json-api

Conversation

@luochen1990

Copy link
Copy Markdown

Motivation

Tools that track channel health (e.g. deciding whether bumping a NixOS host's `nixpkgs` input to a newer eval is safe) need to know which builds changed status between two evals. The diff is already computed server-side, but currently only reachable via the HTML view — which truncates each category to 250 entries and requires parsing HTML tables.

This exposes the existing `buildDiff` result in the JSON response so API clients can consume it directly.

What changes

`GET /eval/:id?compare=X` with `Accept: application/json` now includes a `diff` field alongside the existing eval fields:

Request Before After
`/eval/:id` (no compare) JSON eval fields + build IDs unchanged
`/eval/:id?compare=X` JSON eval fields + build IDs + `diff` field
`/eval/:id` HTML unchanged unchanged

The `diff` field is gated on the explicit `compare` query param (not on whether a previous eval exists). Without `compare`, the JSON response is unchanged — the controller's default behavior of auto-resolving a previous eval for the HTML diff tabs does not leak into the JSON entity.

Per-build field set: `{id, job}`

Each build in the diff is summarized to `{id, job}` — the minimal set that lets consumers:

  • `job`: identify the attrpath + system (e.g. `nixpkgs.python3.x86_64-linux` → attrpath `python3` by stripping the arch suffix)
  • `id`: link to `/build/{id}` for details (no bulk job→id API exists in hydra, so omitting `id` would force consumers back to per-build queries)

The `removed` category has a reduced shape `{job, system}` since those jobs don't exist in the current eval (no `id`).

Example

```bash
curl -H 'Accept: application/json' \
'https://hydra.nixos.org/eval/1827224?compare=1827222'
```

```json
{
"id": 1827224,
"builds": [336650910],
"diff": {
"stillSucceed": [
{"id": 336650910, "job": "tests/api.x86_64-linux"}
],
"stillFail": [],
"nowSucceed": [],
"nowFail": [],
"new": [],
"aborted": [],
"unfinished": [],
"removed": [],
"totalAborted": 0,
"totalFailed": 0,
"totalQueued": 0
}
}
```

Unlike the HTML view, the `diff` field is not truncated — `stillFail` returns every failing build, matching what `buildDiff` (`src/lib/Hydra/Helper/BuildDiff.pm`) already computes.

Testing

New test `t/Hydra/Controller/JobsetEval/diff.t`:

  • without `compare`: no `diff` field (regression guard — the controller auto-resolves eval2 even without compare, so naive gating on eval2 would leak)
  • with `compare`: `diff` field present with all 11 expected keys
  • each build has exactly `{id, job}`; `removed` has `{job, system}`

Full suite (`meson test` + perlcritic severity=1): 99 files, 1558 assertions, 0 failures.

The buildDiff result (stillFail/nowFail/etc.) was already computed and
stashed for the HTML template, but not included in the JSON entity.
API clients had to scrape the HTML, which also truncates each category
to 250 entries.

Add a 'diff' field to the JSON response when compare is specified.
Each build is summarized to {id, job} — the minimal set for consumers
to identify the build and map it to a nixpkgs attrpath.

Without compare, the response is unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant