ci: a classifier that dies must run the gate, not skip it #1972

Merged
jaylouisw merged 1 commit from ci/1118-1020 into main 2026-11-18 10:39:43 +00:00
Owner

A failing Classify the change job used to stand the whole gate down instead of running it: the five jobs below depend on it, so a failed dependency made them skipped, and the pull request carried one red X and zero coverage of its diff.

Baseline (before)

  • In the wild - run 6076, task 15604 (Classify the change, pull #1957, 2026-11-18T08:21:26Z): failed after 855 s, and it is the only task that run ever created - no log survived either (dbfs open "actions_log/jaylouisw/packetforge/f4/15604.log.zst": file does not exist). No test leg, no smoke, no Docker build was ever dispatched.
  • Reproduced on purpose - run 6201, tasks 15979-15985 (branch probe-classify-fail, the same file with a 4-line exit 1 probe in the classify job):
task job duration outcome
15979 Classify the change 2 s failure
15980-15982 Test (Node 18/20/22) 0-1 s skipped
15983 Server smoke test 0 s skipped
15984 Docker build 0 s skipped
15985 Arm auto-merge 0 s skipped

After

always() on the if: of test, smoke and docker. A failed classifier now runs the full matrix - the same conservative answer classify-change.js already prints whenever it cannot read a diff.

Same probe, same 4 lines, on the fixed file - run 6202, tasks 15986-15992:

task job duration outcome
15986 Classify the change 1 s failure
15987 Test (Node 18) 57 s success
15988 Test (Node 20) 49 s success
15989 Test (Node 22) 53 s success
15990 Server smoke test 8 s success
15991 Docker build 20 s success

Normal path unchanged - run 6203 (this branch, workflow_dispatch, 7 tasks, wall clock 66 s, every job green) against the baseline run 6172 (main, dispatch at 10:27, 7 tasks, wall clock 58 s, same 7 tasks, same statuses, arm skipped on both because neither is a pull_request event).

Coverage is unchanged

No check is removed, relaxed or made conditional. live=false and data_only=true are outputs the classifier writes only when it succeeded and positively said so, and both still stand the legs down; an absent output has always meant "run". A cancelled run still cancels its jobs - always() describes a failed dependency, not a cancelled run. The only behaviour that changes is the one where the gate previously produced no verdict at all.

The two probe branches (probe-classify-fail, probe-classify-fail-fixed) are throwaway evidence and are deleted after this opens.

A failing `Classify the change` job used to stand the whole gate down instead of running it: the five jobs below depend on it, so a failed dependency made them `skipped`, and the pull request carried one red X and zero coverage of its diff. ## Baseline (before) * In the wild - **run 6076**, task **15604** (`Classify the change`, pull #1957, 2026-11-18T08:21:26Z): failed after **855 s**, and it is the **only task that run ever created** - no log survived either (`dbfs open "actions_log/jaylouisw/packetforge/f4/15604.log.zst": file does not exist`). No test leg, no smoke, no Docker build was ever dispatched. * Reproduced on purpose - **run 6201**, tasks **15979-15985** (branch `probe-classify-fail`, the same file with a 4-line `exit 1` probe in the classify job): | task | job | duration | outcome | |---|---|---|---| | 15979 | Classify the change | 2 s | failure | | 15980-15982 | Test (Node 18/20/22) | 0-1 s | **skipped** | | 15983 | Server smoke test | 0 s | **skipped** | | 15984 | Docker build | 0 s | **skipped** | | 15985 | Arm auto-merge | 0 s | skipped | ## After `always()` on the `if:` of `test`, `smoke` and `docker`. A failed classifier now runs the full matrix - the same conservative answer `classify-change.js` already prints whenever it cannot read a diff. Same probe, same 4 lines, on the fixed file - **run 6202**, tasks **15986-15992**: | task | job | duration | outcome | |---|---|---|---| | 15986 | Classify the change | 1 s | failure | | 15987 | Test (Node 18) | 57 s | success | | 15988 | Test (Node 20) | 49 s | success | | 15989 | Test (Node 22) | 53 s | success | | 15990 | Server smoke test | 8 s | success | | 15991 | Docker build | 20 s | success | Normal path unchanged - **run 6203** (this branch, workflow_dispatch, 7 tasks, wall clock 66 s, every job green) against the baseline **run 6172** (main, dispatch at 10:27, 7 tasks, wall clock 58 s, same 7 tasks, same statuses, arm skipped on both because neither is a `pull_request` event). ## Coverage is unchanged No check is removed, relaxed or made conditional. `live=false` and `data_only=true` are outputs the classifier writes only when it *succeeded* and positively said so, and both still stand the legs down; an absent output has always meant "run". A cancelled run still cancels its jobs - `always()` describes a failed dependency, not a cancelled run. The only behaviour that changes is the one where the gate previously produced no verdict at all. The two probe branches (`probe-classify-fail`, `probe-classify-fail-fixed`) are throwaway evidence and are deleted after this opens.
ci: a classifier that dies must run the gate, not skip it
All checks were successful
CI / Classify the change (pull_request) Successful in 8s
PR rebase / Rebase onto main and arm auto-merge (pull_request) Successful in 4s
CI / Server smoke test (pull_request) Successful in 5s
CI / Docker build (pull_request) Successful in 21s
CI / Test (Node 20) (pull_request) Successful in 54s
CI / Test (Node 22) (pull_request) Successful in 58s
CI / Test (Node 18) (pull_request) Successful in 1m2s
CI / Arm auto-merge (pull_request) Successful in 10s
ab0421f186
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
jaylouisw/packetforge!1972
No description provided.