test(canonical-urls): a post-probe check answers for its probe, not the tree #2036

Merged
jaylouisw merged 1 commit from fix/1118-2102 into main 2026-11-18 21:12:02 +00:00
Owner

Fixes #2030

Three assertions ended with ok(check().ok, ...) after their probe was removed, and check().ok is the whole tree. One stale URL in a file no probe touches therefore produced four failures: the honest one at the top with its file and line, plus three that claimed "the probe left residue" and "the README is not restored" - while and the README is restored byte for byte, two lines below, passed on the same run.

Reproduced first

With docs/__unrelated_stale.md planted (the issue's own repro):

before:  48 passed, 4 failed
  x no file points at the old repository (1 problems)      <- correct, names file+line
  x and the tree is clean again once the probe is removed   <- wrong cause
  x and the README is restored                             <- wrong cause
  x and the tree is clean again once the probe is removed   <- wrong cause
after:   57 passed, 1 failed
  x no file points at the old repository (1 problems)      <- the only one that should speak
clean:   58 passed, 0 failed

Changed (test/canonical-urls.test.js)

  • problemsFor(rel) / problemsWithCode(code) scope a claim to what the probe touched.
  • drift-probe cleanup: "nothing is reported for the probe once it is removed", plus the probe file is gone from disk.
  • README probe: "no E_NO_CANONICAL_CLONE problem remains", plus a byte-for-byte restore check.
  • badge probe: "no E_BADGE_NO_WORKFLOW problem remains".
  • the whole-tree guard at :47 is untouched - that is where a stray URL belongs, once, with its file and line.
  • new regression block: a stale URL planted outside the probe paths is reported once, against the file and line it is written on, and is not blamed on a probe that is not on disk.

Non-vacuity

Mutating the README cleanup to a silent no-op fails both new README assertions, so they still detect the defect they name.

Gates

  • node scripts/syntax-check.js - 423 files, all OK
  • node test/canonical-urls.test.js - 58 passed, 0 failed
  • node scripts/run-tests.js --filter=canonical - 1 suite, 58 assertions passed, 0 failed

Not touched: .forgejo/workflows/ci.yml:677 still refers to the old assertion name "the tree is clean again" in a historical benchmark note. Left for the CI stream.

Fixes #2030 Three assertions ended with `ok(check().ok, ...)` after their probe was removed, and `check().ok` is the whole tree. One stale URL in a file no probe touches therefore produced four failures: the honest one at the top with its file and line, plus three that claimed "the probe left residue" and "the README is not restored" - while `and the README is restored byte for byte`, two lines below, passed on the same run. ### Reproduced first With `docs/__unrelated_stale.md` planted (the issue's own repro): ``` before: 48 passed, 4 failed x no file points at the old repository (1 problems) <- correct, names file+line x and the tree is clean again once the probe is removed <- wrong cause x and the README is restored <- wrong cause x and the tree is clean again once the probe is removed <- wrong cause after: 57 passed, 1 failed x no file points at the old repository (1 problems) <- the only one that should speak clean: 58 passed, 0 failed ``` ### Changed (`test/canonical-urls.test.js`) - `problemsFor(rel)` / `problemsWithCode(code)` scope a claim to what the probe touched. - drift-probe cleanup: "nothing is reported for the probe once it is removed", plus the probe file is gone from disk. - README probe: "no `E_NO_CANONICAL_CLONE` problem remains", plus a byte-for-byte restore check. - badge probe: "no `E_BADGE_NO_WORKFLOW` problem remains". - the whole-tree guard at `:47` is untouched - that is where a stray URL belongs, once, with its file and line. - new regression block: a stale URL planted outside the probe paths is reported **once**, against the file and line it is written on, and is not blamed on a probe that is not on disk. ### Non-vacuity Mutating the README cleanup to a silent no-op fails both new README assertions, so they still detect the defect they name. ### Gates - `node scripts/syntax-check.js` - 423 files, all OK - `node test/canonical-urls.test.js` - 58 passed, 0 failed - `node scripts/run-tests.js --filter=canonical` - 1 suite, 58 assertions passed, 0 failed Not touched: `.forgejo/workflows/ci.yml:677` still refers to the old assertion name `"the tree is clean again"` in a historical benchmark note. Left for the CI stream.
test(canonical-urls): a post-probe check answers for its probe, not the tree (#2030)
All checks were successful
PR rebase / Rebase onto main and arm auto-merge (pull_request) Successful in 4s
CI / Classify the change (pull_request) Successful in 6s
CI / Server smoke test (pull_request) Successful in 8s
CI / Docker build (pull_request) Successful in 21s
CI / Test (Node 20) (pull_request) Successful in 51s
CI / Test (Node 22) (pull_request) Successful in 53s
CI / Test (Node 18) (pull_request) Successful in 57s
CI / Arm auto-merge (pull_request) Successful in 3s
d678cdc85c
Three assertions ended with `ok(check().ok, ...)` after their probe was removed, and
`check().ok` is the whole tree. So one stale URL anywhere - in a file no probe touches -
made the suite report four failures: the honest one at the top with its file and line,
plus three that claimed "the probe left residue" and "the README is not restored", while
the byte-for-byte README assertion two lines below passed on the same run. That is how
PR #1980 read as broken cleanup instead of one line of prose.

- the probe cleanup checks now assert nothing is reported for the probe's own path
- the README check asserts no E_NO_CANONICAL_CLONE remains (and the bytes match)
- the badge check asserts no E_BADGE_NO_WORKFLOW remains
- the whole-tree guard at :47 is untouched: that is where a stray URL belongs, once
- a regression case plants a stale URL outside the probe paths and pins that it is
  reported once, against its own file and line, and is not blamed on a probe on disk

Measured: with `docs/__unrelated_stale.md` planted, 48 passed / 4 failed before,
57 passed / 1 failed after - the single remaining failure being the :47 guard that
should be the one to speak. Clean tree: 58 passed, 0 failed.
forgejo-actions scheduled this pull request to auto merge when all checks succeed 2026-11-18 21:10:28 +00:00
jaylouisw deleted branch fix/1118-2102 2026-11-18 21:12:03 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
jaylouisw/packetforge!2036
No description provided.