Every show form on the page is typed at a console #1480

Merged
jaylouisw merged 1 commit from test/docs-show-commands into main 2026-11-12 04:34:40 +00:00
Owner

What was missing

test/help-completeness.test.js guards the console's own help index in both directions — but its command list is hand-written on purpose ("it exists to test the console, not to parse prose"). Nothing checked the other paper promise: docs/CLI.md, the page an operator reads on Forgejo before they ever open the app. A row that advertises a spelling the parser no longer knows sends them to a console that answers % Invalid show command — which is the moment the documentation stops being trustworthy.

Measured on 68ec19a0: the page names 46 show forms after splitting alternatives, and typing all of them at a router and a switch is 92 commands. All of them parse today. So this iteration adds the guard, not a bug fix.

What it does

test/docs-show-commands.test.js reads docs/CLI.md, enumerates every backticked show … form, splits the alternatives (show vlan [brief] | spanning-tree), types both spellings of an optional keyword, fills placeholders with real values from a real lab, and runs each one against a router and a switch.

It fails on the three answers that mean the docs and the parser disagree:

% Invalid show command
% Unrecognized …
a throw

A legitimate refusal is not a failure, and the distinction is the point: a switch has no show ip ospf and says so in a sentence. 56 of the 92 answers are refusals of that kind today, and the test asserts none of them complains about the command itself.

Evidence it bites

$ node test/docs-show-commands.test.js
46 show forms, 92 commands typed across a router and a switch
7 passed, 0 failed

# one letter drifted in docs/CLI.md: show spanning-tree -> show spanning-trees
$ node test/docs-show-commands.test.js
✗ no documented form is refused as a spelling the parser does not know
  [[r1] show spanning-trees -> % Invalid show command: spanning-trees | [sw] …]
6 passed, 1 failed

A rule the page now states about itself

Writing this surfaced a self-referential detail worth recording: the prose that documents the guard is itself full of backticked show … examples, so adding the section broke the section's own test twice — once from show … with an ellipsis, once from a deliberately broken example. The rule that resolves it is now written on the page: any show form in backticks is a promise, so breaking examples are written as output in a fenced block, not as commands. The guard is the docs' contract with itself.

Also probed this iteration with no defect found: show power inline on the 152 catalogue models that declare PoE ports but no published budget (it says "budget not modelled on X: the ports are reported, the power they could supply is not" — honest, not a fake zero, and poeUnpowered really does block traffic); and every port's advertised rate against the media its own cage can carry (0 mismatches across 15,300 ports).

Full suite: 238 suites, 8848 assertions, 0 failed. Syntax 306 files OK.

Reviewer note: this changes no engine behaviour — it types the documentation at a real console and fails when the two disagree. The rule stated at the end of docs/CLI.md is what keeps the prose that describes the guard from violating it.

## What was missing `test/help-completeness.test.js` guards the console's own `help` index in both directions — but its command list is hand-written on purpose ("it exists to test the console, not to parse prose"). Nothing checked the *other* paper promise: **`docs/CLI.md`**, the page an operator reads on Forgejo before they ever open the app. A row that advertises a spelling the parser no longer knows sends them to a console that answers `% Invalid show command` — which is the moment the documentation stops being trustworthy. Measured on `68ec19a0`: the page names **46 `show` forms** after splitting alternatives, and typing all of them at a router and a switch is **92 commands**. All of them parse today. So this iteration adds the guard, not a bug fix. ## What it does `test/docs-show-commands.test.js` reads `docs/CLI.md`, enumerates every backticked `show …` form, splits the alternatives (`show vlan [brief] | spanning-tree`), types both spellings of an optional keyword, fills placeholders with real values from a real lab, and runs each one against a router and a switch. It fails on the three answers that mean the docs and the parser disagree: ``` % Invalid show command % Unrecognized … a throw ``` **A legitimate refusal is not a failure**, and the distinction is the point: a switch has no `show ip ospf` and says so in a sentence. 56 of the 92 answers are refusals of that kind today, and the test asserts none of them complains about the command itself. ## Evidence it bites ``` $ node test/docs-show-commands.test.js 46 show forms, 92 commands typed across a router and a switch 7 passed, 0 failed # one letter drifted in docs/CLI.md: show spanning-tree -> show spanning-trees $ node test/docs-show-commands.test.js ✗ no documented form is refused as a spelling the parser does not know [[r1] show spanning-trees -> % Invalid show command: spanning-trees | [sw] …] 6 passed, 1 failed ``` ## A rule the page now states about itself Writing this surfaced a self-referential detail worth recording: the prose that documents the guard is itself full of backticked `show …` examples, so **adding the section broke the section's own test** twice — once from `show …` with an ellipsis, once from a deliberately broken example. The rule that resolves it is now written on the page: *any show form in backticks is a promise, so breaking examples are written as output in a fenced block, not as commands.* The guard is the docs' contract with itself. Also probed this iteration with no defect found: `show power inline` on the 152 catalogue models that declare PoE ports but no published budget (it says "budget not modelled on X: the ports are reported, the power they could supply is not" — honest, not a fake zero, and `poeUnpowered` really does block traffic); and every port's advertised rate against the media its own cage can carry (0 mismatches across 15,300 ports). Full suite: 238 suites, 8848 assertions, 0 failed. Syntax 306 files OK. Reviewer note: this changes no engine behaviour — it types the documentation at a real console and fails when the two disagree. The rule stated at the end of `docs/CLI.md` is what keeps the prose that describes the guard from violating it.
Every show form on the page is typed at a console (#1479)
All checks were successful
CI / Classify the change (pull_request) Successful in 11s
PR rebase / Rebase onto main and arm auto-merge (pull_request) Successful in 13s
CI / Test (Node 20) (pull_request) Successful in 3m6s
CI / Test (Node 18) (pull_request) Successful in 3m24s
CI / Server smoke test (pull_request) Successful in 27s
CI / Docker build (pull_request) Successful in 48s
CI / Test (Node 22) (pull_request) Successful in 2m53s
0de01b4590
help-completeness guards the console's own index, and its command list is
hand-written on purpose - it tests the console, not prose. Nothing checked the
other paper promise: docs/CLI.md, the page an operator reads before they open
the app. A row advertising a spelling the parser no longer knows sends them to
"% Invalid show command", which is the moment the documentation stops being
trustworthy.

The page names 46 show forms after splitting alternatives, and typing all of
them at a router and a switch is 92 commands; all of them parse today, so this
adds the guard rather than fixing a bug.

  * reads docs/CLI.md, enumerates every backticked show form, splits the
    alternatives, types both spellings of an optional keyword, fills the
    placeholders with real values from a real lab, and runs each one against a
    router and a switch;
  * fails on an invalid command, an unrecognised command, or a throw - and on
    nothing else: a legitimate refusal is not a failure. A switch has no
    `show ip ospf` and says so in a sentence, and 56 of the 92 answers are
    refusals of that kind. The test asserts that none of them complains about
    the command itself;
  * proves both directions: an added row would be typed (checked against the
    same enumerator on an edited copy), and one letter drifted in the real page
    fails the suite naming the form and both devices.

Writing it surfaced a self-referential detail now stated on the page: the prose
documenting the guard is full of backticked show examples, so adding the section
broke its own test twice. The rule: any show form in backticks is a promise, so
breaking examples are written as output in a fenced block, not as commands.

Also probed, no defect: `show power inline` on the 152 models that declare PoE
ports but publish no budget (honest "budget not modelled", and poeUnpowered
really does block traffic), and every port's advertised rate against its own
cage's media (0 mismatches in 15,300 ports).

Suite 238 suites, 8848 assertions, 0 failed. Syntax 306 files OK.
jaylouisw scheduled this pull request to auto merge when all checks succeed 2026-11-12 04:34:24 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
jaylouisw/packetforge!1480
No description provided.