Nothing checked that docs/CLI.md's show forms still parse #1479

Closed
opened 2026-11-12 04:32:50 +00:00 by jaylouisw · 1 comment
Owner

The gap

test/help-completeness.test.js guards the console's own help index in both directions — but its command list is hand-written on purpose ("it exists to test the console, not to parse prose"). Nothing checked the other paper promise: docs/CLI.md, the page an operator reads on Forgejo before they ever open the app. A row that advertises a spelling the parser no longer knows sends them to a console that answers % Invalid show command — which is the moment the documentation stops being trustworthy.

Measured on 68ec19a0: the page names 46 show forms after splitting alternatives, and typing all of them at a router and a switch is 92 commands. All of them parse today. So this iteration adds the guard, not a bug fix.

What the test does

test/docs-show-commands.test.js reads docs/CLI.md, enumerates every backticked show … form, splits the alternatives (show vlan [brief] | spanning-tree), types both spellings of an optional keyword, fills placeholders with real values from a real lab, and runs each one against a router and a switch.

It fails on the three answers that mean the docs and the parser disagree:

% Invalid show command
% Unrecognized …
a throw

A legitimate refusal is not a failure, and the distinction is the point: a switch has no show ip ospf and says so in a sentence. 56 of the 92 answers are refusals of that kind today, and the test asserts none of them complains about the command itself.

Evidence it bites

$ node test/docs-show-commands.test.js
46 show forms, 92 commands typed across a router and a switch
7 passed, 0 failed

# one letter drifted in docs/CLI.md: show spanning-tree -> show spanning-trees
$ node test/docs-show-commands.test.js
✗ no documented form is refused as a spelling the parser does not know
  [[r1] show spanning-trees -> % Invalid show command: spanning-trees | [sw] …]
6 passed, 1 failed

A rule the page now states about itself

Writing this surfaced a self-referential detail worth recording: the prose that documents the guard is itself full of backticked show … examples, so adding the section broke the section's own test twice — once from show … with an ellipsis, once from a deliberately broken example. The rule that resolves it is now written on the page: any show form in backticks is a promise, so breaking examples are written as output in a fenced block, not as commands. The guard is the docs' contract with itself.

Also probed this iteration with no defect found: show power inline on the 152 catalogue models that declare PoE ports but no published budget (it says "budget not modelled on X: the ports are reported, the power they could supply is not" — honest, not a fake zero, and poeUnpowered really does block traffic); and every port's advertised rate against the media its own cage can carry (0 mismatches across 15,300 ports).

Full suite: 238 suites, 8848 assertions, 0 failed. Syntax 306 files OK.

## The gap `test/help-completeness.test.js` guards the console's own `help` index in both directions — but its command list is hand-written on purpose ("it exists to test the console, not to parse prose"). Nothing checked the *other* paper promise: **`docs/CLI.md`**, the page an operator reads on Forgejo before they ever open the app. A row that advertises a spelling the parser no longer knows sends them to a console that answers `% Invalid show command` — which is the moment the documentation stops being trustworthy. Measured on `68ec19a0`: the page names **46 `show` forms** after splitting alternatives, and typing all of them at a router and a switch is **92 commands**. All of them parse today. So this iteration adds the guard, not a bug fix. ## What the test does `test/docs-show-commands.test.js` reads `docs/CLI.md`, enumerates every backticked `show …` form, splits the alternatives (`show vlan [brief] | spanning-tree`), types both spellings of an optional keyword, fills placeholders with real values from a real lab, and runs each one against a router and a switch. It fails on the three answers that mean the docs and the parser disagree: ``` % Invalid show command % Unrecognized … a throw ``` **A legitimate refusal is not a failure**, and the distinction is the point: a switch has no `show ip ospf` and says so in a sentence. 56 of the 92 answers are refusals of that kind today, and the test asserts none of them complains about the command itself. ## Evidence it bites ``` $ node test/docs-show-commands.test.js 46 show forms, 92 commands typed across a router and a switch 7 passed, 0 failed # one letter drifted in docs/CLI.md: show spanning-tree -> show spanning-trees $ node test/docs-show-commands.test.js ✗ no documented form is refused as a spelling the parser does not know [[r1] show spanning-trees -> % Invalid show command: spanning-trees | [sw] …] 6 passed, 1 failed ``` ## A rule the page now states about itself Writing this surfaced a self-referential detail worth recording: the prose that documents the guard is itself full of backticked `show …` examples, so **adding the section broke the section's own test** twice — once from `show …` with an ellipsis, once from a deliberately broken example. The rule that resolves it is now written on the page: *any show form in backticks is a promise, so breaking examples are written as output in a fenced block, not as commands.* The guard is the docs' contract with itself. Also probed this iteration with no defect found: `show power inline` on the 152 catalogue models that declare PoE ports but no published budget (it says "budget not modelled on X: the ports are reported, the power they could supply is not" — honest, not a fake zero, and `poeUnpowered` really does block traffic); and every port's advertised rate against the media its own cage can carry (0 mismatches across 15,300 ports). Full suite: 238 suites, 8848 assertions, 0 failed. Syntax 306 files OK.
Author
Owner

Landed as #1480 (0de01b45). Post-merge on main: docs-show-commands 7/0, tree clean.

The detail worth keeping: writing the guard broke the guard twice, because the prose documenting it is full of backticked show examples. The page now states the rule it is held to - any show form in backticks is a promise, so breaking examples are written as output in a fenced block.

Landed as #1480 (`0de01b45`). Post-merge on `main`: `docs-show-commands` 7/0, tree clean. The detail worth keeping: writing the guard broke the guard twice, because the prose documenting it is full of backticked show examples. The page now states the rule it is held to - any show form in backticks is a promise, so breaking examples are written as output in a fenced block.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
jaylouisw/packetforge#1479
No description provided.