the importer asks the engine what a capability is (#1522) #1528

Merged
jaylouisw merged 1 commit from fix/1522-import-devices-capabilities into main 2026-11-15 18:25:06 +00:00
Owner

What was wrong

scripts/import-devices.js kept its own copy of the capability vocabulary — 17 tokens against the engine's 15 — and then stripped any token it did not recognise into spec.role. Three of the five tokens it did not recognise (vlan, lacp, stp) are capabilities the engine does implement, so declaring 802.1Q, LACP or STP made a device poorer and the run reported success.

Reproduced before the fix

Fixture: a switch declaring ["l2-forward","l3-forward","cli","vlan","lacp","stp"], imported with PF_INCOMING/PF_DEVICES pointed at a throwaway catalogue.

$ node scripts/import-devices.js --apply
Scanned 1 file(s), 1 spec(s)
Added: 1  duplicates: 0  rejected: 0
  by vendor: acme=1

Written spec (devices/acme/sw-24g.json) — three capabilities gone, filed under a field nothing reads:

"capabilities": ["l2-forward", "l3-forward", "cli"],
"role": ["vlan", "lacp", "stp"]

Same fixture after the fix

$ node scripts/import-devices.js --apply
Scanned 1 file(s), 1 spec(s)
Added: 1  duplicates: 0  rejected: 0
  by vendor: acme=1
"capabilities": ["l2-forward", "l3-forward", "cli", "vlan", "lacp", "stp"]

And a spec declaring a token the engine does not implement is now named rather than counted:

$ node scripts/import-devices.js --apply
Scanned 1 file(s), 2 spec(s)
Added: 1  duplicates: 0  rejected: 1
  by vendor: acme=1

Rejected:
  - acme/TYPO-8  (devices-acme.json)
      capability "l3-foward" looks like a typo of "l2-forward"

1 spec(s) had warnings (still imported):
  - acme/POE-8
      capability "802.3at" is not one of the 15 capabilities engine/device.js implements; kept as declared, so the simulator ignores it
      capability "midspan" is not one of the 15 capabilities engine/device.js implements; kept as declared, so the simulator ignores it

What changed

  • The vocabulary is the engine's. const { CAPABILITIES } = require('../engine'). engine/index.js already re-exported it, so nothing had to be added to the engine surface. A future change to engine/device.js now reaches the importer with no second edit.
  • The strip is gone. vlan, lacp and stp survive, because the engine says they are capabilities. test/import-devices-capabilities.test.js asserts the importer's accepted set is the engine's set at runtime (set equality and object identity, so an equal-looking copy cannot creep back).
  • Aliases are checked at load. Every value in CAPABILITY_ALIASES must be a token the engine lists, so renaming an engine token fails here instead of silently turning every aliased spec into an unknown.
  • Unknown tokens are kept and named. A token that is not in CAPABILITIES and is not a near-miss of one is a description the engine has no concept of. It stays in spec.capabilities exactly as the source declared it, and the run prints the token and the model. A near-miss is still a typo and still rejected.
  • A tripwire before the write. capabilitiesLost(declared, written) compares what the validator says the source declared against what is about to be written; a difference refuses the spec and exits non-zero. It cannot fire today — which is the point, it is the check that did not exist when the strip was silent.
  • Warnings are named, not counted. "N spec(s) had minor warnings (still imported)" was the line that made the original loss invisible; it now lists each model and each message.
  • scripts/import-devices.js exports validateSpec, CAPABILITIES, CAPABILITY_ALIASES, normaliseCapabilities, capabilitiesLost and editDistance (main still runs only under require.main === module), so the test can measure the real pipeline rather than a copy of it.
  • Docs: docs/DEVICE_SCHEMA.md capability tables now derive from engine/device.js and document the kept-and-named rule; docs/MAINTAINING.md step 3 says the importer never drops a declared capability.

The honest choice on vlan/lacp/stp

#1526 notes that no code branches on the three tokens and that zero specs use them. That is an argument for #1526 (wire them, or remove them from CAPABILITIES — and if removed there, this script loses them automatically, which is exactly the coupling wanted). It is not an argument for one script deleting a token the engine publishes. Until #1526 decides, the importer keeps what the engine lists: the fix here is that the importer stops having an opinion, not that it gains a different one.

Related: the field the tokens were moved to is dead. Top-level spec.role is read by nothing (engine/device.js role is an interface field — 'wan' | 'lan'), so the strip did not even preserve the description it claimed to preserve; 79 catalogue specs carry a role array that no code reads. Nothing under devices/ is edited here.

Tokens that are not capabilities

Deriving from the engine means the importer now reports the power descriptors it used to accept in silence. Keeping them (rather than dropping them) is deliberate: dropping a fact the datasheet declares is the same data loss this issue is about. Census of devices/ at this commit — 2,039 specs, 18 distinct capability tokens:

token specs in CAPABILITIES?
passive-poe 284 no
802.3af 230 no
802.3at 222 no
802.3bt 53 no
poe-injector 10 no
wifi-ap 1 no — it is an alias the importer maps to wireless

588 specs carry at least one of those, so they now produce a named warning on re-import. That is a true statement about the catalogue, and the fix belongs in CAPABILITIES (if the token should grant behaviour) or in a catalogue sweep — not in a second list inside this script. No file under devices/ or catalogue/ is touched by this PR; if Jay wants the descriptors to become real capabilities, or to be swept out of the catalogue, that is a separate decision and a separate PR.

How many copies of a capability token list are in scripts/

Grepped every token-list shape in scripts/:

  • 1 restated vocabulary — scripts/import-devices.js:38, the bug. Now removed.
  • 44 inline capabilities: [...] literal arrays in 3 files: scripts/merge-research.js (42 device records with hand-written token lists), scripts/missing-devices.js:243 (['host'] placeholder for an unresearched stub), scripts/request-queue.js:142 ([] placeholder). Those are uses of the vocabulary, not a copy of it, so they are out of scope here — but merge-research.js writes 42 specs with hand-picked tokens and is the next place this class of drift will show up.

Tests

New test/import-devices-capabilities.test.js, 29 assertions:

  • a switch declaring 802.1Q/LACP/STP imports with all six capabilities intact and no role key;
  • the importer's vocabulary equals the engine's at runtime, compared to require('../engine').CAPABILITIES (not to a hardcoded list), plus identity, plus "every alias lands on a real token";
  • a fixture declaring tokens the engine does not know produces a named warning naming both token and model, in the human output and in --json, keeps the tokens, and does not warn about poe-pse;
  • capabilitiesLost unit-tested in both directions; a near-miss typo still rejected.

npm test: 248 suites, 9,137 assertions, 0 failed before → 249 suites, 9,166 assertions, 0 failed after. npm run syntax: 319 files OK.

Contradictions with the issue text

  • The issue says 1,661 specs and 19 distinct capability tokens; devices/ holds 2,039 specs and 18 distinct tokens at this commit. The engine-side claim is exact: 12 of the engine's 15 tokens are used by some spec, and vlan/lacp/stp by none.
  • The "1 spec(s) had minor warnings" line did not reproduce on my fixture — mine carried label and description, so the run printed no warning line at all. The loss is the same, and it is quieter than the issue says.

Fixes #1522

## What was wrong `scripts/import-devices.js` kept its **own copy** of the capability vocabulary — 17 tokens against the engine's 15 — and then stripped any token it did not recognise into `spec.role`. Three of the five tokens it did not recognise (`vlan`, `lacp`, `stp`) are capabilities the engine *does* implement, so declaring 802.1Q, LACP or STP made a device poorer and the run reported success. ## Reproduced before the fix Fixture: a switch declaring `["l2-forward","l3-forward","cli","vlan","lacp","stp"]`, imported with `PF_INCOMING`/`PF_DEVICES` pointed at a throwaway catalogue. ``` $ node scripts/import-devices.js --apply Scanned 1 file(s), 1 spec(s) Added: 1 duplicates: 0 rejected: 0 by vendor: acme=1 ``` Written spec (`devices/acme/sw-24g.json`) — three capabilities gone, filed under a field nothing reads: ```json "capabilities": ["l2-forward", "l3-forward", "cli"], "role": ["vlan", "lacp", "stp"] ``` ## Same fixture after the fix ``` $ node scripts/import-devices.js --apply Scanned 1 file(s), 1 spec(s) Added: 1 duplicates: 0 rejected: 0 by vendor: acme=1 ``` ```json "capabilities": ["l2-forward", "l3-forward", "cli", "vlan", "lacp", "stp"] ``` And a spec declaring a token the engine does *not* implement is now named rather than counted: ``` $ node scripts/import-devices.js --apply Scanned 1 file(s), 2 spec(s) Added: 1 duplicates: 0 rejected: 1 by vendor: acme=1 Rejected: - acme/TYPO-8 (devices-acme.json) capability "l3-foward" looks like a typo of "l2-forward" 1 spec(s) had warnings (still imported): - acme/POE-8 capability "802.3at" is not one of the 15 capabilities engine/device.js implements; kept as declared, so the simulator ignores it capability "midspan" is not one of the 15 capabilities engine/device.js implements; kept as declared, so the simulator ignores it ``` ## What changed - **The vocabulary is the engine's.** `const { CAPABILITIES } = require('../engine')`. `engine/index.js` already re-exported it, so nothing had to be added to the engine surface. A future change to `engine/device.js` now reaches the importer with no second edit. - **The strip is gone.** `vlan`, `lacp` and `stp` survive, because the engine says they are capabilities. `test/import-devices-capabilities.test.js` asserts the importer's accepted set *is* the engine's set at runtime (set equality **and** object identity, so an equal-looking copy cannot creep back). - **Aliases are checked at load.** Every value in `CAPABILITY_ALIASES` must be a token the engine lists, so renaming an engine token fails here instead of silently turning every aliased spec into an unknown. - **Unknown tokens are kept and named.** A token that is not in `CAPABILITIES` and is not a near-miss of one is a description the engine has no concept of. It stays in `spec.capabilities` exactly as the source declared it, and the run prints the token and the model. A near-miss is still a typo and still rejected. - **A tripwire before the write.** `capabilitiesLost(declared, written)` compares what the validator says the source declared against what is about to be written; a difference refuses the spec and exits non-zero. It cannot fire today — which is the point, it is the check that did not exist when the strip was silent. - **Warnings are named, not counted.** `"N spec(s) had minor warnings (still imported)"` was the line that made the original loss invisible; it now lists each model and each message. - `scripts/import-devices.js` exports `validateSpec`, `CAPABILITIES`, `CAPABILITY_ALIASES`, `normaliseCapabilities`, `capabilitiesLost` and `editDistance` (main still runs only under `require.main === module`), so the test can measure the real pipeline rather than a copy of it. - Docs: `docs/DEVICE_SCHEMA.md` capability tables now derive from `engine/device.js` and document the kept-and-named rule; `docs/MAINTAINING.md` step 3 says the importer never drops a declared capability. ## The honest choice on `vlan`/`lacp`/`stp` #1526 notes that no code branches on the three tokens and that zero specs use them. That is an argument for #1526 (wire them, or remove them from `CAPABILITIES` — and if removed there, this script loses them automatically, which is exactly the coupling wanted). It is **not** an argument for one script deleting a token the engine publishes. Until #1526 decides, the importer keeps what the engine lists: the fix here is that the importer stops having an opinion, not that it gains a different one. Related: the field the tokens were moved to is dead. Top-level `spec.role` is read by nothing (`engine/device.js` `role` is an *interface* field — `'wan' | 'lan'`), so the strip did not even preserve the description it claimed to preserve; 79 catalogue specs carry a `role` array that no code reads. Nothing under `devices/` is edited here. ## Tokens that are not capabilities Deriving from the engine means the importer now *reports* the power descriptors it used to accept in silence. Keeping them (rather than dropping them) is deliberate: dropping a fact the datasheet declares is the same data loss this issue is about. Census of `devices/` at this commit — 2,039 specs, 18 distinct capability tokens: | token | specs | in `CAPABILITIES`? | |---|---|---| | `passive-poe` | 284 | no | | `802.3af` | 230 | no | | `802.3at` | 222 | no | | `802.3bt` | 53 | no | | `poe-injector` | 10 | no | | `wifi-ap` | 1 | no — it is an alias the importer maps to `wireless` | 588 specs carry at least one of those, so they now produce a named warning on re-import. That is a true statement about the catalogue, and the fix belongs in `CAPABILITIES` (if the token should grant behaviour) or in a catalogue sweep — **not** in a second list inside this script. No file under `devices/` or `catalogue/` is touched by this PR; if Jay wants the descriptors to become real capabilities, or to be swept out of the catalogue, that is a separate decision and a separate PR. ## How many copies of a capability token list are in `scripts/` Grepped every token-list shape in `scripts/`: - **1 restated vocabulary** — `scripts/import-devices.js:38`, the bug. Now removed. - **44 inline `capabilities: [...]` literal arrays** in 3 files: `scripts/merge-research.js` (42 device records with hand-written token lists), `scripts/missing-devices.js:243` (`['host']` placeholder for an unresearched stub), `scripts/request-queue.js:142` (`[]` placeholder). Those are *uses* of the vocabulary, not a copy of it, so they are out of scope here — but `merge-research.js` writes 42 specs with hand-picked tokens and is the next place this class of drift will show up. ## Tests New `test/import-devices-capabilities.test.js`, 29 assertions: - a switch declaring 802.1Q/LACP/STP imports with all six capabilities intact and no `role` key; - the importer's vocabulary equals the engine's at runtime, compared to `require('../engine').CAPABILITIES` (not to a hardcoded list), plus identity, plus "every alias lands on a real token"; - a fixture declaring tokens the engine does not know produces a named warning naming both token and model, in the human output and in `--json`, keeps the tokens, and does not warn about `poe-pse`; - `capabilitiesLost` unit-tested in both directions; a near-miss typo still rejected. `npm test`: 248 suites, 9,137 assertions, 0 failed before → 249 suites, 9,166 assertions, 0 failed after. `npm run syntax`: 319 files OK. ## Contradictions with the issue text - The issue says 1,661 specs and 19 distinct capability tokens; `devices/` holds **2,039** specs and **18** distinct tokens at this commit. The engine-side claim is exact: 12 of the engine's 15 tokens are used by some spec, and `vlan`/`lacp`/`stp` by none. - The "1 spec(s) had minor warnings" line did not reproduce on my fixture — mine carried `label` and `description`, so the run printed no warning line at all. The loss is the same, and it is quieter than the issue says. Fixes #1522
the importer asks the engine what a capability is (#1522)
All checks were successful
CI / Classify the change (pull_request) Successful in 14s
PR rebase / Rebase onto main and arm auto-merge (pull_request) Successful in 16s
CI / Test (Node 20) (pull_request) Successful in 3m15s
CI / Test (Node 18) (pull_request) Successful in 3m23s
CI / Server smoke test (pull_request) Successful in 34s
CI / Docker build (pull_request) Successful in 47s
CI / Test (Node 22) (pull_request) Successful in 2m59s
bcc92e738f
jaylouisw scheduled this pull request to auto merge when all checks succeed 2026-11-15 18:24:58 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
jaylouisw/packetforge!1528
No description provided.