main is red: property-validator fails at seed 65 because 58 of the engine's 75 error codes are not in validation.RULES #1654
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
jaylouisw/packetforge#1654
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
mainis red: all threeCI / Testlegs fail ontest/property-validator.test.js, deterministically, at the same seed. This has been true since #1647 and is still true on969c999c(after #1649, #1650).Repro
Green at
30049dc6, red from677cfb0e(#1647).Cause
Not a behaviour change — a fixture draw shift.
test/property-validator.test.js:125asserts every codeconnect()returns is invalidation.RULES:validation.RULESis the connection rule table: 24 entries.code: '...'acrossengine/*.js— 58 are outside that table, because config/link/optical validators each own their codes.E_BIDI_MISMATCHis raised atengine/optical.js:225(one side single-strand BiDi, the other needs two strands). It is aconnect()-time error and it is not inRULES.E_PORT_IN_USEexemption on line 125 is the same bug papered over once already.The five device specs added in #1647 changed the seeded draw so that seed 65 now lands on the BiDi pairing; removing any one of them makes the test pass. Nothing about the engine got worse — the assertion was always too narrow, and every new device spec re-rolls it.
Fix — needs a decision (both files are outside my remit)
e.codeis a well-formed non-empty code (or the union of every code the validators can emit), and delete theE_PORT_IN_USEspecial case. Cheapest, no engine change.E_BIDI_MISMATCH,E_BIDI_SAME_WAVELENGTH,E_UNKNOWN_PART,E_OPTICAL_BUDGET, …) intovalidation.RULESsoconnect()'s code surface is fully documented, and keep the strict assertion.Until one lands,
mainstays red and every PR rebased onto it inherits a failing suite.Main is green again — but not fixed. This is a mask, not a repair.
a9034db1(#1655) is the first green tip in three:CI / Test (Node 18/20/22)all success there, while903a03f2andd8164131fail. Nothing touched the validator or its test in between, so the test did not get better — the sample moved.Verified locally, same machine, same Node:
d8164131:node --test test/property-validator.test.js→ 19 assertions, 1 failed:[seed 65 d0:SFP56-3 -- d2:sfp1 media="console-rs232": unknown code E_BIDI_MISMATCH](720 random link attempts).a9034db1: same command → 19 passed, 0 failed.The mechanism: the property test builds topologies with
specs[Math.floor(rand() * specs.length)]over the real catalogue, so every catalogue change re-rolls what seed 65 means. #1655 added one device (2202 → 2203 specs); the sampling shifted, seed 65 no longer lands on that pair, and the unknown-code failure disappeared with it. Any future device or engine change can bring it straight back, on someone else's PR, with no obvious relationship to their work.Scope is wider than the one code.
validation.RULEScarries 24 codes; the validator path (validation.js,topology.js,media.js,optical.js) emits 51, so 34 codes are outside the test'sRULE_CODESgate and would each be reported as "unknown":E_BAD_LENGTH E_BAUD_MISMATCH E_BIDI_MISMATCH E_BIDI_SAME_WAVELENGTH E_COMBO_OCCUPIED E_FLOW_BAD_RECORD E_FLOW_GHOST_ENDPOINT E_IFACE_BAD_ADDRESS E_IFACE_BAD_MAC E_IFACE_BAD_MASK E_IFACE_DUPLICATE_MAC E_IFACE_NO_MASK E_LAYOUT_BAD_ENTRY E_LAYOUT_MALFORMED E_LAYOUT_UNKNOWN_DEVICE E_LINK E_LINK_BAD_LABEL E_LINK_BAD_STATE E_LINK_DUPLICATE_ID E_LINK_ENDPOINT_MISSING E_LINK_MALFORMED E_LINK_NOT_RESTORED E_LINK_SELF_LOOP E_LINK_UNCHECKABLE E_NOT_A_TOPOLOGY E_NO_DEVICE E_NO_FREE_PORTS E_NO_LINK E_OPTICAL_BUDGET E_SAME_DEVICE E_UNKNOWN E_UNKNOWN_PART E_UNKNOWN_PON_CLASS E_UNKNOWN_SPLITTERSome of those (
E_LINK_*,E_IFACE_*,E_LAYOUT_*,E_FLOW_*) belong to import/restore and are probably never reachable fromvalidateLink, so they may be legitimately absent. But at minimumE_BIDI_MISMATCH,E_BIDI_SAME_WAVELENGTH,E_BAUD_MISMATCH,E_OPTICAL_BUDGET,E_COMBO_OCCUPIED,E_NO_FREE_PORTSandE_SAME_DEVICEarevalidateLinkresults —E_BIDI_MISMATCHis proven reachable by the repro above.Decision still open (unchanged from when this issue was filed — 0 comments so far):
validation.RULES(they get severities and descriptions, and the test's gate becomes meaningful), orKNOWN_CODESlist separate fromRULESin the test, and add the 34, orRULES, so it can never drift.Until one is picked, this test passes by coincidence and any catalogue PR can turn main red for reasons unrelated to its diff. Filing this, not fixing it:
engine/andtest/are not mine to edit from the surgery loop.Surgery loop, 2026-11-16 11:53 — verified by checkout of both tips in
C:/Users/jay/pf-wt/surgery; no worktrees or branches left behind.