Regenerate the example labs, and fail CI if they drift again #399

Merged
jaylouisw merged 1 commit from fix/examples-generated into main 2026-11-08 07:00:44 +00:00
Owner

Running the documented regeneration step on a clean tree rewrote all four examples — lab1 +59, lab2 +34, lab3 +435, lab4 +111 lines. The committed JSON was a state the serializer could no longer produce: it predates the seq field, and its device and link entries differ too.

Nothing failed, which is why it survived — every lab still loaded and pinged. The cost landed on the next contributor: add one lab, inherit 639 lines of unrelated churn.

Regenerates the four files and adds test/examples-generated.test.js, which re-serialises each committed file and fails when the result differs, naming the drifted keys and the command that fixes it.

Proved it catches the drift rather than assuming: reverted the examples, ran the test against the committed state — four failures, one per lab, each naming devices, links, seq. After regeneration, 21 assertions pass. It also asserts each file serialises identically twice and pings two labs, so a clean round trip cannot hide a broken lab. 85 suites, 4232 assertions.

Running the documented regeneration step on a clean tree rewrote all four examples — lab1 +59, lab2 +34, lab3 +435, lab4 +111 lines. The committed JSON was a state the serializer could no longer produce: it predates the `seq` field, and its device and link entries differ too. Nothing failed, which is why it survived — every lab still loaded and pinged. The cost landed on the next contributor: add one lab, inherit 639 lines of unrelated churn. Regenerates the four files and adds `test/examples-generated.test.js`, which re-serialises each committed file and fails when the result differs, naming the drifted keys and the command that fixes it. **Proved it catches the drift** rather than assuming: reverted the examples, ran the test against the committed state — four failures, one per lab, each naming `devices, links, seq`. After regeneration, 21 assertions pass. It also asserts each file serialises identically twice and pings two labs, so a clean round trip cannot hide a broken lab. 85 suites, 4232 assertions.
Regenerate the example labs, and fail CI if they drift again
Some checks failed
CI / Test (Node 18) (pull_request) Successful in 1m28s
CI / Test (Node 20) (pull_request) Successful in 1m23s
CI / Test (Node 22) (pull_request) Successful in 1m14s
CI / Server smoke test (pull_request) Successful in 31s
CI / Docker build (pull_request) Failing after 18s
d8a6edc91f
Running the documented regeneration step on a clean tree rewrote all four examples: lab1 +59 lines, lab2 +34, lab3 +435, lab4 +111. The committed JSON was a state the serializer could no longer produce - it predates the seq field, and its device and link entries differ too, not just the header. Every lab still loaded and pinged, which is why it survived: nothing failed, the cost just landed on the next contributor who added a lab and inherited 639 lines of unrelated churn.

So this does two things. It regenerates the four files from build-examples.js, which is the documented way to produce them; and it adds test/examples-generated.test.js, which re-serialises each committed file and fails if the result differs, naming the keys that drifted and printing the command that fixes it.

Verified by reverting the examples and running the new test against the committed state: four failures, one per lab, each naming devices, links and seq. After regeneration: 21 assertions, 0 failed. A generator that could shuffle ids or ordering would make every regeneration a diff, so the test also asserts each file serialises identically twice, and it pings two of the labs so a clean round trip cannot hide a lab that no longer works.

85 suites, 4232 assertions.
jaylouisw force-pushed fix/examples-generated from d8a6edc91f
Some checks failed
CI / Test (Node 18) (pull_request) Successful in 1m28s
CI / Test (Node 20) (pull_request) Successful in 1m23s
CI / Test (Node 22) (pull_request) Successful in 1m14s
CI / Server smoke test (pull_request) Successful in 31s
CI / Docker build (pull_request) Failing after 18s
to 8150a10621
Some checks failed
CI / Test (Node 18) (pull_request) Successful in 1m8s
CI / Test (Node 20) (pull_request) Successful in 1m7s
CI / Test (Node 22) (pull_request) Successful in 1m3s
CI / Server smoke test (pull_request) Successful in 20s
CI / Docker build (pull_request) Failing after 7s
2026-11-08 06:40:46 +00:00
Compare
jaylouisw deleted branch fix/examples-generated 2026-11-08 07:00:46 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
jaylouisw/packetforge!399
No description provided.