ci(test): the floor leg's extra CPU gets two more workers, so all three legs end together #2355
No reviewers
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
jaylouisw/packetforge!2355
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "ci/1121-0227"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What was slow
The three test legs are the wall clock of every pull request, and they do not do the same amount of work: the same suites cost the Node 18 runtime about a fifth more CPU than Node 22 (571 against 467 CPU-seconds for the 399 suites of run 8265,
/actions/tasks). At one equal width, the floor leg is always the last to report.Baseline (run 8890, tasks 23417/23416/23415, pull #2352,
Test (Node 18/20/22)): 70 s / 66 s / 62 s, in that order - the run ended on the Node 18 leg. Over the last 48 h the medians are 62 s / 58 s / 57 s.The change
PF_TEST_CONCURRENCYis set per leg: the Node 18 floor gets 12, Node 20 and Node 22 keep the 10 that was measured best for them (run 8262/#2262). That is 32 workers of the 40 cores this host has, against the 39 that made every leg 3.2 s slower in run 8271.Measured on this tree at
3387122b, one leg at a time on an idle odin, 430 suites / 14367 assertions green in every run:Coverage is unchanged: the same 430 suites and 14367 assertions run in every leg, on the runtime that leg is named for. Only how many of that leg's suites run at once changes, and the isolated re-run already in the step still decides any suite that loses a race at the new width.
Every leg still prints its width to the log (
suite width: 12 workers on Node 18), so the after-run shows the setting was applied.After
Merged as
aa9963d6after the matrix below went green (PR run 8911, tasks 23483-23488: Node 18 65 s, Node 20 66 s, Node 22 63 s - the floor leg no longer trails its siblings).Post-merge
mainruns, same three legs, paired inside each run:Baseline for the same pair over the ten runs before the merge (02:20-03:08, runs 8884/8888/8890/8897/8901/8905/8907/8913): median +4 s, and +4/+5 in eight of ten. After: -1, -6, 0 - the floor leg now finishes with the others instead of four to five seconds behind them, which is four to five seconds off the wall clock of every run rather than off one leg.
Coverage unchanged: 430 suites / 14367 assertions in every leg, on the runtime that leg is named for, and each leg prints its width (
suite width: 12 workers on Node 18) so the setting is visible in every log.Nothing is red: runs 8911, 8913, 8916, 8920, 8924 and #2357 all green.