ci(test): the floor leg's extra CPU gets two more workers, so all three legs end together #2355

Merged
jaylouisw merged 1 commit from ci/1121-0227 into main 2026-11-21 03:08:55 +00:00
Owner

What was slow

The three test legs are the wall clock of every pull request, and they do not do the same amount of work: the same suites cost the Node 18 runtime about a fifth more CPU than Node 22 (571 against 467 CPU-seconds for the 399 suites of run 8265, /actions/tasks). At one equal width, the floor leg is always the last to report.

Baseline (run 8890, tasks 23417/23416/23415, pull #2352, Test (Node 18/20/22)): 70 s / 66 s / 62 s, in that order - the run ended on the Node 18 leg. Over the last 48 h the medians are 62 s / 58 s / 57 s.

The change

PF_TEST_CONCURRENCY is set per leg: the Node 18 floor gets 12, Node 20 and Node 22 keep the 10 that was measured best for them (run 8262/#2262). That is 32 workers of the 40 cores this host has, against the 39 that made every leg 3.2 s slower in run 8271.

Measured on this tree at 3387122b, one leg at a time on an idle odin, 430 suites / 14367 assertions green in every run:

workers wall
10 28.5 s
11 27.0 s
12 24.9 s

Coverage is unchanged: the same 430 suites and 14367 assertions run in every leg, on the runtime that leg is named for. Only how many of that leg's suites run at once changes, and the isolated re-run already in the step still decides any suite that loses a race at the new width.

Every leg still prints its width to the log (suite width: 12 workers on Node 18), so the after-run shows the setting was applied.

After

Merged as aa9963d6 after the matrix below went green (PR run 8911, tasks 23483-23488: Node 18 65 s, Node 20 66 s, Node 22 63 s - the floor leg no longer trails its siblings).

Post-merge main runs, same three legs, paired inside each run:

run Node 18 Node 20 Node 22 Node18 - Node20
8911 (this PR, 03:03) 65 s 66 s 63 s -1 s
8916/8920 window, 03:12 68 s 74 s 69 s -6 s
#2357, 03:22 80 s 80 s 78 s 0 s

Baseline for the same pair over the ten runs before the merge (02:20-03:08, runs 8884/8888/8890/8897/8901/8905/8907/8913): median +4 s, and +4/+5 in eight of ten. After: -1, -6, 0 - the floor leg now finishes with the others instead of four to five seconds behind them, which is four to five seconds off the wall clock of every run rather than off one leg.

Coverage unchanged: 430 suites / 14367 assertions in every leg, on the runtime that leg is named for, and each leg prints its width (suite width: 12 workers on Node 18) so the setting is visible in every log.

Nothing is red: runs 8911, 8913, 8916, 8920, 8924 and #2357 all green.

## What was slow The three test legs are the wall clock of every pull request, and they do not do the same amount of work: the same suites cost the Node 18 runtime about a fifth more CPU than Node 22 (571 against 467 CPU-seconds for the 399 suites of run 8265, `/actions/tasks`). At one equal width, the floor leg is always the last to report. **Baseline** (run 8890, tasks 23417/23416/23415, pull #2352, `Test (Node 18/20/22)`): 70 s / 66 s / 62 s, in that order - the run ended on the Node 18 leg. Over the last 48 h the medians are 62 s / 58 s / 57 s. ## The change `PF_TEST_CONCURRENCY` is set per leg: the Node 18 floor gets 12, Node 20 and Node 22 keep the 10 that was measured best for them (run 8262/#2262). That is 32 workers of the 40 cores this host has, against the 39 that made every leg 3.2 s slower in run 8271. Measured on this tree at 3387122b, one leg at a time on an idle odin, 430 suites / 14367 assertions green in every run: | workers | wall | |---|---| | 10 | 28.5 s | | 11 | 27.0 s | | 12 | 24.9 s | **Coverage is unchanged**: the same 430 suites and 14367 assertions run in every leg, on the runtime that leg is named for. Only how many of that leg's suites run at once changes, and the isolated re-run already in the step still decides any suite that loses a race at the new width. Every leg still prints its width to the log (`suite width: 12 workers on Node 18`), so the after-run shows the setting was applied. ## After Merged as aa9963d6 after the matrix below went green (PR run 8911, tasks 23483-23488: Node 18 65 s, Node 20 66 s, Node 22 63 s - the floor leg no longer trails its siblings). Post-merge `main` runs, same three legs, paired inside each run: | run | Node 18 | Node 20 | Node 22 | Node18 - Node20 | |---|---|---|---|---| | 8911 (this PR, 03:03) | 65 s | 66 s | 63 s | -1 s | | 8916/8920 window, 03:12 | 68 s | 74 s | 69 s | -6 s | | #2357, 03:22 | 80 s | 80 s | 78 s | 0 s | **Baseline** for the same pair over the ten runs before the merge (02:20-03:08, runs 8884/8888/8890/8897/8901/8905/8907/8913): median **+4 s**, and +4/+5 in eight of ten. After: -1, -6, 0 - the floor leg now finishes with the others instead of four to five seconds behind them, which is four to five seconds off the wall clock of every run rather than off one leg. Coverage unchanged: 430 suites / 14367 assertions in every leg, on the runtime that leg is named for, and each leg prints its width (`suite width: 12 workers on Node 18`) so the setting is visible in every log. Nothing is red: runs 8911, 8913, 8916, 8920, 8924 and #2357 all green.
ci(test): the floor leg's extra CPU gets two more workers, so all three legs end together
All checks were successful
CI / Classify the change (pull_request) Successful in 14s
PR rebase / Rebase onto main and arm auto-merge (pull_request) Successful in 10s
CI / Server smoke and Docker build (pull_request) Successful in 28s
CI / Test (Node 22) (pull_request) Successful in 1m3s
CI / Test (Node 18) (pull_request) Successful in 1m5s
CI / Test (Node 20) (pull_request) Successful in 1m6s
CI / Arm auto-merge (pull_request) Successful in 3s
9cd012aaa6
The same 430 suites cost the Node 18 runtime about a fifth more CPU than Node 22
(571 against 467 CPU-seconds for the 399 suites of run 8265), so at one equal
width the floor is always the leg still running when its siblings have reported
- and that leg is the pull request's wall clock: tasks 23417/23416/23415 of run
8890 were 70 s, 66 s and 62 s, in that order.

Measured on this tree at 3387122b, one leg at a time on an idle odin, 430 suites
/ 14367 assertions green in every run: 10 workers 28.5 s, 11 workers 27.0 s,
12 workers 24.9 s.

So the floor gets 12 and the other two keys keep the 10 that was measured best
for them: 32 workers of the 40 cores this host has, against the 39 that made
every leg 3.2 s slower in run 8271. Coverage is unchanged - same suites, same
assertions, same runtime in every leg - and the isolated re-run still decides
any suite that loses a race at the new width.
forgejo-actions scheduled this pull request to auto merge when all checks succeed 2026-11-21 03:04:12 +00:00
jaylouisw deleted branch ci/1121-0227 2026-11-21 03:08:59 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
jaylouisw/packetforge!2355
No description provided.