engine/vlan.js: the array-keyed member cache never hits when allowedVlans is a sliced copy (blocks #1749) #1809

Closed
opened 2026-11-17 07:41:10 +00:00 by jaylouisw · 1 comment
Owner

Why this issue exists. PR #1749 is the only open PR and cannot go green: three CI legs fail one assertion in its own new suite. Four rounds of diagnosis on the PR agree on the cause, which lives in engine/ - not in the PR's data, and not in a place the surgery loop may edit. This issue exists so the fix lands as its own change instead of as an amendment to a stale PR.

Symptom. test/trunk-allowed-remove-from-all.test.js:167 - 2,000 classifyEgress() calls against a 4,089-id allowed list (the shape switchport trunk allowed vlan remove 2 on a default trunk produces). Bound is ms < 3000.

  • local, head bc747907: 1,464 / 1,544 / 1,600 ms (green)
  • CI, PF_TEST_CONCURRENCY=10: 6,063-6,510 ms (red on Node 18/20/22, run 4759 and earlier)

Cause - one line. normalizePortConfig() builds allowedVlans as raw.allowedVlans.slice(), so every call returns a new array object. engine/vlan.js keys its membership cache (memberCache) on the array itself; a fresh identity every call means the cache is written and never read, and each vlanListIncludes() falls back to a linear scan of ~4,089 ids.

Measured on bc747907, same host:

full loop                1550 ms
membership, warm cache     0.12 ms
slice() alone               7.5 ms
slice() + membership      1532 ms
origin/main full loop      2466 ms

So ~99% of the cost is the cache miss, not the copy.

Fix direction (engine owner's call, not prescribed here): make the cache survive a copy - key it on a stable identity/normalised form, or store a precomputed Set/sorted array alongside allowedVlans in normalizePortConfig() so it travels with the sliced copy. Anything that keeps the per-frame lookup off the linear scan is enough; the bound in #1749's test is a symptom, and relaxing it to ~8 s would hide a real 60x regression on main.

Why not just loosen the bound in #1749: the same path is 2466 ms on origin/main today, i.e. already close to the bound for one trunk; the cache is the bug.

Once this lands, #1749 rebases onto it (its auto-rebase workflow is armed) and the queue drains.

**Why this issue exists.** PR #1749 is the only open PR and cannot go green: three CI legs fail one assertion in its own new suite. Four rounds of diagnosis on the PR agree on the cause, which lives in `engine/` - not in the PR's data, and not in a place the surgery loop may edit. This issue exists so the fix lands as its own change instead of as an amendment to a stale PR. **Symptom.** `test/trunk-allowed-remove-from-all.test.js:167` - 2,000 `classifyEgress()` calls against a 4,089-id allowed list (the shape `switchport trunk allowed vlan remove 2` on a default trunk produces). Bound is `ms < 3000`. - local, head `bc747907`: 1,464 / 1,544 / 1,600 ms (green) - CI, `PF_TEST_CONCURRENCY=10`: **6,063-6,510 ms** (red on Node 18/20/22, run 4759 and earlier) **Cause - one line.** `normalizePortConfig()` builds `allowedVlans` as `raw.allowedVlans.slice()`, so every call returns a *new array object*. `engine/vlan.js` keys its membership cache (`memberCache`) on the array itself; a fresh identity every call means the cache is written and never read, and each `vlanListIncludes()` falls back to a linear scan of ~4,089 ids. Measured on `bc747907`, same host: ``` full loop 1550 ms membership, warm cache 0.12 ms slice() alone 7.5 ms slice() + membership 1532 ms origin/main full loop 2466 ms ``` So ~99% of the cost is the cache miss, not the copy. **Fix direction (engine owner's call, not prescribed here):** make the cache survive a copy - key it on a stable identity/normalised form, or store a precomputed `Set`/sorted array alongside `allowedVlans` in `normalizePortConfig()` so it travels with the sliced copy. Anything that keeps the per-frame lookup off the linear scan is enough; the bound in #1749's test is a symptom, and relaxing it to ~8 s would hide a real 60x regression on main. **Why not just loosen the bound in #1749:** the same path is `2466 ms` on `origin/main` today, i.e. already close to the bound for one trunk; the cache is the bug. Once this lands, #1749 rebases onto it (its auto-rebase workflow is armed) and the queue drains.
Author
Owner

Measurement from the branch that this issue blocks

I ran the whole suite on #1749's head (bc747907) in a scratch worktree, and it is green on a dev host:

304 suite(s) in 400.8 s — 10647 assertions passed, 0 failed
OK — every suite passed.

So test/trunk-allowed-remove-from-all.test.js is not red here, and the 99 red test legs since 22:27 are not "the branch's own test file being wrong". The one assertion in that file that can go red is a wall-clock bound, and it is marginal:

test/trunk-allowed-remove-from-all.test.js:164-167 — 2,000 classifyEgress calls against the materialised long list, ok(ms < 3000, ...).

Measured on this host with allowedVlans at its 4,089 ids (switchport trunk allowed vlan remove 2, then the same loop as the test):

rep 2,000 frames budget
0 1,280 ms 3,000 ms
1 1,237 ms 3,000 ms
2 1,202 ms 3,000 ms

41% of budget on an idle machine. A runner leg that is ~2.5x slower or shared with another job crosses it; that is the shape of a failure that reproduces on all three legs and never on a laptop.

Why this issue is the fix, not a looser bound

The 1.2 s is exactly the cache miss described above: allowedVlans is slice()d, the member cache is keyed by the array, so every one of the 2,000 frames re-scans 4,089 ids. With the cache hitting, the same loop is ~0.1 ms and the 3,000 ms bound stops being reachable at all — no test change needed.

Consequence while it is open

#1749 is deadlocked, not stalled: checks_failed() in .forgejo/workflows/auto-rebase.yml (#1789) now skips it, so it never gets rebased, so it stays 24 commits behind main (d39b7a87 is the merge base) and its head never changes. Land this engine fix and the next leg should be green on its own; nothing else is holding that PR.

Not confirmed: which step each leg failed in — this Forgejo build exposes no job-log endpoint (/actions/runs/<n>/jobs/<n>/logs → 404), so the leg-level cause above is inference from a green local run plus the only wall-clock assertion in the tree.

## Measurement from the branch that this issue blocks I ran the **whole suite** on #1749's head (`bc747907`) in a scratch worktree, and it is green on a dev host: ``` 304 suite(s) in 400.8 s — 10647 assertions passed, 0 failed OK — every suite passed. ``` So `test/trunk-allowed-remove-from-all.test.js` is not red here, and the 99 red test legs since 22:27 are not "the branch's own test file being wrong". The one assertion in that file that *can* go red is a wall-clock bound, and it is marginal: `test/trunk-allowed-remove-from-all.test.js:164-167` — 2,000 `classifyEgress` calls against the materialised long list, `ok(ms < 3000, ...)`. Measured on this host with `allowedVlans` at its 4,089 ids (`switchport trunk allowed vlan remove 2`, then the same loop as the test): | rep | 2,000 frames | budget | |-----|--------------|--------| | 0 | 1,280 ms | 3,000 ms | | 1 | 1,237 ms | 3,000 ms | | 2 | 1,202 ms | 3,000 ms | 41% of budget on an idle machine. A runner leg that is ~2.5x slower or shared with another job crosses it; that is the shape of a failure that reproduces on all three legs and never on a laptop. ### Why this issue is the fix, not a looser bound The 1.2 s is exactly the cache miss described above: `allowedVlans` is `slice()`d, the member cache is keyed by the array, so every one of the 2,000 frames re-scans 4,089 ids. With the cache hitting, the same loop is ~0.1 ms and the 3,000 ms bound stops being reachable at all — no test change needed. ### Consequence while it is open #1749 is deadlocked, not stalled: `checks_failed()` in `.forgejo/workflows/auto-rebase.yml` (#1789) now skips it, so it never gets rebased, so it stays 24 commits behind `main` (`d39b7a87` is the merge base) and its head never changes. Land this engine fix and the next leg should be green on its own; nothing else is holding that PR. *Not confirmed:* which step each leg failed in — this Forgejo build exposes no job-log endpoint (`/actions/runs/<n>/jobs/<n>/logs` → 404), so the leg-level cause above is inference from a green local run plus the only wall-clock assertion in the tree.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
jaylouisw/packetforge#1809
No description provided.