Static assets: 233 KB re-downloaded on every load becomes 72 KB, or zero #1220

Merged
jaylouisw merged 1 commit from perf/static-asset-caching into main 2026-11-09 15:25:22 +00:00
Owner

Static assets: 233 KB re-downloaded on every load becomes 72 KB, or zero

The catalogue was the visible cost; the app itself was the constant one. Five files - app.js at
129 KB, canvas.js 36 KB, faceplate.js 27 KB, style.css 30 KB, index.html 7 KB - went out with
Cache-Control: no-store and no compression, so every load re-downloaded all of it and the
browser was not even allowed to ask whether anything had changed.

Measured against a running server, all five files:

before after
first load 233 KB 72 KB gzipped (3.2×)
reload 233 KB again 0 bytes — five revalidations, five 304s

The bug that made it look like it worked

The first version computed the ETag as e4 && st.mtimeMs ? st.mtimeMs : Date.now(). The first
argument of an fs.stat callback is the error, so that expression was always falsy: every
response stamped a fresh ETag from the clock, and revalidation could never match. The header was
present, well-formed, and completely useless — and a browser test would have looked fine, because
each response was individually correct.

Caught by measuring bytes instead of reading headers: two consecutive requests for app.js had
ETags ten milliseconds apart (W/"128993-1794237477972" then W/"128993-1794237477982").

The test now asserts the boring thing directly: two requests for an unchanged file agree on the
ETag
, which is what makes a 304 possible at all.

What changed

  • ETag + Last-Modified on every asset, with If-None-Match and If-Modified-Since both honoured.
  • Cache-Control: no-cache instead of no-store — still never trusted without asking, but
    allowed to answer 304. The server doubles as the thing being edited, and a browser holding a stale
    app.js after a fix is the most confusing bug report there is.
  • gzip for the text types when the client accepts it, with the compressed copy cached per
    (file, mtime, size)
    so a 129 KB app.js is compressed once per version rather than once per
    request.
  • Content-Length set correctly for both encodings, and Vary: Accept-Encoding.
  • The SPA fallback goes through the same path, so a deep link is revalidatable too.

20 checks in test/server-static-cache.test.js against a real spawned server: ETag stability, both
validators, byte-identical gunzip, Content-Length on both encodings, content types for all five
assets, the SPA fallback, and path traversal still refused. Suite: 126 suites, 5,616 assertions, 0
failed.

Static assets: 233 KB re-downloaded on every load becomes 72 KB, or zero The catalogue was the visible cost; the app itself was the constant one. Five files - `app.js` at 129 KB, `canvas.js` 36 KB, `faceplate.js` 27 KB, `style.css` 30 KB, `index.html` 7 KB - went out with `Cache-Control: no-store` and no compression, so **every** load re-downloaded all of it and the browser was not even allowed to ask whether anything had changed. Measured against a running server, all five files: | | before | after | |---|---|---| | first load | 233 KB | **72 KB** gzipped (3.2×) | | reload | 233 KB again | **0 bytes** — five revalidations, five `304`s | ### The bug that made it look like it worked The first version computed the ETag as `e4 && st.mtimeMs ? st.mtimeMs : Date.now()`. The first argument of an `fs.stat` callback is the **error**, so that expression was always falsy: every response stamped a fresh ETag from the clock, and revalidation could never match. The header was present, well-formed, and completely useless — and a browser test would have looked fine, because each response was individually correct. Caught by measuring bytes instead of reading headers: two consecutive requests for `app.js` had ETags ten milliseconds apart (`W/"128993-1794237477972"` then `W/"128993-1794237477982"`). The test now asserts the boring thing directly: **two requests for an unchanged file agree on the ETag**, which is what makes a 304 possible at all. ### What changed - **ETag + Last-Modified** on every asset, with `If-None-Match` and `If-Modified-Since` both honoured. - **`Cache-Control: no-cache`** instead of `no-store` — still never trusted without asking, but allowed to answer 304. The server doubles as the thing being edited, and a browser holding a stale `app.js` after a fix is the most confusing bug report there is. - **gzip** for the text types when the client accepts it, with the compressed copy **cached per (file, mtime, size)** so a 129 KB `app.js` is compressed once per version rather than once per request. - **`Content-Length`** set correctly for both encodings, and `Vary: Accept-Encoding`. - The **SPA fallback** goes through the same path, so a deep link is revalidatable too. 20 checks in `test/server-static-cache.test.js` against a real spawned server: ETag stability, both validators, byte-identical gunzip, Content-Length on both encodings, content types for all five assets, the SPA fallback, and path traversal still refused. Suite: 126 suites, 5,616 assertions, 0 failed.
Static assets: 233 KB re-downloaded on every load becomes 72 KB, or zero
All checks were successful
CI / Classify the change (pull_request) Successful in 10s
PR rebase / Rebase onto main and arm auto-merge (pull_request) Successful in 13s
CI / Test (Node 18) (pull_request) Successful in 1m37s
CI / Test (Node 20) (pull_request) Successful in 1m32s
CI / Server smoke test (pull_request) Successful in 29s
CI / Docker build (pull_request) Successful in 40s
CI / Test (Node 22) (pull_request) Successful in 1m24s
dd2cc82057
The catalogue was the visible cost; the app itself was the constant one. Five files - `app.js` at
129 KB, `canvas.js` 36 KB, `faceplate.js` 27 KB, `style.css` 30 KB, `index.html` 7 KB - went out with
`Cache-Control: no-store` and no compression, so **every** load re-downloaded all of it and the
browser was not even allowed to ask whether anything had changed.

Measured against a running server, all five files:

| | before | after |
|---|---|---|
| first load | 233 KB | **72 KB** gzipped (3.2×) |
| reload | 233 KB again | **0 bytes** — five revalidations, five `304`s |

### The bug that made it look like it worked

The first version computed the ETag as `e4 && st.mtimeMs ? st.mtimeMs : Date.now()`. The first
argument of an `fs.stat` callback is the **error**, so that expression was always falsy: every
response stamped a fresh ETag from the clock, and revalidation could never match. The header was
present, well-formed, and completely useless — and a browser test would have looked fine, because
each response was individually correct.

Caught by measuring bytes instead of reading headers: two consecutive requests for `app.js` had
ETags ten milliseconds apart (`W/"128993-1794237477972"` then `W/"128993-1794237477982"`).

The test now asserts the boring thing directly: **two requests for an unchanged file agree on the
ETag**, which is what makes a 304 possible at all.

### What changed

- **ETag + Last-Modified** on every asset, with `If-None-Match` and `If-Modified-Since` both honoured.
- **`Cache-Control: no-cache`** instead of `no-store` — still never trusted without asking, but
  allowed to answer 304. The server doubles as the thing being edited, and a browser holding a stale
  `app.js` after a fix is the most confusing bug report there is.
- **gzip** for the text types when the client accepts it, with the compressed copy **cached per
  (file, mtime, size)** so a 129 KB `app.js` is compressed once per version rather than once per
  request.
- **`Content-Length`** set correctly for both encodings, and `Vary: Accept-Encoding`.
- The **SPA fallback** goes through the same path, so a deep link is revalidatable too.

20 checks in `test/server-static-cache.test.js` against a real spawned server: ETag stability, both
validators, byte-identical gunzip, Content-Length on both encodings, content types for all five
assets, the SPA fallback, and path traversal still refused. Suite: 126 suites, 5,616 assertions, 0
failed.
jaylouisw scheduled this pull request to auto merge when all checks succeed 2026-11-09 15:21:40 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
jaylouisw/packetforge!1220
No description provided.