Diagnosing a V8 heap-limit crash that isn't a container OOM-kill

Symptoms

• `FATAL ERROR: ... JavaScript heap out of memory` (or 'Allocation failed - JavaScript heap out of memory') printed before the process exits
• A V8 heap statistics / stack dump is printed alongside it
• The container's or orchestrator's own memory limit shows headroom at the time of the crash

The diagnostic path

9 steps, exactly as this revision published them.

1. Step 1 · StartV8 crashed with a heap-out-of-memory FATAL ERROR

   The process printed this specific message and a heap dump before exiting, and the container's own memory limit (docker stats, cgroups, or the orchestrator) shows it was never approached.

   What happens next

   • passed → step 2, Confirm this is V8's own heap ceiling, not the container being killed

2. Step 2 · TestConfirm this is V8's own heap ceiling, not the container being killed

   A container/cgroup OOM-kill terminates the process abruptly with SIGKILL (exit code 137) and no application-level log line at all — the process never gets a chance to print anything. A V8 heap-limit crash, by contrast, always prints the FATAL ERROR line and a heap statistics dump, because V8 caught its own allocation failure before exiting.

   Read-onlybash

   Copy

   echo $?

   Expected result

   1

   What happens next

   • passed → step 3, V8's heap ceiling is independent of the container's memory limit
   • failed → step 4, This is a container/orchestrator OOM-kill instead
   • unknown → step 4, This is a container/orchestrator OOM-kill instead

3. Step 3 · Root causeV8's heap ceiling is independent of the container's memory limit

   V8 picks a default old-space heap ceiling that is not automatically the same as the container's cgroup memory limit. Even on Node versions where V8 sizes itself against available memory at startup, that sizing happens once at boot, won't grow later even if more memory becomes available, and can still land below the container's true limit depending on how the container reports available memory to the process. The two limits are enforced by different layers and are not kept in sync automatically.

   What happens next

   • passed → step 5, Does memory grow without bound over time, or spike sharply during one operation?

4. Step 4 · EndThis is a container/orchestrator OOM-kill instead

   No V8 heap message, just an abrupt kill with exit code 137 — that failure has no application-level log line at all, because the kernel kills the process rather than the process catching an allocation failure. That's a different problem, at a different layer, than this playbook covers.

5. Step 5 · TestDoes memory grow without bound over time, or spike sharply during one operation?

   Sample memory usage periodically across many requests, not just once.

   Read-onlyjavascript

   Copy

   setInterval(() => console.log(process.memoryUsage()), 30000);

   Expected result

   { rss: 612000000, heapTotal: 480000000, heapUsed: 471000000, external: 9000000, arrayBuffers: 2000000 }

   What happens next

   • passed → step 6, Likely a memory leak, not a sizing problem
   • failed → step 7, Raise V8's heap ceiling to match the container, or stop buffering the whole payload in memory
   • unknown → step 7, Raise V8's heap ceiling to match the container, or stop buffering the whole payload in memory

6. Step 6 · Root causeLikely a memory leak, not a sizing problem

   Raising --max-old-space-size only delays this crash if references are genuinely being retained forever — e.g. a module-level array or Map that's only ever pushed to, event listeners added without matching removal, or closures capturing large objects long after they're needed. Finding the leak itself, e.g. with --inspect and heap snapshots, is outside what this playbook covers.

7. Step 7 · FixRaise V8's heap ceiling to match the container, or stop buffering the whole payload in memory

   Two independent options: set --max-old-space-size explicitly to a value comfortably below the container's actual memory limit, leaving headroom for V8's other memory areas and the OS; and/or change the specific operation to stream data instead of loading it wholesale (e.g. a large query result set or file read in one go).

   Read-onlybash

   Copy

   node --max-old-space-size=3072 server.js

   What happens next

   • passed → step 8, Confirm the crash stops under the same workload

8. Step 8 · Verify the fixConfirm the crash stops under the same workload

   Re-run the same operation/load pattern and confirm there's no further heap-out-of-memory crash, and that the container's own memory limit is still respected rather than trading this crash for a container-level OOM-kill instead.

   What happens next

   • passed → step 9, Resolved
   • failed → step 6, Likely a memory leak, not a sizing problem
   • unknown → step 6, Likely a memory leak, not a sizing problem

9. Step 9 · EndResolved

   The workload completes without a V8 heap crash, within the raised ceiling and within the container's own limit.

Sources

• Command-line API— Node.js