Memory usage spikes with source maps (--enable-source-maps) from v18.10.0 onwards

[joelmukuthu]

Version

18.10.0

Platform

5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Subsystem

No response

What steps will reproduce the bug?

UPDATE: The steps below attempt to create a minimal reproduction of the issue, rather unsuccessfully, since the issue seems to be caused by large source maps and not merely throwing/handing an error. See #46140 (comment) for more info.

Obsolete minimal reproduction

Copy the following code blocks and save them as main.js and main.js.map and then run node --enable-source-maps main.js:

main.js:

// main.ts
var run = (withError) => {
  console.log(
    "Memory usage before (MB):",
    process.memoryUsage().heapUsed / 1024 / 1024
  );
  if (withError) {
    try {
      throw new Error("foo");
    } catch (e) {
      e.stack;
    }
  }
  console.log(
    "Memory usage after (MB):",
    process.memoryUsage().heapUsed / 1024 / 1024
  );
};
console.log("\nWithout error:");
run(false);
console.log("\nWith error:");
run(true);
console.log("\nWithout error:");
run(false);
console.log("\nWith error:");
run(true);
//# sourceMappingURL=main.js.map

main.js.map:

{
  "version": 3,
  "sources": ["main.ts"],
  "sourcesContent": ["const run = (withError) => {\n  console.log(\n    \"Memory usage before (MB):\",\n    process.memoryUsage().heapUsed / 1024 / 1024\n  );\n\n  if (withError) {\n    try {\n      throw new Error(\"foo\");\n    } catch (e) {\n      e.stack;\n    }\n  }\n\n  console.log(\n    \"Memory usage after (MB):\",\n    process.memoryUsage().heapUsed / 1024 / 1024\n  );\n};\n\nconsole.log(\"\\nWithout error:\");\nrun(false);\n\nconsole.log(\"\\nWith error:\");\nrun(true);\n\nconsole.log(\"\\nWithout error:\");\nrun(false);\n\nconsole.log(\"\\nWith error:\");\nrun(true);\n"],
  "mappings": ";AAAA,IAAM,MAAM,CAAC,cAAc;AACzB,UAAQ;AAAA,IACN;AAAA,IACA,QAAQ,YAAY,EAAE,WAAW,OAAO;AAAA,EAC1C;AAEA,MAAI,WAAW;AACb,QAAI;AACF,YAAM,IAAI,MAAM,KAAK;AAAA,IACvB,SAAS,GAAP;AACA,QAAE;AAAA,IACJ;AAAA,EACF;AAEA,UAAQ;AAAA,IACN;AAAA,IACA,QAAQ,YAAY,EAAE,WAAW,OAAO;AAAA,EAC1C;AACF;AAEA,QAAQ,IAAI,kBAAkB;AAC9B,IAAI,KAAK;AAET,QAAQ,IAAI,eAAe;AAC3B,IAAI,IAAI;AAER,QAAQ,IAAI,kBAAkB;AAC9B,IAAI,KAAK;AAET,QAAQ,IAAI,eAAe;AAC3B,IAAI,IAAI;",
  "names": []
}

On my machine, NodeJS v18.11.0 produces:

Without error:
Memory usage before (MB): 5.0794219970703125
Memory usage after (MB): 5.0850372314453125

With error:
Memory usage before (MB): 5.0877227783203125
Memory usage after (MB): 5.37274169921875

Without error:
Memory usage before (MB): 5.37542724609375
Memory usage after (MB): 5.377166748046875

With error:
Memory usage before (MB): 5.379974365234375
Memory usage after (MB): 5.411712646484375

While NodeJS v18.10.0 produces:

Without error:
Memory usage before (MB): 5.038932800292969
Memory usage after (MB): 5.044548034667969

With error:
Memory usage before (MB): 5.047233581542969
Memory usage after (MB): 5.0955810546875

Without error:
Memory usage before (MB): 5.0982666015625
Memory usage after (MB): 5.100006103515625

With error:
Memory usage before (MB): 5.102813720703125
Memory usage after (MB): 5.1345062255859375

When an error is thrown, the increase in memory usage on v18.11.0 spikes a bit higher than on v18.10.0. The same behaviour can be observed in later versions after v18.11.0.

For reference, without --enable-source-maps, this is the output on v18.11.0:

Without error:
Memory usage before (MB): 5.0529632568359375
Memory usage after (MB): 5.0585479736328125

With error:
Memory usage before (MB): 5.0612640380859375
Memory usage after (MB): 5.067131042480469

Without error:
Memory usage before (MB): 5.069816589355469
Memory usage after (MB): 5.071556091308594

With error:
Memory usage before (MB): 5.074363708496094
Memory usage after (MB): 5.080055236816406

And on v18.10.0:

Without error:
Memory usage before (MB): 5.012535095214844
Memory usage after (MB): 5.018119812011719

With error:
Memory usage before (MB): 5.020835876464844
Memory usage after (MB): 5.026702880859375

Without error:
Memory usage before (MB): 5.029388427734375
Memory usage after (MB): 5.0311279296875

With error:
Memory usage before (MB): 5.033935546875
Memory usage after (MB): 5.0396270751953125

How often does it reproduce? Is there a required condition?

It's consistent.

What is the expected behavior?

That memory usage doesn't spike when an error is handled by the application.

What do you see instead?

Memory usage spikes (ref: #46140 (comment)).

Additional information

No response

[bnoordhuis]

The difference looks rather insignificant. I don't think you can say marginally higher memory usage indicates a memory leak. It might, but it probably isn't.

Is this a problem for you somehow or just something you noticed?

[joelmukuthu]

Yes, memory leak might be a bit harsh but on a project I work on, memory usage spikes by about 150MB whenever an error is handled by the application. When that application is run in a container on Kubernetes, whenever an error is handled, the container instantly hits its memory limit and is restarted by K8s. So it's rather serious and blocking upgrading to newer versions.

The application's code is bundled into a single JS file, with only a few external dependencies, so stack traces aren't crazy long. It appears there might be an underlying issue in v18.11.0, it was just hard to prove with a minimal reproduction.

[bnoordhuis]

memory usage spikes by about 150MB whenever an error is handled by the application

That's pretty severe. How does pre-v18.11.0 compare? No spike, or spiky but not as extreme? How big are the source maps?

[joelmukuthu]

How does pre-v18.11.0 compare?

The app is currently running on v18.9.1 and there's no spike there -- memory consumption when an error is handled is just slightly larger than for the no-error scenario.

How big are the source maps?

App script: main.js -> 9.7MB
Source maps: main.js.map -> 12MB

[bnoordhuis]

Can you test with v18.10.0? I don't think there are any changes between v18.10.0 and v18.11.0 that could explain such a jump but between v18.9.1 and v18.10.0 there's at least a2eb55a.

[joelmukuthu]

I think you might be on to something:

Version	Initial (MB)	1st error (MB)	2nd error (MB)	3rd error (MB)	4th error (MB)	5th error (MB)
18.9.1	631	633.8	634 (then 633)	633.4	633.5	633.7
18.10.0	675.2	857.9	999.1	1035 (then 752)	786	906
18.11.0	708.2	728	785	737	865	1012

I ran rudimentary tests and got these numbers from docker stats so they weren't exactly controlled tests. As a control, I also tried a few requests where I didn't generate errors for each of the versions and memory usage stayed rather consistent, no spikes there.

It's interesting that my minimal reproduction (original post) didn't show spikes on 18.10.0!

[bnoordhuis]

@alan-agius4 @aduh95 @legendecas a2eb55a seems to cause excessive memory use. I can't tell if the commit introduced it, or if the issue was already present but latent. You should chime in. :-)

[legendecas]

Thanks for reporting! I'll create a large JS file with about source map of 10M+ size to verify this issue.

[bolt-juri-gavshin]

I think this issue was marked as completed by mistake.
If I understand everything correctly, current LTS (v18.16.0) still has the issue...

[bnoordhuis]

The backport of the fix was blocked on other changes not backporting cleanly but that's been resolved now. I've pinged @juanarbol about it.