Dev mode on Next.js v12 breaks with combination of features

[PabloSzx]

What version of Next.js are you using?

12.0.0 and 12.0.1

What version of Node.js are you using?

v16.13.0 and v14.18.1 and v12.22.7

What browser are you using?

Doesn't apply, breaks on terminal

What operating system are you using?

Arch Linux and macOS Big Sur 11.6

How are you deploying your application?

next dev

Describe the Bug

next dev breaks on v12, but I'm not exactly sure why, I was able to create a minimal reproduction repo with a combination of features, but I tried reproducing the issue with almost every feature (externalDir, chakra-ui, monorepo, typescript of course) of the repository by itself and worked fine, so I'm not completely sure what breaks it

Expected Behavior

dev mode should work??, in v11 everything works fine

To Reproduce

https://github.com/PabloSzx/next-v12-bug

> next

ready - started server on 0.0.0.0:3000, url: http://localhost:3000
warn  - You have enabled experimental feature(s).
warn  - Experimental features are not covered by semver, and may cause unexpected or broken application behavior. Use them at your own risk.

event - compiled successfully in 3.2s (1515 modules)
wait  - compiling /_error...
event - compiled successfully in 322 ms (1516 modules)

<--- Last few GCs --->

[60697:0x4967150]    24587 ms: Scavenge 4046.5 (4123.3) -> 4046.0 (4131.3) MB, 5.2 / 0.0 ms  (average mu = 0.387, current mu = 0.119) allocation failure 
[60697:0x4967150]    24599 ms: Scavenge 4052.6 (4131.3) -> 4051.3 (4131.6) MB, 10.1 / 0.0 ms  (average mu = 0.387, current mu = 0.119) allocation failure 
[60697:0x4967150]    25112 ms: Scavenge 4053.6 (4131.6) -> 4052.5 (4152.6) MB, 511.8 / 0.0 ms  (average mu = 0.387, current mu = 0.119) allocation failure 


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
 1: 0xb02eb0 node::Abort() [node]
 2: 0xa181fb node::FatalError(char const*, char const*) [node]
 3: 0xced75e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
 4: 0xcedad7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
 5: 0xea5d75  [node]
 6: 0xeb544d v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 7: 0xeb814e v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
 8: 0xe7957a v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
 9: 0x11f2d56 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
10: 0x15e7759  [node]

[candidosales]

It's happening the same for me. I increased the heap memory but keep the same issue:

Evidence

node --max-old-space-size=2048 ./node_modules/.bin/next dev

[timneutkens]

We're having a look at the reproduction!

[icopp]

I encountered something very similar on a private project. I haven't put together a separate reproduction yet, but I checked and it has exactly the same versions of each external dependency in the reproduction in this issue.

[timneutkens]

@sokra is investigating this. Please don't post OOM messages in this thread until we post a reply here as they're not useful for investigating the issue and will ping everyone subscribed. If you want to provide a reproduction feel free to post repositories. Just to verify you're all using pnpm?

[mustaphaturhan]

@timneutkens, same error here. I am not using pnpm, only common packages that I use are @chakra-ui/react, @emotion/react, @emotion/styled and framer-motion.

To reproduce this error, I started 2 new project by doing npx create-next-app@latest --typescript and npx create-next-app@latest and installed whole packages but no luck yet.

[timneutkens]

Okay thanks @mustaphaturhan!

[timneutkens]

This seems to be a particular case with Chakra UI and some other deps.
As a temporary workaround you can disable esmExternals:

module.exports = {
  experimental: {
    esmExternals: false
  }
}

[M-e-r-c-u-r-y]

This seems to be a particular case with Chakra UI and some other deps. As a temporary workaround you can disable esmExternals:

module.exports = {
  experimental: {
    esmExternals: false
  }
}

To anyone who isn't sure where to add the module exports stated above as a temporary fix, you can refer to next.config.js

[mustaphaturhan]

@sokra, this error also happens when I install one of this packages: react-intl, firebase or firebase-admin. I looked at their common points but can't find anthing.

[anstapol]

Had the same issue, increased memory, completely re-installed all node related stuff and that didn't helped.

Then I've just removed the .next directory and it worked.
🤦‍♂️

[sangdth]

This seems to be a particular case with Chakra UI and some other deps. As a temporary workaround you can disable esmExternals:

module.exports = {
  experimental: {
    esmExternals: false
  }
}

To anyone who isn't sure where to add the module exports stated above as a temporary fix, you can refer to next.config.js

I'm using Chakra UI too and this work around saved me.

[Navbryce]

Same happening here with Chakra. Seems like it was enabled by default in 11.1.3-canary.69

[arthuryeti]

Same happening here with Chakra.

[PabloSzx]

@timneutkens Since 12.0.1 was released without a fix for this the milestone has to be changed?

I updated the reproduction repo Next.js version to 12.0.1 and the same issue is still there (the workaround of esmExternals: false still works tho)

[cschroeter]

Chakra UI Contributor here. Anything we can do on our side? @timneutkens

[alessandrojcm]

Chakra user here, it also happens to us with Next 12.0.1; in our case, it also happens when trying next build, the build process spans several workers which consume more and more memory until the process eventually dies with a SIGKILL. The esmExternals workaround works, though the bundle size is practically the same as with Next 11 😕

[sokra]

fix in on the way.

Here a little explanation what happend here:

Usually Next.js will not bundle node_modules in the server compiler. It will make them "external" and either require() or import() them. But it has some checks in place to check if a package can be made external.
One of these check is to compare if require() resp. import() leads to the same result in Node.js (otherwise this would break in Node.js).
For import "@chakra-ui/react" it first resolves it to node_modules/@chakra-ui/react/dist/esm/index.js. It checks if this is an ESM or CJS module by extension (.mjs) or "type": "module". That results in "This is not an ESM". That seems incorrect, but note that we are performing that check from Node.js viewpoint and it only check these criteria and @chakra-ui/react is actually missing the "type": "module" here. So Next.js checks if require("@chakra-ui/react") would result in the same module, but it actually don't do that. It would resolve to node_modules/@chakra-ui/react/dist/cjs/index.js.
So Next.js concludes that this module can't be made external. Mainly because the Node.js ESM flags are missing in this package.
As fallback it tries to bundle the package. That's not something we want, mainly for performance reasons.

But that sadly bundling the package also doesn't work correctly in combination with other nested package being made externals. I think it has something to do with async modules and circular dependencies, but I'm still investigating. Actually it shouldn't run into that case anyway.

---

The fix is that Next.js now detects that case and tries to fallback to the CJS version of the package first, before falling back to bundling the package.

---

Chakra UI Contributor here. Anything we can do on our side?

Yes. Please add a package.json with { "type": "module" } to dist/esm/package.json in your packages. That will flag these packages correctly for Node.js (and by using a nested package.json it will only flag that folder).

You can try to import your packages via Node.js to check if they are valid:

$ node -e "import('@chakra-ui/react')"
(node:19932) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.
(Use `node --trace-warnings ...` to show where the warning was created)
.../node_modules/@chakra-ui/react/dist/esm/index.js:1
export * from "./chakra-provider";
^^^^^^

SyntaxError: Unexpected token 'export'

In addition to that file extensions are mandatory in Node.js ESM: https://nodejs.org/api/esm.html#mandatory-file-extensions

-export * from "./chakra-provider";
+export * from "./chakra-provider.js";
-export * from "./extend-theme";
+export * from "./extend-theme.js";
-export * from "./theme-extensions";
+export * from "./theme-extensions.js";

[sokra]

@sokra, this error also happens when I install one of this packages: react-intl, firebase or firebase-admin. I looked at their common points but can't find anthing.

I was not able to reproduce it with these packages, but at least for the firebase case, my PR includes another little bugfix where .mjs where not considered and where bundled instead of made external. So maybe that fixes it too. Maybe wait for the fix and create a reproduction when one of these packages is still broken.

[cschroeter]

@sokra
I think I can add the type property as suggested. Just wonder why Next is not doing the same thing as Rollup and Webpack.

From their documentation:
If your package.json file also has a module field, ES-module-aware tools like Rollup and webpack 2+ will import the ES module version directly.

[sokra]

I will do that for the client-side, but on server-side it tries to run packages unbundled.
btw. when you have an exports field, that's considered over the module field, but that doesn't matter here as they are equal.

Running unbundled means you are subject to Node.js EcmaScript implementation: https://nodejs.org/api/esm.html

[anonymouscatcher]

I'm using Github actions to deploy my app with serverless-nextjs, it's working locally but on deploy environment get JavaScript heap out of memory error...I don't use ChakraUI...but the project is large scale also a lot of dynamic imports. I tried to disable esmExternals but doesn't work.

[patrick-samy]

I'm experiencing the same issue while trying to deploy on Firebase hosting & functions.

The esmExternals fix doesn't seem to address this issue as pointed by @mustaphaturhan. Can anyone think of another way to work around this?

I've pretty much tried everything on Github issues referenced above.

[balazsorban44]

This issue has been automatically locked due to no recent activity. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.