UnhandledSchemeError when dependency uses node:assert imports

[stefanprobst]

What version of Next.js are you using?

11.1.2

What version of Node.js are you using?

14.17.6

What browser are you using?

ff

What operating system are you using?

linux

How are you deploying your application?

vercel

Describe the Bug

In a project which has micromark-util-events-to-acorn@1.0.1 as a dependency, i get the following error when running yarn dev:

Module build failed: UnhandledSchemeError: Reading from "node:assert" is not handled by plugins (Unhandled scheme).

Things to note about this package: it uses an export map with a "development" user condition, and node:assert is only used there, not in the exported production build.

Expected Behavior

No error.

To Reproduce

1. git clone https://github.com/stefanprobst/issue-next-webpack-node-imports.git
2. cd issue-next-webpack-node-imports
3. yarn && yarn add next@latest && yarn dev
4. open http://localhost:3000/
5. See error message

[shimamooo]

Any updates on this issue? Facing the same problem but for xdm.

[shimamooo]

Is the node: protocol supported in the current Next.js webpack configuration?

Things to note about this package: it uses an export map with a "development" user condition, and node:assert is only used there, not in the exported production build.

This is true throughout the micromark ecosystem as far as I know, so support for this would be great!

Edit: mentioned in remarkjs/react-markdown#641, micromark/micromark#87, and remarkjs/react-markdown#632

[stefanprobst]

I'm still getting this error with next.js 12

[raitucarp]

Me too, using remark getting this error

[vashisth00]

Still getting this error

Module build failed: UnhandledSchemeError: Reading from

handled by plugins (Unhandled scheme).
Webpack supports "data:" and "file:" URIs by default.
You may need an additional plugin to handle "https:" URIs.

[balazsorban44]

Looks like webpack itself already supports this. webpack/webpack#12693

[manovotny]

Here's a minimal reproduction repo with no other dependencies besides Next.js and React themselves, if that's helpful.

https://github.com/manovotny/nextjs-node-protocol-bug

[pwan-a]

Also seeing this issue when using MSW with NextJS

[chentsulin]

Next.js adds fallbacks for node built-in modules by using resolve.fallback:

next.js/packages/next/build/webpack-config.ts

Lines 628 to 659 in 384953b

	// https://github.com/webpack/webpack/blob/2a0536cf510768111a3a6dceeb14cb79b9f59273/lib/ModuleNotFoundError.js#L13-L42
	fallback: {
	assert: require.resolve('next/dist/compiled/assert'),
	buffer: require.resolve('next/dist/compiled/buffer/'),
	constants: require.resolve(
	'next/dist/compiled/constants-browserify'
	),
	crypto: require.resolve('next/dist/compiled/crypto-browserify'),
	domain: require.resolve('next/dist/compiled/domain-browser'),
	http: require.resolve('next/dist/compiled/stream-http'),
	https: require.resolve('next/dist/compiled/https-browserify'),
	os: require.resolve('next/dist/compiled/os-browserify'),
	path: require.resolve('next/dist/compiled/path-browserify'),
	punycode: require.resolve('next/dist/compiled/punycode'),
	process: require.resolve('./polyfills/process'),
	// Handled in separate alias
	querystring: require.resolve('next/dist/compiled/querystring-es3'),
	stream: require.resolve('next/dist/compiled/stream-browserify'),
	string_decoder: require.resolve(
	'next/dist/compiled/string_decoder'
	),
	sys: require.resolve('next/dist/compiled/util/'),
	timers: require.resolve('next/dist/compiled/timers-browserify'),
	tty: require.resolve('next/dist/compiled/tty-browserify'),
	// Handled in separate alias
	// url: require.resolve('url/'),
	util: require.resolve('next/dist/compiled/util/'),
	vm: require.resolve('next/dist/compiled/vm-browserify'),
	zlib: require.resolve('next/dist/compiled/browserify-zlib'),
	events: require.resolve('next/dist/compiled/events/'),
	setImmediate: require.resolve('next/dist/compiled/setimmediate'),
	},

And this feature now doesn't support node protocol: webpack/webpack#14166

[OskarGroth]

Not sure if this fix for a similar bug is helpful: webpack/webpack#15577
Can NextJS make sure to update their Webpack module to this version and see if it resolves this?

[timneutkens]

@OskarGroth this went out in webpack 5.71.0, Next.js is using webpack 5.72.0 on the latest stable (12.1.5).

[hrasekj]

I can confirm, that issue is still there in version 12.1.5.

[OskarGroth]

@timneutkens I'm using Next.js 12.1.5 and still experiencing this issue.

More info about the error I'm getting and the troubleshooting I've done: leafac/rehype-shiki#7

[OskarGroth]

No longer experiencing this issue (12.1.5).

[firatgh]

I am facing the same issue, after updating to Next 12.1.5

wait  - compiling...
error - node:buffer
Module build failed: UnhandledSchemeError: Reading from "node:buffer" is not handled by plugins (Unhandled scheme).
Webpack supports "data:" and "file:" URIs by default.
You may need an additional plugin to handle "node:" URIs.

[balazsorban44]

I'm no longer able to reproduce the OP's issue. If you still do, please open a new issue with an attached reproduction, making sure you have tested the latest of next!

Remember to check which Node.js versions support node: protocol imports:

https://nodejs.org/api/esm.html#node-imports

[stefanprobst]

i am still seeing the exact same error message with next@12.1.6 and node@16.14.2:

1. git clone https://github.com/stefanprobst/issue-next-webpack-node-imports.git
2. cd issue-next-webpack-node-imports
3. yarn && yarn add next@latest && yarn dev
4. open http://localhost:3000/
5. See error message

[balazsorban44]

Oh, interesting. The reproduction uses 11.1.2, but upgrading to latest does throw the error after all. 👀

It might be because the client-side code elimination tool isn't able to recognize the module you are using is for server-only.

import fs from "node:fs"

export default function Home() {
  return null
}

export async function getStaticProps() {
  console.log(fs)
  return { props: {} }
}

This works as the code can be analyzed and Next.js will know where the node: import is being used.

Let's reopen and investigate further.

[GabenGar]

Any progress on this? Got hit by this error too while trying to migrate the project to ESM.
Worst part there is no stack trace either.

EDIT: I guess a separately installed node-fetch was the culprit for this, the same as #32935. But I have a suspicion it is not going to be be a proper fix, since I was installing it so fetch could work in jest tests and in my setup. Was getting import errors related to Headers interface, but that was on node 12 so maybe it will work on 16 properly now.

[nbouvrette]

This is related to webpack/webpack#13290 (fresh Next.js comments at the bottom of the thread)

For those looking for a workaround, I wrote a Webpack configuration handler in TypeScript.

You can simply use it by setting your webpack function in your next.config.js to webpackConfigurationHandler using the code below:

import Webpack from 'webpack'

import type { WebpackConfigContext } from 'next/dist/server/config-shared'

/**
 * Next.js did not define any types for its Webpack configs.
 *
 * @see https://github.com/vercel/next.js/blob/canary/packages/next/compiled/webpack/webpack.d.ts
 * @see https://github.com/vercel/next.js/blob/60c8e5c29e4da99ac1aa458b1ba3bdf829111115/packages/next/server/config-shared.ts#L67
 */
export interface WebpackContext extends WebpackConfigContext {
  webpack: typeof Webpack
}

/**
 * Handles the Webpack configuration.
 *
 * @param config - The Webpack configuration options.
 * @param context - The Webpack context
 *
 * @returns A Webpack configuration object.
 */
function webpackConfigurationHandler(
  config: Webpack.Configuration,
  context: WebpackContext
): Webpack.Configuration {
  /**
   * Add support for the `node:` scheme available since Node.js 16.
   *
   * @see https://github.com/vercel/next.js/issues/28774
   */
  config.plugins = config.plugins ?? []
  config.plugins.push(
    new context.webpack.NormalModuleReplacementPlugin(/^node:/, (resource: { request: string }) => {
      resource.request = resource.request.replace(/^node:/, '')
    })
  )

  return config
}

[sebald]

@nbouvrette Thanks for the fix. After that everything works fine, but we get a warning from typescript after adding the plugin.

warn  - ../node_modules/.pnpm/typescript@4.7.4/node_modules/typescript/lib/typescript.js
Critical dependency: the request of a dependency is an expression

Not sure if this is problematic though.

[nbouvrette]

@sebald I recall briefly seeing this error and I think I fixed it by deleting the .next directory. If this doesn't work, you can check my full implementation here and maybe spot something I am missing. I don't have this error on my end with this implementation.

[artivilla]

Slight modification I made so I could use the new format using next@12.3.0 and webpack@5.74.0, if this is helpful:

const webpack = require("webpack");

module.exports = {
  webpack: (config, options) => {
    config.plugins.push(
      new webpack.NormalModuleReplacementPlugin(/^node:/, (resource) => {
        resource.request = resource.request.replace(/^node:/, "");
      })
    );
    return config;
  },
};

[secretmtgdev]

Slight modification I made so I could use the new format using next@12.3.0 and webpack@5.74.0, if this is helpful:

const webpack = require("webpack");

module.exports = {
  webpack: (config, options) => {
    config.plugins.push(
      new webpack.NormalModuleReplacementPlugin(/^node:/, (resource) => {
        resource.request = resource.request.replace(/^node:/, "");
      })
    );
    return config;
  },
};

This is a great workaround!

[pirtlj]

Sill seeing this in next@13.3.1

[Thinkscape]

⚠️ This does not apply to middleware ⚠️

If you see compile errors related to node modules or node: prefix, that is because in Next 13 middleware ALWAYS runs in Edge runtime without a way to opt-out.

This means that whatever you import in middleware.ts, and their dependencies, must not import node packages or use edge-incompatible APIs

1. It is currently impossible to change runtime for middleware with config.experimental.runtime or export const runtime = "nodejs" and they will be ignored.
2. The above webpack workaround will not fix the underlying issue and runtime will still fail with missing packages (such as stream or crypto)
3. Tested on next@13.3.1

Example failures

Even after closing webpack/webpack#13290 and supposed fix in #45149 merged in Jan, it's still failing for code in my middleware.ts that imports a 3rd party module, which seems to have @aws-sdk transient dependency, which dynamically imports node:file

dev:next: wait  - compiling /src/middleware (client and server)...
dev:next: error - node:fs
dev:next: Module build failed: UnhandledSchemeError: Reading from "node:fs" is not handled by plugins (Unhandled scheme).
dev:next: Webpack supports "data:" and "file:" URIs by default.
dev:next: You may need an additional plugin to handle "node:" URIs.
dev:next: Import trace for requested module:
dev:next: node:fs
dev:next: ./node_modules/.pnpm/strtok3@7.0.0/node_modules/strtok3/lib/FsPromise.js
dev:next: ./node_modules/.pnpm/strtok3@7.0.0/node_modules/strtok3/lib/index.js
dev:next: ./node_modules/.pnpm/file-type@18.3.0/node_modules/file-type/index.js
dev:next: ./node_modules/.pnpm/@keystone-6+core@5.2.0_@aws-sdk+abort-controller@3.310.0_@babel+core@7.21.4_@opentelemetry+ap_eexecvfunafgo67t2ljvtiogxm/node_modules/@keystone-6/core/dist/createSystem-a93ebf95.esm.js
dev:next: ./node_modules/.pnpm/@keystone-6+core@5.2.0_@aws-sdk+abort-controller@3.310.0_@babel+core@7.21.4_@opentelemetry+ap_eexecvfunafgo67t2ljvtiogxm/node_modules/@keystone-6/core/context/dist/keystone-6-core-context.esm.js
dev:next: ./src/keystone/context.ts
dev:next: ./src/server/auth.ts

[CanRau]

wow thanks a lot for clarifying this @Thinkscape always felt there's something off with middleware as I couldn't get mysql2 working which I use in dev locally and was always confused but now that not even cuid2 works got me finally investigating.

This is actually reeaally annoying as so far, I think, middleware is the only place to set cookies on a request without user interaction or calling an api, which is already pretty annoying but have this middleware limited like this 🥲

Also I'd really like to be able to pass data from middleware to server components/actions and routes (#34263 (comment)) which is pretty limiting not being able to do this 😬

[balazsorban44]

Hi everyone, this should be fixed in newer versions of Next.js, make sure you upgrade (npm i next@latest)! I tested the reproduction on top of this issue and it worked.

If you still see an issue and you think it's related, please open a new one with an attached reproduction, thanks! 💚

[nbouvrette]

@balazsorban44 which version should have the fix exactly? I can't find any mention in the release notes of the 2 latest version (13.4.8 or 13.4.9)

I tried updating to 13.4.9 and got the following error when removing my fix:

- error node:path
Module build failed: UnhandledSchemeError: Reading from "node:path" is not handled by plugins (Unhandled scheme).
Webpack supports "data:" and "file:" URIs by default.
You may need an additional plugin to handle "node:" URIs.
Import trace for requested module:
node:path
../lib/index.js

[timneutkens]

@nbouvrette It was fixed many releases ago. Keep in mind that node: is not supported in client-side code because it's Node.js-only. So e.g. server components or getServerSideProps/getStaticProps is fine, other places it'll still error.

[nbouvrette]

@timneutkens for the explanation - I think I understand what is going on here. The error I am getting is on import { sep as pathSeparator } from 'node:path' which is used both by the client and server side.

The reason why it works with my workaround is because the node:path module, and specifically the sep property, is a bit of a special case. The sep property provides the path separator character for the current operating system, which is a simple string ('/' for Unix-based systems, '\' for Windows). This doesn't rely on any Node.js-specific functionality, so it can be used in the browser without any issues.

Webpack has shims for some Node.js core modules that provide equivalent functionality in the browser, and path is one of them. The path shim provides the sep property, so my code works as expected.

Ideally I would hope that this would work our of the box without having me using special Webpack config. What do you think @timneutkens @balazsorban44 ? Or should I open another issue for this use case?

[timneutkens]

Webpack has shims for some Node.js core modules that provide equivalent functionality in the browser, and path is one of them. The path shim provides the sep property, so my code works as expected.

We eventually want to remove these default shims because they increase bundle size massively in most cases. node: is meant to signal "In the Node.js environment" so hence why we intentionally do not add shims for those.

[nbouvrette]

Ok I get your point @timneutkens - I suspect for my specific use case the shims are pretty small but I could always try to replace them without having to rely on Webpack so that if you ever decide to drop the default shims my code still work.

[timneutkens]

Definitely yeah 👍

[github-actions]

This closed issue has been automatically locked because it had no new activity for a month. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.