Build fails to compile - module not found: can't resolve "child_process" 12.1.6

[wadehammes]

Verify canary release

• I verified that the issue exists in Next.js canary release

Provide environment information

Operating System:
Platform: linux
Arch: x64
Version: #1 SMP Wed Mar 2 00:30:59 UTC 2022
Binaries:
Node: 14.18.1
npm: 6.14.15
Yarn: 1.22.18
pnpm: N/A
Relevant packages:
next: 12.1.6
react: 18.1.0
react-dom: 18.1.0

What browser are you using? (if relevant)

Firefox

How are you deploying your application? (if relevant)

Vercel

Describe the Bug

In the latest 12.1.6, I can't get my site to compile on Vercel. Everything works fine in previous release. See log below. I am not sure if this is a Sentry issue or an issue with next.

Expected Behavior

Site compiles like in 12.1.5

To Reproduce

Have a next site set up with Sentry initiated at webpack level.

[balazsorban44]

Related: #36486, #36434

Note that you should not use Node-specific APIs in your Middleware as it uses the Edge Runtime. (See the docs: https://nextjs.org/docs/api-reference/edge-runtime#unsupported-apis).

Sentry does not seem to be compatible at the moment.

Maybe related #36237 (comment)

Could you add a small reproduction to verify?

[wadehammes]

Sentry works fine in middleware in 12.1.5 @balazsorban44 wondering what changed?

[wadehammes]

These nextRuntime does not work for me in #36237

[sergeyshaykhullin]

It also break axios usage inside _middleware

experimental: {
  runtime: 'nodejs'
},

Doesn't help ;\

error - ./node_modules/axios/lib/helpers/toFormData.js:26:0
Module not found: Can't resolve 'buffer'

Import trace for requested module:
./node_modules/axios/lib/axios.js
./node_modules/axios/index.js
./src/api/index.ts
./src/pages/_middleware.ts

https://nextjs.org/docs/messages/module-not-found

You're using a Node.js module (buffer) which is not supported in the Edge Runtime.
Learn more: https://nextjs.org/docs/api-reference/edge-runtime

[liamross]

Seems to be breaking Prisma, Upstash, etc within Middleware files, not sure if this is a new "stricter" middleware setup or if the version is broken.

[balazsorban44]

Sentry works fine in middleware in 12.1.5 @balazsorban44 wondering what changed?

So up until recently, we have polyfilled a few Node.js APIs even for the Edge Runtime. This has been changed in #36190 as it was the original intention as documented: https://nextjs.org/docs/api-reference/edge-runtime#unsupported-apis

Any library relying on that did so by mistake.

[balazsorban44]

@sergeyshaykhullin the switchable runtime does not concern Middleware, we added a note about this in the docs to make it more clear: #36862

Also in the Middleware docs there is this:

Middleware uses a strict runtime

So this should explain your situation. I'm closing this as the result is expected. A final word on Sentry, they don't have official Middleware support yet: getsentry/sentry-javascript#4206

[wadehammes]

@balazsorban44 thanks! Ive stripped sentry out of the API calls we were making at the middleware level and all is working now.

[sergeyshaykhullin]

@balazsorban44 Does that mean i can't do http requests inside middleware?

[liamross]

@sergeyshaykhullin You can still use fetch (https://nextjs.org/docs/api-reference/edge-runtime#fetch)

Also as a side note: I think the issues I was facing may have been due to a custom webpack config within the next config file, will do more testing later

[balazsorban44]

Improved the documentation here: #36862

[straube]

@balazsorban44 In case I fetch CMS data (we use Sanity.io) in a middleware/Edge function and their client relies on unsupported Node modules, I have only two choices:

1. Ask the CMS client maintainer to "fix" their package to not use native Node modules (unlikely).
2. Write our own client/integration to the CMS API.

Is that right?

[balazsorban44]

You should not be using Middleware to fetch CMS data, check out our data fetching documentation for that.

[straube]

Maybe I should've explained the use case: the redirects are managed by content editors -- for SEO, etc. -- then they're stored in the CMS. We have a site-wide middleware that looks up for redirects using the Sanity.io client. That's why we fetch CMS data in a middleware.

Perhaps, we could add polyfills for the required node modules to our own codebase?

[wadehammes]

Maybe I should've explained the use case: the redirects are managed by content editors -- for SEO, etc. -- then they're stored in the CMS. We have a site-wide middleware that looks up for redirects using the Sanity.io client. That's why we fetch CMS data in a middleware.

Perhaps, we could add polyfills for the required node modules to our own codebase?

Very curious in this use case as well, as I have wanted to do something similar in our Contentful setup to allow my team to handle setting up redirects in the CMS. Currently, it's a manual process to add them to next.config.js and then go through a deploy process. Would be cool to automate this some how. 🤔

[straube]

@wadehammes Here is a simplified version of the middleware I mentioned, just to illustrate how it can be done:

import { NextRequest, NextResponse } from "next/server";

const getRedirectUrl = (target: string, req: NextRequest) => {
  const isAbsoluteUrl = Boolean(target.match(/^https?:\/\//));
  if (isAbsoluteUrl) {
    return target;
  }

  /*
   * Relative URL.
   *
   * https://nextjs.org/docs/messages/middleware-relative-urls
   */
  const url = req.nextUrl.clone();
  url.pathname = target;
  return url;
};

export async function middleware(req: NextRequest) {
  const { pathname } = req.nextUrl;

  // This is where you'd fetch the redirect from your CMS
  const redirect = await getRedirectByPath(pathname);

  if (redirect) {
    const url = getRedirectUrl(redirect.target, req);
    return NextResponse.redirect(url, redirect.permanent ? 301 : 302);
  }
};

The type of the redirect object returned by getRedirectByPath looks like this:

type Redirect = {
  target: string;
  permanent: boolean;
};

But I guess you get the idea.

[wadehammes]

@wadehammes Here is a simplified version of the middleware I mentioned, just to illustrate how it can be done:

import { NextRequest, NextResponse } from "next/server";

const getRedirectUrl = (target: string, req: NextRequest) => {
  const isAbsoluteUrl = Boolean(target.match(/^https?:\/\//));
  if (isAbsoluteUrl) {
    return target;
  }

  /*
   * Relative URL.
   *
   * https://nextjs.org/docs/messages/middleware-relative-urls
   */
  const url = req.nextUrl.clone();
  url.pathname = target;
  return url;
};

export async function middleware(req: NextRequest) {
  const { pathname } = req.nextUrl;

  // This is where you'd fetch the redirect from your CMS
  const redirect = await getRedirectByPath(pathname);

  if (redirect) {
    const url = getRedirectUrl(redirect.target, req);
    return NextResponse.redirect(url, redirect.permanent ? 301 : 302);
  }
};

The type of the redirect object returned by getRedirectByPath looks like this:

type Redirect = {
  target: string;
  permanent: boolean;
};

But I guess you get the idea.

@straube awesome implementation, we use middleware for AB test identification and page redirects (with Launch Darkly) which works perfect (we use the fetch API to hit custom internal endpoints that handle the Launch Darkly identify and flag retrieval), wondering if you could leverage something similar (not familiar with Sanity) but the more I think about it I could probably do the same with Contentful and just hit their endpoint (and pass the pathname as a param) instead of using their SDK (which would fail due to the node module issue).

[straube]

Perhaps, we could add polyfills for the required node modules to our own codebase?

Just answering my own question here: no. I tried that and some polyfills rely on eval which is also not available on the Edge environment. We'll probably have to update the code where we use client libraries (such as Sanity) to dispatch requests using fetch.

[github-actions]

This closed issue has been automatically locked because it had no new activity for a month. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.