Cannot build with .node files and Vercel adapter #1547

Nick-Mazuk · 2021-05-24T20:28:35Z

Describe the bug

Cannot build Sveltekit while using Sharp and the Vercel adapter. Sharp works in development. I suspect this issue applies to all dependencies with .node files.

This seems unrelated to the "error when including packages" question in the FAQ or #737

Logs

> Using @sveltejs/adapter-vercel
 > node_modules/.pnpm/sharp@0.28.2/node_modules/sharp/lib/output.js:4:22: error: No loader is configured for ".node" files: node_modules/.pnpm/sharp@0.28.2/node_modules/sharp/build/Release/sharp.node
    4 │ const sharp = require('../build/Release/sharp.node');

To Reproduce

Initialize new Sveltekit project
Install sharp pnpm i sharp
Create a new endpoint and import sharp
Add Vercel adapter
pnpm build

Reproduction repo: https://github.com/Nick-Mazuk/sveltekit-sharp

Expected behavior

Site would build without issues.

Stacktraces
If you have a stack trace to include, we recommend putting inside a <details> block for the sake of the thread's readability:

Stack trace

pnpm build

> ~TODO~@0.0.1 build /Users/nickmazuk/Documents/Github/sveltekit-sharp
> svelte-kit build

vite v2.3.3 building for production...
✓ 14 modules transformed.
.svelte-kit/output/client/_app/manifest.json                    1.21kb
.svelte-kit/output/client/_app/layout.svelte-39a4acb4.js        0.49kb / brotli: 0.29kb
.svelte-kit/output/client/_app/pages/index.svelte-af73b2f3.js   0.79kb / brotli: 0.42kb
.svelte-kit/output/client/_app/error.svelte-8eafbf20.js         1.19kb / brotli: 0.57kb
.svelte-kit/output/client/_app/assets/start-a8cd1609.css        0.16kb / brotli: 0.10kb
.svelte-kit/output/client/_app/chunks/vendor-3681a316.js        5.05kb / brotli: 1.98kb
.svelte-kit/output/client/_app/start-22b096ce.js                17.18kb / brotli: 5.50kb
vite v2.3.3 building SSR bundle for production...
transforming (1) .svelte-kit/build/app.js"default" is imported from external module "sharp" but never used in "src/routes/endpoint.js".
✓ 10 modules transformed.
.svelte-kit/output/server/app.js   9.40kb

Run npm run preview to preview your production build locally.

> Using @sveltejs/adapter-vercel
 > node_modules/.pnpm/sharp@0.28.3/node_modules/sharp/lib/utility.js:7:22: error: No loader is configured for ".node" files: node_modules/.pnpm/sharp@0.28.3/node_modules/sharp/build/Release/sharp.node
    7 │ const sharp = require('../build/Release/sharp.node');
      ╵                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

> Build failed with 1 error:
node_modules/.pnpm/sharp@0.28.3/node_modules/sharp/lib/utility.js:7:22: error: No loader is configured for ".node" files: node_modules/.pnpm/sharp@0.28.3/node_modules/sharp/build/Release/sharp.node
Error: Build failed with 1 error:
node_modules/.pnpm/sharp@0.28.3/node_modules/sharp/lib/utility.js:7:22: error: No loader is configured for ".node" files: node_modules/.pnpm/sharp@0.28.3/node_modules/sharp/build/Release/sharp.node
    at failureErrorWithLog (/Users/nickmazuk/Documents/Github/sveltekit-sharp/node_modules/.pnpm/esbuild@0.11.23/node_modules/esbuild/lib/main.js:1443:15)
    at /Users/nickmazuk/Documents/Github/sveltekit-sharp/node_modules/.pnpm/esbuild@0.11.23/node_modules/esbuild/lib/main.js:1125:28
    at runOnEndCallbacks (/Users/nickmazuk/Documents/Github/sveltekit-sharp/node_modules/.pnpm/esbuild@0.11.23/node_modules/esbuild/lib/main.js:1043:65)
    at buildResponseToResult (/Users/nickmazuk/Documents/Github/sveltekit-sharp/node_modules/.pnpm/esbuild@0.11.23/node_modules/esbuild/lib/main.js:1123:7)
    at /Users/nickmazuk/Documents/Github/sveltekit-sharp/node_modules/.pnpm/esbuild@0.11.23/node_modules/esbuild/lib/main.js:1230:14
    at /Users/nickmazuk/Documents/Github/sveltekit-sharp/node_modules/.pnpm/esbuild@0.11.23/node_modules/esbuild/lib/main.js:606:9
    at handleIncomingPacket (/Users/nickmazuk/Documents/Github/sveltekit-sharp/node_modules/.pnpm/esbuild@0.11.23/node_modules/esbuild/lib/main.js:703:9)
    at Socket.readFromStdout (/Users/nickmazuk/Documents/Github/sveltekit-sharp/node_modules/.pnpm/esbuild@0.11.23/node_modules/esbuild/lib/main.js:573:7)
    at Socket.emit (node:events:365:28)
    at addChunk (node:internal/streams/readable:314:12)
 ERROR  Command failed with exit code 1.

Information about your SvelteKit Installation:

Diagnostics

System:
OS: macOS 11.3
CPU: (16) x64 Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
Memory: 8.95 GB / 64.00 GB
Shell: 3.2.57 - /bin/bash
Binaries:
Node: 16.1.0 - ~/.nvm/versions/node/v16.1.0/bin/node
Yarn: 1.22.10 - /usr/local/bin/yarn
npm: 7.11.2 - ~/.nvm/versions/node/v16.1.0/bin/npm
Browsers:
Brave Browser: 90.1.24.86
Firefox: 87.0
Safari: 14.1
npmPackages:
@sveltejs/kit: next => 1.0.0-next.109
svelte: ^3.34.0 => 3.38.2

Adapter: "@sveltejs/adapter-vercel": "^1.0.0-next.19"
Sharp: "sharp": "^0.28.3"

Severity

Blocking deployment of new feature.

The text was updated successfully, but these errors were encountered:

benmccann · 2021-05-25T00:10:58Z

Since I wasn't aware of them, .node files are the extension for native addons: https://nodejs.org/api/addons.html

Does this work with adapter-node? Does Vercel support .node files independent of SvelteKit?

Nick-Mazuk · 2021-05-25T00:58:50Z

Building works with adapter-node, and I'd assume Vercel supports .node files since Next.js uses the same sharp library. It's also worth mentioning that I also run into issues building with the Vercel adapter locally, so it's not isolated to building on their servers.

benmccann · 2021-05-25T04:12:35Z

I wonder if it has anything to do with adapter-vercel's use of esbuild:

kit/packages/adapter-vercel/index.js

Line 30 in 6372690

await esbuild.build({

Nick-Mazuk · 2021-05-25T04:47:46Z

Looks like esbuild is probably the culprit: evanw/esbuild#1051

Luckily, the issue also has a code snippet for an ESBuild plugin to fix this. Not sure if adding it to the adapter itself is the direction Sveltekit should go, but it would likely work.

Alternatively, sharp (and equivalent packages) can also be marked as external. I don't think it makes sense to embed a list of external packages like sharp into the adapter. Perhaps there's some way to configure this? Perhaps the adapter could also follow the config kit.vit.optimizeDeps.exclude?

cupcakearmy · 2021-06-16T21:59:53Z

Hitting this issue as well

7antra · 2021-06-23T22:06:38Z

Oupsi, let's go for a selfmade linode deploy as always :'(

alteredorange · 2021-06-24T18:08:08Z

Is there a current workaround for this? I can't get sharp to work, even trying:

 vite: {
      optimizeDeps: {
     //   include: ['sharp']  - also tried include
        exclude: ['sharp']
      },
      ssr: {
        external: ['sharp']
      },

Any help would be appreciated!

Nick-Mazuk · 2021-06-24T18:14:06Z

@alteredorange, I couldn't figure out a workaround so I ended up creating a custom serverless function on Vercel. It's very much not ideal, though.

7antra · 2021-06-27T14:53:18Z

Did someone find a solution here ?

haishanh · 2021-06-29T04:06:18Z

Try add a postbuild inside scripts in package.json

In my case, I'm using Prisma with N-API I have:

  "postbuild": "cp node_modules/.prisma/client/*.node .vercel_build_output/functions/node/render/;cp prisma/schema.prisma .vercel_build_output/functions/node/render/;ls .vercel_build_output/functions/node/render/",

see also this comment I posted in Prisma repo.

Nick-Mazuk · 2021-06-30T22:08:23Z

@haishanh, with Primsa, did you also have errors during the build step? It looks like the issue you commented on in the Prisma repo didn't experience build-time errors. If you did also get those build-time errors, how'd you fix that?

Nick-Mazuk · 2021-06-30T22:08:43Z

Looks like this issue might be resolved once #1732 is merged.

kilianhln · 2021-07-02T20:23:10Z

Same Issue with the netlify-adapter here.

Nick-Mazuk · 2021-07-18T21:53:01Z

I haven't tried it yet, but it looks like #1914 is merged and should fix this issue for all the official adapters (except static)

7antra · 2021-08-19T08:05:11Z

Did someone try to use sharp with SvelteKit since ~v.151 ? Because I'm not sure if that is possible : Issue #1051 in esbuild github

antony · 2021-08-27T16:10:26Z

I'm not having any luck with the vercel adapter and sharp as it can't find ../build/Release/sharp-linux-x64.node.

vercel bundles your app into a single binary when it deploys, so we need to get this file on the filesystem alongside it in order for that to work.

7antra · 2021-08-27T16:56:54Z

Hey @antony, it seems there is a hacky fix : esbuild config ; but i didn't try yet

hmaesta · 2021-09-01T20:06:43Z

I am not using Vercel adapter but facing this problem as well.

I was running svimg on a Codesandbox and it was fine until I updated some packages to try something. Now everything is broken 😖

Even if I set old packages versions in package.json it is still returning a sharp error 🤔

antony · 2021-09-02T11:03:16Z

I got the .node files to work by rewriting the path in the lambda and copying the .node file to the lambda dir:

#!/bin/bash

LAMBDA_DIR=.vercel_build_output/functions/node/render/
cp node_modules/sharp/build/Release/*.node $LAMBDA_DIR
sed -i 's#\.\./build/Release#\.#g' $LAMBDA_DIR/index.js

However, it looks like once deployed to Vercel's lambda, there is no vips library available.

This seems odd because nextjs uses it.

Perhaps @leo can shine some light here - how is next using libvips? Is it possible for us to access it at runtime?

UPDATE: As usual, a millisecond after pinging Leo, this now seems to work (essentially, placing the build shared object right next to the .node file and lambda:

#!/bin/bash

LAMBDA_DIR=.vercel_build_output/functions/node/render/
cp node_modules/sharp/build/Release/*.node $LAMBDA_DIR
cp node_modules/sharp/vendor/8.11.3/linux-x64/lib/libvips-cpp.so.42 $LAMBDA_DIR
sed -i 's#\.\./build/Release#\.#g' $LAMBDA_DIR/index.js

Of course it's a total hack - I'd be interested to know if there was a better way of achieving this.

leo · 2021-09-02T13:04:18Z

Am I correctly understanding that you're in need of a native dependency in the Vercel Build Step? If so, which one exactly and what would it be used for?

If you're talking about optimizing images, have you considered using the respective edge network API for that?

antony · 2021-09-13T14:24:31Z

@leo what we'd want, as I understand it, is for Vercel's build step to handle unbundled native dependencies, or somehow provide a way to allow these dependencies to become available to projects.

Libraries like sharp dynamically load node files at runtime, so Vercel's bundling doesn't factor them in and they are missing after deployment.

The fix above makes a manual move of the required files into the lambda dir, which is quite painful to figure out.

Didn't know about the edge network API though - that could save me a lot of time in a few places!

leo · 2021-09-14T14:04:41Z

I see. Thanks for clarifying!

When using the File System API, Vercel won't do any automatic bundling based on resolving what a Serverless Function needs like we do for our user-facing API. Instead, as described on the documentation page, we just place the file we're provided with directly in the final Lambda, and place any files next to it that were put into the same folder next to that entrypoint file, so that's where you could place them.

Does that help?

When it comes to this specific case, however, I recommend using the Image Optimization API, as anything else, I think, would be a waste of performance both in the build step and at runtime.

antony · 2021-09-29T17:30:58Z

Hi Leo - Ok so I think this makes sense. If I move the files into the right place, everything seems to work fine. I think the new issue I'm encountering is to do with vite's handing of .node files, which is a different issue I need to look at!

Thanks for your help!

james-gardner · 2021-10-27T10:07:44Z

Hi Leo - Ok so I think this makes sense. If I move the files into the right place, everything seems to work fine. I think the new issue I'm encountering is to do with vite's handing of .node files, which is a different issue I need to look at!

Thanks for your help!

@antony my understanding is that's related to ESBuild? Struggling with this at the moment.

benmccann added the pkg:adapter-vercel Pertaining to the Vercel adapter label May 25, 2021

benmccann closed this as completed Jul 22, 2021

quagliero mentioned this issue Jul 30, 2021

Build fails - Cannot find module 'konva' netlify/next-runtime#571

Closed

hmaesta mentioned this issue Sep 1, 2021

Image break if using nested routes xiphux/svimg#11

Closed

benmccann mentioned this issue Oct 27, 2021

[breaking] do a single bundling with Vite removing esbuild #2580

Closed

KATO-Hiro mentioned this issue Oct 14, 2023

チーム開発ができるようにVercelへのデプロイをGitHub Actions経由でできるようにしましょう AtCoder-NoviSteps/AtCoderNoviSteps#121

Closed

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cannot build with .node files and Vercel adapter #1547

Cannot build with .node files and Vercel adapter #1547

Nick-Mazuk commented May 24, 2021

benmccann commented May 25, 2021

Nick-Mazuk commented May 25, 2021

benmccann commented May 25, 2021

Nick-Mazuk commented May 25, 2021

cupcakearmy commented Jun 16, 2021

7antra commented Jun 23, 2021

alteredorange commented Jun 24, 2021

Nick-Mazuk commented Jun 24, 2021

7antra commented Jun 27, 2021

haishanh commented Jun 29, 2021

Nick-Mazuk commented Jun 30, 2021

Nick-Mazuk commented Jun 30, 2021

kilianhln commented Jul 2, 2021

Nick-Mazuk commented Jul 18, 2021

7antra commented Aug 19, 2021 •

edited

Loading

antony commented Aug 27, 2021

7antra commented Aug 27, 2021 •

edited

Loading

hmaesta commented Sep 1, 2021

antony commented Sep 2, 2021 •

edited

Loading

leo commented Sep 2, 2021

antony commented Sep 13, 2021

leo commented Sep 14, 2021

antony commented Sep 29, 2021

james-gardner commented Oct 27, 2021

Cannot build with .node files and Vercel adapter #1547

Cannot build with .node files and Vercel adapter #1547

Comments

Nick-Mazuk commented May 24, 2021

benmccann commented May 25, 2021

Nick-Mazuk commented May 25, 2021

benmccann commented May 25, 2021

Nick-Mazuk commented May 25, 2021

cupcakearmy commented Jun 16, 2021

7antra commented Jun 23, 2021

alteredorange commented Jun 24, 2021

Nick-Mazuk commented Jun 24, 2021

7antra commented Jun 27, 2021

haishanh commented Jun 29, 2021

Nick-Mazuk commented Jun 30, 2021

Nick-Mazuk commented Jun 30, 2021

kilianhln commented Jul 2, 2021

Nick-Mazuk commented Jul 18, 2021

7antra commented Aug 19, 2021 • edited Loading

antony commented Aug 27, 2021

7antra commented Aug 27, 2021 • edited Loading

hmaesta commented Sep 1, 2021

antony commented Sep 2, 2021 • edited Loading

leo commented Sep 2, 2021

antony commented Sep 13, 2021

leo commented Sep 14, 2021

antony commented Sep 29, 2021

james-gardner commented Oct 27, 2021

7antra commented Aug 19, 2021 •

edited

Loading

7antra commented Aug 27, 2021 •

edited

Loading

antony commented Sep 2, 2021 •

edited

Loading