rebuilding take too long

[lili21]

why it take 4 seconds to rebuild a file? is it incremental rebuild?

[xHomu]

The team is still optimizing the new dev server. Couple things to try:

• Change your remix dependencies to: "nightly".
• Turn off tailwind or postcss in remix.config if you're not using them.

Make sure you're not double-purging with incorrectly configurated server.js, reference https://github.com/xHomu/remix-v2-server/branches

[lili21]

after some digging in, I found some interest things.

I'm using ESM module format. and there is a dependency, let's call it tux.

import { TUXProvider } from 'tux';

export default function Index() {
  return <TUXProvider>Hello</TUXProvider>;
}

the entry file of tux is just reexporting many components.

export { TUXProvider } from './components/TUXProvider/TUXProvider.js'
export { TUXButton } from './components/TUXButton/TUXButton.js'
// and so on, 1000 more lines.

the file is pretty big, but it is tree shakeable.

I changed my code

export default function Index() {
  return <div>Hello</div>;
}

the rebuilding only takes like 200~300ms.

I tried to import it with full path,

import { TUXProvider } from 'tux/dist/components/TUXProvider/TUXProvider.js';

export default function Index() {
  return <TUXProvider>Hello</TUXProvider>;
}

but the build failed.

SyntaxError: Named export 'TUXProvider' not found. The requested module 'tux/dist/components/TUXProvider/TUXProvider.js' is a CommonJS module, which may not support all module.exports as named exports.

[dmarkow]

So if you're saying just importing that component library is the difference between 200ms and 4-6 seconds, it would be helpful to know the exact component library you're using -- I assume it isn't actually tux, as that package installs something different from what you showed.

[lili21]

it's an internal package. I will try some open sourced library and create a reproduce repo.

[lili21]

https://github.com/lili21/remix-building-time
I created a repo that can reproduce the problem.

[pcattori]

Thank you for the repro! I took a look and build/rebuild times are about the same with unstable_dev enabled and disabled. Locally

I'm getting ~150ms builds and rebuilds without antd and ~1s builds/rebuilds with antd on an M1 Max MBP.
I also did some perf profiling with and without antd:

👆without antd

👆with antd

Looking at those perf profiles, the root cause is just that there's way more files for esbuild to process when pulling in that library. Not sure what we can do about that in the short-term.

[lili21]

Thanks, that's helpful.

it looks like esbuild incremental build doesn't help a lot. does esbuild provide some way to use the cache? I just changed my source code, no need to rebuild the antd library.

[lili21]

comparing to my other remix project which is not using HMR and got more dependencies. the rebuilding only take like 1s.

[lili21]

too many esbuild plugin I guess.

[lili21]

my other remix project still using deferred version( @remix-run/react@deferred）, and the rebuild is way more faster.

[vidrepar]

Thank you for the repro! I took a look and build/rebuild times are about the same with unstable_dev enabled and disabled. Locally

I'm getting ~150ms builds and rebuilds without antd and ~1s builds/rebuilds with antd on an M1 Max MBP. I also did some perf profiling with and without antd:

👆without antd 👆with antd

Looking at those perf profiles, the root cause is just that there's way more files for esbuild to process when pulling in that library. Not sure what we can do about that in the short-term.

My observations:

1. There seems to be a significant difference between M chips vs. Intel ones (only tested on Macs). 1s becomes 40s - 60s for incremental rebuilds (eg. adding a console.log) on MBP 2017 3,1 GHz Dual-Core Intel Core i5 when migrating a project from CRA. To be fair, Next (app dir, with webpack; not tested with --turbo) rebuilds also take longer for the same migration, but not more than 40s.
2. Fixing imports for MUI (https://mui.com/) to a tree-shake-able version made a significant difference in a Next migration test (didn't try it with Remix). This is probably related to @pcattori 's comment about antd - in MUI (https://mui.com/material-ui/guides/minimizing-bundle-size/#development-environment) they claim that the development build can be up to 6x slower depending on the type of import ( slower: import { Delete } from '@mui/icons-material';, faster: import Delete from '@mui/icons-material/Delete'; )
3. I like to use namespaced imports import * as stripeClient from './stripe-client'; - they are tree-shakeable (https://esbuild.github.io/api/#tree-shaking) but I'm not sure how they affect incremental rebuilds. Would be great to understand this.
4. This discussion revolves around 2 things atm, tree shaking and incremental builds. I think that the true issue to solve here is incremental rebuilds (they might be connected to tree-shaking partially or fully, not sure).
5. Found some issues regarding incremental rebuilds with esbuild that might be worth looking into: Incremental build / watch mode should be probably taken more seriously evanw/esbuild#1204, Bundled React is having worse performance evanw/esbuild#2951

Possible short-term action points:

1. Add a section to the docs about the imports and how they affect DX (see point 2 in the "My observations" section)
2. Introduce some kind of a warning (eslint?) that prevents named imports from specific (?) 3rd party libraries. Not sure how to do this yet.
3. Add a note to the docs mentioning that the DX on older computers can be significantly degraded and to not expect the legendary DX (atm).

Other:

• We expect bundling and/or rebuilding frontend, backend, and whatnot in large codebases, and not on the newest computers in under 1s - this is challenging ofc. It seems that Next encountered some of these problems as well, thus slowly migrating (?) from webpack to their own Rust-based thing https://vercel.com/blog/turbopack
• I'm not aware of what's cooking with the new dev server, the Remix team may already be aware of all of these issues and is trying to solve them, or even made significant progress. This is just my 50 cents.
• Keep up the great work, @pcattori and the team 💪

[lili21]

it looks like esbuild will treat the dependency as CMD, and the tree shaking wouldn't work IMO. maybe that's why import Delete from '@mui/icons-material/Delete is faster.

will serverModuleFormat: "esm" help here? make the server side bunlde tree shakeable.

[kiliman]

@vidrepar great summary. I'm sure the Remix team appreciates the feedback.

[flemzord]

We're also experiencing dev performance problems, with pages taking 15s to reload. Has anyone found a solution?

[storycoding]

Could we get an official answer from the Remix team here? Seems like some clarity is due in regards to the reasoning behind this issue

[deftomat]

Issue is in esbuild. It doesn't support the REAL incremental rebuilds. Every rebuild is basically from scratch. We solved this in our build process by doing the build it 2 phases:

1. run esbuild with custom plugin which will replace every import from node_modules with simple "facade like" file which just imports the required module from node_modules. Every facade is flagged as external, so esbuild will not include it in bundle.
2. find all imported facades in a produced bundle and use them as entry points for a second esbuild run.

By doing this, we can bundle our code and everything from node_modules in two separate processes. So, each change in our code will trigger just the first step of the build process which is often just a few milliseconds instead of bundling all the node_modules every time which could easily took 4~5 seconds. There are situations like adding new external dependency which could trigger the second step too but they don't happen often.

[gustavorino]

@deftomat this is very interesting, can you share a repo or gist with that setup?

[deftomat]

Hi, sorry I cannot. It is a private repo with our own custom build chain tailored for aws-cdk.

[lili21]

have you tried vite ? @deftomat,

[deftomat]

Yes, but it was too slow for us at that time due to Rollup.

[pcattori]

Happy to report that with our new Vite plugin, HMR times are vastly improved. The HMR w/ AntD that used to take ~1s on my MBP M1 Max now takes <100ms with my (lower powered) MBP M1 Pro. Anyone seeing slow rebuild times should switch to the Remix Vite plugin, which is on the cusp of being marked as stable (#8713).

---

To reproduce, I grabbed the Vite template (npx create-remix@latest --template remix-run/remix/templates/unstable-vite), and installed antd and added ConfigProvider to my app/root.tsx:

import {
  Links,
  Meta,
  Outlet,
  Scripts,
  ScrollRestoration,
} from "@remix-run/react";
import { ConfigProvider } from "antd";

export default function App() {
  return (
    <html lang="en">
      <head>
        <meta charSet="utf-8" />
        <meta name="viewport" content="width=device-width,initial-scale=1" />
        <Meta />
        <Links />
      </head>
      <body>
        <ConfigProvider>
          <Outlet />
        </ConfigProvider>
        <ScrollRestoration />
        <Scripts />
      </body>
    </html>
  );
}

Then I hit p + enter in terminal to start profiling, edited app/routes/_index.tsx, and then hit p + enter in terminal again to stop profiling.

[pcattori]

For any other perf related issue, please open GitHub issues with reproductions and profiles included to help expedite any fixes!

[deftomat]

Nice!

[lopugit]

Wow, wow, wow, wow, wow, wow, Vite is..... Vite is.... Wow 🤤🤯🙏

[kiliman]

It's quite literally "Magic" 🪄🧙✨

[LuckyArdhika]

How to use vite config with server.ts but still maintain HMR?

[paul-vd]

you can follow the guide of https://github.com/remix-run/remix/blob/main/templates/express/server.js and then if you want typescript use tsx to launch the server

[paul-vd]

NOTE: HMR would not work on the server.ts file.

If you need reload on the server file it's better to implement chokidar to run the tsx command.