metro web builds cache key collision

[leggomuhgreggo]

Current Behavior

After upgrading to Expo SDK 51, and adpting expo web projects from webpack to metro web, we noticed that sometimes one project's build would have environment variables from a different project in the monorepo.

The issue was intermittent and only happened in pipelines with multiple web builds.

We determined that — if the metro.config.js file is identical between two or more projects — then it uses the same cache key. And when multiple projects are being built — after the first project is completed — subsequent projects will resolve the first project's compiled/cached app config (etc) from metro cache — because they have the same cache-key.

Note

This is also true for start commands, and (presumably) other build related commands.

EDIT: updated to improve clarity

Expected Behavior

The builds should be isolated and config from one build should not leak into subsequent builds.

GitHub Repo

https://github.com/leggomuhgreggo/metro-web-cache-issue-2024-08-09

Steps to Reproduce

Follow instructions in README.

Nx Report

Node           : 18.20.2
OS             : darwin-arm64
Native Target  : aarch64-macos
yarn           : 1.22.22

nx                 : 19.5.6
@nx/js             : 19.5.6
@nx/jest           : 19.5.6
@nx/linter         : 19.5.6
@nx/eslint         : 19.5.6
@nx/workspace      : 19.5.6
@nx/cypress        : 19.5.6
@nx/detox          : 19.5.6
@nx/devkit         : 19.5.6
@nx/esbuild        : 19.5.6
@nx/eslint-plugin  : 19.5.6
@nx/expo           : 19.5.6
@nx/nest           : 19.5.6
@nx/node           : 19.5.6
@nx/playwright     : 19.5.6
@nx/plugin         : 19.5.6
@nx/react          : 19.5.6
@nx/storybook      : 19.5.6
@nrwl/tao          : 19.5.6
@nx/vite           : 19.5.6
@nx/web            : 19.5.6
@nx/webpack        : 19.5.6
typescript         : 5.5.3
---------------------------------------
Community plugins:
@nx-go/nx-go           : 2.7.0
@nx-tools/nx-container : 5.1.0
@nxkit/playwright      : 3.0.2
@nxrs/cargo            : 0.3.4
---------------------------------------
Local workspace plugins:
         @workspace/sg-codepipeline-plugin
         @workspace/sg-codedeploy-plugin
         @workspace/sg-typeorm-plugin
         @workspace/sg-rover-plugin
         @workspace/sg-expo-plugin
         @workspace/repo
         @workspace/sg-vite-plugin
         @sg-monorepo/workspace-plugin

Failure Logs

No response

Package Manager Version

No response

Operating System

• macOS
• Linux
• Windows
• Other (Please specify)

Additional Information

I've also opened up an issue on the expo repo here:
expo/expo#30930

[xiongemi]

this seems to be something to do with expo cache. the easist way to fix is just run the command with --clear flag to clear the expo cache when export. for example:

nx run mysterious-macropod-app:export --skip-nx-cache -- --platform web --clear
nx run hopping-heron-app:export --skip-nx-cache -- --platform web --clear

[xiongemi]

the other thing you can do is in project.json, add targets:

  "targets": {
    "export": {
      "options": {
        "args": ["--clear"]
      }
    }
  }

so it will always clear the expo cache when you can export command.

[leggomuhgreggo]

Thanks for taking a look at this!

this seems to be something to do with expo cache. the easist way to fix is just run the command with --clear flag to clear the expo cache when export. for example:

Yes, this is in line with our findings here:

We determined that — if the metro.config.js file is identical between two or more projects — then it uses the same cache key. And when multiple projects are being built — after the first project is completed — subsequent projects will resolve the first project's compiled/cached app config (etc) from metro cache — because they have the same cache-key.

EDIT: updated to improve clarity

I think it'd be useful to add something to the metro.config.js — maybe part of the withNxMetro function — to distinguish the configs — so that when they're hashed, the corresponding cache-key is differentiated.

You can see an example of what Im talking about here, which leverages the cacheVersion option

[xiongemi]

Thanks for taking a look at this!

this seems to be something to do with expo cache. the easist way to fix is just run the command with --clear flag to clear the expo cache when export. for example:

Yes, this is in line with our findings here:

We determined that — if the metro.config.js file is identical between two or more projects — then it uses the same cache key. And when multiple projects are being built — after the first project is completed — subsequent projects will resolve the first project's compiled/cached app config (etc) from metro cache — because they have the same cache-key.

EDIT: updated to improve clarity

I think it'd be useful to add something to the metro.config.js — maybe part of the withNxMetro function — to distinguish the configs — so that when they're hashed, the corresponding cache-key is differentiated.

You can see an example of what Im talking about here, which leverages the cacheVersion option

this solves the metro cache key between 2 different apps. however, when I change the .env file or app.config.js, it does not refresh the output.

[leggomuhgreggo]

however, when I change the .env file or app.config.js, it does not refresh the output.

Nice find! Just confirmed on my end too.

Oddly passing --clear for one of the projects, seems to bust the cache for all projects.

yarn nx run hopping-heron-app:export --clear --platform web --skip-nx-cache
yarn nx run mysterious-macropod-app:export --platform web --skip-nx-cache

I'm guessing the expo app config is being compiled separately from other app files — because changes to App.tsx make it into the build, where changes to app.config.js do not.

(I tried looking at the expo/metro source code to see how that was working, but no luck, so far)

I can imagine why they might want to cache it separately, but yeah would seem like an oversight that modifying the app config file wouldn't break its cache. 🤷

I think the "sticky app config cache" issues could probably be considered separately from the "cross-project cache contamination" issue. If its agreeable to y'all, I'd vote in favor of shipping the cacheVersion fix and following up on the remaining items.