Detect & fix yarn / npm incompatibilities

[hunterloftis]

For most projects with a yarn.lock file, yarn is working well, but if it doesn't work, users have to either lock to an old version of the buildpack or search to discover .slugignore to deploy their project, which is frustrating. Some simple things could make this experience smoother:

• add ".yarn-cache" to the default cache list instead of caching node_modules, and clear node_modules on any yarn project. This should mitigate some missing module issues with yarn at the moment.
• add yarn --version to the cache signature to clear the cache whenever yarn is added / updated
• make YARN_PRODUCTION inherit from NPM_CONFIG_PRODUCTION unless it's explicitly set, such that yarn and npm both treat devDependencies equivalently again
• add a warning on failure that checks for $YARN and links to build behavior docs.
• add a warning on failure that checks for $YARN and looks for private modules, and log that they are currently unsupported by yarn.
• recommend running with YARN_PRODUCTION=false until yarn install --production doesn't install correct dependencies yarnpkg/yarn#761 is resolved
• when yarn fails, detect the failure and show how to use slugignore to opt out of yarn on Heroku
• use yarn list --depth=0 instead of yarn ls

Possibly also follow this practice for npm: cache the global module cache, but clear the local node_modules on each build.

[adamreisnz]

I would say that this Yarn issue is currently the biggest problem for production builds: yarnpkg/yarn#761

The problem is that with the production flag, yarn doesn't find deep dependencies of modules sometimes and doesn't install them, causing production builds to break consistently.

Perhaps a warning around missing dependencies for production should be added to the checklist above.

[hunterloftis]

@adamreisnz done

[itaysabato]

@hunterloftis is there a recommended way to use .yarn-cache on Heroku until the first item on the list is implemented?

Thanks!

[hunterloftis]

@itaysabato you could just add it to your cached directories:

• https://devcenter.heroku.com/articles/nodejs-support#custom-caching

[itaysabato]

@hunterloftis I can't seem to make caching work with yarn. Please let me know if this is a bug or something I am doing wrong.

I tried setting the cache directory to .yarn-cach by adding the line
"cacheDirectories": [".yarn-cache" ], to my package.json but the result was:

-----> Caching build
       Clearing previous node cache
       Saving 1 cacheDirectories (package.json):
       - .yarn-cache (nothing to cache)

And indeed on the next build I got:

-----> Restoring cache
       Loading 1 from cacheDirectories (package.json):
       - .yarn-cache (not cached - skipping)

After invoking yarn cache dir during the Heroku build I found that the actual cache directory path used is reportedly /app/.cache/yarn. I tried using both that and .cache/yarn but got the same result.

Any ideas?

Thanks

[hunterloftis]

Hey @itaysabato thanks for sharing those results; I'll take a look at it this week.

[itaysabato]

yarnpkg/yarn#761 is now solved in v0.19.1.

Also dev dependencies are built when NPM_CONFIG_PRODUCTION is false so there is no need to make YARN_PRODUCTION inherit from it explicitly. Thus the following warning is also obsolete:

Warning: when NODE_ENV=production, yarn will NOT install any devDependencies
(even if NPM_CONFIG_PRODUCTION is false)

P.s. @hunterloftis any news regarding yarn cache? :)

Thanks!