We can probably make webpack go much much faster.

[ara4n]

Thins to consider:

• Babel runs single-threaded, and webpack loaders run in series, so only uses one core. We could look at something like https://github.com/amireh/happypack to act as a webpack loader which then hands out to a pool of node worker processes to parallelise Babel etc.
• Switch js-sdk and react-sdk to use webpack rather than just babel, so we can then use things like happypack and babel-loader?cacheDirectory to parallelise the build process for them
• Check we've tuned Uglify correctly (e.g. turned on parallel) when in production mode (something like:

  optimization: {
    runtimeChunk: false,
    splitChunks: {
      cacheGroups: {
        commons: {
          test: /[\\/]node_modules[\\/]/,
          name: 'vendors',
          chunks: 'all',
        },
      },
    },
    minimizer: [
      new UglifyJsPlugin({
        cache: true,
        parallel: true,
        sourceMap: true
      })
    ]
  },

Currently uglify doesn't seem to use all available cores.

• Do all the things (other than the multiple configs optimisations, given we only have one config) from https://blog.box.com/blog/how-we-improved-webpack-build-performance-95

[turt2live]

Something else to consider if the js-sdk and react-sdk change build systems is to actually publish a compiled output on npm. Currently when pulling the packages down, both SDKs rebuild themselves to give the developer a fresh copy.

[bwindels]

I think we already publish compiled output (index.js requires stuff from lib/, and the prepublish step builds the SDK), not sure about rebuilding after installation though ...

[MTRNord]

As I currently tried using splitChunks:

Doing it on everything breaks most of the result (css and libs fail to load)
This seems to work but is not fully optimized:

// Automatically split vendor and commons
        // https://twitter.com/wSokra/status/969633336732905474
        // https://medium.com/webpack/webpack-4-code-splitting-chunk-graph-and-the-splitchunks-optimization-be739a861366
        splitChunks: {
            cacheGroups: {
                vendor: {
                    test: /[\\/]node_modules[\\/](react|react-dom|react-addons-perf|bluebird|matrix-js-sdk|emojione)[\\/]/,
                    name: 'vendor',
                    chunks: 'all',
                }
            }
        },
        // Keep the runtime chunk seperated to enable long term caching
        // https://twitter.com/wSokra/status/969679223278505985
        runtimeChunk: true,

It is a modified version of: https://github.com/facebook/create-react-app/blob/52449c34eedc53e50a2a159d38604ea7df5bd997/packages/react-scripts/config/webpack.config.prod.js#L158-L167

(I added the regexp to limit where it gets run on)

[MTRNord]

So I got a build using matrix-react-sdk transpiled using webpack and using #7391 (comment)

This generates this stats.json: (Cant upload as 50MB is way to big for github issues or gists :/ )

The bundle folder itself now has a size of 5,17 MB according to windows.

[MTRNord]

Also the cache and parallel options are already set to true on Uglify: https://github.com/webpack/webpack/blob/v4.20.2/lib/WebpackOptionsDefaulter.js#L309-L310

[MTRNord]

Branches of the things I was able to do are up at:

https://github.com/MTRNord/riot-web/tree/webpack4-optimizations
https://github.com/MTRNord/matrix-react-sdk/tree/webpack4-optimizations

Small TODO List I discovered by reviewing my changes:

• The react-sdk component-index needs ONLY to change if we build via webpack and ONLY if that is the case reference src instead of lib. To leave direct babel builds working

[turt2live]

We might also want to take a look at tree shaking ( https://webpack.js.org/guides/tree-shaking/ ) to cut even more off the app. According to chrome, about 30% of the bundle is unused in normal operation of Riot.

Chrome also thinks that 82% of our CSS is unused though, which I have a harder time believing.

[MadLittleMods]

@turt2live Is this issue still useful to the team to track aspirations?

[turt2live]

yes - webpack is still one of the slower parts of our devx

[t3chguy]

Closing in favour of https://github.com/vector-im/wat-internal/issues/67