[technical exploration] js-ipfs bundle optimized #398
Comments
|
I agree we should investigate how we can minimize the size of js-ipfs, but I strongly disagree to do it at the cost of developer productivity.
|
|
Using https://webpack.github.io/analyse/ and http://ipfs.io/ipfs/QmSoAbNz1QWwioCzGHtV97Mtx23u19r3oVmznhk2bMLxSp you can see and analyze the largest offenders in terms of size. The one that are visible immediately are
Just for comparison, the core shim is 7 KiB large. |
|
Some more work allowed me to bring down the size to Also analysis of modules size of the new bundle is below |
daviddias
changed the title
|
Some interesting data here: https://nolanlawson.com/2016/08/15/the-cost-of-small-modules/ |
|
There is also: https://chrisbateman.github.io/webpack-visualizer/ which helps analyse dependencies from webpack a bit better. In addition there is this article about using nsolid to do runtime analysis of dependencies https://nodesource.com/blog/is-guy-fieri-in-your-node-js-packages/ |
|
Using the webpack visualizer I found out that we are currently bringing in 12 versions of readable-stream cough which should be fixed after #403 |
|
@dignifiedquire I would love if there was a section explaining how you hook those tools to help you visualise, and if possible that they can be run with just a script :) |
|
@diasdavid this can not be put into a script, but
|
|
@diasdavid can we change the name of the topic to "optimize the size of js-ipfs"? |
|
Some feedback from elsewhere (cc @pelle). @diasdavid @dignifiedquire make sure to note the comment re. dependencies.
|
|
Perhaps this is not the right thread but since js-ipfs-api and js-ipfs are connected and we're on the topic of size optimization: ConsenSys is using https://github.com/pelle/browser-ipfs for their IPFS wrapper. Their reason is simply the size: 900kb (js-ipfs-api) vs. 2kb (browser-ipfs). @diasdavid @dignifiedquire this begs the question: why is js-ipfs-api so large? What are the dependencies that make it so large? If js-ipfs-api can't use less dependencies, should we talk about a light api lib, much like browser-ipfs? |
|
@haadcode Thank you for bringing that up. It should go, however, into a This makes me thing that it would be pretty dope if |
|
Moving conversation re. js-ipfs-api to ipfs-inactive/js-ipfs-http-client#353 |
|
A lot of good things are in this article: https://pouchdb.com/2016/01/13/pouchdb-5.2.0-a-better-build-system-with-rollup.html. PouchDB has struggled with similar issues in terms of optimisation in size but also ensuring things are usable in node, webpack & browserify. |
|
More interesting things about using a monorepo and many small packages from PouchDB: https://pouchdb.com/2016/06/06/introducing-pouchdb-custom-builds.html |
|
JS IPFS is a large collection of modules that aim to implement IPFS in Node.js and the browser. As such the distributions of these modules has a specific set of constraints. Our current setup is not bad, and does generate bundles usable in Node.js and the browser, but there are some pain points that need work. Current Pain Points
Optimization Goals
Module FormatsThere are two different module formats for JavaScript modules in main use today.
The current code base uses CommonJS. Available ToolingThe tooling landscape is quite large today, with things developing and changing quite rapidly. The for us currently relevant tooling is listed below. Module BundlersA module bundler can take in many JavaScript files and generate a bundle, which is usable in the browser.
ES2015 TranspilersTranspilers can transform code written with ES2015 features and output code that is usable in ES5 (and lower) environments. A good comparision of the differences in size and runtime can be found in The cost of transpiling ES2015 in 2016. ProposalGiven the set of constraints mentioned above, the following is a list of steps I suggest to improve and solve our current pain points. 1. Improve build artifactsSimilar to what PouchDB does, the end result for Node.js and the browser should be a single file. If there are differences between Node.js and browser, modules use two different entry points
For the builds we target the same places as currently
but To make tooling aware of what is avaliable, the following should fields should be in "main": "./lib/index.js",
"jsnext:main": "./src/index.js",
"browser": {
"./lib/index.js": "./dist/index.js"
},
"jspm": {
"main": "dist/index.js"
}Benefits
Drawbacks
2. Test webpack & browserify in CI
Benefits
Drawbacks
3. Move to ES2015 Modules
Benefits
Drawbacks
4. Carefully audit the dependency tree
Benefits
Drawbacks
ResourcesBlog Posts
Issues on IPFS |
|
Very good suggestions. It would also be useful if we could create a lighter weight configuration of the library ideally just using the packager like browserify or webpack implementing only common use cases for the browser. Another packager to add to the list is React Native's https://github.com/facebook/react-native/tree/master/packager It presents a bunch of new headaches in that it doesn't go in and try to replace any node infrastructure. This causes many problems using libraries that assume something like browserify will automatically add React Native may seem like a corner case, but I know many developers are starting to use it now so it will be increasingly common. |
|
Thanks @pelle for bringing up react-native, the browser bundle in the above suggestion should work in react-native but it's important to test and check. |
|
Pretty alarmed by this proposal. This type of optimization (removing code) should not be so complex. And I'm not convinced it has to be.
|
|
@dignifiedquire should move this proposal into its own threat to capture discussion there. It's going to get big |
|
I did some more investigation, and it looks like rollup has quite the large savings for us. I generated a bundle which transforms commonjs to es6 modules and then uses rollbar and got the unminified size down from |
daviddias
changed the title
|
Let's continue this discussion on #429 |
As of today, the js-ipfs dist version is considerably large (~4MB), which is a lot of JavaScript for a module designed for the browser. It is true that we have to inject a lot of code to support things like spdy, multiple transports, crypto channels and so on, but there are things that can be avoided/optimized, namely: shims everywhere for es6->es5 transpilation.
The 'promise' is that in the future, webpack's 'tree shaking' will remove duplicate code paths by reusing dependencies, but as far as I can tell, this won't work for CommonJS modules (anytime soon or ever).
Also, supporting several API's natively, namely callbacks and promises, forces every user to require everything, even if they decide to use just one, paying the cost associated.
My intuition is that the benchmarking tests will also reveal that we can optimize js-ipfs by reducing the overhead introduced by shims and that straight ES5 code will be more optimized than ES6, we will see.
So, here are some things that we might want to reconsider meanwhile:
I'm sure there will be other things we can do too.
The text was updated successfully, but these errors were encountered: