Javascript / WASM File Size in custom builds #164
Replies: 6 comments 10 replies
-
Looks like a cool project again! As you say, it seems to be very specific for your setup, so I'm guessing it is going to be hard to generalize this (if it is worth the savings in the first place). I'm guessing that, besides gzipping, there's not much else you can do to minify the package? |
Beta Was this translation helpful? Give feedback.
-
I would like to drop my 50 cents here. I cannot stress enough how important the package size is in web game dev. Every 50 kb is worth saving. Majority of gamers today are accessing the games from potato devices with some crappy network. It takes not-inconsiderable time to download the game. The players simply don't wait. Maybe it comes from the fear that if it takes long to load it will eat their monthly plan (many regions don't have unlimited plans and pay per Mb). Maybe we get used to the fact that websites are insta-loading and web games are expected as well. No matter the reasons, our stats show that at most you have is 3 seconds for the "potential" player to load the game. Every second after that you lose half of your players. By second 10 only your friends will actually see and try the game. Unfortunately, "its only for the first load" is a misconception - there is simply no second load, if it takes too long for the first one. This is the main reason we did multiple IDL variants for Ammo, stripping out features that are not needed for a type of game. When you say 300 kb of saving - its already an amazing number worth going for, not mentioning even potentially larger improvements. I will definitely try it, @PhoenixIllusion, even if a generalized solution will be hard to create. I am totally fine to have an unofficial and brittle one that requires maintenance with every new release, but the size savings are worth it. |
Beta Was this translation helpful? Give feedback.
-
I found that instead of trying to manually transform the generated wasm JS-file, compiling with I need to confirm it works for every feature we've been using with all potential flags, especially multi-threading, but it does perform the step of stripping exposure of all the Extra Stub to add before our glue function assert(x) {
if (!x) throw 'failed assert';
}
var runtimeInitialized = false;
var __ATINIT__ = [];
function addOnInit(cb) {
__ATINIT__.unshift(cb);
}
function callRuntimeCallbacks(callbacks) {
while (callbacks.length > 0) {
// Pass the module as the first argument.
callbacks.shift()(Module);
}
}
ready = function() {
updateMemoryViews();
const heapKey = ['HEAP8','HEAPU8','HEAP16','HEAPU16','HEAP32','HEAPU32','HEAPF32','HEAPF64'];
const heapVal = [HEAP8,HEAPU8,HEAP16,HEAPU16,HEAP32,HEAPU32,HEAPF32,HEAPF64];
heapKey.forEach((k,i) => {
Module[k] = heapVal[i];
});
Module['_webidl_malloc']=_webidl_malloc;
Module['_webidl_free']=_webidl_free;
callRuntimeCallbacks(__ATINIT__);
readyPromiseResolve(Module);
} This produces the Distribution wasm-JS uncompressed file size of about 550k vs the existing 1.1MB (or a wasm-compat of 3.2MB compared to 3.7 MB) Regarding the wasm binary itself, You would be required to do a special compile and run your application through all your test cases (including create/run/destroy of everything), and store a log-file the special build produces after each test case that flags what functions were called. These profile files are then all combined together during a fresh build to produce 2x WASM files.
Ideally you don't need to ever load module 2. https://emscripten.org/docs/optimizing/Module-Splitting.html |
Beta Was this translation helpful? Give feedback.
-
Is there some way to see which C++ symbols are taking up the most space? As it happens, I was replacing std::vector with my own Array class and I just integrated that version into JoltPhysics.js. This makes jolt-physics.wasm.wasm 137 KB smaller than it was before and jolt-physics.wasm-compat.js 548 KB smaller (the C++ STL library is known for its bulkiness). I can imagine some of the other STL classes I'm using also add a lot of weight. |
Beta Was this translation helpful? Give feedback.
-
That screenshot already suggests another possible saving: Defining |
Beta Was this translation helpful? Give feedback.
-
FYI I made another change that removes C++ RTTI and exception code (unused by Jolt). This reduces the build by another 1%. See: f858ba9 |
Beta Was this translation helpful? Give feedback.
-
This post is not about any request to change any part of the library, just discussing custom builds.
Posting TLDR Conclusion:
With all of the additions to the library that have been done, I looked into file size change of the project, with regard to both the JS file (base + glue) and stand-alone WASM file.
Due to server-side support for file compression via GZ, and browser built-in compression support of gz compression-streams, compressed sizes are used for reference on bandwidth.
Currently, with v0.23.0, these both stand at:
Compared with version version v0.10.0
As a web-developer, this isn't much growth and not unreasonable compared to most other libraries, or the size of a couple detailed background JPGs. The biggest JS growth appears to purely be method-names, either of classes and functions or repeated usage of the raw emscripten export names. With ServiceWorkers, this is also just a one-time-only download of files.
Since I was going to make custom builds for my projects anyway, for multi-threading at least and possibly bake in additional event-listeners, I looked into if I could minify my builds and see how much smaller it would be.
I looked into the vanilla TypeScript package for its language features, before turning to a transformation and analysis library
ts-morph
. I loaded up my library directory in the script, and located thetypings.ts
file for jolt-physics. I then traversed the Jolt module for every class, method, property, return-types, parameter-types, inherited types, and variable. I used project-wide-reference-lookup on each item, flagging every item as 'required' if it existed outside of the typings.ts and logging to a JSON file.This did break in cases where Key-Value notation was used on a class, or classes were assigned to an interface, then methods invoked on that interface, ex:
I then loaded up the existing JoltJS.idl file and processed it top-top-bottom, parsing enum's and interfaces and seeing if they were 'required', breaking apart enums and interfaces to only their required fields and outputting to a new IDL file. I ran this through the IDL type-gen code, and tested compiles against it to confirm that all my code still compiled and type-checks were flagging any missing methods or properties.
Once built and validated against library's examples, my totals were:
Conclusion: Quite a bit of overhead and risk in generating new builds, with savings under gzip of ~300k, so I'm not sure its worth it to me yet.
I may look into modifying the existing web_idl_binder.py generated JS file only so it can be tree-shaken, and possibly cut that down by during bundle-time.
Files used for my minify, having several edge cases and the need to add an extra "override" file to add types not originally included:
https://gist.github.com/PhoenixIllusion/68f3796be91fdf26aa667290f6782692
Beta Was this translation helpful? Give feedback.
All reactions