Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wasm packager #9009

Merged
merged 5 commits into from
Oct 3, 2023
Merged

Wasm packager #9009

merged 5 commits into from
Oct 3, 2023

Conversation

mischnic
Copy link
Member

@mischnic mischnic commented May 14, 2023

This came up in #8964
So the usecase is a transformer that outputs a wasm asset with a sourcemap

  1. Wasm is a binary format but still has a sourceMappingURL section, which is however encoded using some [strLength, ...strBytes] constructs. So just replacing HASH_REF_fcf6ff1d86220c0f to 37ce1cf4 changes the bytes but not the length description and thus corrupts the binary. The workaround here is to hardcode that these hash references become 17 bytes shorter (9+16 chars to 8 chars) and set strLength to store the length after the hash replacement.
  2. The hash replacement itself (replaceStream) is text-based but runs on all bundles, and the current implementation breaks these bundles because a buffer->string->buffer round trip will modify non-utf8 bytes. replaceStream can be rewritten using buffers , but I haven't benchmarked yet whether this is slower for JS bundles. It appears to be equal or faster (DATA_* contain varying numbers of hash reference, from none to one every 10 bytes)
replaceStream       DATA_0, chunk=8192 x 13,247 ops/sec ±2.59% (139 runs sampled)
replaceStream       DATA_1, chunk=8192 x 14,338 ops/sec ±2.03% (142 runs sampled)
replaceStream       DATA_2, chunk=8192 x 17,329 ops/sec ±2.76% (138 runs sampled)
replaceStream       DATA_3, chunk=8192 x 14,752 ops/sec ±1.58% (143 runs sampled)
replaceStreamBuffer DATA_0, chunk=8192 x 19,405 ops/sec ±1.18% (158 runs sampled)
replaceStreamBuffer DATA_1, chunk=8192 x 18,978 ops/sec ±1.38% (156 runs sampled)
replaceStreamBuffer DATA_2, chunk=8192 x 18,650 ops/sec ±2.11% (154 runs sampled)
replaceStreamBuffer DATA_3, chunk=8192 x 19,190 ops/sec ±1.49% (155 runs sampled)
  • Tests

@mischnic mischnic force-pushed the wasm-packager branch 2 times, most recently from 8ef43b6 to 216a545 Compare May 18, 2023 08:51
Comment on lines +27 to +30
sourcemapReference.includes('HASH_REF_')
? // HASH_REF_\w{16} -> \w{8}
sourcemapReference.length - (9 + 16 - 8)
: undefined,
Copy link
Member Author

@mischnic mischnic May 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is quite hacky...

@mischnic mischnic marked this pull request as ready for review May 18, 2023 09:32
@parcel-benchmark
Copy link

parcel-benchmark commented Sep 18, 2023

Benchmark Results

Kitchen Sink ✅

Timings

Description Time Difference
Cold 1.45s -24.00ms
Cached 240.00ms -7.00ms

Cold Bundles

No bundle changes detected.

Cached Bundles

Bundle Size Difference Time Difference
dist/legacy/index.ff03421b.js 1.48kb +0.00b 408.00ms +43.00ms ⚠️
dist/legacy/index.e9bb1616.js 1.06kb +0.00b 408.00ms +44.00ms ⚠️
dist/modern/index.4a29d309.js 921.00b +0.00b 408.00ms +43.00ms ⚠️
dist/legacy/index.html 826.00b +0.00b 424.00ms +34.00ms ⚠️
dist/modern/index.html 749.00b +0.00b 423.00ms +33.00ms ⚠️

React HackerNews ✅

Timings

Description Time Difference
Cold 3.81s -62.00ms
Cached 409.00ms +8.00ms

Cold Bundles

Bundle Size Difference Time Difference
dist/PermalinkedComment.e9dc4a75.js 3.92kb +0.00b 367.00ms +41.00ms ⚠️
dist/UserProfile.8945a243.js 1.38kb +0.00b 367.00ms +41.00ms ⚠️
dist/NotFound.8b44a81d.js 269.00b +0.00b 367.00ms +41.00ms ⚠️

Cached Bundles

Bundle Size Difference Time Difference
dist/logo.8dd07848.png 244.00b +0.00b 272.00ms +40.00ms ⚠️

AtlasKit Editor ✅

Timings

Description Time Difference
Cold 32.65s -745.00ms
Cached 2.20s +7.00ms

Cold Bundles

Bundle Size Difference Time Difference
dist/media-viewer.38e3999a.js 536.13kb +0.00b 8.21s -2.75s 🚀
dist/ConfigPanelFieldsLoader.182d39bc.js 303.43kb +0.00b 8.21s +526.00ms ⚠️
dist/card.d06de810.js 138.91kb +0.00b 8.20s +527.00ms ⚠️
dist/ElementBrowser.e8f01080.js 61.94kb +0.00b 8.21s +528.00ms ⚠️
dist/archive.c374f622.js 59.90kb +0.00b 8.20s -2.76s 🚀
dist/esm.bfca2115.js 59.30kb +0.00b 8.22s +539.00ms ⚠️
dist/component-lazy.51d1dec9.js 58.94kb +0.00b 5.68s +635.00ms ⚠️
dist/ConfigPanelFieldsLoader.2b7c03be.js 15.74kb +0.00b 8.21s +528.00ms ⚠️
dist/ui.8c117104.js 14.48kb +0.00b 8.21s +528.00ms ⚠️
dist/ConfigPanelFieldsLoader.5dfde67d.js 13.63kb +0.00b 8.21s +528.00ms ⚠️
dist/pdfRenderer.01deafa1.js 12.04kb +0.00b 8.20s +527.00ms ⚠️
dist/mobile-upload.3baad8e4.js 7.79kb +0.00b 8.21s +528.00ms ⚠️
dist/mobile-upload.7a892a37.js 7.79kb +0.00b 8.21s +527.00ms ⚠️
dist/media-viewer-analytics-error-boundary.60bdaa4c.js 3.18kb +0.00b 8.21s -2.75s 🚀
dist/ru.0cf3f40e.js 2.81kb +0.00b 5.73s -1.95s 🚀
dist/uk.282f23b1.js 2.76kb +0.00b 8.21s +528.00ms ⚠️
dist/codeViewerRenderer.51140ec8.js 2.61kb +0.00b 8.20s +527.00ms ⚠️
dist/th.137e1013.js 2.60kb +0.00b 8.21s +528.00ms ⚠️
dist/vi.b46097db.js 2.09kb +0.00b 8.21s +527.00ms ⚠️
dist/tr.c85d90a9.js 2.03kb +0.00b 8.21s +528.00ms ⚠️
dist/sv.1c06c95c.js 1.98kb +0.00b 8.21s +528.00ms ⚠️
dist/zh_TW.b7c55aa6.js 1.86kb +0.00b 8.21s +527.00ms ⚠️
dist/zh.b01fe721.js 1.84kb +0.00b 8.21s +527.00ms ⚠️
dist/workerHasher.540c9790.js 1.56kb +0.00b 8.21s +528.00ms ⚠️
dist/workerHasher.c840c607.js 1.56kb +0.00b 8.21s +528.00ms ⚠️
dist/heading3.73972e33.js 1.35kb +0.00b 5.67s +621.00ms ⚠️
dist/heading2.c27d912d.js 1.17kb +0.00b 5.67s +621.00ms ⚠️
dist/heading4.be08fc9e.js 1.12kb +0.00b 5.67s +620.00ms ⚠️
dist/heading1.d7caae19.js 1.02kb +0.00b 5.67s +621.00ms ⚠️
dist/sk.4be9c93f.js 656.00b +0.00b 5.73s -1.95s 🚀
dist/simpleHasher.c14e20b4.js 589.00b +0.00b 8.21s +528.00ms ⚠️
dist/simpleHasher.23db7a52.js 589.00b +0.00b 8.21s +527.00ms ⚠️
dist/index.html 248.00b +0.00b 5.03s -6.01s 🚀

Cached Bundles

Bundle Size Difference Time Difference
dist/media-viewer-analytics-error-boundary.60bdaa4c.js 3.18kb +0.00b 7.94s -2.48s 🚀
dist/ru.0cf3f40e.js 2.81kb +0.00b 5.68s -2.18s 🚀
dist/heading5.d2f94d9d.js 1.23kb +0.00b 5.05s -642.00ms 🚀
dist/index.html 248.00b +0.00b 4.77s -268.00ms 🚀

Three.js ✅

Timings

Description Time Difference
Cold 2.82s +11.00ms
Cached 323.00ms +11.00ms

Cold Bundles

No bundle changes detected.

Cached Bundles

Bundle Size Difference Time Difference
dist/Three.js 572.25kb +0.00b 945.00ms +50.00ms ⚠️

Click here to view a detailed benchmark overview.

@devongovett devongovett merged commit 12f4809 into v2 Oct 3, 2023
15 of 16 checks passed
@devongovett devongovett deleted the wasm-packager branch October 3, 2023 03:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants