Change the encoding of wit definitions #1252
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
crates/wit-component/tests/interfaces/upstream-deps-same-name/foo.wit.print
Outdated
Show resolved
Hide resolved
|
@alexcrichton I've switched the fuzzer over to using the V2 format, and also modified it to never produce an empty package. |
alexcrichton
approved these changes
Oct 18, 2023
alexcrichton
added a commit
to alexcrichton/wasmtime
that referenced
this pull request
Oct 19, 2023
This is primarily to pull in bytecodealliance/wasm-tools#1252 to get decoding of the new format into a Wasmtime release ASAP.
alexcrichton
added a commit
to bytecodealliance/wasmtime
that referenced
this pull request
Oct 19, 2023
This is primarily to pull in bytecodealliance/wasm-tools#1252 to get decoding of the new format into a Wasmtime release ASAP.
alexcrichton
added a commit
to alexcrichton/wasm-tools
that referenced
this pull request
Oct 30, 2023
Currently there's a few encoding changes in-flight in wasm-tools. Two relevant ones here are: * How WIT packages are encoded (`WIT_COMPONENT_ENCODING_V2`, bytecodealliance#1252) * How type information is embedded in core wasm files (`WIT_COMPONENT_NEW_ENCODE`, bytecodealliance#1260) Currently these don't end up playing well together. If the new WIT package encoding is used then it breaks when combined with the new metadata encoding. This commit seeks to rectify this situation. Currently this wasn't previously tested on CI due to this particular combination of flags not being on in the tests. The fixes here largely amount to refactoring to delete some duplicated code. Previously the v1/v2 split introduced an `encode_world` that was defined in both modules, but the output was different for both modules and only one would work when fed to decoding. This commit removes this split and leaves only one canonical `encode_world` function since it should be the same for both as well. This then additionally updates `WIT_COMPONENT_NEW_ENCODE=1` to preserve a lookalike format of a WIT package. Furthermore `decode_world` is updated to expect this structure as well. The intention is that the encoded metadata is as-if there was a single world in a WIT package. These refactorings fix the prior buggy behavior between the two new encodings. Note that this was only exposed through `WIT_COMPONENT_NEW_ENCODE=1` which was only very recently added. Previous builds using only `WIT_COMPONENT_ENCODING_V2=1` should continue to work fine and produce the same output as before.
alexcrichton
added a commit
that referenced
this pull request
Oct 31, 2023
Currently there's a few encoding changes in-flight in wasm-tools. Two relevant ones here are: * How WIT packages are encoded (`WIT_COMPONENT_ENCODING_V2`, #1252) * How type information is embedded in core wasm files (`WIT_COMPONENT_NEW_ENCODE`, #1260) Currently these don't end up playing well together. If the new WIT package encoding is used then it breaks when combined with the new metadata encoding. This commit seeks to rectify this situation. Currently this wasn't previously tested on CI due to this particular combination of flags not being on in the tests. The fixes here largely amount to refactoring to delete some duplicated code. Previously the v1/v2 split introduced an `encode_world` that was defined in both modules, but the output was different for both modules and only one would work when fed to decoding. This commit removes this split and leaves only one canonical `encode_world` function since it should be the same for both as well. This then additionally updates `WIT_COMPONENT_NEW_ENCODE=1` to preserve a lookalike format of a WIT package. Furthermore `decode_world` is updated to expect this structure as well. The intention is that the encoded metadata is as-if there was a single world in a WIT package. These refactorings fix the prior buggy behavior between the two new encodings. Note that this was only exposed through `WIT_COMPONENT_NEW_ENCODE=1` which was only very recently added. Previous builds using only `WIT_COMPONENT_ENCODING_V2=1` should continue to work fine and produce the same output as before.
alexcrichton
added a commit
to alexcrichton/wasm-tools
that referenced
this pull request
Nov 1, 2023
This commit enables the new encoding for WIT packages in WebAssembly components described in WebAssembly/component-model#248 and originally implemented in bytecodealliance#1252. Support for the new encoding has been in a wasm-tools for a bit and it's also released with Wasmtime 14. This switch means that infrastructure will start being exposed to it by default now. Support for the old encoding remains to assist with interop as well. In a release or two support for creating the old encoding will be removed.
pchickey
pushed a commit
that referenced
this pull request
Nov 1, 2023
This commit enables the new encoding for WIT packages in WebAssembly components described in WebAssembly/component-model#248 and originally implemented in #1252. Support for the new encoding has been in a wasm-tools for a bit and it's also released with Wasmtime 14. This switch means that infrastructure will start being exposed to it by default now. Support for the old encoding remains to assist with interop as well. In a release or two support for creating the old encoding will be removed.
alexcrichton
added a commit
to alexcrichton/wasm-tools
that referenced
this pull request
Sep 11, 2024
This commit removes support for the "v1" encoding of WIT packages into WebAssembly. This was originally migrated to v2 in bytecodealliance#1252 and v2 was enabled by default in bytecodealliance#1274. This finishes the transition by removing support for creating the old encoding.
github-merge-queue bot
pushed a commit
that referenced
this pull request
Sep 12, 2024
* Remove v1 encoding support for WIT documents/worlds This commit removes support for the "v1" encoding of WIT packages into WebAssembly. This was originally migrated to v2 in #1252 and v2 was enabled by default in #1274. This finishes the transition by removing support for creating the old encoding. * Fix fuzzer build * Update outdated comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Address WebAssembly/component-model#248 by implementing support for the new encoding proposal. Additionally, this PR maintains backwards compatibility with decoding the previous encoding, by detecting when the newer encoding was used.
The rollout of the v2 encoding scheme will follow roughly the same plan as for semicolons in wit: there's a new environment variable introduced that controls the encoding scheme,
WIT_COMPONENT_ENCODING_V2that can be used to control when we encode in the new format, with the default behavior being what's done on main. After tooling has caught up to expecting the new format we can switch the default behavior to be the new format, and then completely remove support for v1 after we're confident nothing depends on it anymore.