-
Notifications
You must be signed in to change notification settings - Fork 191
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Spin out host generators into their own repositories #395
Comments
bytecodealliance/wasm-tools#819 contains the first step of this, splitting out the |
alexcrichton
added a commit
to alexcrichton/witx-bindgen
that referenced
this issue
Nov 16, 2022
These crates have been migrated to the `wasm-tools` repository as part of bytecodealliance/wasm-tools#819 to handle the first step of bytecodealliance#395
alexcrichton
added a commit
that referenced
this issue
Nov 17, 2022
* Remove wit-{component,parser} from this repository These crates have been migrated to the `wasm-tools` repository as part of bytecodealliance/wasm-tools#819 to handle the first step of #395 * Use `wasm-tools` for demo instead of `wit-component`
alexcrichton
added a commit
to alexcrichton/witx-bindgen
that referenced
this issue
Nov 18, 2022
This commit removes the host Python generation from this project. The support here has all moved upstream to https://github.com/bytecodealliance/wasmtime-py and is now supported through: python -m wasmtime.bindgen my-component.wasm --out-dir ./my-component Some documentation is in the upstream `README.md` about how to use. Necessary tests have been mirrored to the upstream repository to the best of my ability, and this is intended to represent an inflection point where the growth of the Python generator is no longer tied up with all the other languages in this repository. This is a step further along the path of bytecodealliance#395.
bytecodealliance/wasmtime-py#104 and #436 are the extraction of the Python host generator. |
alexcrichton
added a commit
that referenced
this issue
Nov 18, 2022
This commit removes the host Python generation from this project. The support here has all moved upstream to https://github.com/bytecodealliance/wasmtime-py and is now supported through: python -m wasmtime.bindgen my-component.wasm --out-dir ./my-component Some documentation is in the upstream `README.md` about how to use. Necessary tests have been mirrored to the upstream repository to the best of my ability, and this is intended to represent an inflection point where the growth of the Python generator is no longer tied up with all the other languages in this repository. This is a step further along the path of #395.
alexcrichton
added a commit
to alexcrichton/witx-bindgen
that referenced
this issue
Nov 22, 2022
This bindings generator has been moved to the `wasmtime` crate upstream as part of bytecodealliance/wasmtime#5317. This is another step along the road of bytecodealliance#395. The Wasmtime-hosted tests are all removed here as well but may get re-added in the future for testing as necessary.
#439 is the removal of Wasmtime, moved upstream as part of bytecodealliance/wasmtime#5317 |
alexcrichton
added a commit
to alexcrichton/witx-bindgen
that referenced
this issue
Nov 29, 2022
This bindings generator has been moved to the `wasmtime` crate upstream as part of bytecodealliance/wasmtime#5317. This is another step along the road of bytecodealliance#395. The Wasmtime-hosted tests are all removed here as well but may get re-added in the future for testing as necessary.
Closed
alexcrichton
added a commit
that referenced
this issue
Dec 6, 2022
This bindings generator has been moved to the `wasmtime` crate upstream as part of bytecodealliance/wasmtime#5317. This is another step along the road of #395. The Wasmtime-hosted tests are all removed here as well but may get re-added in the future for testing as necessary.
alexcrichton
added a commit
to alexcrichton/witx-bindgen
that referenced
this issue
Jan 25, 2023
This commit removes the JS host generator from this repository as it now lives at bytecodealliance/js-component-tools. The runtime tests will be updated in a subsequent commit to use Wasmtime as a host for executing components. This additionally removes the demo as it's no longer possible to build it in just this repository. The demo has slowly been getting less useful over time as generators are removed from this repository. Additionally it's always been a demo and not much more so it's more-or-less served its purpose and doesn't seem to have a great place to fit any more. This can possibly exist out-of-tree eventually in a separate location that pulls together all known generators, but until that time it seems best to just remove it for now. Closes bytecodealliance#395
rvolosatovs
pushed a commit
to bytecodealliance/wrpc
that referenced
this issue
May 23, 2024
* Remove wit-{component,parser} from this repository These crates have been migrated to the `wasm-tools` repository as part of bytecodealliance/wasm-tools#819 to handle the first step of bytecodealliance/wit-bindgen#395 * Use `wasm-tools` for demo instead of `wit-component`
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This was discussed at the wit-bindgen meeting last Friday, and after some futher discussion with @pchickey I've come up with a proposal which I personally feel is the direction that wit-bindgen should go in. I wanted to open an issue though to get feedback from others, however.
To recap, the core issue at hand here is that the Wasmtime repository would like to vendor an implementation of
wasi-preview2
. This would necessitate Wasmtime depending onwit-bindgen
for host bindings generation, however, butwit-bindgen
depends back on the Wasmtime repository. This circular dependency between git repositories cannot be resolved and isn't a maintainable way to implement this code.After much delibration and possible ways to go about this, I would like to propose that all host generators be "split out" of the
wit-bindgen
repository into their own repositories, in time possibly doing the same thing for guests. Concretely what I'm envisioning is:wit-component
andwit-parser
into thewasm-tools
repository. This sort of low-level tooling is the "base" of the dependency tree and would live next to all our other low level tools. We could create commands such aswasm-tools wit-validate
or similar to validate the structure of*.wit
files or generate the markdown documentation that's there today. No host/guest generator would live in thewasm-tools
repository. Tests here would solely be about name resolution and syntactical correctness of*.wit
files plus the*.wat
tests thatwit-component
has today.wasm-tools
and would be tested with raw*.wat
inputs, unlike today where it's tested with compiled guest inputs.wasmtime-py
repository (exact location and naming TBD).wit-bindgen
as-is today could be compiled to a*.wasm
and published to NPM from this location.wit-bindgen
repository, for now. While these may eventually move to their own repository it seems like it may make the most sense to leave them in one location now for discoverability and in general there's no pressing need for splitting them out. Guest generators would be tested with a host generator.With this organization it solves the original problem of implementing and testing
wasi-preview2
in the Wasmtime repository by having the Wasmtime generator live there and being able to depend on the Rust generator from this repository. Otherwise the benefits of this organization I believe are:wit-parser
andwit-component
so I don't think there will be too much meaningful duplication.Despite all this I don't think that this is universally a net win. There are still a number of downsides to this new organization, chiefly among them the loss of discoverability. There will no longer be a single place to go and interact with
wit-bindgen
and supported languages. This can be somewhat recovered with a page of documentation but it will likely not be quite as good as the story of today. In reality, though, wit-bindgen is not currently documented for its language support and having such documentation anyway would be a relatively large undertaking to keep maintained over time.Another major issue though with splitting repositories is versioning. It's not possible to atomically update all wit-bindgen-related repositories meaning that there will be a number of
wit-parser
versions in the wild and possibly incompatible host/guest generators. The hopeful solution for this, though, is strictly defined interchanges between guest/host generators. By using the official component binary format as an interchange along with*.wit
which is also defined in thecomponent-model
repository itself it's hoped that most of the churn will be of a non-breaking sort where the Rust APIs towit-parser
may change but the underlying text format will not change as much. Nevertheless it's expected that this will be more of an issue than it would otherwise be with a monorepo-style approach.Finally we don't actually have dozens of folks clamoring to maintain everything in
wit-bindgen
at this time. This means that splitting out some repositories runs the risk of actually decreasing maintenance in the short-to-medium term due to the same people being spread more thinly across repositories. I don't really know of a great answer for this other than "hope it gets better over time".Personally I feel that this is the "least bad" option at this time. I think it also has a number of benefits over the current organization, though, slightly outweighing the downsides I can think of. Despite all that though I'd like to confirm with others that this is a reasonable approach to take and see if others have ideas as well.
If this is the route decided on (splitting host generators out) then I don't think that anything will be split out before worlds are implemented in the text format, so it will still be some time before this is all implemented.
The text was updated successfully, but these errors were encountered: