Wasmer bindings

[syrusakbary]

I'm opening this issue to follow up on the PR that we created a few months ago adding support for Wasmer bindings: #173, given that the project has already matured a bit.

Right now the wit-bindgen repo is supporting different bindings over runtimes that are actually not maintained by the Bytecode Alliance, namely:

• JS API
• Spidermonkey
• Rust

In general, we believe that adding official support for more runtimes is not only desired for the community but also beneficial, as it standardizes all bindings into one repo with one name and place to go for support.

Given the comment from @tschneidereit on the PR, I understood that it might not be on the willingness of the project maintainers to add support for Wasmer bindings, but I want to be clear that this will probably imply a hard fork on the project, which I believe might not be on the best interests of the community.
I also would like to mention that Wasmer will fully cover the maintenance of the its bindings on this repo in case we agree that convergence (of one repo hosting all bindings) is the most optimal path, so no one else will need to assume the maintenance burden.

I'm opening this issue to see if we can converge on an agreement before doing any hard fork, so in the end, the Wasm users are the main beneficiaries.
Looking forward to hearing your thoughts!

Note: for anyone searching for Wasmer support on wit-bindgen in the future, here's the repo! https://github.com/wasmerio/wit-bindgen

[esoterra]

Is there a reason you aren't implementing Wasmer bindings in your own repo depending on anything you need from this one through cargo dependencies? I don't see why having your own 3rd-party bindings requires forking the entirety of wit-bindgen.

[syrusakbary]

There are a few reasons.

First one is discoverability: if people search for wit bindgen, it's very likely that they will land on this repo first. Which will make them more likely to use the runtimes supported in the "official repo", and harder for them to search for the one that is not on the main repo.
Even if Wasmer has the wit-bindgen repo, google deprioritizes forks in their search engine which makes it harder for that repo to appear in the results (try searching for "wasmer wit bindgen" and see how easy is for you to find the wasmer fork).

Second one is installation. Right now wit-bindgen is an empty crate, but it's likely that the default way of using it will be through cargo install wit-bindgen.
Once the crate is published, people will gravitate towards installing the base wit-bindgen, which will also make them gravitate mainly towards the runtimes supported there.

I frankly don't see any strong technical reason on why Wasmer bindings couldn't be on this repo. But here are some reasons on why I think it will be beneficial to have the Wasmer bindings integrated:

• Both Wasmer bindings are already fully tested (we have the tests already integrated in both Wasmer Rust and Wasmer Python)
• It doesn't add any more overhead for maintainers compared to other runtime integrations such as the JS or SpiderMonkey.
• it doesn't require people maintaining them actively since we (Wasmer) will actively collaborate on making them up to date (we are super happy to also jump on any PR that might be refactoring stuff and update wasmer bindings accordingly, so there's 0 overhead for current maintainers)
• It also incentivizes more entities to collaborate more with the repo rather than maintaining our ideas separately.
• It unifies community instead of split it (a hard fork will likely not help towards unification)

In general, collaboration is usually better than not collaboration. So that's the gist ;)

[tschneidereit]

There's still not been any material change to the key reasons I gave last time for not adding 3rd-party bindings to this repository.

The move to the full component model implementation is still very much in full swing, so lots of churn will necessarily still happen.

More importantly, nothing has changed about the more fundamental reasons I gave last time either. Here's the relevant part of my reply from last time:

To manage expectations for the future though, while we haven't decided on any kind of formal policy, I'm not sure we'll ever want to maintain an open-ended list of 3rd-party bindings in this repository. wit-bindgen is a project with specific stakeholders working on it with specific goals in mind, and we need to ensure that we stay focused on these. We'll base decisions around which bindings to host here strictly on whether we see it as beneficial to wit-bindgen as a project and its stakeholders, on a case-by-case basis.

The stakeholders of wit-bindgen see the bindings maintained in this repository as valuable and furthering the goals we're focused on, which is why they're included here.

As @Kylebrown9 points out, you should be able to use wit-bindgen as a dependency instead of forking it. I gave more detail on that last time, too:

Since we do appreciate that it's attractive to make use of the work done here without having to invest the considerable effort to do and maintain a full separate implementation, we structured wit-bindgen in a way that makes it easy to maintain bindings externally and make use of wit-bindgen through an API. While that API will necessarily undergo churn for a while, too, we aim to eventually have that settle down and follow semver versioning. I'd encourage you to investigate making use of this for your bindings.

If that for whatever reason doesn't work for you, then a fork might indeed be the right choice. That'd also have the added benefit of allowing you to catch up to latest developments at your own pace.

Alternatively, you could also consider doing your own completely independent implementation under a different name, which would have two significant benefits: one is that it'd allow you to fully tailor it to your needs. The other is that, as I mentioned to you in a WASI subgroup call a few months ago, having independent implementations of standardized concepts is tremendously useful in a standardization process, so by going this route you'd be able to actually support the process. It'd also solve the discoverability problem, as people intending to use Wasmer would naturally use the bindings generator you're promoting.

I hope that answers the questions you had, so I'll go ahead and close this issue.

[syrusakbary]

Alternatively, you could also consider doing your own completely independent implementation under a different name

I'm not sure if you understood what I meant with a hard fork, so let me clarify with an example.
A hard fork happened with node.js some time ago: the project was forked and distributed under a different name (io.js) because of internal disagreements on how things were moving forward (io.js was not a new implementation of node.js, was simply a fork with a different name, driven by different goals).

Hard forks are usually not beneficial for the community, that's why I'm trying to steer things towards collaboration before doing it.
But if a hard fork is the only option given, we'll go with it 😉 (there is no shame in doing it since everything is already open-source).

If you don't want to have Wasmer bindings in wit-bindgen, that's ok... we don't want to force anyone.
But that will have some implications, I just wanted to make sure you fully understood those, and we all try to put community first!

Feel free to re-open the issue if at some moment you decide that community interests should prevail :)

[willemneal]

How about providing a link to your project in the README? It is not much effort for this project to list third party implementations of the binding generators. Plus the current implementation of the wit-bindgen binary is also a bit underbaked. For example, it could use the same extension strategy as cargo, e.g. cargo-expand to add subcommands.

I still don't understand the cause for a fork. Your library is dependent on the wit parser and core generator. If you find issues with these, then a PR would add value to all current and future generators. Is it because you want people to find wasmer's implementation when searching for wit-bindgen? If so, then adding your generator to the readme would help increase your SEO.

[syrusakbary]

How about providing a link to your project in the README?

Unfortunately, and as I explained in previous comments on this issue, that will not work.

In any case, we just released and open-sourced our hard fork wai today:
https://github.com/wasmerio/wai

If the time comes when the wit-bindgen project maintainers decide that they would like to integrate support for Wasmer upstream, we would love to aim unifying projects again (assuming we can converge on a stable format, as stability is one of the main golas of WAI). Although, we understand and realize that this decision is not in our realm and we can only hope for others to put the Wasm community first.

Here's an article that we have created showcasing WebAssembly Interfaces ("WAI"), as it might be helpful for future readers:
https://wasmer.io/posts/wasmer-takes-webassembly-libraries-manistream-with-wai

[alexcrichton]

It looks like @syrusakbary you may be out of the loop a bit with this repository so I'll try to summarize the current state of things:

• As Till mentioned there's still lots of work to do in this repository and Reimplement host generators in terms of components #314 is an example of a large amount of churn now which, while done now, happened since you opened this issue.
• Work in this repository is pivoting towards standards-based collaboration in the upstream component model repository with the start of a description here for *.wit syntax/semantics.
• The work on *.wit has always intended to align with the component model. This isn't simply some random IDL we conjured up to work with a few guests and hosts, and this is codified in the wit-component library which allows generating components from *.wasm and extracting *.wit from components. Furthermore this is the basis for collaboration and features in *.wit.
• For example collaboration resource types were removed from this repository temporarily and is being designed upstream now.
• All host generators are being removed from this repository for example here and here. The JS generator will eventually move to its own repository as well, just takes time.

Overall if you're genuinely interested in collaborating on wit-bindgen I would encourage you to follow issues/discussions here. Chats happen on zulip and there are weekly in-person meetings on zoom to attend as well. The wheels aren't always turning at a lightning-fast pace over here so progress may seem slow sometimes but we're all always trying to march towards the shared vision of the component model.