Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add wasm32-wasi-musl target #165

Merged
merged 2 commits into from
Feb 22, 2024
Merged

Add wasm32-wasi-musl target #165

merged 2 commits into from
Feb 22, 2024

Conversation

rockwotj
Copy link
Contributor

@rockwotj rockwotj commented Feb 19, 2024

I would like to be able to compile wasm/wasi with zig + bazel. This adds
a toolchain to be able to do so. Wasm/wasi is a pretty rapidly changing
space, but I've choosen the most specific naming that I could (WASI
preview 1).

I've confirmed this works by running the compiled binary with wasmtime,
a popular wasm/wasi runtime.

bazel run --run_under=wasmtime //test/c:which_libc_wasip1_wasm

@CLAassistant
Copy link

CLAassistant commented Feb 19, 2024

CLA assistant check
All committers have signed the CLA.

@motiejus
Copy link
Collaborator

Since you are changing the final archive, you will need to add this here:

"v2.2.1": "3b8107de0d017fe32e6434086a9568f97c60a111b49dc34fc7001e139c30fdea",

"v3.0.0": "a6dd921a52e40efb88d816d7d0bc79fd2501a2a1de219f69fbfd546fe4044fd1"

@rockwotj
Copy link
Contributor Author

"v3.0.0": "a6dd921a52e40efb88d816d7d0bc79fd2501a2a1de219f69fbfd546fe4044fd1"

Ah thanks! I see what's happening now. That was non obvious 😄

Copy link
Collaborator

@motiejus motiejus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is good, but we need a way to run and verify that the binary runs at least on one platform.

I see the following options:

  1. Use a system package and run with --run_under. MacOS and Windows seems to be available, or curl <...> | bash on Ubuntu.
  2. Get a "lightweight" runtime and use it. The most lightweight I found and was able to quickly run was from zig itself:
code/zig/stage1$ zig cc -O3 -target x86_64-linux-musl wasm2c.c -o wasm2c
code/zig/stage1$ ./wasm2c ~/code/hermetic_cc_toolchain/bazel-bin/test/c/which_libc-__platform_wasip1_wasm hello.c
code/zig/stage1$ zig cc -O3 -target x86_64-linux-musl hello.c wasi.c -o hello
code/zig/stage1$ ./hello foo
wasi non-glibc

This one should work on all platforms. You are welcome to explore any options.

toolchain/private/defs.bzl Outdated Show resolved Hide resolved
toolchain/private/defs.bzl Outdated Show resolved Hide resolved
toolchain/private/defs.bzl Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
test/c/BUILD Outdated Show resolved Hide resolved
tools/releaser/main.go Outdated Show resolved Hide resolved
tools/releaser/main.go Outdated Show resolved Hide resolved
tools/releaser/main.go Outdated Show resolved Hide resolved
@rockwotj
Copy link
Contributor Author

This is good, but we need a way to run and verify that the binary runs at least on one platform.

I have used a go based wasm runtime to verify this within Bazel. PTAL

@rockwotj
Copy link
Contributor Author

I see snyk is unhappy. Please advise 😄

@motiejus
Copy link
Collaborator

wazero is a great find!

This is coming together pretty nicely. I am replacing test/c/test.sh with a Go version, which will:

  1. replace the dependency on system shell.
  2. be a good place to add wazero as an optional executor. That way, wasm tests can be executed on any platform.

My WIP branch is in https://github.com/motiejus/hermetic_cc_toolchain/tree/test_go , feel free to keep an eye for a PR here too.

ghost pushed a commit to motiejus/hermetic_cc_toolchain that referenced this pull request Feb 21, 2024
- run all tests on all platforms.
- enable more CI checks on more platforms. Turns out I just missed quite
  a few.
- replace `test.sh` with `test.go`, which will be useful in uber#165
ghost pushed a commit to motiejus/hermetic_cc_toolchain that referenced this pull request Feb 21, 2024
- run all tests on all platforms.
- enable more CI checks on more platforms. Turns out I just missed quite
  a few.
- replace `test.sh` with `test.go`, which will be useful in uber#165
ghost pushed a commit to motiejus/hermetic_cc_toolchain that referenced this pull request Feb 21, 2024
- run all tests on all platforms.
- enable more CI checks on more platforms. Turns out I just missed quite
  a few.
- replace `test.sh` with `test.go`, which will be useful in uber#165
ghost pushed a commit to motiejus/hermetic_cc_toolchain that referenced this pull request Feb 21, 2024
- run all tests on all platforms.
- enable more CI checks on more platforms. Turns out I just missed quite
  a few.
- replace `test.sh` with `test.go`, which will be useful in uber#165.
ghost pushed a commit to motiejus/hermetic_cc_toolchain that referenced this pull request Feb 21, 2024
- run all tests on all platforms.
- enable more CI checks on more platforms. Turns out I just missed quite
  a few.
- replace `test.sh` with `test.go`, which will be useful in uber#165.
motiejus pushed a commit that referenced this pull request Feb 21, 2024
- run all tests on all platforms.
- enable more CI checks on more platforms. Turns out I just missed quite
  a few.
- replace `test.sh` with `test.go`, which will be useful in #165.
@motiejus
Copy link
Collaborator

I landed #166, which opens up a path for you to add a WASI runner to https://github.com/uber/hermetic_cc_toolchain/blob/a5dcb8c486ecbbbbf82f70f189a35fe85a0a9810/test/c/c_test.go .

Can you rebase and move your executor there? That way we will be able to test WASI on all platforms (via c_runner.go) and avoid adding tests/wasi.

@rockwotj
Copy link
Contributor Author

@motiejus updated - thanks!

test/c/c_test.go Outdated Show resolved Hide resolved
I would like to be able to compile wasm/wasi with zig + bazel. This adds
a toolchain to be able to do so. Wasm/wasi is a pretty rapidly changing
space, but I've choosen the most specific naming that I could (WASI
preview 1).

I've confirmed this works by running the compiled binary with wazero,
a popular golang based wasm/wasi runtime in the tests.

Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
@rockwotj
Copy link
Contributor Author

Consider updating

Good callout. Done thanks!

@motiejus motiejus merged commit 2ce914b into uber:main Feb 22, 2024
8 checks passed
@motiejus
Copy link
Collaborator

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants