Enable wasm LLVM backend #42571

tlively · 2017-06-09T21:11:35Z

Enables compilation to WebAssembly with the LLVM backend using the target triple "wasm32-unknown-unknown". This is the beginning of my work on #38804.

edit: The new new target is now wasm32-experimental-emscripten instead of wasm32-unknown-unknown.

rust-highfive · 2017-06-09T21:11:48Z

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @nikomatsakis (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

nikomatsakis · 2017-06-09T21:26:13Z

r? @brson

eddyb · 2017-06-09T22:43:05Z

You have a merge commit: you should always pass --rebase to git pull.

brson · 2017-06-09T23:38:29Z

Thanks @tlively! I'm amazed you've made so much progress. I'm not going to look at this tonight, but next monday.

chicoxyzzy · 2017-06-11T01:30:33Z

src/librustc_back/target/wasm32_unknown_unknown.rs

+        dynamic_linking: false,
+        executables: true,
+        // Today emcc emits two files - a .js file to bootstrap and
+        // possibly interpret the wasm, and a .wasm file


Perhaps it'll be better to use WebAssembly Standalone mode

I agree that we should not always produce JavaScript, and I would like to be able to get there eventually. However, we probably want to produce the JavaScript file at least when compiling executables, since wasm is not executable without a JavaScript wrapper to launch it. When compiling libraries, emitting only wasm makes much more sense, but we have to make sure we can do that in a way that will allow them to be linked and used naturally, and in a way that does not strengthen our dependency on emscripten's LLVM modifications. I plan to do this work in a future PR.

I'm not arguing against producing JavaScript along with wasm but you always can wrap you wasm by hands and have customization that produced JS doesn't provide.

brson · 2017-06-12T22:56:24Z

src/bootstrap/native.rs

@@ -94,6 +94,7 @@ pub fn llvm(build: &Build, target: &str) {
       .profile(profile)
       .define("LLVM_ENABLE_ASSERTIONS", assertions)
       .define("LLVM_TARGETS_TO_BUILD", llvm_targets)
+       .define("LLVM_EXPERIMENTAL_TARGETS_TO_BUILD", "WebAssembly")


See how LLVM_TARGETS_TO_BUILD can be configured by config.llvm_targets? I think we better do similar by adding config.llvm_experimental_targets - it doesn't seem right to allow all the targets to be configured out except the experimental one. It should be reasonably obvious how to make that change, but it will also require updating the config file example.

brson · 2017-06-12T22:58:38Z

src/librustc_back/target/mod.rs

@@ -214,6 +214,7 @@ supported_targets! {
    ("le32-unknown-nacl", le32_unknown_nacl),
    ("asmjs-unknown-emscripten", asmjs_unknown_emscripten),
    ("wasm32-unknown-emscripten", wasm32_unknown_emscripten),
+    ("wasm32-unknown-unknown", wasm32_unknown_unknown),


Using "unknown" for the OS here is a bit odd, since this platform does link with emcc and runs on the emscripten runtime. I think, as we discussed previously, this target is destined to replace the existing 'wasm32-unknown-emscripten' target. Do you see this name as temporary?

wasm32-unknown-unknown is the triple that LLVM uses for its WebAssembly backend, and it will eventually be possible to link wasm with lld. If the Emscripten runtime were to be replaced with a smaller Rust-specific runtime, we could remove all dependencies on Emscripten for WebAssembly, which I think would be ideal. Given that, I wasn't thinking of wasm32-unknown-unknown as temporary, although another possible name for an Emscripten-free target would be wasm32-unknown-wasm.

On the other hand, this could be a temporary name, and if we decide not to replace the Emscripten linker and runtime then it would make sense to rename this target to replace the old one once it is ready. Another option would be to eventually have different targets for each of the possible wasm paths: fastcomp-emscripten (the current path), llvm-emscripten (this PR), and llvm-lld.

In the meantime, it might make sense for me to rename the target in this PR wasm32-unknown-experimental, since it really is not yet a usable target.

cc @eholk

To add, WASM supports non-web target, and as a developer of an alternate wasm backend myself, the emscripten-free target is ideal.

In any case I think this name is fine for now.

In the Rust system I would expect an emscripten-wasm target and a non-emscripten-wasm target to have two different target triples.

brson · 2017-06-12T23:12:37Z

src/tools/build-manifest/src/main.rs

@@ -81,6 +81,7 @@ static TARGETS: &'static [&'static str] = &[
    "s390x-unknown-linux-gnu",
    "sparc64-unknown-linux-gnu",
    "wasm32-unknown-emscripten",
+    "wasm32-unknown-unknown",


This line is not necessary.

This list is defining the targets we distribute as part of Rust releases, and this target will not be built and distributed at this time.

brson · 2017-06-12T23:18:18Z

@tlively This looks like a fine start to me. Thanks a bunch. I left some comments to be addressed.

Can you say a bit about what your next steps are? I believe you are going to be experimenting with this backend out of tree, probably with an upgraded LLVM, and slowly upstreaming Rust changes that can be upstreamed until you are ready to do the big LLVM upgrade.

Since this backend is not useful generally right now, it could be worth making the "llvm_experimental_targets" empty by default and requiring it to be configured explicitly. We are configuring in by default things like PTX and hexagon, surely nobody is doing anything productive with PTX today.

Yeah, I think I am inclined to make the wasm backend off by default and require devs (you :)) to configure it in explicitly. Will you make that change?

@alexcrichton can you peek at this and see what you think?

brson · 2017-06-12T23:19:29Z

I guess I shouldn't say it's not generally useful. I just assume that. Does this target produce working wasm using the in-tree LLVM today?

brson · 2017-06-12T23:22:10Z

Please rebase and squash these commits.

tlively · 2017-06-13T01:28:02Z

I guess I shouldn't say it's not generally useful. I just assume that. Does this target produce working wasm using the in-tree LLVM today?

It does, with some big caveats. It produces usable #[no_core] binaries, and with some effort produces usable #[no_std] binaries. With more effort it can produce binaries linked with std, but due to a version mismatch somewhere in my toolchain these are broken. I would definitely describe this as "not generally useful."

it could be worth making the "llvm_experimental_targets" empty by default and requiring it to be configured explicitly.

This sounds like a good idea. I will submit a separate PR that makes this change for existing experimental targets, and once/if that lands I will update this PR as well.

Can you say a bit about what your next steps are? I believe you are going to be experimenting with this backend out of tree, probably with an upgraded LLVM, and slowly upstreaming Rust changes that can be upstreamed until you are ready to do the big LLVM upgrade.

That sounds about right, except that I'm going to try using the in-tree LLVM backend as long as possible. I will probably have to do an upgrade at some point, but since my impression is that that could be difficult, I'm trying to implement as much new upstreamable functionality as possible first.

My immediate next steps will be to get tests building and running and make sure the #[no_core] tests pass. I will also clean up or document assumptions the build system has about the environment to build rustc with this target enabled. After that I will try to get the rest of the tests to pass, which may involve updating LLVM. Once the tests pass, I will be able to work on removing the Emscripten linking and runtime dependency if that's something we want to do.

tlively · 2017-06-15T18:33:47Z

ping @brson, this is ready for another look. Should PRs always be squashed to one commit, or was it only because of merge commits and other dirtiness that the original commits had to be squashed?

tlively · 2017-06-16T19:24:11Z

It turns out that changing the name of the target to wasm32-experimental-emscripten means that I automatically pick up all the special cases for the emscripten target everywhere else in the code base, so hold on for an update for that.

The new target is wasm32-experimental-emscripten. Adds a new configuration option to opt in to building experimental LLVM backends such as the WebAssembly backend. The target name was chosen to be similar to the existing wasm32-unknown-emscripten target so that the build and tests would work with minimal other code changes. When/if the new target replaces the old target, simply renaming it should just work.

tlively · 2017-06-16T22:55:30Z

ping @brson. This should be good to go now. The next steps will be to get testing working without all the manual intervention needed now, then possibly to create a bot for this target. Getting all the tests to pass may involve updating LLVM and fixing bugs in the backend.

SpaceManiac · 2017-06-18T06:29:07Z

There seems to be some disagreement about target triple in the code. Is it wasm32-unknown-unknown (opening comment), wasm32-experimental-emscripten (filename and registration), or wasm32-unknown-emscripten (the Target struct)?

tlively · 2017-06-19T05:18:47Z

Is it wasm32-unknown-unknown (opening comment), wasm32-experimental-emscripten (filename and registration), or wasm32-unknown-emscripten (the Target struct)?

The Rust target this PR introduces is wasm32-experimental-emscripten (in a previous iteration it was wasm32-unknown-unknown). The target triple that gets passed to LLVM is still wasm32-unkown-unknown, because that is what LLVM knows about. There is also still the other wasm target, wasm32-unknown-emscripten, that uses Emscripten for code generation. The new experimental target will probably be renamed to replace the wasm32-unknown-emscripten target once it is stable and good enough.

eholk · 2017-06-19T18:14:25Z

@brson or @alexcrichton - could you take another look at this?

alexcrichton · 2017-06-19T22:35:56Z

Sorry for the delay, looks great @tlively!

@bors: r+

FWIW we recently added a disabled builder for wasm32-unknown-emscripten, so perhaps this could reuse and/or copy a lot of that for a wasm32-experimental-emscripten target? We unfortunately have capacity issues right now so we're not in a position to add more builders, but that shouldn't stop you making them!

bors · 2017-06-19T22:35:57Z

📌 Commit a1981a6 has been approved by alexcrichton

bors · 2017-06-20T09:44:46Z

⌛ Testing commit a1981a6 with merge 380100c...

Enable wasm LLVM backend Enables compilation to WebAssembly with the LLVM backend using the target triple "wasm32-unknown-unknown". This is the beginning of my work on #38804. **edit:** The new new target is now wasm32-experimental-emscripten instead of wasm32-unknown-unknown.

bors · 2017-06-20T12:24:42Z

☀️ Test successful - status-appveyor, status-travis
Approved by: alexcrichton
Pushing 380100c to master...

eholk · 2017-06-23T17:33:58Z

FYI @vadimcn

rust-highfive assigned nikomatsakis Jun 9, 2017

rust-highfive assigned brson and unassigned nikomatsakis Jun 9, 2017

chicoxyzzy reviewed Jun 11, 2017

View reviewed changes

Mark-Simulacrum added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 11, 2017

frewsxcv added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Jun 11, 2017

brson reviewed Jun 12, 2017

View reviewed changes

tlively force-pushed the wasm-dev branch from 18fecac to 4a90f77 Compare June 13, 2017 01:39

This was referenced Jun 13, 2017

Migrate wasm target to LLVM wasm backend #38804

Closed

Add config option for experimental LLVM targets #42643

Closed

arielb1 added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 13, 2017

tlively force-pushed the wasm-dev branch from d01a3e7 to a1981a6 Compare June 16, 2017 22:51

bors merged commit a1981a6 into rust-lang:master Jun 20, 2017

lilianmoraru mentioned this pull request Aug 19, 2017

Addition of the new wasm32-experimental-emscripten target rust-lang-deprecated/rust-buildbot#158

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable wasm LLVM backend #42571

Enable wasm LLVM backend #42571

tlively commented Jun 9, 2017 •

edited

Loading

rust-highfive commented Jun 9, 2017

nikomatsakis commented Jun 9, 2017

eddyb commented Jun 9, 2017

brson commented Jun 9, 2017

chicoxyzzy Jun 11, 2017

tlively Jun 11, 2017

chicoxyzzy Jun 11, 2017

brson Jun 12, 2017

brson Jun 12, 2017

tlively Jun 13, 2017 •

edited

Loading

cretz Jun 13, 2017 •

edited

Loading

brson Jun 13, 2017

brson Jun 12, 2017

brson commented Jun 12, 2017

brson commented Jun 12, 2017

brson commented Jun 12, 2017

tlively commented Jun 13, 2017

tlively commented Jun 15, 2017

tlively commented Jun 16, 2017

tlively commented Jun 16, 2017

SpaceManiac commented Jun 18, 2017

tlively commented Jun 19, 2017

eholk commented Jun 19, 2017

alexcrichton commented Jun 19, 2017

bors commented Jun 19, 2017

bors commented Jun 20, 2017

bors commented Jun 20, 2017

eholk commented Jun 23, 2017

Enable wasm LLVM backend #42571

Enable wasm LLVM backend #42571

Conversation

tlively commented Jun 9, 2017 • edited Loading

rust-highfive commented Jun 9, 2017

nikomatsakis commented Jun 9, 2017

eddyb commented Jun 9, 2017

brson commented Jun 9, 2017

chicoxyzzy Jun 11, 2017

Choose a reason for hiding this comment

tlively Jun 11, 2017

Choose a reason for hiding this comment

chicoxyzzy Jun 11, 2017

Choose a reason for hiding this comment

brson Jun 12, 2017

Choose a reason for hiding this comment

brson Jun 12, 2017

Choose a reason for hiding this comment

tlively Jun 13, 2017 • edited Loading

Choose a reason for hiding this comment

cretz Jun 13, 2017 • edited Loading

Choose a reason for hiding this comment

brson Jun 13, 2017

Choose a reason for hiding this comment

brson Jun 12, 2017

Choose a reason for hiding this comment

brson commented Jun 12, 2017

brson commented Jun 12, 2017

brson commented Jun 12, 2017

tlively commented Jun 13, 2017

tlively commented Jun 15, 2017

tlively commented Jun 16, 2017

tlively commented Jun 16, 2017

SpaceManiac commented Jun 18, 2017

tlively commented Jun 19, 2017

eholk commented Jun 19, 2017

alexcrichton commented Jun 19, 2017

bors commented Jun 19, 2017

bors commented Jun 20, 2017

bors commented Jun 20, 2017

eholk commented Jun 23, 2017

tlively commented Jun 9, 2017 •

edited

Loading

tlively Jun 13, 2017 •

edited

Loading

cretz Jun 13, 2017 •

edited

Loading