Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement sock_accept and basic networking #3711

Merged
merged 1 commit into from
Feb 1, 2022

Conversation

haraldh
Copy link
Contributor

@haraldh haraldh commented Jan 21, 2022

Together with rust-lang/rust#93158 this enables basic networking in rust.

See #3730 for the tracking issue.

With the addition of sock_accept() in wasi-0.11.0, wasmtime can now implement basic networking for pre-opened sockets.

For Windows AsHandle was replaced with AsRawHandleOrSocket to cope with the duality of Handles and Sockets.

For Unix a wasi_cap_std_sync::net::Socket enum was created to handle the {Tcp,Unix}{Listener,Stream} more efficiently in
WasiCtxBuilder::preopened_socket().

The addition of that many WasiFile implementors was mainly necessary, because of the difference in the num_ready_bytes() function.

A known issue is Windows now busy polling on sockets, because except for stdin, nothing is querying the status of windows handles/sockets.

Another know issue on Windows, is that there is no crate providing support for fcntl(fd, F_GETFL, 0) for sockets, so WasiFile::get_fdflags() always returns 0.

Todo

  • Test cases

@github-actions
Copy link

Subscribe to Label Action

cc @kubkon

This issue or pull request has been labeled: "wasi"

Thus the following users have been cc'd because of the following labels:

  • kubkon: wasi

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.

src/commands/run.rs Outdated Show resolved Hide resolved
@haraldh haraldh force-pushed the wasmtime_sock_accept branch 8 times, most recently from 598194d to e6121ae Compare January 26, 2022 15:17
src/commands/run.rs Outdated Show resolved Hide resolved
@haraldh haraldh force-pushed the wasmtime_sock_accept branch 2 times, most recently from 998f051 to e16a969 Compare January 26, 2022 17:14
@npmccallum
Copy link
Member

@haraldh Is this still a draft?

@haraldh haraldh force-pushed the wasmtime_sock_accept branch 2 times, most recently from 0f85778 to 60109b5 Compare January 27, 2022 10:25
@haraldh haraldh changed the title DRAFT: Implement sock_accept and basic networking Implement sock_accept and basic networking Jan 27, 2022
@haraldh haraldh marked this pull request as ready for review January 27, 2022 12:01
@haraldh
Copy link
Contributor Author

haraldh commented Jan 27, 2022

@npmccallum now it's ready for a review :)

Copy link
Member

@sunfishcode sunfishcode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! Just one potential question:

UnixStream(cap_std::os::unix::net::UnixStream),
#[cfg(unix)]
UnixListener(cap_std::os::unix::net::UnixListener),
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another option here would be to make Socket be a simple struct that holds an OwnedFd, and uses rustix::net calls to implement the functions rather than cap_std::net/std::net. Rustix even supports the socket APIs on Windows, so this ought to work. And you could probably unify the "stream" vs. "listener" implementations below. That said, either way seems fine for now, so feel free to make a judgement call here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I didn't want to change all the other parts too much.

What you are proposing can be done in a separate PR, which will change much more, I guess.

With the Socket abstraction, I think I didn't make the "user" facing API incompatible with your proposal.

@haraldh
Copy link
Contributor Author

haraldh commented Jan 28, 2022

Changed one more thing to make fd_fdstat_get() work on the sockets.

             async fn get_fdflags(&self) -> Result<FdFlags, Error> {
-                Err(Error::badf())
+                let fdflags = self.0.as_filelike().get_fd_flags()?;
+                Ok(from_sysif_fdflags(fdflags))
             }

@haraldh haraldh force-pushed the wasmtime_sock_accept branch 2 times, most recently from 5cc7139 to 0d4b911 Compare January 28, 2022 13:30
With the addition of `sock_accept()` in `wasi-0.11.0`, wasmtime can now
implement basic networking for pre-opened sockets.

For Windows `AsHandle` was replaced with `AsRawHandleOrSocket` to cope
with the duality of Handles and Sockets.

For Unix a `wasi_cap_std_sync::net::Socket` enum was created to handle
the {Tcp,Unix}{Listener,Stream} more efficiently in
`WasiCtxBuilder::preopened_socket()`.

The addition of that many `WasiFile` implementors was mainly necessary,
because of the difference in the `num_ready_bytes()` function.

A known issue is Windows now busy polling on sockets, because except
for `stdin`, nothing is querying the status of windows handles/sockets.

Another know issue on Windows, is that there is no crate providing
support for `fcntl(fd, F_GETFL, 0)` on a socket.

Signed-off-by: Harald Hoyer <harald@profian.com>
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Jan 28, 2022
wasi: implement `sock_accept` and enable networking

With the addition of `sock_accept()` to snapshot1, simple networking via a passed `TcpListener` is possible. This PR implements the basics to make a simple server work.

See also:
* [wasmtime tracking issue](bytecodealliance/wasmtime#3730)
* [wasmtime PR](bytecodealliance/wasmtime#3711)

TODO:
* [ ] Discussion of `SocketAddr` return value for `::accept()`

```rust
        Ok((
            TcpStream::from_inner(unsafe { Socket::from_raw_fd(fd as _) }),
            // WASI has no concept of SocketAddr yet
            // return an unspecified IPv4Addr
            SocketAddr::new(Ipv4Addr::UNSPECIFIED.into(), 0),
        ))
```
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Jan 28, 2022
wasi: implement `sock_accept` and enable networking

With the addition of `sock_accept()` to snapshot1, simple networking via a passed `TcpListener` is possible. This PR implements the basics to make a simple server work.

See also:
* [wasmtime tracking issue](bytecodealliance/wasmtime#3730)
* [wasmtime PR](bytecodealliance/wasmtime#3711)

TODO:
* [ ] Discussion of `SocketAddr` return value for `::accept()`

```rust
        Ok((
            TcpStream::from_inner(unsafe { Socket::from_raw_fd(fd as _) }),
            // WASI has no concept of SocketAddr yet
            // return an unspecified IPv4Addr
            SocketAddr::new(Ipv4Addr::UNSPECIFIED.into(), 0),
        ))
```
@haraldh
Copy link
Contributor Author

haraldh commented Jan 31, 2022

All right.. rust-lang PR merged 🥳

@npmccallum
Copy link
Member

@sunfishcode Is this ready for merge?

@sunfishcode sunfishcode merged commit 853a025 into bytecodealliance:main Feb 1, 2022
@sunfishcode
Copy link
Member

Yes, looks good to me!

haraldh added a commit to haraldh/mio that referenced this pull request Feb 4, 2022
With
* bytecodealliance/wasmtime#3711
* rust-lang/rust#93158
merged, mio can have limited support for networking for the
`wasm32-wasi` target.

Signed-off-by: Harald Hoyer <harald@profian.com>
haraldh added a commit to haraldh/mio that referenced this pull request Feb 15, 2022
With
* bytecodealliance/wasmtime#3711
* rust-lang/rust#93158
merged, mio can have limited support for networking for the
`wasm32-wasi` target.

Signed-off-by: Harald Hoyer <harald@profian.com>
haraldh added a commit to haraldh/mio that referenced this pull request Feb 15, 2022
With
* bytecodealliance/wasmtime#3711
* rust-lang/rust#93158
merged, mio can have limited support for networking for the
`wasm32-wasi` target.

Signed-off-by: Harald Hoyer <harald@profian.com>
haraldh added a commit to haraldh/mio that referenced this pull request Feb 15, 2022
With
* bytecodealliance/wasmtime#3711
* rust-lang/rust#93158
merged, mio can have limited support for networking for the
`wasm32-wasi` target.

Signed-off-by: Harald Hoyer <harald@profian.com>
haraldh added a commit to haraldh/mio that referenced this pull request Feb 18, 2022
With
* bytecodealliance/wasmtime#3711
* rust-lang/rust#93158
merged, mio can have limited support for networking for the
`wasm32-wasi` target.

Signed-off-by: Harald Hoyer <harald@profian.com>
haraldh added a commit to haraldh/mio that referenced this pull request Feb 21, 2022
Based on tokio-rs#1395

And with
* bytecodealliance/wasmtime#3711
* rust-lang/rust#93158
merged, mio can have limited support for networking for the
`wasm32-wasi` target.

Co-authored-by: Thomas de Zeeuw <thomasdezeeuw@gmail.com>
Signed-off-by: Harald Hoyer <harald@profian.com>
haraldh added a commit to haraldh/mio that referenced this pull request Mar 8, 2022
Based on tokio-rs#1395

And with
* bytecodealliance/wasmtime#3711
* rust-lang/rust#93158
merged, mio can have limited support for networking for the
`wasm32-wasi` target.

Co-authored-by: Thomas de Zeeuw <thomasdezeeuw@gmail.com>
Signed-off-by: Harald Hoyer <harald@profian.com>
Thomasdezeeuw added a commit to tokio-rs/mio that referenced this pull request Mar 8, 2022
Based on #1395

And with
* bytecodealliance/wasmtime#3711
* rust-lang/rust#93158
merged, mio can have limited support for networking for the
`wasm32-wasi` target.

Co-authored-by: Thomas de Zeeuw <thomasdezeeuw@gmail.com>
Signed-off-by: Harald Hoyer <harald@profian.com>
bdbai pushed a commit to YtFlow/mio-noafd that referenced this pull request May 15, 2022
Based on tokio-rs#1395

And with
* bytecodealliance/wasmtime#3711
* rust-lang/rust#93158
merged, mio can have limited support for networking for the
`wasm32-wasi` target.

Co-authored-by: Thomas de Zeeuw <thomasdezeeuw@gmail.com>
Signed-off-by: Harald Hoyer <harald@profian.com>
stlankes added a commit to hermit-os/mio that referenced this pull request Jun 30, 2022
Based on tokio-rs#1395

And with
* bytecodealliance/wasmtime#3711
* rust-lang/rust#93158
merged, mio can have limited support for networking for the
`wasm32-wasi` target.

Co-authored-by: Thomas de Zeeuw <thomasdezeeuw@gmail.com>
Signed-off-by: Harald Hoyer <harald@profian.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wasi Issues pertaining to WASI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants