Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document that the SocketAddr memory representation is not stable #83524

Merged
merged 1 commit into from
Mar 27, 2021

Conversation

faern
Copy link
Contributor

@faern faern commented Mar 26, 2021

Intended to help out with #78802. Work has been put into finding and fixing code that assumes the memory layout of SocketAddrV4 and SocketAddrV6. But it turns out there are cases where new code continues to make the same assumption (example).

The memory layout of a type in std is never part of the public API. Unless explicitly stated I guess. But since that is invalidly relied upon by a considerable amount of code for these particular types, it might make sense to explicitly document this. This can be temporary. Once #78802 lands it does not make sense to rely on the layout any longer, and this documentation can also be removed.

@rust-highfive
Copy link
Collaborator

r? @Mark-Simulacrum

(rust-highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 26, 2021
@faern
Copy link
Contributor Author

faern commented Mar 26, 2021

r? @sfackler

Because you are the reviewer of #78802

@sfackler
Copy link
Member

@bors r+ rollup

@bors
Copy link
Contributor

bors commented Mar 26, 2021

📌 Commit 147316a has been approved by sfackler

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 26, 2021
JohnTitor added a commit to JohnTitor/rust that referenced this pull request Mar 27, 2021
…t, r=sfackler

Document that the SocketAddr memory representation is not stable

Intended to help out with rust-lang#78802. Work has been put into finding and fixing code that assumes the memory layout of `SocketAddrV4` and `SocketAddrV6`. But it turns out there are cases where new code continues to make the same assumption ([example](spacejam/seaslug@96927dc#diff-917db3d8ca6f862ebf42726b23c72a12b35e584e497ebdb24e474348d7c6ffb6R610-R621)).

The memory layout of a type in `std` is never part of the public API. Unless explicitly stated I guess. But since that is invalidly relied upon by a considerable amount of code for these particular types, it might make sense to explicitly document this. This can be temporary. Once rust-lang#78802 lands it does not make sense to rely on the layout any longer, and this documentation can also be removed.
@the8472
Copy link
Member

the8472 commented Mar 27, 2021

I am slightly concerned about explicitly mentioning the absence of a guarantee. Does that now mean that every type that doesn't say this implicitly has such a guarantee? Of course it doesn't, but it would be nice if it could be phrased in a way that makes it clear that we just happen to spell out what already is and has been the default policy.

bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 27, 2021
Rollup of 9 pull requests

Successful merges:

 - rust-lang#83239 (Remove/replace some outdated crates from the dependency tree)
 - rust-lang#83328 (Fixes to inline assmebly tests)
 - rust-lang#83343 (Simplify and fix byte skipping in format! string parser)
 - rust-lang#83388 (Make # pretty print format easier to discover)
 - rust-lang#83431 (Tell GitHub to highlight `config.toml.example` as TOML)
 - rust-lang#83508 (Use the direct link to the platform support page)
 - rust-lang#83511 (compiletest: handle llvm_version with suffix like "12.0.0libcxx")
 - rust-lang#83524 (Document that the SocketAddr memory representation is not stable)
 - rust-lang#83525 (fix doc comment for `ty::Dynamic`)

Failed merges:

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit d340f63 into rust-lang:master Mar 27, 2021
@rustbot rustbot added this to the 1.53.0 milestone Mar 27, 2021
@faern faern deleted the document-socketaddr-mem-layout branch March 27, 2021 10:46
@faern
Copy link
Contributor Author

faern commented Mar 27, 2021

@the8472 Is that properly documented somewhere? The fact that the memory layout is not part of a types stable API. If so, we could link to it maybe? The SocketAddr documentation does not feel like the correct place to state properties about the entire std IMHO.

@the8472
Copy link
Member

the8472 commented Mar 27, 2021

https://doc.rust-lang.org/reference/type-layout.html#the-default-representation

The Default Representation

Nominal types without a repr attribute have the default representation. Informally, this representation is also called the rust representation.
There are no guarantees of data layout made by this representation.

The rustonomicon says roughly the same.

There are more details in the UCG, but they're not normative yet.

But there is a much simpler fact that establishes that the internals of SocketAddr are not public: The fields are not pub, and the absence of that means they're private, aka not part of the API.

@faern
Copy link
Contributor Author

faern commented Mar 28, 2021

But there is a much simpler fact that establishes that the internals of SocketAddr are not public: The fields are not pub, and the absence of that means they're private, aka not part of the API.

I'm not disagreeing with you. But it's a fact that at least nine different crates have assumed they can rely on the memory layout of these types. So just as a temporary precaution while trying to make the change of implementation happen we can try to not continue making the same mistake.

@the8472
Copy link
Member

the8472 commented Mar 28, 2021

But it's a fact that at least nine different crates have assumed they can rely on the memory layout of these types.

We'd have to ask the authors, but my guess it's due to a lack of conversion APIs between the std SocketAddr type to the libc types. You need them if you actually want to make syscalls and writing those conversions is a bit tedious if you want to unpack IPv4, IPv6, ports, scope IDs, ...
So people take the shortcut and ignore the distinction between implementation details and API surface to fill that gap.

Looking at the fixup PRs that's 100LoC replicated over several crates.

@faern
Copy link
Contributor Author

faern commented Mar 28, 2021

People should bring in socket2 to get the conversion, if they don't want to implement it by themselves: https://docs.rs/socket2/0.4.0/socket2/struct.SockAddr.html#impl-From%3CSocketAddr%3E.

std should not offer conversion to and from libc types as std does not depend on libc (publicly). And since the entire point of this is to move the SocketAddr types to core, the types become even more foreign to libc.

@faern
Copy link
Contributor Author

faern commented Mar 28, 2021

socket2 has provided conversion to and from the system representation since basically forever. A bunch of the fix PRs I have submitted to various libraries to patch this up could probably have used socket2 instead of duplicating a lot of conversion code indeed!

@the8472
Copy link
Member

the8472 commented Mar 28, 2021

socket2 is owned by rust-lang, could we just link there?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants