Implement From<char> for u64 and u128.

[Julian-Wollersberger]

With this PR you can write

let u = u64::from('👤');
let u = u128::from('👤');

Previously, you could already write as conversions (Playground link):

// Lossless conversions
dbg!('👤' as u32);    // Prints 128100
dbg!('👤' as u64);    // Prints 128100
dbg!('👤' as u128);   // Prints 128100

// truncates, thus no `From` impls.
dbg!('👤' as u8);     // Prints 100
dbg!('👤' as u16);    // Prints 62564

// These `From` impls already exist.
dbg!(u32::from('👤'));               // Prints 128100
dbg!(u64::from(u32::from('👤')));    // Prints 128100

The idea is from @gendx who opened this Internals thread, and @withoutboats responded that someone should open a PR for it.
Some people mentioned From<char> impls for f32 and f64, but that doesn't seem correct to me, so I didn't include them here.

I don't know what the feature should be named. Must it be registered somewhere, like unstable features?

r? @withoutboats

[Julian-Wollersberger]

@rustbot modify labels: +T-libs +relnotes +C-enhancement

[rustbot]

Error: Label relnotes can only be set by Rust team members

Please let @rust-lang/release know if you're having trouble with this bot.

[scottmcm]

Some people mentioned From<char> impls for f32 and f64, but that doesn't seem correct to me, so I didn't include them here.

Could you elaborate on why? It seems like the same "These From impls already exist" argument would apply just as well for f64, since f64::from(u32::from('👤')) also works.

[withoutboats]

Conversion to u64 and u128 have pretty obvious utility (you want the integer representation of the char in a format which has reserved a certain number of bits for this data), to float not so much. Conversion to floats can be added by someone who has need of the feature.

[withoutboats]

@rfcbot fcp merge

[rfcbot]

Team member @withoutboats has proposed to merge this. The next step is review by the rest of the tagged team members:

• @Amanieu
• @BurntSushi
• @KodrAus
• @dtolnay
• @m-ou-se
• @sfackler
• @withoutboats

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[withoutboats]

I don't know what the feature should be named. Must it be registered somewhere, like unstable features?

What you've done is perfect, thanks!

[m-ou-se]

I don't think these From implementations fit with the rest of the interface of char.

char isn't seen anywhere in Rust as a numeric/integral type, like the integer types are. As @scottmcm said on the internals thread, it's somewhat like a struct char(u32);; a type that uses u32 as the underlying representation. It is Into<u32> and TryFrom<u32> because of this special relationship with u32. It even has a char::from_u32 and from_u32_unchecked.

If we really want to make char behave more like an integer (which I don't think we should), then conversions to only u64 and u128 would be incomplete. Every code point is also representable as a i32, i64 and i128. And we should then probably also have TryFrom<u64> for char, etc. And (as @scottmcm already mentioned) possibly conversions to and from floating point values too. I don't think this is the direction char should go. (Would we expect num::BigInt to implement From<char>?)

It's unfortunate that currently the way to get the codepoint as u32 from a char is c as u32 or u32::from(c). This makes u64::from(u32::from(c)) just look redundant. If char had a .as_u32() or .codepoint() (just like char::from_u32), c.as_u32().into() or u64::from(c.codepoint()) would look more reasonable.

[withoutboats]

I think this entire line of discussion is overthinking it.. as was pointed out on internals, we already have From<u8> for char, which has nothing inherently to do with u8 having a "special relationship" with char, as it losslessly converts bit patterns that are non-ASCII into chars.

This sort of philosophical discussion about "what a char is" is not very interesting to me. Users sometimes have code which has guaranteed that their value is within the valid bitrange for char (and are guaranteeing that by using our char type) but need to pass that value to an interface accepting a broader range of values. I see no compelling reason to make this more difficult for them.

[m-ou-se]

We don't have a From<u16> for char, and the From<u8> for char does exists because of the special relationship between u8 and char. u8 is used for ascii characters; it even has methods specifically for ascii. Unicode codepoints (char) are a superset of ascii on purpose. The decision to have that as a non-faillible conversion, and thus include non-ascii characters as well, was well thought out and made based on the fact that unicode codepoints 128-255 match ISO-8859-1.

code which has guaranteed that their value is within the valid bitrange for char (and are guaranteeing that by using our char type) but need to pass that value to an interface accepting a broader range of values

Then why only u64 and u128? Plenty of C code uses int/i32 for these things, char in C is often signed, and if you want to pass a codepoint as number to javascript/json, it'd be a f64.

[withoutboats]

Then why only u64 and u128? Plenty of C code uses int/i32 for these things, char in C is often signed, and if you want to pass a codepoint as number to javascript/json, it'd be a f64.

If anyone has strong enough desire for those APIs to make a PR, I would check my box to add them as well. What I favor is pragmatism: just as we should not not add these APIs for philosophical reasons, we also should not add them just for the sake of being "complete" as well.

[m-ou-se]

Storing a codepoint in an int doesn't seem like a very philosophical situation.

I would check my box to add them as well.

I probably wouldn't check mine for floats or TryFrom<u64>, which makes me hesitant to check it here, unless there is something fundamentally different about the conversions in this PR.

[BurntSushi]

@m-ou-se I'd like to try understanding your position better. Is it a "these don't make conceptual sense," or do you see some practical disadvantage here?

When I saw this proposal, my thinking was on similar lines as @withoutboats. In particular, the "I need to guarantee some value is within a range but also need to pass it to some interface with a broader range" made sense to me. Granted, it probably doesn't come up often for char (it's already somewhat rare to even bother with the u32 version of char), so the upside here is pretty small. But I guess I also perceive the downside as pretty small. What do we gain by making the user do u64::from(u32::from(some_char))?

Also, @m-ou-se, as a practical matter, depending on how you feel on the matter, you may want to register a concern with the FCP bot. I think that will block the FCP from passing even if all-but-two members check their boxes. And I only say this since you're new to the team and I'm not sure if you know about it or not!

[m-ou-se]

Also, @m-ou-se, as a practical matter, depending on how you feel on the matter, you may want to register a concern with the FCP bot. I think that will block the FCP from passing even if all-but-two members check their boxes. And I only say this since you're new to the team and I'm not sure if you know about it or not!

Thanks :) I'm aware. I didn't raise a concern with the bot on purpose, if the majority of the team agrees, I don't want to block this.

I'd like to try understanding your position better. Is it a "these don't make conceptual sense," or do you see some practical disadvantage here?

I just want to have an idea of which other char conversions we would and would not consider okay to add in the future. With most changes, we'd have a period of unstability in which we can see what other changes come in, and have more time to consider how it fits in with the rest of the (future) interface. Since this is insta-stable, we should at least discuss the (potential) surrounding APIs a bit, such as char::from_u32, TryFrom<u64> and From<char> for f64. I very much agree with boats that we shouldn't block things on philosophical concerns, but I don't think we can make decisions like this entirely in isolation. Using signed integers or floats for codepoints is a very common and real thing.

So far, u32 has been given the special treatment as the integer type for codepoints, with a dedicated char::from_u32, etc. Adding conversions to other integer types raises the question if u32 is still special, and what that means for the rest of the interface.

What do we gain by making the user do u64::from(u32::from(some_char))?

I mentioned above I think that's not great. u64::from(some_char.codepoint()) or something would look nicer. And maybe u64::from(some_char) is fine after all.

I just want to make sure we consider the surroundings of this API a bit. I can't vote on these two specific conversions without at least considering what this means for the related ones. (TryFrom, signed integers, floats, and from_u32.) Especially since multiple people already raised questions about this on the internals thread.

[BurntSushi]

@m-ou-se Gotya. That makes sense to me.

As somewhat of an irrelevant aside, when I build APIs like char in other libraries, I usually don't bother too much with From impls. Instead, I'll do as you suggested: add specific methods like as_codepoint instead. If only because it makes it more discoverable. This doesn't mean I don't also add From impls eventually. Just depends on whether someone wants it or not.

Otherwise, yeah, if we add these then I guess it does make some sense to think about other cases.

Using signed integers or floats for codepoints is a very common and real thing.

If we take "common" here to be "relatively common", then I can see signed integers maybe because of FFI or other interfaces that require signed integers? But I'm having a hard time coming up with cases where floats are useful or even relatively common. Can you say more about those?

Otherwise, yeah, I would personally be okay with From impls for signed integers. I'm less sure about floats, but don't feel that strongly about them. In particular, I just don't see much downside from adding them.

So far, u32 has been given the special treatment as the integer type for codepoints, with a dedicated char::from_u32, etc. Adding conversions to other integer types raises the question if u32 is still special, and what that means for the rest of the interface.

Yes, I think u32 is still special. The fact that char is a u32 is very nearly guaranteed by the API docs. And when you do need to deal with a codepoint as an integer, u32 is almost certainly the most common. I do see a special relationship between char and u32 and wouldn't want to add things like from_u64. (Is that what you're getting at here? Not sure.)

[Julian-Wollersberger]

I don't have a strong opinion here, and no personal use case. I made this PR because it was an easy contribution.

Some people mentioned From<char> impls for f32 and f64, but that doesn't seem correct to me, so I didn't include them here.

Could you elaborate on why? It seems like the same "These From impls already exist" argument would apply just as well for f64, since f64::from(u32::from('👤')) also works.

My intuition is that From and Into are only for the "obviously correct" conversions. When the conversion can be lossy, you have to use as or TryFrom / TryInto. Also, as conversions have non-obvious semantics, in the sense that I always need to look up if a conversion is defined to truncate, panic, saturate or bitwise reinterpret.

Since char is defined as a 4 byte Unicode scalar, the conversion to u32 is the most obvious: "Give me that Unicode scalar as a number." Conversion to u64 and u128 is IMO also obvious and unambiguous: "Give me that Unicode scalar as a number, but use more bits than necessary."

For converting char to f32 or f64, it's not obvious to me what it should do. Can the sign be negative? Can there be a NaN character? What impact would rounding have? Apparently this can be answered with "No, no and doesn't happen", but to avoid this confusion I'm against adding From impls here.

I'm not so sure about From<char> for {i32, i64, i128}, though I'd argue there can also be confusion if negative scalar values could be a thing. Again, you would need to look up what will happen precisely to learn that Unicode is specified up to 24 bits, so no actual problems would arise.

The biggest use case I see for char conversions to u64 and u128 is to give better control over the memory layout. Additional conversions wouldn't give any more control. They can still be done with as anyway.

[withoutboats]

I basically agree that the problem with float is that it doesn't feel as completely obvious as u64 and u128. But if there were someone trying to bind to some JavaScript API that takes unicode scalars as a float and wanted Into<f64>, I wouldn't object to adding it.

I also think a char method called codepoint or something similar that is equivalent to casting to u32 would be a positive addition to std, because sometimes code is clearer if you use a specifically named function instead of the generic conversion.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[Julian-Wollersberger]

This slipped a release and I updated the version to 1.51.
@withoutboats can you merge this? Thanks!

[withoutboats]

@bors r+

[bors]

📌 Commit e8cb72c has been approved by withoutboats

[bors]

⌛ Testing commit e8cb72c with merge 34628e5...