Enable gimli-symbolize to be used by libstd

[alexcrichton]

Currently the gimli-symbolize feature also enables the std feature. This means that it's not suitable for inclusion into the standard library. This also means that now that we've switched this crate to using gimli by default the standard library may languish! All-in-all, let's put the final nail in the coffin of libbacktrace and switch to gimli by enabling libstd to use gimli.

Some initial words have been said about this on #189, but the general state of the world is that the gimli implementation in this crate only uses a bunch of functions from std. This ranges from:

• std::env::current_exe() - used to parse own debuginfo
• std::fs::* - used to find, locate, and open debuginfo. Often the current executable, macOS is much more involved though
• std::{path,ffi}::* - manipulating paths to get passed around in various places, along with reading C strings on Linux.

I don't think it is currently clear at this time how we're going to implement this. There's a few possible strategies that have been proposed so far:

1. Vendor an implementation of everything into this crate. This would involve duplicating code from libstd into this crate, calling all the raw syscalls ourselves. One major downside of this is that as the debuginfo-finding process gets more complicated this syscall layer is undoubtedly going to get more complicated. Additionally std::env::current_exe() is already significantly complicated to run on all platforms.
2. Have some sort of trait which libstd implements. The standard library would then pass down its implementation into this crate. This trait would be a bit of a pain to maintain over time, however.
3. Include this library as a submodule into libstd. This is actually much easier after the recent rejiggering of features, so this may be a a somewhat plausible route. It would involve sync'ing dependencies with libstd, however, and that may not be trivial over time. Another downside of this approach is that it's somewhat difficult to maintain this style of coding over time. It's pretty nonstandard and would likely turn away possible contributors.
4. Split the sys module out of libstd into a separate crate. The backtrace crate would then un-stably depend on this sys module. This is a huge change for libstd, however, and highly unlikely to happen.

I'm definitely open to more ideas if folks have them :)

[alexcrichton]

One thing I should also mention though is that we likely want to let the gimli support bake on crates.io for some time before actually moving on this. There's undoubtedly various porting issues and/or bugs that will arise, and it'd be great to weed those out before we move into libstd.

[fitzgen]

FWIW, option (2) seems like the best course of action to me, based on gut feeling.

[alexcrichton]

Taking the submodule route is actually surprisingly easy to get working.

This is the change to this crate which can be summarized with:

• Remove usage of use crate:: to use relative super paths instead
• Add a #[cfg] to define a local mod std { pub use crate::* } which "acts like libstd"
• Route all use std through that module

This is the change to libstd which looks like:

• Add some [patch] for object/addr2line/gimli, but that's just temporary. Those patches are easily upstreamed.
• Inline dependencies of the backtrace crate directly into libstd's Cargo.toml
• Feign the "gimli-symbolize" feature in libstd's Cargo.toml
• Create a backtrace_rs module which points to the source of the crate. Adjust all paths in libstd referencing the previous backtrace_rs crate to reference this submodule of libstd instead.
• Allow dead code/unused attributes in the backtrace_rs module to avoid dealing with warnings we don't want to fix.

With those two applied I was able to get everything working locally.

I was thinking again about the impact of developers on this crate, and I'm actually less worried than I was before. You'll still have access to all of core and alloc and such, it's just the few use std:: paths in gimli.rs which are a bit wonky. In that sense it's not really all that bad. The lack of use crate:: is pretty bad, but a CI lint could enforce that here in this crate, and it's easy enough to work around.

Given the level of ease to get this working, I'm pretty hesitant to use traits/duplication/vendoring code. This seems like it might be the way to go. It means updates are a bit more of a pain because it's a git submodule rather than a crates.io dependency, but code-wise it's quite light...

[est31]

The submodule method is also future-compatible with a possible (future) std/core unification.

[alexcrichton]

With the merge of #344 I've additionally worked to get two more crates working as part of a dependency of libstd:

• Mark library as #![no_std] remram44/adler32-rs#15
• Flag miniz_oxide as a #![no_std] library Frommi/miniz_oxide#81

My assumption is that we'll want to enable zlib decompression by default like libstd does today, so I think we'll need to merge these before including into rust-lang/rust.

[luser]

Do you think this is a viable path for this crate and any others that libstd may want to include in the future, or do you think there would be value in exploring whether some set of language/library changes could make this easier to do?

[alexcrichton]

Personally, no, I don't think this is viable for many other crates to use. It's a real pain managing submodules and architecting the crate such that it may be included as a submodule but also builds as a standalone crate. I think that this strategy only really makes sense if it applies to a small handful of crates (e.g. 1 or 2).

I think it seems reasonable to explore changes to make this more viable, but I don't really know what the best change would be. Everything is pretty large scale compared to the maintenance burden on one crate.

[alexcrichton]

This was done awhile back!