don't make intra-crate calls to exported functions go through the PLT or similar

[froydnj]

Apologies for the mouthful of an issue title. Here's some example Rust for the issue at hand:

pub struct Ex {
    d: u32
}

pub fn call(x: &Ex)
{
    call_with_flags(x, 0)
}

pub fn call_with_flags(x: &Ex, flags: u32)
{
    println!("d: {}, flags: {}", x.d, flags)
}

Compiling this with rustc -O -C inline-threshold=0 results in the following on x86-64 Linux (the use of -C inline-threshold=0 is a bit artificial, but I've seen the same sort of issue when disassembling std, which presumably doesn't modify inline-threshold):

0000000000000000 <plt::call::h4fbe4633d0f48375>:
   0:   31 f6                   xor    %esi,%esi
   2:   e9 00 00 00 00          jmpq   7 <plt::call::h4fbe4633d0f48375+0x7>
            3: R_X86_64_PLT32   plt::call_with_flags::he191eb6ceb0f1f21-0x4

That R_X86_64_PLT32 relocation is going to get resolved to a call into the PLT, which introduces a small amount of overhead on every call, in addition to taking up unneeded space with the PLT entry and the function pointer in the GOT. It would be better to use a R_X86_64_PC32 relocation there, which will get turned into a direct jump at link time. glibc uses this technique to great effect so that all intra-libc calls (except for a few things like malloc, etc.) don't go through the PLT.

Folks might want the ability to override public functions of a crate via LD_PRELOAD or similar, but doing so seems a little tricky with the current name mangling scheme. Perhaps a -C option could be added?

[alexcrichton]

AFAIK we don't explicitly ask for this anywhere, so it's just an accident or a consequence of something we're passing down to LLVM. We pass down a few "position independent please" flags, but other than that I can't recall anything offhand.

In any case this is probably a pretty simple fix to just pass a flag or two to LLVM.

[Aatch]

Looking at the LLVM docs, I think "protected" visibility style is what we want: http://llvm.org/docs/LangRef.html#visibility-styles

On ELF, protected visibility indicates that the symbol will be placed in the dynamic symbol table, but that references within the defining module will bind to the local symbol. That is, the symbol cannot be overridden by another module.

Making it the default doesn't seem like a bad idea, pulling weird LD_PRELOAD-style tricks is probably rare (and difficult) enough to assume it's not happening. We could add an attribute to get the previous behaviour back ("default" visibility style), not that I know what we'd call that...

[alexcrichton]

Nice find! We may want to hold off on changing #[no_mangle] symbols without some more investigation, but everything else should be fine to basically change at will (so long as it works) as they're all basically internal implementation details anyway.

[Aatch]

So this is actually much harder than originally expected. Turns out the behaviour of "protected" visibility is complicated, and might run into GCC bugs. The specific problem I'm running into is this:

#[inline]
pub fn foo() -> fn() {
    fn bar() {}

   bar
}

The creation of the function pointer is the problem.

There seems to be some weird behaviour going from an rlib to a so which is confusing something along the line.

Also, my research suggests that using protected visibility may slow down users of the library, though it's unclear if the issue affects us or not.

[MarkSwanson]

R_X86_64_PC32 may not work.
I'm currently unable to create a Rust DSO because of a R_X86_64_PC32 problem and the only thing that might work (?) is X86_64_64.

This seems to be a good description of _PC32 vs _64:
http://eli.thegreenplace.net/2011/11/11/position-independent-code-pic-in-shared-libraries-on-x64/

Strangely, I can't get rustc to generate X86_64_64. I've tried -C code-model=large.

I think I should probably create a new issue.

[Mark-Simulacrum]

I cannot reproduce this anymore, so I'm going to close -- I think #38117 fixed this.