Wrong and missing information in PDB file

[MolecularMatters]

With both Rust 1.60 stable as well as the latest nightly (rustc 1.62.0-nightly (082e4ca 2022-04-26)), rustc currently does not emit correct S_OBJNAME records and LF_BUILDINFO records into the PDB on Windows.

Steps to reproduce:

• cargo new repro
• cd repro
• cargo build
• dump the PDB using either DIA2Dump or llvm pdbutil

S_OBJNAME:
The S_OBJNAME record is stored in the PDB for each module stream, and contains the full path to the compiled .o file. This is supported by LLVM at: https://github.com/llvm/llvm-project/blob/9592e88f59cfd399e9285cfec50a23675f43a43a/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.cpp#L783

At the moment, this record is empty for all .o files stored in a PDB.

LF_BUILDINFO:
Similarly, the LF_BUILDINFO record stores the following information for each module stream in the PDB:

• the current working directory
• path to the build tool, e.g. rustc
• source file
• PDB file (not applicable in this case)
• command-line used to compile the corresponding .o

This is supported by LLVM at: https://github.com/llvm/llvm-project/blob/9592e88f59cfd399e9285cfec50a23675f43a43a/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.cpp#L920

At the moment, with rustc there are the following defects in the LF_BUILDINFO record:

• path to the build tool is missing, i.e. uses an invalid type index, which makes DIA2Dump choke and llvm pdbutil crash
• command-line is missing, again uses an invalid type index
• source file is wrong, e.g. it is "src\main.rs@\2j9iw9kw1zbigor6" where it should be "src\main.rs"

Tools like Live++ (https://liveplusplus.tech) need this information in order to be able to recompile individual .o files.
The relevant Zulip discussion can be found here: https://rust-lang.zulipchat.com/#narrow/stream/182449-t-compiler.2Fhelp/topic/Live.2B.2B.20for.20Rust

[nebulark]

I've take a shot fixing this issue and have a working fix. Note that have very little knowlege in that code area, so this is probably not ideal:

The problem seems that Arg0 and CommandLineArgs of the MCTargetOptions cpp class are not set here:

rust/compiler/rustc_llvm/llvm-wrapper/PassWrapper.cpp

Line 378 in bb548f9

extern "C" LLVMTargetMachineRef LLVMRustCreateTargetMachine(

My fix would now set thise member in the mentioned function. However, due to pointer lifetime requirements I had to add another function which saves the commandline args for the programs duration in the passwrapper.

I have a draft here:
https://github.com/nebulark/rust_custom/commit/8acc93124c6de80fea5c7efc648c5dd56775d74a
Patchfile in case the link dies:
draft-added-CL-and-CMD-to-pdb.patch

This should now fix the missing path to the build tool, as well as missing commandline.

I have no clue what to do about the 3rd point 'source file is wrong, e.g. it is "src\main.rs@\2j9iw9kw1zbigor6" where it should be "src\main.rs"'. As far as I understand there can only be a single file, which would be the cpp in C++. Due to n:m mapping of sourcfiles to cgus, the current value of root rs file + cgu name makes sense to me as it is.

However, I found a "workaround": Using Dia2Dump one can find all .rs files for a given cgu in the *** FILES section. So while the information might not be as nice, the info must be present in the PDB so this should be good enough?

[MolecularMatters]

I have no clue what to do about the 3rd point 'source file is wrong, e.g. it is "src\main.rs@\2j9iw9kw1zbigor6" where it should be "src\main.rs"'. As far as I understand there can only be a single file, which would be the cpp in C++. Due to n:m mapping of sourcfiles to cgus, the current value of root rs file + cgu name makes sense to me as it is.

You are correct, the compiland environment in the PDB only supports one file being specified as source file.
If we can guarantee that the source file specified in the PDB is always of the form "sourceFilename@cguName" then that should be fine.

By the way, how are these names of cgus generated? Is this a hash of the contents? Or just some unique identifier?

However, I found a "workaround": Using Dia2Dump one can find all .rs files for a given cgu in the *** FILES section. So while the information might not be as nice, the info must be present in the PDB so this should be good enough?

Yes, that should be enough.
We use the information stored in the module information stream and individual module streams (named *** FILES in Dia2Dump output) for building a dependency tree used by Live++.
If this information is present for all cgus associated with an .rs file, then we should be able to (re)build the full N:M dependency mapping, essentially giving us enough information to recompile individual .rs files.

I can give your changes a try as soon as they land in a nightly build.

[bjorn3]

By the way, how are these names of cgus generated? Is this a hash of the contents? Or just some unique identifier?

The cgu name is a unique identifier. The exact format depends on if incremental mode is enabled or not and if the flag to use human readable cgu names is used or not. I'm not sure where the src\main.rs@\2j9iw9kw1zbigor6 string is generated though. It may be generated by LLVM rather than rustc. As such I don't have any clue how stable this format is.

Yes, that should be enough.
We use the information stored in the module information stream and individual module streams (named *** FILES in Dia2Dump output) for building a dependency tree used by Live++.
If this information is present for all cgus associated with an .rs file, then we should be able to (re)build the full N:M dependency mapping, essentially giving us enough information to recompile individual .rs files.

If you include_bytes!() a file I believe it doesn't end up in the debuginfo. And dependencies may not end up in there either. The .d files emitted by rustc when using --emit dep-info (as cargo uses by default for it's own recompilation logic) will contain all information necessary to determine if a crate should be recompiled if -Zbinary-dep-depinfo is used (without library dependencies are missing, but all source files of the crate itself are still mentioned) To actually recompile it you do need the full commandline invocation though. Cargo passes -Cmetadata with an unpredictable value which affects the symbol names of everything defined inside the crate.

[ChrisDenton]

It is generally better to open a new issue rather than on an old closed one.

[capickett]

Apologies for that. I've moved this to #128842.