Create a derive macro for HashStable and allow proc macros in rustc #58013

Zoxc · 2019-01-30T20:54:28Z

A combination of #56864 and #56795. There were complications with using serde_derive as rustc doesn't know which crate to use for the host when there is a serde_derive in the sysroot and cargo passes another on the command line built from crates.io.

r? @eddyb (for proc macro changes) @alexcrichton (for rustbuild changes) @michaelwoerister (for the macro itself)

Zoxc · 2019-01-30T22:06:54Z

Blocked on rust-lang/cargo#6547

michaelwoerister · 2019-01-31T13:18:09Z

The macro looks awesome. This should let us get rid of so much boilerplate. Also, the synstructure crate looks pretty sweet :)

alexcrichton · 2019-02-01T16:40:33Z

I haven't been following this too too closely, but @Zoxc can you detail a bit more what some of the changes are to linking and such?

Zoxc · 2019-02-01T17:44:29Z

I'm just going to list the changes enabling proc macros.

We pass -Zdual-proc-macros to both cargo and rustc when cross compiling. Cargo will then build proc macros both for the host and the target and pass both to rustc for crates which depend upon them. Rustc will link to the target proc macro crate in metadata, but load the host proc macro crate and use that to execute the macros.
Bootstrap will now move proc macro crates compiled to the host into the sysroot at lib\rustlib\<host-triple>\lib in addition to the target crates which are moved to lib\rustlib\<target-triple>\lib
We require the Test target for the compiler, since that compiles libproc_macro and we need that to build proc macros used by the stage1 compiler to build stage2 compiler crates.
We make -Z force-unstable-if-unmarked affect proc macros too, to avoid an ICE in stability checking

mark-i-m · 2019-02-04T20:48:38Z

src/librustc_macros/Cargo.toml

+[package]
+name = "rustc_macros"
+version = "0.1.0"
+authors = ["The Rust Project Developers"]


Is it possible to make this a rust 2018 crate? Otherwise, somebody will come along and do that later for #58099 anyway, leading to more churn...

we could/should enforce this via tidy once we have all of them ported

alexcrichton · 2019-02-14T16:28:16Z

FWIW I don't think I'll have much to add any more to this, so long as it works and bootstraps seems fine by me, I wouldn't block on me for this.

Zoxc · 2019-02-15T11:22:09Z

We'll need to get rust-lang/cargo#6547 into beta before this can be merged.

vext01 · 2019-02-20T09:40:23Z

This PR is now unblocked :)

mati865 · 2019-02-20T11:45:49Z

@vext01 it's not that easy.
Updated Cargo has to make it's way to beta branch first, then bootstrap compiler has to updated.
After it's done this PR will be unblocked.

Zoxc · 2019-02-23T13:16:55Z

Ping @eddyb for review of the proc macro parts.

Zoxc · 2019-03-01T15:21:19Z

@rust-lang/compiler Do we have someone else who can review the proc macro changes in rustc here?

src/librustc_macros/src/hash_stable.rs

… a host proc macro to execute them

rust-highfive · 2019-03-07T06:29:59Z

The job i686-gnu of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[02:58:33] Documenting book redirect pages (i686-unknown-linux-gnu)
[02:58:34] [TIMING] TheBook { compiler: Compiler { stage: 2, host: "i686-unknown-linux-gnu" }, target: "i686-unknown-linux-gnu", name: "book" } -- 0.967
[02:58:34] Documenting stage2 std (i686-unknown-linux-gnu)
[02:58:35]     Checking core v0.0.0 (/checkout/src/libcore)
The job exceeded the maximum time limit for jobs, and has been terminated.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

pietroalbini · 2019-03-07T06:32:22Z

@bors retry

bors · 2019-03-07T07:17:03Z

⌛ Testing commit f2ef283 with merge c1187c488edaf9d9874c5ca40840552b436cf003...

pietroalbini · 2019-03-07T07:17:54Z

@bors retry

rust-highfive · 2019-03-07T07:20:06Z

Your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

Cloning into 'rust-lang/rust'...
travis_time:end:01a0ba21:start=1551943099700463655,finish=1551943105949872037,duration=6249408382
$ cd rust-lang/rust
$ git checkout -qf c1187c488edaf9d9874c5ca40840552b436cf003
fatal: reference is not a tree: c1187c488edaf9d9874c5ca40840552b436cf003
The command "git checkout -qf c1187c488edaf9d9874c5ca40840552b436cf003" failed and exited with 128 during .
Your build has been stopped.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

Zoxc · 2019-03-07T18:58:52Z

@bors p=0

Zoxc · 2019-03-08T05:46:03Z

@bors p=500

bors · 2019-03-08T05:46:17Z

⌛ Testing commit f2ef283 with merge 0547ceb...

@eddyb

Create a derive macro for HashStable and allow proc macros in rustc A combination of #56864 and #56795. There were complications with using `serde_derive` as rustc doesn't know which crate to use for the host when there is a serde_derive in the sysroot and cargo passes another on the command line built from crates.io. r? @eddyb (for proc macro changes) @alexcrichton (for rustbuild changes) @michaelwoerister (for the macro itself)

pietroalbini · 2019-03-08T06:43:03Z

@bors treeclosed- p=2

bors · 2019-03-08T08:39:06Z

☀️ Test successful - checks-travis, status-appveyor
Approved by: oli-obk
Pushing 0547ceb to master...

michaelwoerister · 2019-03-11T09:31:04Z

🎉

eddyb · 2019-03-13T16:21:24Z

src/librustc_metadata/locator.rs

@@ -258,7 +260,7 @@ pub struct Context<'a> {
    pub extra_filename: Option<&'a str>,
    // points to either self.sess.target.target or self.sess.host, must match triple
    pub target: &'a Target,
-    pub triple: &'a TargetTriple,
+    pub triple: TargetTriple,


cc @nnethercote

eddyb · 2019-03-13T16:24:16Z

src/librustc/session/config.rs

@@ -1232,6 +1232,8 @@ options! {DebuggingOptions, DebuggingSetter, basic_debugging_options,
         Use with RUST_REGION_GRAPH=help for more info"),
    parse_only: bool = (false, parse_bool, [UNTRACKED],
        "parse only; do not compile, assemble, or link"),
+    dual_proc_macros: bool = (false, parse_bool, [TRACKED],
+        "load proc macros for both target and host, but only link to the target"),


Sorry for being so late to see this, but can an issue be opened to track this "feature"?

Opened #64671

jyn514 · 2021-05-01T05:04:40Z

@Zoxc what's the reason for the last commit ("make cargo a rustc tool again")? It prevents building cargo without building the whole compiler first.

eddyb · 2021-05-03T12:21:55Z

@jyn514 looks like it resolves this failure #58013 (comment), and it's just a revert of e501a87 in this PR - if you look at the full diff, this PR doesn't change that part of src/bootstrap/tool.rs - sounds like you took a wrong turn during a git blame?

jyn514 · 2021-05-03T12:55:45Z

Hmm ok - I was hoping to remove that dependency in #84780 (comment), I guess I should test freebsd locally.

Yes, probably wrong git blame, thanks for the pointer :)

eddyb · 2021-05-03T13:58:40Z

I guess I should test freebsd locally.

I suspect this will show up on linux too, it looks like some sort of Cargo feature conflict between tools.

EDIT: keep in mind src/tools/cargo is used by everything that uses Cargo as a library (e.g. RLS or clippy):

rust/Cargo.toml

Lines 87 to 94 in c825bc4

    
           # We want the RLS to use the version of Cargo that we've got vendored in this 
        
           # repository to ensure that the same exact version of Cargo is used by both the 
        
           # RLS and the Cargo binary itself. The RLS depends on Cargo as a git repository 
        
           # so we use a `[patch]` here to override the github repository with our local 
        
           # vendored copy. 
        
           [patch."https://github.com/rust-lang/cargo"] 
        
           cargo = { path = "src/tools/cargo" } 
        
           cargo-util = { path = "src/tools/cargo/crates/cargo-util" }

rust-highfive assigned eddyb Jan 30, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jan 30, 2019

Zoxc assigned alexcrichton and michaelwoerister Jan 30, 2019

This comment has been minimized.

Sign in to view

mark-i-m reviewed Feb 4, 2019

View reviewed changes

vext01 mentioned this pull request Feb 11, 2019

The first incarnation of the metadata serialiser/deserialiser. softdevteam/ykpack#1

Closed

Zoxc force-pushed the stable-hash-macro-simple branch from c5b0065 to 441e350 Compare February 15, 2019 05:25

alexcrichton mentioned this pull request Feb 15, 2019

Add an unstable option to build proc macros for both the host and the target rust-lang/cargo#6547

Merged

This comment has been minimized.

Sign in to view

Zoxc force-pushed the stable-hash-macro-simple branch from 441e350 to 756241d Compare February 25, 2019 19:37

This comment has been minimized.

Sign in to view

Zoxc force-pushed the stable-hash-macro-simple branch from 756241d to 5fae5f7 Compare February 26, 2019 11:44

This comment has been minimized.

Sign in to view

Zoxc force-pushed the stable-hash-macro-simple branch from 5fae5f7 to 8746683 Compare March 1, 2019 00:22

oli-obk reviewed Mar 1, 2019

View reviewed changes

src/librustc_macros/src/hash_stable.rs Outdated Show resolved Hide resolved

Zoxc added 4 commits March 5, 2019 00:36

Create a derive macro for HashStable

97431a4

Allow linking to a proc macro on the target in metadata and still use…

c3c1c8d

… a host proc macro to execute them

Bootstrap changes

e501a87

Encode proc macro stability

70156fb

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Mar 7, 2019

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 7, 2019

bors added the merged-by-bors This PR was explicitly merged by bors. label Mar 8, 2019

bors merged commit f2ef283 into rust-lang:master Mar 8, 2019

Zoxc deleted the stable-hash-macro-simple branch March 8, 2019 17:57

mati865 mentioned this pull request Mar 11, 2019

error: unknown debugging option: dual-proc-macros when running cross tests #59110

Closed

eddyb reviewed Mar 13, 2019

View reviewed changes

Xanewok mentioned this pull request Apr 19, 2019

save-analysis: Use serde instead of libserialize to dump JSON data #60053

Merged

ehuss mentioned this pull request Sep 21, 2019

Tracking issue for dual-proc-macros #64671

Open

michaelwoerister removed their assignment Jun 10, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create a derive macro for HashStable and allow proc macros in rustc #58013

Create a derive macro for HashStable and allow proc macros in rustc #58013

Zoxc commented Jan 30, 2019

Zoxc commented Jan 30, 2019

michaelwoerister commented Jan 31, 2019

alexcrichton commented Feb 1, 2019

Zoxc commented Feb 1, 2019

This comment has been minimized.

mark-i-m Feb 4, 2019

oli-obk Mar 5, 2019

alexcrichton commented Feb 14, 2019

Zoxc commented Feb 15, 2019

vext01 commented Feb 20, 2019

mati865 commented Feb 20, 2019

Zoxc commented Feb 23, 2019

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Zoxc commented Mar 1, 2019

rust-highfive commented Mar 7, 2019

pietroalbini commented Mar 7, 2019

bors commented Mar 7, 2019

pietroalbini commented Mar 7, 2019

rust-highfive commented Mar 7, 2019

Zoxc commented Mar 7, 2019

Zoxc commented Mar 8, 2019

bors commented Mar 8, 2019

pietroalbini commented Mar 8, 2019

bors commented Mar 8, 2019

michaelwoerister commented Mar 11, 2019

eddyb Mar 13, 2019

eddyb Mar 13, 2019

ehuss Sep 21, 2019

jyn514 commented May 1, 2021

eddyb commented May 3, 2021

jyn514 commented May 3, 2021

eddyb commented May 3, 2021 •

edited

Loading

Create a derive macro for HashStable and allow proc macros in rustc #58013

Create a derive macro for HashStable and allow proc macros in rustc #58013

Conversation

Zoxc commented Jan 30, 2019

Zoxc commented Jan 30, 2019

michaelwoerister commented Jan 31, 2019

alexcrichton commented Feb 1, 2019

Zoxc commented Feb 1, 2019

This comment has been minimized.

mark-i-m Feb 4, 2019

Choose a reason for hiding this comment

oli-obk Mar 5, 2019

Choose a reason for hiding this comment

alexcrichton commented Feb 14, 2019

Zoxc commented Feb 15, 2019

vext01 commented Feb 20, 2019

mati865 commented Feb 20, 2019

Zoxc commented Feb 23, 2019

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Zoxc commented Mar 1, 2019

rust-highfive commented Mar 7, 2019

pietroalbini commented Mar 7, 2019

bors commented Mar 7, 2019

pietroalbini commented Mar 7, 2019

rust-highfive commented Mar 7, 2019

Zoxc commented Mar 7, 2019

Zoxc commented Mar 8, 2019

bors commented Mar 8, 2019

pietroalbini commented Mar 8, 2019

bors commented Mar 8, 2019

michaelwoerister commented Mar 11, 2019

eddyb Mar 13, 2019

Choose a reason for hiding this comment

eddyb Mar 13, 2019

Choose a reason for hiding this comment

ehuss Sep 21, 2019

Choose a reason for hiding this comment

jyn514 commented May 1, 2021

eddyb commented May 3, 2021

jyn514 commented May 3, 2021

eddyb commented May 3, 2021 • edited Loading

eddyb commented May 3, 2021 •

edited

Loading