Clean Compile times

[semtexzv]

While bevy seems to be much faster than Amethyst to compile, It still takes several minutes to chew through all the dependencies. On my laptop, I can build the simple game that references bevy in around 3 minutes. Most of these 3 minutes however, are spent on 4 different crates:

• ash - 47s
• wgpu - 40s
• syn - 37s
• bevy_ecs - 29s

It would be nice to use the current inflow of new users and contributors to optimize some of these crates in order to reduce biggest offenders in compile time.

Also bevy_ecs currently uses 3 nested macro calls, in order to generate necessary impls supporting fun.system() syntax. If possible, we should try to improve this/find an alternative, since the display messages aren't that good if user uses something that isn't resource. Also touches on #25

[cart]

First I want to point out that while clean compile times are a nice thing to improve, doing so isn't my primary focus because what developers really "feel" when developing is iterative compile times (making small changes in an already-compiled project, then testing those changes).

I also want to be very careful with how we approach the topic of "improving clean compile times" in other projects. Each project owner is operating under different constraints, and it is unfair to expect them to operate under our constraints in the interest of slightly faster clean compiles.

A great example of this is the syn crate. Yes, it adds ~40 seconds to a clean compile, but it also makes writing proc macros much easier. Many useful libs also use it, making it very hard to remove from a dependency tree. It would be unfair to ask a project like wgpu to remove syn from their dependency tree because it would make their lives much harder. I've looked into removing syn from the Bevy dependency tree in the past and it would require rewriting basically every foundational library, which is not worth it to me and bad for the ecosystem as a whole.

On the topic of bevy_ecs, the only real alternatives to the approach we're using:

1. use proc macros (which also have a compile time cost, which would then get paid at every system declaration which could easily be a net loss in clean compile times and more importantly, would increase iterative compile times).
2. use a single "tuple impl" for all resources / components :

fn system((a, b, c): (Res<A>, ResMut<B>, Res<C>), (x, y, z) (&X, Mut<Y>, &Z)) { }

This cuts down on the implementations required (bevy_ecs would compile in ~6 seconds), but it also reduces the ergonomics and clarity of systems.

[semtexzv]

I'd say that the long compile times are maybe not felt directly but correlate with amount of bloat the package brings, and reducing this bloat would be beneficial nonetheless. This could be done by more granularly enabling features, pruning nested dependencies.

Yes, those were the 2 alternatives I've encountered in my own research into the ecs ergonomics. However, I was thinking, whether it would be possible to unify different system types by flattening into 2 levels. Implementing SystemArg for different argument types - Commands, Res<T>/ResMut<T> where T : Resource, Query<(C1, C2,...)> and implementing System for SystemArg tuples.

[cart]

Thats an interesting idea I haven't considered in the past. But I think that would require making queries less statically typed? How would we know how to convert a function signature like

fn system(c: Commands, r1: Res<R1>, r2: Res<R2>, query: Query<(&A, &mut B)>))

Which the system impl would read like this: (SystemArg, SystemArg, SystemArg, SystemArg)

Back into these types: (Commands, ResourceQuery<(&R1, &R2)>, query: Query<(&A, &mut B)>)

which are what the ecs needs to operate? World expects a Query type to iterate, Resources need a ResourceQuery, and we need to run setup operations on Commands.

[zicklag]

Cargo Watt might be a way to reduce syn compile times?

[cart]

It can definitely do that, but we only save on compile time if every dependency I'm the tree also uses cargo watt. Otherwise it will actually increase compile times because we'll compile normal syn and the watt runtime.

What I really want is for rust distributions to include a precompiled syn. That solves the problem in a cleaner / absolute way. Syn is a foundational lib with a high compile time price. It would be cool if the Rust team acknowledged that.

[Plecra]

If the bevy_ecs compile times mostly come down to those macro expansions, would you expect it to slowly improve over time as Rust's type system gets stronger with variadic generics e.t.c.?

It's hardly a perfect solution, but I think it makes sense to accept some flaws as temporary limitations of Rust which don't need to solved in the short term.

[zicklag]

Just a note on something that I noticed. I'm pretty sure that the bevy_ecs compile time ( at least the re-compile time ) shot up quite a bit with the latest addition of lockless queries, or something very close to that commit. I haven't done any scientific testing, but rust analyzer and cargo builds of that crate seemed to take a good chunk longer than it did before from an unscientific "feeling".

I just noticed it because I was working on the bevy_ecs and after pulling master compiles felt much slower.

[cart]

Yup it definitely went up because there are more versions of function impls being generated. We have a plan to (hopefully) significantly cut down bevy_ecs compile times by impling for (Param, Param, Param) instead of (Query, Query, QuerySet), but idk yet if it will actually work.

[zicklag]

OK, cool. I'd for sure rather have functionality than faster compile times for something that should be so rarely touched by actual users, but glad it's on the radar. 👍

[cart]

We just got a nice compile time boost by using the fn system(Param, Param, Param) {} approach (#798).

Clean compiles will always be a moving target and I don't want to keep issues open with no tangible closure criteria, so I'm closing this issue. If anyone wants to open issues for specific clean-compile improvement areas, feel free to do so.

[semtexzv]

Definitely closable. Thank you for the hard work 👍

[razaamir]

What are you guys working on? Raza

On Mon, Nov 9, 2020 at 1:39 AM Carter Anderson ***@***.***> wrote: We just got a nice compile time boost by using the fn system(Param, Param, Param) {} approach (#798 <#798>). Clean compiles will always be a moving target and I don't want to keep issues open with no tangible closure criteria, so I'm closing this issue. If anyone wants to open issues for specific clean-compile improvement areas, feel free to do so. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#604 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHGK5TAEZ3UHMKHFL3RS443SO36YBANCNFSM4R6I6UOQ> .

-- Raza Amir