guaranteed tail call elimination

[thestinger]

No description provided.

[bstrie]

When I asked @thestinger how this proposal deals with the widly-publicized email from a year ago explaining why Rust can't support TCE (https://mail.mozilla.org/pipermail/rust-dev/2013-April/003557.html), his response was along the lines of "LLVM supports this now, so everything is okay." But I'd still like to see the following point from Graydon's email elaborated upon in more detail:

Tail calls "play badly" with deterministic destruction. Including deterministic drop of ~ boxes. Not to say that they're not composable, but the options for composing them are UI-awkward, performance-penalizing, semantics-complicating or all of the above.

What does he mean by "play badly", and what implications does this have if this feature is accepted?

[emberian]

I am in favor of this, however should probably clarify what happens to locals. The way I envision it, locals will be destroyed before the tailcall.

[bstrie]

Of course, I'm hugely in favor of this if we can pull it off. It promises to be a PR windfall as well.

[thestinger]

@bstrie: The semantics are equivalent to return immediately after the call, so there's no way to destroy locals after the callee runs. Instead, locals could be destroyed before the call unless they are passed into it by-value but that's a bit weird.

[bstrie]

Not that I'm against accepting this feature merely out of concern for some hypothetical alternative implementation, but how much work are we talking here? Using GCC as an example, say.

[thestinger]

The commit adding x86 support for musttail was quite small. I don't know how much work this would involve for GCC. The feature is designed to be portable, but involves work for each new architecture.

[bstrie]

Thanks. Basically I just want to make sure that it's not so much work that our choices effectively boil down to "never implement Rust on any other backend" vs. "force all alternative implementations to completely ignore a language feature".

[bstrie]

Re: using become rather than the long-reserved be, @thestinger's reply on IRC is that be gives no indication as to its meaning, whereas become implies "this function becomes that function". Personally I don't care either way, but become gives me an ineffable satisfaction due to having the same number of characters as return (though this could easily be construed as a mark against it as well).

[bstrie]

Can you elaborate on this last point, of our current calling convention getting in the way?

[thestinger]

The current convention is to perform a copy in the caller and pass a pointer to the alloca, which is clearly not allowed. AFAICT only passing large values by-value would work.

[steveklabnik]

I love become for reasons that don't really matter.

[matthieu-m]

This seems evil: imagine converting a return into a become and suddenly you get issues because the behavior changed ?

Instead, I would propose to make it visible: if you want to have locals that have a destructor: no problem, but wrap them in an explicit {} block so that it's visible they get destroyed before the become instruction.

[thestinger]

Well that's why it's in unresolved questions rather than part of the proposal. The restriction is painful, and it's unclear if there's a sane way of relaxing it.

[bstrie]

I recommend that we retain all of the restrictions for the purpose of this RFC. Seeing TCE get used in practice will give us data on which relaxations to pursue, and there's no rush since they're all totally backwards-compatible.

[dherman]

I don't think it's evil; because become is a control effect, as return is, it's explicit. But it's a different control effect and does things in different orders.

• return: evaluate argument, then return
• become: return, then evaluate argument

Edit: note that the alternative is even more of a refactoring hazard: if you simply can't do a become in the presence of destructors then non local changes break code.

[dherman]

One thing to keep in mind: in languages with implicit proper tail calls, you can do convenient things like branch on two alternative calls in tail position:

if p() then { foo(a, b) } else { bar(x, y, z) }

Doing that kind of thing requires you to proliferate become keywords:

if p() then { become foo(a, b) } else { become bar(x, y, z) }

The thing is, I don't see a way to allow become to accept arbitrary expressions in its argument (as opposed to syntactically requiring a function call expression). You can't have the language statically reject function calls inside that expression that appear in non-tail position since e.g. you want to allow p() in

become (if p() then { foo(a, b) } else { bar(x, y, z) })

And frankly this defeats the purpose of having an explicit tail calling form in the first place; tail calls inside the body become implicit tail calls. So ISTM the only reasonable syntax is to require the argument of become to be a call.

Since we'd expect tail calls to be less common (and loops more common) in Rust than in languages like Scheme that rely on tail calls as the only form of iteration, it might be fine to have a bit of extra become noise. But I don't know if in practice tail-recursive algorithms might get unwieldy.

Probably that was the intention of the proposal (maybe it said so and I missed it?) but anyway I'm spelling out some consequences.

[matthieu-m]

I do not understand why become if p() { foo(a, b) } else { bar(x, y, z) }; could not be accepted.

In a way, I see become if p() { foo(a, b) } else { bar(x, y, z) }; as syntactic sugar for if p() { become foo(a, b); } else { become bar(x, y, z); } and I could see the Rust front-end "lowering" become easily either in if expressions or match expressions.

Of course, I would suggest starting without it, it is backward compatible anyway.

[thestinger]

It would be quite confusing if become before an expression caused the calls later in the expression to destroy stuff earlier.

[bstrie]

I must be missing something, how would it be possible to have both become foo(a, b) and become bar(x, y, z) in the same function? Isn't the function signature of the new function required to match that of the current function?

[thestinger]

@bstrie: That's right, I didn't notice from the example.

[dherman]

@bstrie: "Isn't the function signature of the new function required to match that of the current function?"

That's a shame. You can't make it work for arbitrary signatures (obviously with matching return type), with an appropriate amount of stack frame shuffling? Is this based on an LLVM limitation, a lack of desire to implement the shuffling, or a belief that the shuffling is "un-Rusty"? I understand if the latter but it is an unproven middle ground of expressiveness. It also needs to be spelled out just what constitutes matching signatures (eg a monomorphic function tail calling a compatible polymorphic function).

[dherman]

@matthieu-m: The purpose of an explicit tail calling operator (become) in the first place, as distinct from just automatically making function calls in tail position behave as proper tail calls, is that the semantics of performing a function call as a tail call or not is observably different, and the performance model is radically different. If we didn't care about that we wouldn't design it as an explicit operator in the first place. But allowing function call nested inside a become expression to be implicit tail calls would be going right back to an implicit design. Only, it'd be a mucky middle ground, where some function calls are implicitly tail calls and others aren't. So I think the most sensible design is to make become syntactically restricted to take a call expression.

[thestinger]

@dherman: The ABI has to match for musttail in order to make the cross-platform guarantee. LLVM had tail call optimization before musttail, but it wasn't reliable or portable. In Rust the sanest way to expose this is requiring the types to precisely match. It would be a severe backwards compatibility issue if adding a private field invalidated code using the type.

[alexcrichton]

Closing, this was discussed a few weeks ago where we decided to postpone this.

[graue]

Updated discussion link for any other curious people reading this :)

[mrhota]

updated link for the meeting minutes where the postponement was discussed: https://github.com/rust-lang/meeting-minutes/blob/master/weekly-meetings/2014-05-20.md

[petrochenkov]

The next iteration - #1888 (also postponed).