SIP-47 - Clause Interleaving #47

Sporarum · 2022-08-17T08:32:00Z

Edit: Other PRs have since modified this document, for the latest version, go here

We propose to generalize method signatures to allow any number of type parameter lists, interleaved with term parameter lists and using parameter lists. As a simple example, it would allow to define

def pair[A](a: A)[B](b: B): (A, B) = (a, b)

Here is also a more complicated and contrived example that highlights all the possible interactions:

def foo[A][B <: A](using a: A)(b: B)[C <: a.type, D](cd: (C, D))[E]: Foo[A, B, C, D, E]

(taken from the summary section)

kitbellew · 2022-08-29T18:55:48Z

We propose to generalize method signatures to allow any number of type parameter lists, interleaved with term parameter lists and using parameter lists. As a simple example, it would allow to define
def pair[A](a: A)[B](b: B): (A, B) = (a, b)
Here is also a more complicated and contrived example that highlights all the possible interactions:
def foo[A][B <: A](using a: A)(b: B)[C <: a.type, D](cd: (C, D))[E]: Foo[A, B, C, D, E]
(taken from the summary section)

what about

def pair[A1, A2](a1: A1, a2: A2)[B1, B2, B3](b1: B1, b2: B2, b3: B3): (A1, B3) = (a1, b3)

or

def pair[A1, A2][B1, B2, B3](a1: A1, a2: A2)(b1: B1, b2: B2, b3: B3): (A1, B3) = (a1, b3)

Sporarum · 2022-08-29T19:07:18Z

Yes of course !
If you haven't already, do read the "changed file" (even though it's really an added file), it should give you a full picture of what this feature allows

And if anything's not clear, feel free to ask
(if possible as a PR comment, so I can reply more easily)

Kordyjan · 2022-09-01T09:10:56Z

While I agree that what is proposed in this SIP would be a valuable addition to Scala. I remember myself having to write intermediate classes with apply methods to circumvent the single type param clause limitation.

However, right now in Scala 3 there are some ways to achieve that. For example, if I'm not mistaken:

def getOrElse(key: Key)[V >: key.Value](default: V): V = ???

from the proposal, can be right now defined as

def getOrElse: (k:Key) => [V >: k.Value] => (default: V) => V = 
    (k: Key) => [V] => (default: V) => ???

Of course, the current approach is much less visually pleasing and has some (quite arbitrary) limitations. For example, it is not legal to declare

def makePair: [A] => [B] => (A, B) = ???

while I assume your proposal would allow

def makePair[A][B]: (A, B) = ???

I think the proposal itself should address that. It should make a statement about why we should add a new thing to the language that will have only very niche use cases, instead of improving already existing mechanisms.

Sporarum · 2022-09-02T05:56:52Z

@Kordyjan, thank you for your feedback !
I added a section on that, let me know what you think

Sporarum · 2022-09-08T12:23:39Z

@Kordyjan I added a section addressing what you asked, what do you think ?

Kordyjan · 2022-09-08T12:53:21Z

Sorry for not answering earlier. The section indeed addresses what I thought was missing in the proposal.

About the proposal itself: I think it is well thought-out. It feels much more like lifting an arbitrary limitation than adding a new feature to the language. As I said, it would have made much code that I wrote in the past clearer and easier to follow. My only fear was that it would make it possible to write unnecessarily complicated signatures for a small benefit, making libraries relying heavily on this feature scary for the newcomers. However, I haven't found anything worse than what the current language features allow library authors to create.

Therefore I'm provisionally voting to accept this proposal. However, I'm curious about the opinions of other reviewers.

Sporarum · 2022-09-09T08:04:19Z

@soronpo, @chrisandrews-ms, I have not heard from you, what do you think of the proposal ?

soronpo · 2022-09-09T09:17:28Z

Apologies, I remembered I was not assigned to it at the beginning, and didn't notice I was assigned after.
For motivation, you should mention that it resolves an extension method feature limitation (need to find the relevant issue).
I'm in favor of accepting this proposal, but I should mention that to accept it after the implementation stage is a much greater challenge. The feature interaction and possible combinations here are massive! To be able to properly test all that and report good error messages may be very difficult.

Sporarum · 2022-09-09T10:10:35Z

extension method feature limitation

Do you mean extension [T](...) def foo[U](...) ?
If so this has already been added, and at the time, the codebase was refactored to deal with arbitrarily many type clauses, so your point:

accept it after the implementation stage is a much greater challenge

Might not actually be that challenging, as the implementation boils down to "modify the parser", since the generated tree is already valid in the rest of the compiler

Sporarum · 2022-09-09T10:16:10Z

I will add two things to the proposal later today:
The fact that

def getOrElse(k:Key): [V >: k.Value] => (default: V) => V = 
    (k: Key) => [V] => (default: V) => ???

Poses problem with overloading resolution, if there is another def getOrElse(k:Key)...

Add a section clarifying that Clause Interleaving also applies to the right-hand side of extension methods

soronpo · 2022-09-09T10:43:18Z

Do you mean extension [T](...) def foo[U](...) ?
If so this has already been added, and at the time, the codebase was refactored to deal with arbitrarily many type clauses

I remembered scala/scala3#14451 but now I see it will not be resolved.

Might not actually be that challenging, as the implementation boils down to "modify the parser", since the generated tree is already valid in the rest of the compiler

My guess is that type inference and overloading checks are not that generic. Hoping to be proven wrong.

Also explains how overloading resolutions make the alternatives unattractive

chrisandrews-ms · 2022-09-13T19:41:28Z

I'm in favor of accepting too. I can see the utility of this proposal, especially for partial inference of type parameters, a problem that has come up a few times in our Scala 2 codebase. The SIP seems well thought out and uncontroversial.

A couple minor questions:

Am I right in assuming that methods with interleaved clauses wouldn't be callable from Scala 2.13 (using the TASTy reader)? I don't see that as a problem (since it also applies to other Scala 3 features) but it would be good to document that limitation.
I note that there's already an experimental implementation PR. Is that ready to merge if this SIP passes the first vote? It would be good to be able to experiment and look for edge cases.

Sporarum · 2022-09-14T08:17:25Z

1. Am I right in assuming that methods with interleaved clauses _wouldn't_ be callable from Scala 2.13 (using the TASTy reader)? I don't see that as a problem (since it also applies to other Scala 3 features) but it would be good to document that limitation.

That is correct, I'll update the proposal

2. I note that there's already an experimental implementation PR. Is that ready to merge if this SIP passes the first vote? It would be good to be able to experiment and look for edge cases.

The implementation is pretty much ready, yes, but it adds it under experimental, so it will only be available in nightly and with an import, which would give us some time to experiment

Kordyjan · 2022-09-27T09:59:51Z

I maintain my recommendation that the Committee should vote during the next meeting and accept the proposal.

soronpo · 2022-09-27T10:01:37Z

I maintain my recommendation that the Committee should vote during the next meeting and accept the proposal.

Same

chrisandrews-ms · 2022-09-27T10:10:23Z

I also recommend to accept

raulraja · 2022-09-27T10:32:27Z

I maintain my recommendation that the Committee should vote during the next meeting and accept the proposal.

Also, recommend accepting

adampauls · 2022-10-02T11:37:00Z

The committee believes that Named Typed Parameter Application (a future SIP) is the better approach to answer use-cases where adjacent type parameter clauses are useful.

Out of curiosity, why does the committee believe these two answers should be mutually exclusive? They are both available for non-type parameters.

soronpo · 2022-10-02T12:04:23Z

The committee believes that Named Typed Parameter Application (a future SIP) is the better approach to answer use-cases where adjacent type parameter clauses are useful.

Out of curiosity, why does the committee believe these two answers should be mutually exclusive? They are both available for non-type parameters.

To prevent API entropy and abuse by "creative" developers. In short, it was easier to get approval for the strict version than the general version. It's always possible to submit the following SIP in the future, if indeed there are good use-cases. Small well-calculated and steady increments are good for languages.

odersky

I also recommend accept.

Kordyjan · 2022-10-21T14:13:27Z

The proposal was accepted during our last SIP meeting!

This means that the Committee agrees that the proposal can be implemented as an experimental feature.

Sporarum · 2022-10-21T14:42:49Z

Thank you, I'm happy to hear that !
The PR for the feature (already in experimental) can be found here:
scala/scala3#14019
I believe it is ready or almost ready, and therefore requires only review from the compiler team
(And I will address the reviews personally)

kitbellew · 2022-10-25T23:34:01Z

content/clause-interleaving.md

+DefDef                 ::=  DefSig [‘:’ Type] ‘=’ Expr
+DefSig                 ::=  id [DefParamClauses] [DefImplicitClause]
+DefParamClauses        ::=  DefParamClauseChunk {DefParamClauseChunk}
+DefParamClauseChunk    ::=  [DefTypeParamClause] TermOrUsingParamClause {TermOrUsingParamClause}


does this mean that this is not valid anymore:

def foo[A](implicit a: A) = ...

Oh, nicely spotted, my intent was for it to be allowed,
but it appears I made a mistake with the grammar

Since the implementation team (which is also myself) can make changes to the proposal, I believe this can fall under this umbrella
I will think about what's the clearest way to fix the grammar

This should fix it:

DefDef ::= DefSig [‘:’ Type] ‘=’ Expr - DefSig ::= id [DefParamClauses] [DefImplicitClause] + DefSig ::= id [DefParamClauses] [DefTypeParamClause] [DefImplicitClause] DefParamClauses ::= DefParamClauseChunk {DefParamClauseChunk} DefParamClauseChunk ::= [DefTypeParamClause] TermOrUsingParamClause {TermOrUsingParamClause}

There is another quirk of this definition however, it allows to parse def f(a: Int)(b: Int) in two different ways (one vs two DefParamClauseChunk), is this an issue ?

I didn’t have a look at the complete grammar but what you propose does not look correct, what about the following?

DefDef ::= DefSig [‘:’ Type] ‘=’ Expr DefSig ::= id [DefParamClauses] DefParamClauses ::= DefParamClauseChunk {DefParamClauseChunk} DefParamClauseChunk ::= DefTypeParamClause | TermOrUsingParamClause | DefImplicitClause

Alas, this allows two consecutive type clauses, or multiple implicit clauses.

here's another option:

DefSig ::= id [DefParamClauses] [DefImplicitClause] DefParamClauses ::= DefParamClauseChunkTypeOpt {DefParamClauseChunkTypeReq} DefParamClauseChunkTypeOpt ::= [DefTypeParamClause] {TermOrUsingParamClause} DefParamClauseChunkTypeReq ::= DefTypeParamClause {TermOrUsingParamClause}

ah, that doesn't work either... perhaps this:

DefSig ::= id [DefParamClauses] [DefImplicitClause] DefParamClauses ::= [DefTypeParamClause] {DefParamClauseChunk} [TermOrUsingParamClause] DefParamClauseChunk ::= TermOrUsingParamClause {TermOrUsingParamClause} DefTypeParamClause

This should fix it:

DefDef ::= DefSig [‘:’ Type] ‘=’ Expr - DefSig ::= id [DefParamClauses] [DefImplicitClause] + DefSig ::= id [DefParamClauses] [DefTypeParamClause] [DefImplicitClause] DefParamClauses ::= DefParamClauseChunk {DefParamClauseChunk} DefParamClauseChunk ::= [DefTypeParamClause] TermOrUsingParamClause {TermOrUsingParamClause}

There is another quirk of this definition however, it allows to parse def f(a: Int)(b: Int) in two different ways (one vs two DefParamClauseChunk), is this an issue ?

I didn’t have a look at the complete grammar but what you propose does not look correct

@julienrf Could you expand on this, I can't manage to find a failing example with the above syntax

I talked about it some more with @julienrf and we came to a very easy and clean solution:
Use the pre-"no type currying" syntax, and add an end of line comment that says "and type clauses cannot be adjacent"

- DefSig ::= id [DefParamClauses] [DefImplicitClause] + DefSig ::= id [DefParamClauses] [DefImplicitClause] -- and two DefTypeParamClause cannot be adjacent - DefParamClauses ::= DefParamClauseChunk {DefParamClauseChunk} - DefParamClauseChunk ::= [DefTypeParamClause] TermOrUsingParamClause {TermOrUsingParamClause} - TermOrUsingParamClause ::= DefTermParamClause - | UsingParamClause + DefParamClauses ::= DefParamClause { DefParamClause } + DefParamClause ::= DefTypeParamClause + | DefTermParamClause + | UsingParamClause

I think this syntax is easier to understand than the formal one

@julienrf How should I go about updating the proposal ?

Please open a PR in this repository with your changes.

kitbellew · 2022-10-27T09:32:44Z

content/clause-interleaving.md

+DefDef                 ::=  DefSig [‘:’ Type] ‘=’ Expr
+DefSig                 ::=  id [DefParamClauses] [DefImplicitClause]
+DefParamClauses        ::=  DefParamClauseChunk {DefParamClauseChunk}
+DefParamClauseChunk    ::=  [DefTypeParamClause] TermOrUsingParamClause {TermOrUsingParamClause}


Alas, this allows two consecutive type clauses, or multiple implicit clauses.

here's another option:

DefSig ::= id [DefParamClauses] [DefImplicitClause] DefParamClauses ::= DefParamClauseChunkTypeOpt {DefParamClauseChunkTypeReq} DefParamClauseChunkTypeOpt ::= [DefTypeParamClause] {TermOrUsingParamClause} DefParamClauseChunkTypeReq ::= DefTypeParamClause {TermOrUsingParamClause}

kitbellew · 2022-10-27T09:33:01Z

content/clause-interleaving.md

+UsingParamClause       ::=  [nl] ‘(’ ‘using’ (DefTermParams | FunArgTypes) ‘)’
+DefImplicitClause      ::=  [nl] ‘(’ ‘implicit’ DefTermParams ‘)’


I'd also rename

TermOrUsingParamClause ::= DefTermParamClause | UsingParamClause UsingParamClause ::= [nl] ‘(’ ‘using’ (DefTermParams | FunArgTypes) ‘)’ DefImplicitClause ::= [nl] ‘(’ ‘implicit’ DefTermParams ‘)’

to

DefNonTypeParamClause ::= DefTermParamClause | DefUsingParamClause DefUsingParamClause ::= [nl] ‘(’ ‘using’ (DefTermParams | FunArgTypes) ‘)’ DefImplicitParamClause ::= [nl] ‘(’ ‘implicit’ DefTermParams ‘)’

As noted in scala#47, the grammar did not correspond to the rest of the proposal, this PR solves that

@smarter

This aims to add the ability to declare functions with many clauses of type parameters, instead of at most one, and to allow those clauses to be interleaved with term clauses: ```scala def foo[A](x: A)[B](y: B) = (x, y) ``` All user-facing details can be found in the [Scala 3 new features doc](https://github.com/lampepfl/dotty/blob/2a079f92bfc05420e90987d908c41589ac94418d/docs/_docs/reference/other-new-features/generalized-method-syntax.md), and in the [SIP proposal](scala/improvement-proposals#47) The implementation details are described below in the commit messages The community's opinion of the feature can be found on [the scala contributors forum](https://contributors.scala-lang.org/t/clause-interweaving-allowing-def-f-t-x-t-u-y-u) (note however that the description there is somewhat outdated) ### Dependencies This feature has been [accepted by the SIP committee for implementation](scala/improvement-proposals#47 (comment)), it can therefore become part of the language as an experimental feature at any time. The feature will be available with `import scala.language.experimental.clauseInterleaving` ### How to make non-experimental `git revert` the commits named "Make clause interleaving experimental" and "Add import to tests" ### Future Work 1. Implement given aliases with clause interweaving: (to have types depends on terms) ```scala given myGiven[T](using x: T)[U](using y: U) = (x, y) ``` 2. Add interleaved clauses to the left-hand side of extension methods: ```scala extension (using Int)[A](using A)(a: A)[B](using B) def foo: (A, B) = ??? ``` 3. Investigate usefulness/details of clause interweaving for classes and type currying for types: ```scala class Foo[A](a: A)[B](b: B) new Foo(0)("Hello!") // type: Foo[Int][String] ? type Bar[A][B] = Map[A, B] Bar[Char] // should this mean [B] =>> Bar[Char][B] ? ``` (Started as a semester project with supervision from @smarter, now part of my missions as an intern at the scala center)

As noted in scala#47, the grammar did not correspond to the rest of the proposal, this PR solves that

Add clause-interleaving.md

f025daa

sjrd changed the title ~~Clause Interleaving~~ SIP-47 Clause Interleaving Aug 17, 2022

sjrd changed the title ~~SIP-47 Clause Interleaving~~ SIP-47 - Clause Interleaving Aug 17, 2022

julienrf added stage:design status:under-review labels Aug 18, 2022

julienrf assigned smarter, soronpo, chrisandrews-ms and Kordyjan and unassigned smarter Aug 18, 2022

This was referenced Aug 29, 2022

Adding Clause Interleaving to method definitions scala/scala3#14019

Merged

WIP - Add support for Clause Interleaving scalameta/scalameta#2820

Closed

Add polmorphic function workaround

3486db4

anatoliykmetyuk added status:vote-requested and removed status:under-review labels Sep 9, 2022

Clarify effect on extension methods

d7068db

Also explains how overloading resolutions make the alternatives unattractive

anatoliykmetyuk added the recommendation:accept label Sep 12, 2022

Update clause-interleaving.md

722bbb4

julienrf added recommendation:accept status:vote-requested and removed status:under-review labels Sep 27, 2022

odersky approved these changes Oct 21, 2022

View reviewed changes

Kordyjan added stage:implementation status:waiting-for-implementation and removed stage:design recommendation:accept status:vote-requested labels Oct 21, 2022

Kordyjan merged commit 3ca173c into scala:main Oct 21, 2022

kitbellew reviewed Oct 25, 2022

View reviewed changes

kitbellew reviewed Oct 27, 2022

View reviewed changes

soronpo mentioned this pull request Nov 8, 2022

SIP-49 - Polymorphic Eta-Expansion #49

Merged

Sporarum added a commit to Sporarum/improvement-proposals that referenced this pull request Nov 29, 2022

Fix and simplify grammar in SIP - scala#47

0f2f635

As noted in scala#47, the grammar did not correspond to the rest of the proposal, this PR solves that

Sporarum mentioned this pull request Nov 29, 2022

Fix and simplify grammar in SIP - 47 #53

Merged

odersky pushed a commit to odersky/improvement-proposals that referenced this pull request Jul 18, 2024

Fix and simplify grammar in SIP - scala#47

974a5d7

As noted in scala#47, the grammar did not correspond to the rest of the proposal, this PR solves that

kyouko-taiga added stage:completed status:shipped and removed status:waiting-for-implementation stage:implementation labels Aug 16, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SIP-47 - Clause Interleaving #47

SIP-47 - Clause Interleaving #47

Sporarum commented Aug 17, 2022 •

edited

Loading

kitbellew commented Aug 29, 2022 •

edited

Loading

Sporarum commented Aug 29, 2022

Kordyjan commented Sep 1, 2022 •

edited

Loading

Sporarum commented Sep 2, 2022

Sporarum commented Sep 8, 2022

Kordyjan commented Sep 8, 2022

Sporarum commented Sep 9, 2022

soronpo commented Sep 9, 2022

Sporarum commented Sep 9, 2022

Sporarum commented Sep 9, 2022 •

edited

Loading

soronpo commented Sep 9, 2022

chrisandrews-ms commented Sep 13, 2022

Sporarum commented Sep 14, 2022

Kordyjan commented Sep 27, 2022 •

edited

Loading

soronpo commented Sep 27, 2022

chrisandrews-ms commented Sep 27, 2022

raulraja commented Sep 27, 2022

adampauls commented Oct 2, 2022

soronpo commented Oct 2, 2022

odersky left a comment

Kordyjan commented Oct 21, 2022

Sporarum commented Oct 21, 2022

kitbellew Oct 25, 2022

Sporarum Oct 26, 2022

Sporarum Oct 26, 2022 •

edited

Loading

julienrf Oct 27, 2022 •

edited

Loading

kitbellew Oct 27, 2022

kitbellew Oct 27, 2022

Sporarum Nov 28, 2022 •

edited

Loading

Sporarum Nov 28, 2022

Sporarum Nov 29, 2022 •

edited

Loading

julienrf Nov 29, 2022

kitbellew Oct 27, 2022

kitbellew Oct 27, 2022

		UsingParamClause ::= [nl] ‘(’ ‘using’ (DefTermParams \| FunArgTypes) ‘)’
		DefImplicitClause ::= [nl] ‘(’ ‘implicit’ DefTermParams ‘)’

SIP-47 - Clause Interleaving #47

SIP-47 - Clause Interleaving #47

Conversation

Sporarum commented Aug 17, 2022 • edited Loading

Edit: Other PRs have since modified this document, for the latest version, go here

kitbellew commented Aug 29, 2022 • edited Loading

Sporarum commented Aug 29, 2022

Kordyjan commented Sep 1, 2022 • edited Loading

Sporarum commented Sep 2, 2022

Sporarum commented Sep 8, 2022

Kordyjan commented Sep 8, 2022

Sporarum commented Sep 9, 2022

soronpo commented Sep 9, 2022

Sporarum commented Sep 9, 2022

Sporarum commented Sep 9, 2022 • edited Loading

soronpo commented Sep 9, 2022

chrisandrews-ms commented Sep 13, 2022

Sporarum commented Sep 14, 2022

Kordyjan commented Sep 27, 2022 • edited Loading

soronpo commented Sep 27, 2022

chrisandrews-ms commented Sep 27, 2022

raulraja commented Sep 27, 2022

adampauls commented Oct 2, 2022

soronpo commented Oct 2, 2022

odersky left a comment

Choose a reason for hiding this comment

Kordyjan commented Oct 21, 2022

Sporarum commented Oct 21, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Sporarum Oct 26, 2022 • edited Loading

Choose a reason for hiding this comment

julienrf Oct 27, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Sporarum Nov 28, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Sporarum Nov 29, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Sporarum commented Aug 17, 2022 •

edited

Loading

kitbellew commented Aug 29, 2022 •

edited

Loading

Kordyjan commented Sep 1, 2022 •

edited

Loading

Sporarum commented Sep 9, 2022 •

edited

Loading

Kordyjan commented Sep 27, 2022 •

edited

Loading

Sporarum Oct 26, 2022 •

edited

Loading

julienrf Oct 27, 2022 •

edited

Loading

Sporarum Nov 28, 2022 •

edited

Loading

Sporarum Nov 29, 2022 •

edited

Loading