"binary patterns cannot be matched in parallel using =" on an assignment #6348

RobinMorisset · 2022-10-05T14:35:42Z

foo(X) ->
  <<>> = (<<>> = 42).

is rejected by erlc with test.erl:5:3: binary patterns cannot be matched in parallel using '='.

This surprised me because I expected it to be parsed as follows:

foo(X) ->
  X = (<<>> = 42),
  <<>> = X.

Which is accepted.

Even weirder, the following is accepted:

foo(X) ->
  <<>> = begin <<>> = 42 end.

Showing that (Expr) and begin Expr end are not treated equivalently by the compiler.

Looking at the documentation in

I could not find anything warning of this rather surprising behavior.

So I don't know whether this is a bug in erlc or just a missing piece of documentation (or maybe it is documented somewhere else?).

The text was updated successfully, but these errors were encountered:

bjorng · 2022-10-06T04:39:43Z

You have encountered an undocumented limitation in the pattern matching compilation.

Pattern matching of multiple patterns is done in parallel. So if we have:

Pat1 = Pat2 = Pat3 ... PatN = Expr

all the patterns will be consolidated into a single pattern:

Pat = Expr

For example:

{Tag, Value} = E = {ok, N} = Expr

will be consolidated like so:

{ok=Tag, Value=N}=E = Expr

Parentheses in the source code are not explicitly represented in the parse tree (abstract code). Parentheses used in patterns will only change in which order multiple patterns are consolidated. The end result will always be the same.

begin ... end, on the other hand, has an explicit representation in the parse tree, and when used in patterns will force the pattern matching to be evaluated sequentially.

The consolidation of patterns breaks down for binary patterns in parallel. When the binary syntax was first introduced in Erlang/OTP R7, there was no way to express the consolidation of parallel binary patterns in Core Erlang for patterns such as the one in this example:

bar(Bin) ->
    <<A:8,B:24>> = <<C:16,D:16>> = Bin,
    {A, B, C, D}.

I will have to think about what do. I see two possibilities:

Document the limitation.
Make parallel matching work for binary patterns (and map patterns). Changes made to the compiler in EEP 52: Allow key and size expressions in map and binary matching #2521 has probably made that feasible without too much effort.

Closes erlang#6348

There has always been an implementation limitation for matching of binaries (for technical reasons). For example: foo(Bin) -> <<A:8>> = <<X:4,Y:4>> = Bin, {A,X,Y}. This would fail to compile with the following message: t.erl:5:5: binary patterns cannot be matched in parallel using '=' % 5| <<A:8>> = <<X:4,Y:4>> = Bin, % | ^ This commit lifts this restriction, making the example legal. A restriction for map matching is also lifted, but before we can describe that, we'll need a digression to talk about the `=` operator. The `=` operator can be used for two similar but slightly differently purposes. When used in a pattern, for example in a function head, both the left-hand and right-hand side operands must be patterns: Pattern1 = Pattern2 For example: bar(#{a := A} = #{b := B}) -> {A, B}. The following example will not compile because the right-hand side is not a pattern but an expression: wrong(#{a := A} = #{b => B}) -> {A, B}. t.erl:4:23: illegal pattern % 4| wrong(#{a := A} = #{b => B}) -> {A, B}. % | ^ Used in this context, the `=` operator does not imply that the two patterns are matched in any particular order. Attempting to use a variable matched out on the left-hand side on the right-hand side, or vice versa, will fail: also_wrong1(#{B := A} = #{b := B}) -> {A,B}. also_wrong2(#{a := A} = #{A := B}) -> {A,B}. t.erl:6:15: variable 'B' is unbound % 6| also_wrong1(#{B := A} = #{b := B}) -> {A,B}. % | ^ t.erl:7:27: variable 'A' is unbound % 7| also_wrong2(#{a := A} = #{A := B}) -> {A,B}. % | ^ The other way to use `=` is in a function body. For example: foobar(Value) -> #{a := A, b := B} = #{b => Value}, {A, B}. Used in a function body, the right-hand side of `=` must **not** be a pattern: illegal_foobar(Value) -> #{a := A, b := B} = #{b := Value}, {A, B}. t.erl:20:29: only association operators '=>' are allowed in map construction % 17| #{a := A} = #{b := B}. % | ^ When used in a body context, the value of the `=` operator is the value of its right-hand side operand. When multiple `=` operators are combined, they are evaluted from right to left. That means that any number of patterns can be matched at once: Pattern1 = Pattern2 = ... = PatternN = Expr Given that there is a well-defined evaluation order, one would except that the following example would be legal: baz(M) -> #{K := V} = #{k := K} = M, V. It is not. In Erlang/OTP 25 or earlier, the compilation fails with the following message: t.erl:28:7: variable 'K' is unbound % 28| #{K := V} = #{k := K} = M, % | ^ That restriction is now lifted, making the example legal. Closes erlang#6348

There has always been an implementation limitation for matching of binaries (for technical reasons). For example: foo(Bin) -> <<A:8>> = <<X:4,Y:4>> = Bin, {A,X,Y}. This would fail to compile with the following message: t.erl:5:5: binary patterns cannot be matched in parallel using '=' % 5| <<A:8>> = <<X:4,Y:4>> = Bin, % | ^ This commit lifts this restriction, making the example legal. A restriction for map matching is also lifted, but before we can describe that, we'll need a digression to talk about the `=` operator. The `=` operator can be used for two similar but slightly differently purposes. When used in a pattern, for example in a function head, both the left-hand and right-hand side operands must be patterns: Pattern1 = Pattern2 For example: bar(#{a := A} = #{b := B}) -> {A, B}. The following example will not compile because the right-hand side is not a pattern but an expression: wrong(#{a := A} = #{b => B}) -> {A, B}. t.erl:4:23: illegal pattern % 4| wrong(#{a := A} = #{b => B}) -> {A, B}. % | ^ Used in this context, the `=` operator does not imply that the two patterns are matched in any particular order. Attempting to use a variable matched out on the left-hand side on the right-hand side, or vice versa, will fail: also_wrong1(#{B := A} = #{b := B}) -> {A,B}. also_wrong2(#{a := A} = #{A := B}) -> {A,B}. t.erl:6:15: variable 'B' is unbound % 6| also_wrong1(#{B := A} = #{b := B}) -> {A,B}. % | ^ t.erl:7:27: variable 'A' is unbound % 7| also_wrong2(#{a := A} = #{A := B}) -> {A,B}. % | ^ The other way to use `=` is in a function body. Used in this way, the right-hand side must be an expression: Pattern = Expression For example: foobar(Value) -> #{a := A} = #{a => Value}, A. Used in this context, the right-hand side of `=` must **not** be a pattern: illegal_foobar(Value) -> #{a := A} = #{a := Value}, A. t.erl:18:21: only association operators '=>' are allowed in map construction % 18| #{a := A} = #{a := Value}, % | ^ When used in a body context, the value of the `=` operator is the value of its right-hand side operand. When multiple `=` operators are combined, they are evaluted from right to left. That means that any number of patterns can be matched at once: Pattern1 = Pattern2 = ... = PatternN = Expr Given that there is a well-defined evaluation order, one would except that the following example would be legal: baz(M) -> #{K := V} = #{k := K} = M, V. It is not. In Erlang/OTP 25 or earlier, the compilation fails with the following message: t.erl:28:7: variable 'K' is unbound % 28| #{K := V} = #{k := K} = M, % | ^ That restriction is now lifted, making the example legal. Closes erlang#6348

bjorng · 2022-11-02T08:49:27Z

I have lifted the restrictions on binary and map patterns in the linked pull request.

There has always been an implementation limitation for matching of binaries (for technical reasons). For example: foo(Bin) -> <<A:8>> = <<X:4,Y:4>> = Bin, {A,X,Y}. This would fail to compile with the following message: t.erl:5:5: binary patterns cannot be matched in parallel using '=' % 5| <<A:8>> = <<X:4,Y:4>> = Bin, % | ^ This commit lifts this restriction, making the example legal. A restriction for map matching is also lifted, but before we can describe that, we'll need a digression to talk about the `=` operator. The `=` operator can be used for two similar but slightly differently purposes. When used in a pattern, for example in a function head, both the left-hand and right-hand side operands must be patterns: Pattern1 = Pattern2 For example: bar(#{a := A} = #{b := B}) -> {A, B}. The following example will not compile because the right-hand side is not a pattern but an expression: wrong(#{a := A} = #{b => B}) -> {A, B}. t.erl:4:23: illegal pattern % 4| wrong(#{a := A} = #{b => B}) -> {A, B}. % | ^ Used in this context, the `=` operator does not imply that the two patterns are matched in any particular order. Attempting to use a variable matched out on the left-hand side on the right-hand side, or vice versa, will fail: also_wrong1(#{B := A} = #{b := B}) -> {A,B}. also_wrong2(#{a := A} = #{A := B}) -> {A,B}. t.erl:6:15: variable 'B' is unbound % 6| also_wrong1(#{B := A} = #{b := B}) -> {A,B}. % | ^ t.erl:7:27: variable 'A' is unbound % 7| also_wrong2(#{a := A} = #{A := B}) -> {A,B}. % | ^ The other way to use `=` is in a function body. Used in this way, the right-hand side must be an expression: Pattern = Expression For example: foobar(Value) -> #{a := A} = #{a => Value}, A. Used in this context, the right-hand side of `=` must **not** be a pattern: illegal_foobar(Value) -> #{a := A} = #{a := Value}, A. t.erl:18:21: only association operators '=>' are allowed in map construction % 18| #{a := A} = #{a := Value}, % | ^ When used in a body context, the value of the `=` operator is the value of its right-hand side operand. When multiple `=` operators are combined, they are evaluted from right to left. That means that any number of patterns can be matched at once: Pattern1 = Pattern2 = ... = PatternN = Expr Given that there is a well-defined evaluation order, one would except that the following example would be legal: baz(M) -> #{K := V} = #{k := K} = M, V. It is not. In Erlang/OTP 25 or earlier, the compilation fails with the following message: t.erl:28:7: variable 'K' is unbound % 28| #{K := V} = #{k := K} = M, % | ^ That restriction is now lifted, making the example legal. Closes erlang#6348

…into bjorn/master/primary_preloaded * bjorn/compiler/parallel-match/erlangGH-6348/OTP-18297: Lift restrictions for matching of binaries and maps

There has always been an implementation limitation for matching of binaries (for technical reasons). For example: foo(Bin) -> <<A:8>> = <<X:4,Y:4>> = Bin, {A,X,Y}. This would fail to compile with the following message: t.erl:5:5: binary patterns cannot be matched in parallel using '=' % 5| <<A:8>> = <<X:4,Y:4>> = Bin, % | ^ This commit lifts this restriction, making the example legal. A restriction for map matching is also lifted, but before we can describe that, we'll need a digression to talk about the `=` operator. The `=` operator can be used for two similar but slightly differently purposes. When used in a pattern, for example in a function head, both the left-hand and right-hand side operands must be patterns: Pattern1 = Pattern2 For example: bar(#{a := A} = #{b := B}) -> {A, B}. The following example will not compile because the right-hand side is not a pattern but an expression: wrong(#{a := A} = #{b => B}) -> {A, B}. t.erl:4:23: illegal pattern % 4| wrong(#{a := A} = #{b => B}) -> {A, B}. % | ^ Used in this context, the `=` operator does not imply that the two patterns are matched in any particular order. Attempting to use a variable matched out on the left-hand side on the right-hand side, or vice versa, will fail: also_wrong1(#{B := A} = #{b := B}) -> {A,B}. also_wrong2(#{a := A} = #{A := B}) -> {A,B}. t.erl:6:15: variable 'B' is unbound % 6| also_wrong1(#{B := A} = #{b := B}) -> {A,B}. % | ^ t.erl:7:27: variable 'A' is unbound % 7| also_wrong2(#{a := A} = #{A := B}) -> {A,B}. % | ^ The other way to use `=` is in a function body. Used in this way, the right-hand side must be an expression: Pattern = Expression For example: foobar(Value) -> #{a := A} = #{a => Value}, A. Used in this context, the right-hand side of `=` must **not** be a pattern: illegal_foobar(Value) -> #{a := A} = #{a := Value}, A. t.erl:18:21: only association operators '=>' are allowed in map construction % 18| #{a := A} = #{a := Value}, % | ^ When used in a body context, the value of the `=` operator is the value of its right-hand side operand. When multiple `=` operators are combined, they are evaluted from right to left. That means that any number of patterns can be matched at once: Pattern1 = Pattern2 = ... = PatternN = Expr Given that there is a well-defined evaluation order, one would except that the following example would be legal: baz(M) -> #{K := V} = #{k := K} = M, V. It is not. In Erlang/OTP 25 or earlier, the compilation fails with the following message: t.erl:28:7: variable 'K' is unbound % 28| #{K := V} = #{k := K} = M, % | ^ That restriction is now lifted, making the example legal. Closes erlang#6348

…into bjorn/master/primary_preloaded * bjorn/compiler/parallel-match/erlangGH-6348/OTP-18297: Lift restrictions for matching of binaries and maps

There has always been an implementation limitation for matching of binaries (for technical reasons). For example: foo(Bin) -> <<A:8>> = <<X:4,Y:4>> = Bin, {A,X,Y}. This would fail to compile with the following message: t.erl:5:5: binary patterns cannot be matched in parallel using '=' % 5| <<A:8>> = <<X:4,Y:4>> = Bin, % | ^ This commit lifts this restriction, making the example legal. A restriction for map matching is also lifted, but before we can describe that, we'll need a digression to talk about the `=` operator. The `=` operator can be used for two similar but slightly differently purposes. When used in a pattern, for example in a function head, both the left-hand and right-hand side operands must be patterns: Pattern1 = Pattern2 For example: bar(#{a := A} = #{b := B}) -> {A, B}. The following example will not compile because the right-hand side is not a pattern but an expression: wrong(#{a := A} = #{b => B}) -> {A, B}. t.erl:4:23: illegal pattern % 4| wrong(#{a := A} = #{b => B}) -> {A, B}. % | ^ Used in this context, the `=` operator does not imply that the two patterns are matched in any particular order. Attempting to use a variable matched out on the left-hand side on the right-hand side, or vice versa, will fail: also_wrong1(#{B := A} = #{b := B}) -> {A,B}. also_wrong2(#{a := A} = #{A := B}) -> {A,B}. t.erl:6:15: variable 'B' is unbound % 6| also_wrong1(#{B := A} = #{b := B}) -> {A,B}. % | ^ t.erl:7:27: variable 'A' is unbound % 7| also_wrong2(#{a := A} = #{A := B}) -> {A,B}. % | ^ The other way to use `=` is in a function body. Used in this way, the right-hand side must be an expression: Pattern = Expression For example: foobar(Value) -> #{a := A} = #{a => Value}, A. Used in this context, the right-hand side of `=` must **not** be a pattern: illegal_foobar(Value) -> #{a := A} = #{a := Value}, A. t.erl:18:21: only association operators '=>' are allowed in map construction % 18| #{a := A} = #{a := Value}, % | ^ When used in a body context, the value of the `=` operator is the value of its right-hand side operand. When multiple `=` operators are combined, they are evaluted from right to left. That means that any number of patterns can be matched at once: Pattern1 = Pattern2 = ... = PatternN = Expr Given that there is a well-defined evaluation order, one would expect that the following example would be legal: baz(M) -> #{K := V} = #{k := K} = M, V. It is not. In Erlang/OTP 25 or earlier, the compilation fails with the following message: t.erl:28:7: variable 'K' is unbound % 28| #{K := V} = #{k := K} = M, % | ^ That restriction is now lifted, making the example legal. Closes erlang#6348

There has always been an implementation limitation for matching of binaries (for technical reasons). For example: foo(Bin) -> <<A:8>> = <<X:4,Y:4>> = Bin, {A,X,Y}. This would fail to compile with the following message: t.erl:5:5: binary patterns cannot be matched in parallel using '=' % 5| <<A:8>> = <<X:4,Y:4>> = Bin, % | ^ This commit lifts this restriction, making the example legal. A restriction for map matching is also lifted, but before we can describe that, we'll need a digression to talk about the `=` operator. The `=` operator can be used for two similar but slightly differently purposes. When used in a pattern in a clause, for example in a function head, both the left-hand and right-hand side operands must be patterns: Pattern1 = Pattern2 For example: bar(#{a := A} = #{b := B}) -> {A, B}. The following example will not compile because the right-hand side is not a pattern but an expression: wrong(#{a := A} = #{b => B}) -> {A, B}. t.erl:4:23: illegal pattern % 4| wrong(#{a := A} = #{b => B}) -> {A, B}. % | ^ Used in this context, the `=` operator does not imply that the two patterns are matched in any particular order. Attempting to use a variable matched out on the left-hand side on the right-hand side, or vice versa, will fail: also_wrong1(#{B := A} = #{b := B}) -> {A,B}. also_wrong2(#{a := A} = #{A := B}) -> {A,B}. t.erl:6:15: variable 'B' is unbound % 6| also_wrong1(#{B := A} = #{b := B}) -> {A,B}. % | ^ t.erl:7:27: variable 'A' is unbound % 7| also_wrong2(#{a := A} = #{A := B}) -> {A,B}. % | ^ The other way to use `=` is in a function body. Used in this way, the right-hand side must be an expression: Pattern = Expression For example: foobar(Value) -> #{a := A} = #{a => Value}, A. Used in this context, the right-hand side of `=` must **not** be a pattern: illegal_foobar(Value) -> #{a := A} = #{a := Value}, A. t.erl:18:21: only association operators '=>' are allowed in map construction % 18| #{a := A} = #{a := Value}, % | ^ When used in a body context, the value of the `=` operator is the value of its right-hand side operand. When multiple `=` operators are combined, they are evaluted from right to left. That means that any number of patterns can be matched at once: Pattern1 = Pattern2 = ... = PatternN = Expr which is equivalent to: Var = Expr PatternN = Var . . . Pattern2 = Var Pattern1 = Var Given that there is a well-defined evaluation order from right to left, one would expect that the following example would be legal: baz(M) -> #{K := V} = #{k := K} = M, V. It is not. In Erlang/OTP 25 or earlier, the compilation fails with the following message: t.erl:28:7: variable 'K' is unbound % 28| #{K := V} = #{k := K} = M, % | ^ That restriction is now lifted, making the example legal. Closes erlang#6348 Closes erlang#6444

There has always been an implementation limitation for matching of binaries (for technical reasons). For example: foo(Bin) -> <<A:8>> = <<X:4,Y:4>> = Bin, {A,X,Y}. This would fail to compile with the following message: t.erl:5:5: binary patterns cannot be matched in parallel using '=' % 5| <<A:8>> = <<X:4,Y:4>> = Bin, % | ^ This commit lifts this restriction, making the example legal. A restriction for map matching is also lifted, but before we can describe that, we'll need a digression to talk about the `=` operator. The `=` operator can be used for two similar but slightly differently purposes. When used in a pattern in a clause, for example in a function head, both the left-hand and right-hand side operands must be patterns: Pattern1 = Pattern2 For example: bar(#{a := A} = #{b := B}) -> {A, B}. The following example will not compile because the right-hand side is not a pattern but an expression: wrong(#{a := A} = #{b => B}) -> {A, B}. t.erl:4:23: illegal pattern % 4| wrong(#{a := A} = #{b => B}) -> {A, B}. % | ^ Used in this context, the `=` operator does not imply that the two patterns are matched in any particular order. Attempting to use a variable matched out on the left-hand side on the right-hand side, or vice versa, will fail: also_wrong1(#{B := A} = #{b := B}) -> {A,B}. also_wrong2(#{a := A} = #{A := B}) -> {A,B}. t.erl:6:15: variable 'B' is unbound % 6| also_wrong1(#{B := A} = #{b := B}) -> {A,B}. % | ^ t.erl:7:27: variable 'A' is unbound % 7| also_wrong2(#{a := A} = #{A := B}) -> {A,B}. % | ^ The other way to use `=` is in a function body. Used in this way, the right-hand side must be an expression: Pattern = Expression For example: foobar(Value) -> #{a := A} = #{a => Value}, A. Used in this context, the right-hand side of `=` must **not** be a pattern: illegal_foobar(Value) -> #{a := A} = #{a := Value}, A. t.erl:18:21: only association operators '=>' are allowed in map construction % 18| #{a := A} = #{a := Value}, % | ^ When used in a body context, the value of the `=` operator is the value of its right-hand side operand. When multiple `=` operators are combined, they are evaluted from right to left. That means that any number of patterns can be matched at once: Pattern1 = Pattern2 = ... = PatternN = Expr which is equivalent to: Var = Expr PatternN = Var . . . Pattern2 = Var Pattern1 = Var Given that there is a well-defined evaluation order from right to left, one would expect that the following example would be legal: baz(M) -> #{K := V} = #{k := K} = M, V. It is not. In Erlang/OTP 25 or earlier, the compilation fails with the following message: t.erl:28:7: variable 'K' is unbound % 28| #{K := V} = #{k := K} = M, % | ^ That restriction is now lifted, making the example legal. Closes erlang#6348 Closes erlang#6444 Closes erlang#6467

…-6348/OTP-18297 Lift restrictions for matching of binaries and maps

RobinMorisset added the bug Issue is reported as a bug label Oct 5, 2022

bjorng added the team:VM Assigned to OTP team VM label Oct 6, 2022

bjorng self-assigned this Oct 6, 2022

bjorng added a commit to bjorng/otp that referenced this issue Nov 1, 2022

Allow parallel matching of binaries

0daccdf

Closes erlang#6348

bjorng mentioned this issue Nov 2, 2022

Lift restrictions for matching of binaries and maps #6415

Merged

bjorng closed this as completed in #6415 Dec 13, 2022

bjorng added a commit that referenced this issue Dec 13, 2022

Merge pull request #6415 from bjorng/bjorn/compiler/parallel-match/GH…

02384cf

…-6348/OTP-18297 Lift restrictions for matching of binaries and maps

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

"binary patterns cannot be matched in parallel using =" on an assignment #6348

"binary patterns cannot be matched in parallel using =" on an assignment #6348

RobinMorisset commented Oct 5, 2022

bjorng commented Oct 6, 2022

bjorng commented Nov 2, 2022

"binary patterns cannot be matched in parallel using =" on an assignment #6348

"binary patterns cannot be matched in parallel using =" on an assignment #6348

Comments

RobinMorisset commented Oct 5, 2022

bjorng commented Oct 6, 2022

bjorng commented Nov 2, 2022