Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

+no_copt changes the result of a comprehension #7494

Closed
RobinMorisset opened this issue Jul 12, 2023 · 0 comments · Fixed by #7538
Closed

+no_copt changes the result of a comprehension #7494

RobinMorisset opened this issue Jul 12, 2023 · 0 comments · Fixed by #7538
Assignees
Labels
bug Issue is reported as a bug team:VM Assigned to OTP team VM

Comments

@RobinMorisset
Copy link
Contributor

RobinMorisset commented Jul 12, 2023

Describe the bug
Running the following code:

f() ->
    <<0 || _ <- [], _ <- ok, false>>.

wrapper0() ->
    io:format("~kp", [catch f()]).

by doing

erlc -W0 ~/minimized/test237800.erl
erl -pa . -noshell -s test237800 wrapper0 -s init stop

results in the following output:

{'EXIT',{{bad_generator,ok},
         [{test237800,f,0,
                      [{file,"/home/rmorisset/minimized/test237800.erl"},
                       {line,9}]},
          {test237800,wrapper0,0,
                      [{file,"/home/rmorisset/minimized/test237800.erl"},
                       {line,12}]},
          {init,start_em,1,[]},
          {init,do_boot,3,[]}]}}

But adding +no_copt to erlc insteads makes it output <<>>.

Expected behavior
I don't know exactly which behavior is correct: should generators and filters be checked even after an empty generator?
I could not find anything about this question in the documentation (https://www.erlang.org/doc/reference_manual/expressions.html#comprehensions), beyond the following sentence:

A bit string comprehension returns a bit string, which is created by concatenating the results of evaluating BitStringExpr for each combination of bit string generator elements for which all filters are true

But since later generators can shadow variables in earlier ones, I expected them to be evaluated left to right.

It is possible that both behaviors are allowed on purpose, but in that case I'd like to suggest a warning in the documentation as it is a very non-obvious (and in my view not very useful) source of non-determinism.

Affected versions
master

@RobinMorisset RobinMorisset added the bug Issue is reported as a bug label Jul 12, 2023
@jhogberg jhogberg added team:VM Assigned to OTP team VM stalled waiting for input by the Erlang/OTP team labels Jul 12, 2023
@bjorng bjorng self-assigned this Jul 31, 2023
@bjorng bjorng removed the stalled waiting for input by the Erlang/OTP team label Aug 3, 2023
bjorng added a commit to bjorng/otp that referenced this issue Aug 4, 2023
bjorng added a commit to bjorng/otp that referenced this issue Aug 4, 2023
Consider the following function:

    bug() ->
        << <<C:8>> || C <- [], _ <- ok >>.

The first generator is an empty list, so the second generator should
never be evaluated, and thus the expected result is an empty
binary. That is indeed the result when evaluating the comprehension in
the shell:

    1> << <<C:8>> || C <- [], _ <- ok >>.
    <<>>

(And also when turning off compiler optimizations.)

However, when running compiled code there will be an exception:

    2> t:bug().
    ** exception error: bad generator ok
         in function  t:bug/0 (t.erl, line 8)

The reason for the exception is that the compiler as an optimization
inserts code to calculate the size of the resulting binary. As part of
the size calculation, `length(ok)` is evaluated, which will obviously
fail. When the size calculation fails, a `bad_generator` exception is
raised.

This commit changes the error handling for the size calculation to
use a default size for the binary instead of raising an exception.

Fixes erlang#7494
bjorng added a commit to bjorng/otp that referenced this issue Aug 7, 2023
Consider the following function:

    bug() ->
        << <<C:8>> || C <- [], _ <- ok >>.

The first generator is an empty list, so the second generator should
never be evaluated, and thus the expected result is an empty
binary. That is indeed the result when evaluating the comprehension in
the shell:

    1> << <<C:8>> || C <- [], _ <- ok >>.
    <<>>

(And also when turning off compiler optimizations.)

However, when running compiled code there will be an exception:

    2> t:bug().
    ** exception error: bad generator ok
         in function  t:bug/0 (t.erl, line 8)

The reason for the exception is that the compiler as an optimization
inserts code to calculate the size of the resulting binary. As part of
the size calculation, `length(ok)` is evaluated, which will obviously
fail. When the size calculation fails, a `bad_generator` exception is
raised.

This commit changes the error handling for the size calculation to
use a default size for the binary instead of raising an exception.

Fixes erlang#7494
bjorng added a commit that referenced this issue Aug 10, 2023
…H-7494/OTP-18703

Correct evaluation order for binary comprehensions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Issue is reported as a bug team:VM Assigned to OTP team VM
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants