Can’t define type “product()” in OTP 26.0.2

[snakeduse]

Describe the bug

Hello! I have a strange behaviour of dialyzer.

When I try to create type product() then dialyzer reported an error:

===> Error in dialyzing apps: Analysis failed with error:
Unable to find type 'fun'/2

Last messages in the log cache:
  Reading files and computing callgraph... done in 0.13 secs
  Removing edges... done in 0.00 secs

To Reproduce

To reproduce you can use example:

-module(test_product_app).

-behaviour(application).

-export([start/2, stop/1, test/0]).

-type product() :: binary().

start(_StartType, _StartArgs) ->
    test_product_sup:start_link().

stop(_State) ->
    ok.

-spec test() -> Result when
    Result :: product().
test() ->
    <<"123">>.

If you remove -spec for function test, then error will not reproduce.

Affected versions
I use Erlang/OTP 26.0.2 (installed from esl-erlang_26.0.2-1~debian~buster_amd64.deb) on Debian 10 with rebar3 v.3.22.1

Additional context
First of all I asked on the forum and the people have confirmed the same behaviour. So I think it looks like a bug.

Unfortunately I don't know the code of dialyzer and can't create a PR. Sorry for that.

[TD5]

iirc, this happened to me when I updated to OTP 26 from OTP 25, and I didn't clear my build artefacts and PLT. If you completely rebuild your code from scratch and create a new PLT, do you still see this error?

If this is indeed a backwards compatibility issue with OTP 26, a change to make any Dialyzer modifications invalidate the PLT, like:

-spec implementation_module_paths() -> module_file_path_lookup().
implementation_module_paths() ->
  Dir = code:lib_dir(dialyzer),
  Files1 = filelib:wildcard("**/ebin/**.beam",Dir),
  Files2 = [filename:join([Dir, F]) || F <- Files1],
  #{beam_file_to_module(File) => File || File <- Files2}.

in dialyzer_iplt.erl and dialyzer_cplt.erl might help? Possibly also something similar for changes to the core erlang representation too.

[starbelly]

FWIW @TD5 I tried this work-around, no go.

[snakeduse]

If you completely rebuild your code from scratch and create a new PLT, do you still see this error?

I used clean machine without OTP 25 (I have installed only OTP 26) and error happens always with and without "_build" directory and other artifacts.

[TD5]

I notice you're using rebar3. From what I recall, rebar3 uses it's own pre-built PLT file of OTP to start a Dialyzer analysis. It's possible that is providing an incompatible PLT file for your OTP release.

One way to see whether that is the case would be by running Dialyzer without rebar3 to see whether that works, e.g.:

# Delete any existing PLT from the default location, e.g. something like rm `$HOME/.cache/erlang/*plt`
dialyzer --incremental --apps dialyzer stdlib kernel crypto compiler

If this fails with the same error, then I am confident this is a bug in OTP. If this doesn't fail with that error, I suspect it's to do with an out-of-date PLT or .beam file being provided by rebar3.

My gut says this is related to rebar3 providing out-dated build artefacts, since I saw a similar issue myself (where I don't use rebar3) and clearing all my caches fixed it, iirc.

N.B. if this is determined to be an issue of incompatible artefacts coming from rebar3 or otherwise, then there's a bug in Dialyzer too, since it should invalidate the PLT at that point and re-build it from scratch, or reject the input .erl/.beam files. In that case, my suggestion above might help.

[TD5]

always with and without "_build" directory and other artifacts

It may also be worth clearing your global rebar3 cache dir, where PLT files per OTP version are stored. This is in $HOME/.cache/rebar3 by default (see https://github.com/erlang/rebar3/blob/main/apps/rebar/src/rebar_dir.erl#L172).

[starbelly]

always with and without "_build" directory and other artifacts

It may also be worth clearing your global rebar3 cache dir, where PLT files per OTP version are stored. This is in $HOME/.cache/rebar3 by default (see https://github.com/erlang/rebar3/blob/main/apps/rebar/src/rebar_dir.erl#L172).

Right, I blew everything away, I'm happy to double check. I also tried with just dialyzer (to be clear it blows up trying to build the plt, not update or or use it).

Edit:

$ ls /Users/me/Library/Caches/erlang/
$ 

^ nothing

$ dialyzer --build_plt _build/default/lib/whoops/ebin/  
  Creating PLT /Users/me/Library/Caches/erlang/.dialyzer_plt ...
dialyzer: Analysis failed with error:
whoops.erl:7:2: Unable to find type 'fun'/2

Last messages in the log cache:
  Reading files and computing callgraph... done in 0.03 secs
  Removing edges... done in 0.00 secs

So this seems like something goes wrong in going through the abstract code in building the initial plt. I'm happy to look further (provided I can find the time).

[paulo-ferraz-oliveira]

Here's a simple reference that shows the issue: https://github.com/paulo-ferraz-oliveira/setup-beam-example/actions/runs/5978380986/job/16220351170.

Erlang/OTP is installed alongside rebar3, then we run rebar3 dialyzer. The issue is as-described @snakeduse, for OTP 26.

===> Error in dialyzing apps: Analysis failed with error:
Unable to find type 'fun'/2

There's no cache involved, in this example, and the instance is clear of PLTs (well at least before the rebar3 command). Re-running the jobs yields the same results.

@TD5, I also run your suggestion for incremental Dialyzer, but it fails (not sure if that's what's causing the overall failure with lots of unknown). Edit: I'll redo it with -Wno_unknown.. (later)... it seems to show no errors, in this case.

[paulo-ferraz-oliveira]

Not sure if it helps, but there's a few mentions to product in the AST and @starbelly already referred to this in the past. Maybe linked to the issue?

[starbelly]

Not sure if it helps, but there's a few mentions to product in the AST and @starbelly already referred to this in the past. Maybe linked to the issue?

I couldn't find any issues directly related to this problem. To be clear product in the abstract code is normal and nothing new prior to OTP 26, but yeah 😄

[paulo-ferraz-oliveira]

To be clear product in the abstract code is normal and nothing new prior to OTP 26, but yeah 😄

I know it's not new 👍 but something changed around this particular word, so I'm just curious.

[starbelly]

To be clear product in the abstract code is normal and nothing new prior to OTP 26, but yeah 😄

I know it's not new 👍 but something changed around this particular word, so I'm just curious.

I knew you did, just for other reads who come along 😄

FWIW, I did some manual CI and found that in the latest patch of OTP 25, we're good, no issue. This started in OTP-26-rc1.

[starbelly]

I narrowed this down to two commits :

1. 963c7d5
   • @TD5 introducing incremental dialyzer support (btw thank you!) This commit doesn't actually cause the problem, but it's surely related.
2. 982de29
   • After @bjorng fixed a problem with interpretation of [none()] and at that point a type called product won't work.

Without actually going through the code, I can not say why, I will probably look at this later as it interests me 😄

[starbelly]

I found the problem, it's arguable where it should be fixed, but here's one patch that would work :

---
 lib/dialyzer/src/erl_types.erl | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/lib/dialyzer/src/erl_types.erl b/lib/dialyzer/src/erl_types.erl
index 8068baf8e8..a81989af7d 100644
--- a/lib/dialyzer/src/erl_types.erl
+++ b/lib/dialyzer/src/erl_types.erl
@@ -4410,6 +4410,11 @@ from_form({type, _Anno, 'fun', [{type, _, product, Domain}, Range]},
   {Dom1, L1, C1} = list_from_form(Domain, S, D, L, C),
   {Ran1, L2, C2} = from_form(Range, S, D, L1, C1),
   {t_fun(Dom1, Ran1), L2, C2};
+from_form({type, _Anno, 'fun', [{user_type, _, product, Domain}, Range]},
+          S, D, L, C) ->
+  {Dom1, L1, C1} = list_from_form(Domain, S, D, L, C),
+  {Ran1, L2, C2} = from_form(Range, S, D, L1, C1),
+  {t_fun(Dom1, Ran1), L2, C2};
 from_form({type, _Anno, identifier, []}, _S, _D, L, C) ->
   {t_identifier(), L, C};
 from_form({type, _Anno, integer, []}, _S, _D, L, C) ->
--

Less fancier than what I had originally thought, but I got to know dialyzer code more! 😄