Build failure with OCaml 4.02.0 MinGW

[camlspotter]

The following build trial fails with OCaml 4.02.0 MinGW:

# in Cygwin shell
$ ./configure
$ make all
...
''C:/ocamlmgw/bin/ocamldep.opt -pp '''camlp4/boot/camlp4boot.native -D OPT' -modules camlp4/Camlp4/Debug.mli > camlp4/Camlp4/Debug.mli.depends
'camlp4' is not recognized as an internal or external command,
operable program or batch file.
File "camlp4/Camlp4/Debug.mli", line 1:
Error: Error while running external preprocessor
Command line: camlp4/boot/camlp4boot.native -D OPT "camlp4/Camlp4/Debug.mli" > C:\cygwin\tmp\ocamlpp08f0af

Exit code 2 while executing this command:
  ''C:/ocamlmgw/bin/ocamldep.opt -pp '''camlp4/boot/camlp4boot.native -D OPT' -modules camlp4/Camlp4/Debug.mli > camlp4/Camlp4/Debug.mli.depends
Makefile:9: recipe for target 'byte' failed

The string "camlp4/boot/camlp4boot.native" is sent to Sys.command and cmd.exe does not understand it. It must be "camlp4\boot\camlp4boot.native":

$ C:/ocamlmgw/bin/ocamldep.opt -pp '_build/camlp4/boot/fcamlp4boot.native -D OPT' -modules camlp4/Camlp4/Debug.mli
'_build' is not recognized as an internal or external command,
operable program or batch file.
File "camlp4/Camlp4/Debug.mli", line 1:
Error: Error while running external preprocessor
Command line: _build/camlp4/boot/fcamlp4boot.native -D OPT "camlp4/Camlp4/Debug.mli" > C:\cygwin\tmp\ocamlpp005928

camlp4/Camlp4/Debug.mli:

Using "_build\camlp4\boot\camlp4boot.native" resolves the problem:

$ C:/ocamlmgw/bin/ocamldep.opt -pp '_build\\camlp4\\boot\\camlp4boot.native -D OPT' -modules camlp4/Camlp4/Debug.mli
camlp4/Camlp4/Debug.mli: Format

[camlspotter]

So far, I use the following workaround but it is too dirty. Do not ask me why we need 6 backslashes:

Not sure this is a bug of p4 or ocamlbuild.

diff --git a/myocamlbuild.ml b/myocamlbuild.ml
index be02e79..4e675bc 100644
--- a/myocamlbuild.ml
+++ b/myocamlbuild.ml
@@ -95,7 +95,7 @@ let () =

     if not use_external_camlp4boot then
       dep ["camlp4boot"] [hot_camlp4boot];
-    flag ["ocaml"; "pp"; "camlp4boot"] (P hot_camlp4boot);
+    flag ["ocaml"; "pp"; "camlp4boot"] (P "camlp4\\\\\\boot\\\\\\camlp4boot.native");
     flag ["ocaml"; "pp"; "camlp4boot"; "native"] (S[A"-D"; A"OPT"]);
     flag ["ocaml"; "pp"; "camlp4boot"; "pp:dep"] (S[A"-D"; A"OPT"]);
     flag ["ocaml"; "pp"; "camlp4boot"; "pp:doc"] (S[A"-printer"; A"o"]);

[ghost]

I would say it's an ocamlbuild problem: hot_camlp4boot is defined as "camlp4"/"boot"/"camlp4boot" where / should do the appropriate thing according to the platform.

[protz]

I've run into the issue as well while re-generating the windows installer. More precisely, the issue lies with the command

  ''C:/ocamlmgw/bin/ocamldep.opt -pp '''camlp4/boot/camlp4boot.native -D OPT' -modules camlp4/Camlp4/Debug.mli > camlp4/Camlp4/Debug.mli.depends

This command uses forward-slashes, which is fine for both the windows kernel and the unixy environment that camlp4 expects for its build to happen properly. What happens, however, is that ocamldep calls (probably) into Sys.command, which in turns calls cmd.exe, which interprets camlp4/boot as camlp4 followed by the /boot option, hence the error message.

What I'm not getting is: how that did even work properly in 4.01?

I'll try to apply Jun's workaround.

CC @planar

[protz]

For the record, / expands to filename_concat which is defined in ocaml's ocamlbuild/my_std.ml. The function certainly does not do the right thing on non-unix platforms.

[ghost]

In 4.01 the myocamlbuild.ml was always using ocamlrun camlp4/boot/camlp4boot.byte. I changed this to use the .native when possible and dropped the ocamlrun. Not sure why ocamlbuild doesn't define / as Filename.concat.

I'll add a fix in camlp4 in the meantime.

[protz]

That explains it. I'll try to run an experiment and see if, by any chance, dropping ocamlbuild's custom implementation of filename concatenation would work.

[camlspotter]

4.02.0 is released and the failure becomes now a bit different:

''C:/ocamlmgw/bin/ocamldep.opt -pp ''''camlp4\boot\camlp4boot.native' -D OPT' -modules camlp4/Camlp4/Debug.mli > camlp4/Camlp4/Debug.mli.depends
'camlp4bootcamlp4boot.native' is not recognized as an internal or external command,
operable program or batch file.
File "camlp4/Camlp4/Debug.mli", line 1:
Error: Error while running external preprocessor
Command line: camlp4bootcamlp4boot.native -D OPT "camlp4/Camlp4/Debug.mli" > C:\cygwin\tmp\ocamlpp241848

Exit code 2 while executing this command:
  ''C:/ocamlmgw/bin/ocamldep.opt -pp ''''camlp4\boot\camlp4boot.native' -D OPT' -modules camlp4/Camlp4/Debug.mli > camlp4/Camlp4/Debug.mli.depends
Makefile:9: recipe for target 'byte' failed

''''camlp4\boot\camlp4boot.native' -D OPT' seems to be created as follows:

Now camlp4/boot/camlp4boot.native is replaced by camlp4\boot\camlp4boot.native but quoted. This additional quote does not work as intended, since we have another quote to add -D OPT. `''''camlp4\boot\calmp4boot.native' -D OPT' is parsed by cmd.exe as:

• ''
• ''
• camlp4bootcamlp4boot.native (backslashes are thrown away)
• ' -D OPT'

[camlspotter]

The previous workaround still works somehow:

diff --git a/myocamlbuild.ml b/myocamlbuild.ml
index 1c95e10..2102359 100644
--- a/myocamlbuild.ml
+++ b/myocamlbuild.ml
@@ -105,7 +105,7 @@ let () =
      | Some fn ->
        dep ["camlp4boot"] [fn]
      | None -> ());
-    flag ["ocaml"; "pp"; "camlp4boot"] (P hot_camlp4boot_cmd);
+    flag ["ocaml"; "pp"; "camlp4boot"] (P "camlp4\\\\\\boot\\\\\\camlp4boot.native"); (* never ask me why 6 *)
     flag ["ocaml"; "pp"; "camlp4boot"; "native"] (S[A"-D"; A"OPT"]);
     flag ["ocaml"; "pp"; "camlp4boot"; "pp:dep"] (S[A"-D"; A"OPT"]);
     flag ["ocaml"; "pp"; "camlp4boot"; "pp:doc"] (S[A"-printer"; A"o"]);

[camlspotter]

Reason is an over quoting. camlp4\boot\camlp4boot.native should not be quoted since a quote is applied to its external including -D OPT.

[damiendoligez]

Note that the quoting is not done properly either: it fails to quote the quotes.

[modlfo]

I'm having the same problem when building the MSVC port. The workaround works as well.

[dra27]

The problem is clearly ocamlbuild - invoking cmd.exe (via Sys.command) in order to invoke bash -c is a special kind of self-flagellation; it would clearly be much better if ocamlbuild on Windows used CreateProcessEx directly to call bash and at least avoid one worry for escaping. But the number of single quote marks suggest that escaping is being done incorrectly anyway (see dubious comment in Command.string_of_command_spec_with_calls).

However, it's irritating (to be as polite as possible) that camlp4 doesn't build out of the box for Windows. The following patch to myocamlbuild.ml 4.01.1+3 is working well for me:

@@ -86,7 +86,7 @@
         let dep = "camlp4"/"boot"/exe in
         let cmd =
           let ( / ) = Filename.concat in
-          "camlp4"/"boot"/exe
+          String.escaped (String.escaped ("camlp4"/"boot"/exe))
         in
         (Some dep, cmd)
     in

Can this be incorporated? It's totally clear in the context that for non-Windows platforms the calls to String.escaped will do nothing (so we don't break other platforms) and even if/when ocamlbuild is fixed, multiple backslashes are not an error in Windows paths (i.e. C:\\\Windows\\system32 is perfectly valid)

[ghost]

@dra27 the patch looks good. Can you create a pull-request?

[dra27]

Yes - which branch would you prefer it in - 4.02 or trunk?

[ghost]

@dra27 any will do, you can do it on trunk

[dra27]

See #94 (should fix #93 as well)

[ghost]

I applied @dra27's patch. I'll do a minor release for windows next week unless someone complains

[msprotz]

Looks like this problem just got worse.

Exit code 2 while executing this command:
   ocamldep.opt -pp ' '\''camlp4\\\\boot\\\\camlp4boot.native'\'' -D OPT' -modules camlp4/Camlp4/Debug.mli > camlp4/Camlp4/Debug.mli.depends

This is with a freshly-built ocamlbuild.

protz@Joprotze-Z420:~/Code/camlp4 ((4.03+1)) $ opam show ocamlbuild
             package: ocamlbuild
             version: 0.9.2

[msprotz]

the ' '\'' in particular is really suspicious