pkgsMusl.firefox: fix build

[yu-re-ka]

Description of changes

Depends on #190796

I had some discussion in the musl irc channel about the malloc problem. There is two problems actually: lld crashes when built with the default stack size. But even when built with a larger stack size, it would crash when built with the musl mallocng allocator, because that allocator produces a very bad result for this allocation pattern and would exceed the default max_map_count value while linking firefox. Building musl with the oldalloc allocator works, but is very slow. Instead, I am LD_PRELOADing mimalloc here to work around this issue.

There is still an issue with dynamic linker path when running the resulting firefox. it is necessary to put /path-of-unwrapped-firefox/lib/firefox in the LD_LIBRARY_PATH manually, otherwise it will not start. i don't know why exactly. Fixed

Thunderbird builds and works as well.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 22.11 Release Notes (or backporting 22.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
  • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
• Fits CONTRIBUTING.md.

[S-NA]

There is still an issue with dynamic linker path when running the resulting firefox. it is necessary to put /path-of-unwrapped-firefox/lib/firefox in the LD_LIBRARY_PATH manually, otherwise it will not start. i don't know why exactly.

That seems oddly familiar to something I encountered awhile back when adding LTO support. It ended up that Mozilla had a Python script that tripped up on whitespace differences from GNU readelf and LLVM readelf which in turn caused /path-of-unwrapped-firefox/lib/firefox/dependentlibs.list to fail to be properly generated. I contributed a fix upstream awhile back for it so it is not the issue here but I suggest looking at the contents of dependentlibs.list and seeing if it has a list of shared objects. If it is missing items then something in the pkgsMusl environment is breaking an assumption of dependentlibs.py. Otherwise Firefox is failing to load dependentlibs.list.

[yu-re-ka]

The dependentlibs.list was generated correctly. It is the same between the glibc and musl build. An strace reveals very different behavior:

musl

[yuka@yaya:~/musl-nixos/firefox-104.0.2]$ rg mozsandbox log
81:read(3, "libmozsandbox.so\nliblgpllibs.so\n"..., 1024) = 89
82:open("/home/yuka/musl-nixos/outputs/out/lib/firefox/libmozsandbox.so", O_RDONLY|O_LARGEFILE) = 4
87:open("/home/yuka/musl-nixos/outputs/out/lib/firefox/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 4
1281:open("/nix/store/x5x7f006wbs09ww83pki17wli7qljaj2-icu4c-71.1/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1282:open("/nix/store/67i5mqcag7r35ydgh6g4yij0vg2z6xrd-libffi-3.4.2/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1283:open("/nix/store/0qyvydcbx9hfhmrzlfk7j7sfvyg2mwbv-nspr-4.34.1/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1284:open("/nix/store/vn1l1a1yxbwmp8nawc03p20vldla2r3f-zlib-1.2.12/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1285:open("/nix/store/7nirw85kh6zkjxlijjhwf2m9cd6iviiw-nss-3.82/lib64/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1286:open("/nix/store/sksgkzmxnj1q7ym2yzhhfr3f1rqyjji2-freetype-2.12.1/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1287:open("/nix/store/7ywq4vsqgl99p9vg06akay3yg1ri3hl0-fontconfig-2.14.0-lib/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1288:open("/nix/store/kzqrav33fgal77hv1rryby0k2pj0mph7-glib-2.72.3/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1289:open("/nix/store/zzhyvzgmxhhhx13gcspwwb2ffir6gmyi-gtk+3-3.24.34/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1290:open("/nix/store/k5bfc8lvj8ks9bil5m1hcpd4pjqfd4rs-atk-2.38.0/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1291:open("/nix/store/m1l9cwkxvpabklf29x021k7rxc3jqrfx-cairo-1.16.0/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1292:open("/nix/store/mi47j9c9ixhjbzn1zq86y9l0lg4z39xh-gdk-pixbuf-2.42.8/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1293:open("/nix/store/b3c2ikzmzcj76ixds4i55xpzwc0wx5rw-pango-1.50.8/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1294:open("/nix/store/nbf5i7m3l8n56j0c5h6pciygz5xcxgin-harfbuzz-5.1.0/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1295:open("/nix/store/kcmiv0h368pgz3fic5p311hj97s1vy1c-libpng-apng-1.6.37/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1296:open("/nix/store/dgk6wjwsq004n8rmkmaydpyz4ly8azmp-libwebp-1.2.4/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1297:open("/nix/store/sd70as1lqg1nvysing79lcp89fi02bii-libevent-2.1.12/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1298:open("/nix/store/5nk3wcm7q3zdqp9z7wfk108i15yhzcaa-libvpx-1.12.0/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1299:open("/nix/store/qlvc1ympanf4qalp2pwqfjs8wpx5nczh-pixman-0.38.4/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1300:open("/nix/store/z055cqq1sxl6bmzxhkg19lj0dwvlmpj3-dbus-1.14.0-lib/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1301:open("/nix/store/l8h95clnh22vvpjn9wb147jcq2w2q2yg-dbus-glib-0.112/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1302:open("/nix/store/2044lgh7qxpjkabb0i97y5clly5b3lpw-libXext-1.3.4/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1303:open("/nix/store/87k78lavc9ndkihjvvm8wrhqh9wl4jcs-libX11-1.7.2/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1304:open("/nix/store/yqx8yjbna5c9ndfq4bgkwd1w1qx77sp8-libxcb-1.14/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1305:open("/nix/store/8aspn3pkbj1a49hmqx81nkqxr2hq8sqr-libXcomposite-0.4.5/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1306:open("/nix/store/jxn57fm0jyf9qqnr2i7m1ckaw25ysgwh-libXfixes-6.0.0/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1307:open("/nix/store/bmmldybsss3d83w146lxib7sy6q1lyb0-libXcursor-1.2.0/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1308:open("/nix/store/k1x2rx52nbl9a67yrf7j9kk71gr5avbw-libXi-1.8/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1309:open("/nix/store/63wqbdk9zsb5gxpd05wgss8hcqrgrj87-libXrandr-1.5.2/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1310:open("/nix/store/lw0gsp8p4ws7mrhm4fxvybkbkxfjihns-libXdamage-1.1.5/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1311:open("/nix/store/via7achzpkbxh4pl4q0jmapp49r04bg4-libXtst-1.2.3/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1312:open("/nix/store/347fqzxbcm48ajcwhc80qyrppw2qldb4-gcc-11.3.0-lib/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1313:open("/nix/store/lbyphf5vcnsjv9n1kx2h4rvws3hz0m6g-libXrender-0.9.10/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1314:open("/nix/store/fjmgvqwwj52bvk4nb64j3diyb5fwm9bl-alsa-lib-1.2.7.2/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1315:open("/nix/store/40fiaz98s2dzgi0pp87zkvx8hhv2i6c2-musl-1.2.3/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1316:open("/nix/store/347fqzxbcm48ajcwhc80qyrppw2qldb4-gcc-11.3.0-lib/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1317:open("/nix/store/40fiaz98s2dzgi0pp87zkvx8hhv2i6c2-musl-1.2.3/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1319:open("/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1320:open("/usr/local/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1321:open("/usr/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1326:Error loading shared library libmozsandbox.so: No such file or directory (needed by /home/yuka/musl-nixos/outputs/out/lib/firefox/libxul.so)) = 142

[yuka@yaya:~/musl-nixos/firefox-104.0.2]$ 

glibc

[yuka@mari:~]$ rg mozsandbox log
148:read(3, "libmozsandbox.so\nliblgpllibs.so\n"..., 4096) = 89
149:openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/libmozsandbox.so", O_RDONLY) = 4
154:openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = 4
27593:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/glibc-hwcaps/x86-64-v3/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27595:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/glibc-hwcaps/x86-64-v2/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27597:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/tls/haswell/x86_64/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27599:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/tls/haswell/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27601:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/tls/x86_64/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27603:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/tls/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27605:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/haswell/x86_64/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27607:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/haswell/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27609:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/x86_64/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27611:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = 11
27932:[pid 17397] read(11, "libmozsandbox.so\nliblgpllibs.so\n"..., 4096) = 89

[yuka@mari:~]$ 

Note that glibc is only trying to open the libmozsandbox.so a second time in a child process

[yu-re-ka]

I fucked up the rebase, opening a new PR to not create any more notifications

[ghost]

I fucked up the rebase, opening a new PR to not create any more notifications

It's not your fault; github's UI for this is atrocious.

Hint: before changing the base, mark your PR as a draft. That way if this happens it won't ping half of humanity.