jpeg-xl 0.7.0

[filbranden]

• Have you followed the guidelines for contributing?
• Have you ensured that your commits follow the commit style guide?
• Have you checked that there aren't other open pull requests for the same formula update/change? NO, there's PR jpeg-xl 0.7.0 #111355 from @owine, but this PR solves the issue with brew linkage --cached --test --strict jpeg-xl from that PR (this PR also preserves the original commits from that one.)
• Have you built your formula locally with brew install --build-from-source <formula>, where <formula> is the name of the formula you're submitting?
• Is your test running fine brew test <formula>, where <formula> is the name of the formula you're submitting?
• Does your build pass brew audit --strict <formula> (after doing brew install --build-from-source <formula>)? If this is a new formula, does it pass brew audit --new <formula>?

---

Bump jpeg-xl to version 0.7.0

[SMillerDev]

it seems aom needs a revision bump

[carlocab]

Make sure to rebase before the revision bump, since aom was just updated and it'll be outdated on your branch.

[filbranden]

@SMillerDev Ok I bumped revision of aom and also vips and webkitgtk which have an explicit depends_on "jpeg-xl".

(Should there be a brew command or perhaps an option argument to brew bump-formula-pr to automatically bump the revision of formulas that depend on the one being bumped, for cases in which the ABI changes like this?)

@carlocab Indeed and also webkitgtk had a conflict that needed manual resolution. Rebased the commit stack and all seems clear now. Thanks!

[SMillerDev]

Brew commands don't help here, because not everything always breaks.

[carlocab]

Ok I bumped revision of aom and also vips and webkitgtk which have an explicit depends_on "jpeg-xl".

Are you sure they're needed? Having a depends_on alone is not sufficient to require one.

[filbranden]

Are you sure they're needed? Having a depends_on alone is not sufficient to require one.

Yes, pretty much sure.

See previous failed run at:
https://github.com/Homebrew/homebrew-core/actions/runs/3110096393/jobs/5040961111

Commands brew linkage --test vips and brew linkage --test webkitgtk were also failing, with output that includes:

  Missing libraries:
    unexpected (libjxl.so.0.6)

[filbranden]

I'm seeing breakage in formulas that depend on aom too.

In particular vips-heif.so module indicates libheif needs to be bumped, and ffmpeg binary seems to be affected too. (There's libavif, I haven't seen breakage due to it yet, but I guess it probably needs a bump too.) I'll add one more commit to bump these three here.

[carlocab]

This project tags commits (and is at 1.0+ now), so it would be good to make this its own formula, I think.

[filbranden]

@carlocab @SMillerDev This now passed the CI tests (at least on 12-arm64).

It seems the others were cancelled after a (long) run, not sure if this was some quota or what.

It's odd that I had to bump revisions on, say, geeqie, since the dependency is through geeqie -> imagemagick -> libheif -> aom -> jpeg-xl, but the error message was clear on a geeqie component linking against libjxl.so.0.6, so it seemed like the right thing to do. (Even though imagemagick itself didn't seem to need a revision bump.)

Anyways, please take a look and let me know if you think anything else is still needed for this PR, or if it should be ready to merge at this point.

Thanks!

[filbranden]

@carlocab Thanks for adding the long build label! But tests are now failing with PR requires the CI-long-timeout label but it is not set, so can you please add that label as well? I'm happy to do a force push to trigger a new build once that's in place. Thanks!

[carlocab]

Error: PR requires the CI-long-timeout label but it is not set!
Error: If the longer timeout is not required, remove the "long build" label.
Error: Otherwise, add the "CI-long-timeout" label.
Error: No more than 2 PRs at a time may use "CI-long-timeout".

There are already 2 PRs using "CI-long-timeout": #111543, #111434

In any case, your commits need fixing up, since you have commits that modify multiple formulae (which is why there is an "automerge-skip" label).

Do

cd $(brew --repo homebrew/core)
git checkout bump-jpeg-xl-0.7.0
git reset --hard HEAD~3
brew bump-revision --message "(jpeg-xl 0.7.0)" aom ffmpeg{,@4} geeqie libavif libheif vips webkitgtk
git push --force

[filbranden]

Done recreating the commits using brew bump-revision.

[carlocab]

webkitgtk failed to build on Linux. CC @Homebrew/linux

It's the same problem we saw for gobject-introspection:
/tmp/webkitgtk-20220925-222614-fmmcru/webkitgtk-2.38.0/build/WebKit2Gtk/DerivedSources/ModernMediaControlsGResourceBundle.xml: Failed to close file descriptor for child process (Operation not permitted).

[BrewTestBot]

@carlocab has triggered a merge.

[carlocab]

Will dispatch a build for webkitgtk on a GitHub runner separately.

[BrewTestBot]

❌ @carlocab bottle request for webkitgtk failed.

[carlocab]

It happened on a GitHub runner too:
2022-09-26T04:35:19.3645163Z /tmp/webkitgtk-20220926-40157-m5mblh/webkitgtk-2.38.0/build/WebKit2Gtk/DerivedSources/ModernMediaControlsGResourceBundle.xml: Failed to close file descriptor for child process (Operation not permitted).

[danielnachun]

I think we have to tweak our workflow files: https://gist.github.com/nathabonfim59/b088db8752673e1e7acace8806390242. It seems there is a glibc change that requires an additional parameter to be used when calling docker.

[carlocab]

Nice find. We probably want to work out which syscalls we want to allow instead of just passing unconfined.

[danielnachun]

The plot thickens here: https://gitlab.gnome.org/GNOME/glib/-/commit/ce04a124040be091407e070280d86ca810bacb8c indicates this is due to a bug in which undefined syscalls are incorrectly returning EPERM (Operation not permitted) instead of ENOSYS, which in glib automatically falls back to using a different syscall to close the file handle.

It appears the root cause of this is running a glibc 2.34 or newer image on a host OS running an older kernel. Normally when you do this, you get back ENOSYS and your code is supposed to fall back to using a different syscall. But because the aforementioned bug, EPERM is returned instead.

@Homebrew/linux is our docker host ubuntu-latest (which would be Ubuntu 20.04) or ubuntu-22.04? If not we might be able to fix this changing to ubuntu-22.04, as these failures imply that somewhere in the virtualization stack, an older kernel is being used that doesn't implement some syscalls that are expected in glibc.

Alternatively, I think the offending syscall is clone, and would be fixed by setting --cap-add CAP_SYS_ADMIN to our docker calls. I'm not sure how easy this will be to test though.

[carlocab]

Yes, this was mentioned on Slack a while ago: moby/moby#43595

@Homebrew/linux is our docker host ubuntu-latest (which would be Ubuntu 20.04) or ubuntu-22.04?

We should be using ubuntu-22.04 on GitHub-hosted runners,

homebrew-core/.github/workflows/tests.yml

Line 88 in e42f141

core.setOutput('linux-runner', 'ubuntu-22.04')

but who knows if that's not virtualised on a machine with an older kernel.

Alternatively, I think the offending syscall is clone, and would be fixed by setting --cap-add CAP_SYS_ADMIN to our docker calls. I'm not sure how easy this will be to test though.

We could try brew install --build-bottle webkitgtk in a Docker container with that enabled. Or we could just add it to the workflow and see if it helps.

[danielnachun]

We could try brew install --build-bottle webkitgtk in a Docker container with that enabled. Or we could just add it to the workflow and see if it helps.

I think we might have to do the latter. This failure is very host-dependent and I would guess on Windows and macOS hosts where they basically emulate the syscalls for the newest Linux kernel it would work, as would any Linux host with a new enough kernel.

One useful piece of documentation that would be nice to have for the future is the "anatomy" of the virtualization stack so we actually know what version of the kernel is actually executing our syscalls, though I have no idea how easy it is get to get that information.