Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

db: fix build on Darwin and clang 16 #241178

Merged
merged 2 commits into from
Jul 5, 2023
Merged

Conversation

reckenrode
Copy link
Contributor

@reckenrode reckenrode commented Jul 2, 2023

Description of changes

This PR updates Berkley DB’s configure scripts to build with clang 16. These changes were done to db5 then back and forward-ported to db4 and db6 as necessary.

  • It adds missing implicit int to main in configure tests.
  • It adds missing headers that cause tests to fix a false negative results with clang 16.
  • It corrects the call to signal and the signal handler’s definition in the mmap growth test to fix a false negative result with clang 16.

In addition, it replaces the mutex implementation with os_unfair_lock on Darwin. _spin_lock_try and _spin_unlock are private APIs and deprecated. The recommended replacement according to the deprecation message is os_unfair_lock. This also changes the db4 mutex implementation from pthreads to os_unfair_lock (even though db4 supported the other implementation, it wasn’t being used).

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.11 Release Notes (or backporting 23.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

The configure scripts frequently use `main` returning an implicit `int`,
which causes spurious failures when CC is clang 16+. This is fixed by
patching the provided macros and regenerating the scripts with
autoreconfHook, though it requires some manual processing (see below).

The upstream `configure.ac` is written in such a way that it requires
fixups and post-processing.

* Fixups are required because the original build process just cats the
  macros together into one file. When `aclocal` is run, it does not pick
  up all of them. This is worked around by catting the missing macros to
  a file that is picked up by autoreconfHook.
* Post-processing is required to populate the version information. This
  is done in a subshell to avoid polluting the environment with the
  contents of `RELEASE`. Otherwise, the build will fail because the
  version C macros it expects will not be defined.
Both `_spin_lock_try` and `_spin_unlock` are private and deprecated
APIs, which are not exported by any headers in the SDK. The build fails
because the configure script does not define the functions before
calling them, which is treated as error by clang 16.

This patch replaces use of those APIs with `os_unfair_lock`, which is
the recommended replacement per the deprecation messages.
@ofborg ofborg bot added the 6.topic: darwin Running or building packages on Darwin label Jul 2, 2023
@reckenrode
Copy link
Contributor Author

@ofborg eval

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/2390

@wegank
Copy link
Member

wegank commented Jul 5, 2023

Would it be possible to remove subversion as well as its dependencies (cyrus_sasl, db, openldap, serf) from bootstrap, if no other packages depend on them?

@reckenrode
Copy link
Contributor Author

xcbuild creates a compatible “toolchain” that includes ctags, which uses fetchsvn. Dropping Subversion from the bootstrap would require either removing xcbuild from any of the things it builds (which is not necessarily a bad idea, but it’s a non-trivial amount of work) or updating ctags not to use fetchsvn. However, that seems like a separate issue.

This PR is not required for the stdenv bootstrap because nothing in the final stdenv depends on Berkeley DB. However, anything post-bootstrap that does will need these patches. That’s why this is one of the related (but not prerequisite) PRs in the tracking issue. I’m trying to address problems before the actual update is merged to minimize the amount of breakage.

@wegank wegank merged commit ff9ba20 into NixOS:staging Jul 5, 2023
Comment on lines +6 to +7
extraPatches = [ ./clang-4.8.patch ./CVE-2017-10140-4.8-cwd-db_config.patch ]
++ lib.optionals stdenv.isDarwin [ ./darwin-mutexes-4.8.patch ];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't make the patch unconditional?

@@ -19,10 +22,48 @@ stdenv.mkDerivation (rec {
sha256 = sha256;
};

# The provided configure script features `main` returning implicit `int`, which causes
# configure checks to work incorrectly with clang 16.
nativeBuildInputs = lib.optionals stdenv.cc.isClang [ autoreconfHook ];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we just reconfigure on all platforms? It would make things more uniform and bugs would be easier to spot on linux.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can see if it works on Linux and open a PR to make it unconditional if there are no issues.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would be awesome!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opened #242060 to make them unconditional.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants