Merge #270: Upstream PRs 1391, 1290, 1389, 1397, 1399, 1400, 1348, 14…

…02, 1274, 1394, 1404, 1062, 1401, 1373, 1403, 1398, 1405, 1396, 1406, 1410, 1409, 1411, 1412, 1414, 1413, 1415, 1417, 1390, 1416, 1422, 1424, 1395

b41caaa bppp: replace memcmp in tests with secp256k1_memcmp_var (Jonas Nick)
6a3aae8 group_parse: use secp256k1_memcmp_var instead of memcmp (Jonas Nick)
e9d522f ci: turn on -zkp modules in macos-native job (Jonas Nick)
c45b7c4 refactor: introduce testutil.h (deduplicate `random_fe_`, `ge_equals_` helpers) (Sebastian Falbesoner)
dc55141 tests: simplify `random_fe_non_zero` (remove loop limit and unneeded normalize) (Sebastian Falbesoner)
d9d80fd ci: Bump major versions for docker actions (Hennadii Stepanov)
421d848 ci: Align Autotools/CMake `CI_INSTALL` directory names (Hennadii Stepanov)
9f005c6 cmake: Install `libsecp256k1.pc` file (Hennadii Stepanov)
b0f7bfe doc: Do not mention soname in CHANGELOG.md "ABI Compatibility" section (Hennadii Stepanov)
bd9d98d doc: Align documented scripts with CI ones (Hennadii Stepanov)
7030364 tests: add CHECK_ERROR_VOID and use it in scratch tests (Jonas Nick)
f8d7ea6 tests: Replace counting_illegal_callbacks with CHECK_ILLEGAL_VOID (Jonas Nick)
a1d52e3 tests: remove unnecessary test in run_ec_pubkey_parse_test (Jonas Nick)
875b0ad tests: remove unnecessary set_illegal_callback (Jonas Nick)
9b118bc release cleanup: bump version after 0.4.0 (Jonas Nick)
1633980 release: Prepare for 0.4.0 (Tim Ruffing)
d9a8506 changelog: Catch up in preparation of release (Tim Ruffing)
8659a01 ci: Add `release` job (Hennadii Stepanov)
f9b3889 ci: Update `actions/checkout` version (Hennadii Stepanov)
2635068 ci/gha: Let MSan continue checking after errors in all jobs (Tim Ruffing)
e78c7b6 ci/Dockerfile: Reduce size of Docker image further (Tim Ruffing)
2f0d3bb ci/Dockerfile: Warn if `ulimit -n` is too high when running Docker (Tim Ruffing)
4b8a647 ci/gha: Add ARM64 QEMU jobs for clang and clang-snapshot (Tim Ruffing)
6ebe7d2 ci/Dockerfile: Always use versioned clang packages (Tim Ruffing)
c223d7e ci: Switch macOS from Ventura to Monterey and add Valgrind (Hennadii Stepanov)
cce0456 ci: Make repetitive command the default one (Hennadii Stepanov)
317a4c4 ci: Move `git config ...` to `run-in-docker-action` (Hennadii Stepanov)
676ed8f ci: Move "C++ (public headers)" from Cirrus to GitHub Actions (Hennadii Stepanov)
61fc3a2 ci: Move "C++ -fpermissive..." from Cirrus to GitHub Actions (Hennadii Stepanov)
d51fb0a ci: Move "MSan" from Cirrus to GitHub Actions (Hennadii Stepanov)
c22ac27 ci: Move sanitizers task from Cirrus to GitHub Actions (Hennadii Stepanov)
ee1be62 ci: Use concurrency for pull requests only (Hennadii Stepanov)
fc3dea2 ci: Move "ppc64le: Linux..." from Cirrus to GitHub Actions (Hennadii Stepanov)
7782dc8 ci: Move "ARM64: Linux..." from Cirrus to GitHub Actions (Hennadii Stepanov)
0a16de6 ci: Move "ARM32: Linux..." from Cirrus to GitHub Actions (Hennadii Stepanov)
ea33914 ci: Move "s390x (big-endian): Linux..." from Cirrus to GitHub Actions (Hennadii Stepanov)
880be8a ci: Move "i686: Linux (Debian stable)" from Cirrus to GiHub Actions (Hennadii Stepanov)
e10878f ci, gha: Drop `driver-opts.network` input for `setup-buildx-action` (Hennadii Stepanov)
4ad4914 ci, gha: Add `retry_builder` Docker image builder (Hennadii Stepanov)
6617a62 ci: Remove "x86_64: Linux (Debian stable)" task from Cirrus CI (Hennadii Stepanov)
03c9e65 ci, gha: Add "x86_64: Linux (Debian stable)" GitHub Actions job (Hennadii Stepanov)
ad3e65d ci: Remove GCC build files and sage to reduce size of Docker image (Tim Ruffing)
ef9fe95 ci: Drop no longer needed workaround (Hennadii Stepanov)
87d35f3 ci: Rename `cirrus.sh` to more general `ci.sh` (Hennadii Stepanov)
d6281dd ci: Remove Windows tasks from Cirrus CI (Hennadii Stepanov)
2b6f9cd ci, gha: Add Windows jobs based on Linux image (Hennadii Stepanov)
b0886fd ci, gha: Ensure only a single workflow processes `github.ref` at a time (Hennadii Stepanov)
d23da6d use secp256k1_scalar_verify checks (stratospher)
c7d0454 add verification for scalars (stratospher)
ad15215 update max scalar in scalar_cmov_test and fix schnorrsig_verify exhaustive test (stratospher)
d78bec7 ci: Remove Windows MSVC tasks from Cirrus CI (Hennadii Stepanov)
3545dc2 ci, gha: Run all MSVC tests on Windows natively (Hennadii Stepanov)
54058d1 field: remove `secp256k1_fe_equal_var` (siv2r)
bb4efd6 tests: remove unwanted `secp256k1_fe_normalize_weak` call (siv2r)
c2f6435 ci: Add comment about switching macOS to M1 on GHA later (Tim Ruffing)
4a24fae ci: Remove "arm64: macOS Ventura" task from Cirrus CI (Hennadii Stepanov)
8e54a34 ci, gha: Run "x86_64: macOS Ventura" job on GitHub Actions (Hennadii Stepanov)
747ada3 test: Silent noisy clang warnings about Valgrind code on macOS x86_64 (Hennadii Stepanov)
d62db57 ci: Use Homebrew's gcc in native macOS task (Hennadii Stepanov)
b7c685e Save _normalize_weak calls in group add methods (Peter Dettman)
c83afa6 Tighten group magnitude limits (Peter Dettman)
173e8d0 Implement current magnitude assumptions (Peter Dettman)
49afd2f Take use of _fe_verify_magnitude in field_impl.h (Sebastian Falbesoner)
4e9661f Add _fe_verify_magnitude (no-op unless VERIFY is enabled) (Peter Dettman)
690b0fc add missing group element invariant checks (Sebastian Falbesoner)
9c91ea4 ci: Enable ellswift module where it's missing (Tim Ruffing)
db32a24 ctimetests: Use new SECP256K1_CHECKMEM macros also for ellswift (Tim Ruffing)
8408dfd Revert "ci: Run sage prover on CI" (Hennadii Stepanov)
c8d9914 ci, gha: Run "SageMath prover" job on GitHub Actions (Hennadii Stepanov)
f1774e5 ci, gha: Make MSVC job presentation more explicit (Hennadii Stepanov)
5ee039b ci: Remove "Windows (VS 2022)" task from Cirrus CI (Hennadii Stepanov)
a2f7ccd ci: Run "Windows (VS 2022)" job on GitHub Actions (Hennadii Stepanov)
175db31 ci: Drop no longer needed `PATH` variable update on Windows (Hennadii Stepanov)
116d2ab cmake: Set `ENVIRONMENT` property for examples on Windows (Hennadii Stepanov)
cef3739 cmake, refactor: Use helper function instead of interface library (Hennadii Stepanov)
a1bd497 refactor: take use of `secp256k1_scalar_{zero,one}` constants (part 2) (Sebastian Falbesoner)

Pull request description:

  [bitcoin-core/secp256k1#1391]: refactor: take use of `secp256k1_scalar_{zero,one}` constants (part 2)
  [bitcoin-core/secp256k1#1290]: cmake: Set `ENVIRONMENT` property for examples on Windows
  [bitcoin-core/secp256k1#1389]: ci: Run "Windows (VS 2022)" job on GitHub Actions
  [bitcoin-core/secp256k1#1397]: ci: Remove "Windows (VS 2022)" task from Cirrus CI
  [bitcoin-core/secp256k1#1399]: ci, gha: Run "SageMath prover" job on GitHub Actions
  [bitcoin-core/secp256k1#1400]: ctimetests: Use new SECP256K1_CHECKMEM macros also for ellswift
  [bitcoin-core/secp256k1#1348]: tighten group magnitude limits, save normalize_weak calls in group add methods (revival of #1032)
  [bitcoin-core/secp256k1#1402]: ci: Use Homebrew'\''s gcc in native macOS task
  [bitcoin-core/secp256k1#1274]: test: Silent noisy clang warnings about Valgrind code on macOS x86_64
  [bitcoin-core/secp256k1#1394]: ci, gha: Run "x86_64: macOS Ventura" job on GitHub Actions
  [bitcoin-core/secp256k1#1404]: ci: Remove "arm64: macOS Ventura" task from Cirrus CI
  [bitcoin-core/secp256k1#1062]: Removes `_fe_equal_var`, and unwanted `_fe_normalize_weak` calls (in tests)
  [bitcoin-core/secp256k1#1401]: ci, gha: Run all MSVC tests on Windows natively
  [bitcoin-core/secp256k1#1373]: Add invariant checking for scalars
  [bitcoin-core/secp256k1#1403]: ci, gha: Ensure only a single workflow processes `github.ref` at a time
  [bitcoin-core/secp256k1#1398]: ci, gha: Add Windows jobs based on Linux image
  [bitcoin-core/secp256k1#1405]: ci: Drop no longer needed workaround
  [bitcoin-core/secp256k1#1396]: ci, gha: Add "x86_64: Linux (Debian stable)" GitHub Actions job
  [bitcoin-core/secp256k1#1406]: ci, gha: Move more non-x86_64 tasks from Cirrus CI to GitHub Actions
  [bitcoin-core/secp256k1#1410]: ci: Use concurrency for pull requests only
  [bitcoin-core/secp256k1#1409]: ci: Move remained task from Cirrus to GitHub Actions
  [bitcoin-core/secp256k1#1411]: ci: Make repetitive command the default one
  [bitcoin-core/secp256k1#1412]: ci: Switch macOS from Ventura to Monterey and add Valgrind
  [bitcoin-core/secp256k1#1414]: ci/gha: Add ARM64 QEMU jobs for clang and clang-snapshot
  [bitcoin-core/secp256k1#1413]: ci: Add `release` job
  [bitcoin-core/secp256k1#1415]: release: Prepare for 0.4.0
  [bitcoin-core/secp256k1#1417]: release cleanup: bump version after 0.4.0
  [bitcoin-core/secp256k1#1390]: tests: Replace counting_illegal_callbacks with CHECK_ILLEGAL_VOID
  [bitcoin-core/secp256k1#1416]: doc: Align documented scripts with CI ones
  [bitcoin-core/secp256k1#1422]: cmake: Install `libsecp256k1.pc` file
  [bitcoin-core/secp256k1#1424]: ci: Bump major versions for docker actions
  [bitcoin-core/secp256k1#1395]: tests: simplify `random_fe_non_zero` (remove loop limit and unneeded normalize)

  This PR can be recreated with `./contrib/sync-upstream.sh -b master range ee7aaf2`.
  Tip: Use `git show --remerge-diff` to show the changes manually added to the merge commit.' --web

  - Replace fe_equal_var with fe_equal
  - Use CHECK_ILLEGAL instead of CHECK/ecount
  - Turn on secp256k1-zkp specific modules in CI

ACKs for top commit:
  real-or-random:
    ACK b41caaa

Tree-SHA512: e8994fa2590d6e277d60a130d3f11b0256dc7fbe67e8a2b85463d9849d3f9fbfccf366b17b3904f1974f8b7b27a24128d2a04a3337c1b8409d000d136714bf90

.github/actions/install-homebrew-valgrind/action.yml

@@ -0,0 +1,33 @@
+name: "Install Valgrind"
+description: "Install Homebrew's Valgrind package and cache it."
+runs:
+  using: "composite"
+  steps:
+    - run: |
+        brew tap LouisBrunner/valgrind
+        brew fetch --HEAD LouisBrunner/valgrind/valgrind
+        echo "CI_HOMEBREW_CELLAR_VALGRIND=$(brew --cellar valgrind)" >> "$GITHUB_ENV"
+      shell: bash
+
+    - run: |
+        sw_vers > valgrind_fingerprint
+        brew --version >> valgrind_fingerprint
+        git -C "$(brew --cache)/valgrind--git" rev-parse HEAD >> valgrind_fingerprint
+        cat valgrind_fingerprint
+      shell: bash
+
+    - uses: actions/cache@v3
+      id: cache
+      with:
+        path: ${{ env.CI_HOMEBREW_CELLAR_VALGRIND }}
+        key: ${{ github.job }}-valgrind-${{ hashFiles('valgrind_fingerprint') }}
+
+    - if: steps.cache.outputs.cache-hit != 'true'
+      run: |
+        brew install --HEAD LouisBrunner/valgrind/valgrind
+      shell: bash
+
+    - if: steps.cache.outputs.cache-hit == 'true'
+      run: |
+        brew link valgrind
+      shell: bash

.github/actions/run-in-docker-action/action.yml

@@ -0,0 +1,49 @@
+name: 'Run in Docker with environment'
+description: 'Run a command in a Docker container, while passing explicitly set environment variables into the container.'
+inputs:
+  dockerfile:
+    description: 'A Dockerfile that defines an image'
+    required: true
+  tag:
+    description: 'A tag of an image'
+    required: true
+  command:
+    description: 'A command to run in a container'
+    required: false
+    default: ./ci/ci.sh
+runs:
+  using: "composite"
+  steps:
+    - uses: docker/setup-buildx-action@v3
+
+    - uses: docker/build-push-action@v5
+      id: main_builder
+      continue-on-error: true
+      with:
+        context: .
+        file: ${{ inputs.dockerfile }}
+        tags: ${{ inputs.tag }}
+        load: true
+        cache-from: type=gha
+
+    - uses: docker/build-push-action@v5
+      id: retry_builder
+      if: steps.main_builder.outcome == 'failure'
+      with:
+        context: .
+        file: ${{ inputs.dockerfile }}
+        tags: ${{ inputs.tag }}
+        load: true
+        cache-from: type=gha
+
+    - # Tell Docker to pass environment variables in `env` into the container.
+      run: >
+        docker run \
+          $(echo '${{ toJSON(env) }}' | jq -r 'keys[] | "--env \(.) "') \
+          --volume ${{ github.workspace }}:${{ github.workspace }} \
+          --workdir ${{ github.workspace }} \
+          ${{ inputs.tag }} bash -c "
+            git config --global --add safe.directory ${{ github.workspace }}
+            ${{ inputs.command }}
+          "
+      shell: bash