Add On-demand WAL Download to logicalfuncs

[save-buffer]

We implemented on-demand WAL download for walsender, but other things that may want to read the WAL from safekeepers don't do that yet. This PR makes it do that by adding the same set of hooks to logicalfuncs.

Addresses #7959

Also relies on:
neondatabase/postgres#438
neondatabase/postgres#437
neondatabase/postgres#436

[github-actions]

3204 tests run: 3062 passed, 0 failed, 142 skipped (full report)

---

Flaky tests (2)

Postgres 16

• test_scrubber_tenant_snapshot[4]: release
• test_storage_controller_smoke: debug

Code coverage* (full report)

• functions: 31.6% (6625 of 20990 functions)
• lines: 48.6% (51490 of 106045 lines)

* collected from Rust tests only

---

_{The comment gets automatically updated with the latest test results
93c79c3 at 2024-06-11T21:13:39.625Z :recycle:}

[jcsp]

I'm a reviewer on this PR, but I have no idea what it's about. Please always write a description that fills out the PR template -- that's helpful even if a linked issue is provided, but in this case the link goes to a PR that has an empty description too.

[save-buffer]

Sorry, I did have an issue that links to a slack thread but I didn't paste correctly. I've added a description

[kelvich]

@save-buffer that PR would benefit from a test that can reproduce the issue without a fix and would pass with fix :-)

[kelvich]

Error message looks like that:

PG:2024-06-06 16:22:46.537 GMT ttid=1d880e086b24349586dc1ee10ebea34e/1921f4934124c56e8e937fefffe88bfa
sqlstate=XX000 [6370]
ERROR:  invalid magic number 0000 in log segment 00000001000000000000000F, offset 12206080

(posting here so that search on the error message could find this issue; originally i went in https://github.com/neondatabase/cloud/issues/12992).

[kelvich]

Also double-checked that user faces invalid magic number 0000 in log segment when calling pg_logical_slot_peek_binary_changes, which is turn calls pg_logical_slot_get_changes_guts. So everything checks out. https://neonprod.grafana.net/goto/D-SdG_sIR?orgId=1

[save-buffer]

Added a test

[save-buffer]

Without callbacks:

psycopg2.errors.InternalError_: invalid record length at 0/2ADA620: wanted 24, got 0

With callbacks: test passes

[kelvich]

@knizhnik and/or @ars pls take a look too as you are marked as code owners here

[arssher]

Thanks! Let's also run pgindent on C part.