Fix bug in prefetch cleanup

[knizhnik]

Problem

Running test_pageserver_restarts_under_workload in POR #7275 I get the following assertion failure in prefetch:

#5  0x00005587220d4bf0 in ExceptionalCondition (
    conditionName=0x7fbf24d003c8 "(ring_index) < MyPState->ring_unused && (ring_index) >= MyPState->ring_last", 
    fileName=0x7fbf24d00240 "/home/knizhnik/neon.main//pgxn/neon/pagestore_smgr.c", lineNumber=644)
    at /home/knizhnik/neon.main//vendor/postgres-v16/src/backend/utils/error/assert.c:66
#6  0x00007fbf24cebc9b in prefetch_set_unused (ring_index=1509) at /home/knizhnik/neon.main//pgxn/neon/pagestore_smgr.c:644
#7  0x00007fbf24cec613 in prefetch_register_buffer (tag=..., force_latest=0x0, force_lsn=0x0)
    at /home/knizhnik/neon.main//pgxn/neon/pagestore_smgr.c:891
#8  0x00007fbf24cef21e in neon_prefetch (reln=0x5587233b7388, forknum=MAIN_FORKNUM, blocknum=14110)
    at /home/knizhnik/neon.main//pgxn/neon/pagestore_smgr.c:2055

(gdb) p ring_index
$1 = 1509
(gdb) p MyPState->ring_unused
$2 = 1636
(gdb) p MyPState->ring_last
$3 = 1636

Summary of changes

Check status of prefetch_wait_for

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.
• Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
• If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

• Do not forget to reformat commit message to not include the above checklist

[knizhnik]

@MMeent please look at prefetch_set_unused code:

prefetch_set_unused(uint64 ring_index)
{
	PrefetchRequest *slot = GetPrfSlot(ring_index);

	if (ring_index < MyPState->ring_last)
		return;					/* Should already be unused */

	Assert(MyPState->ring_unused > ring_index);

GetPrfSlot contains assertion check:

#define GetPrfSlot(ring_index) ( \
	( \
		AssertMacro((ring_index) < MyPState->ring_unused && \
					(ring_index) >= MyPState->ring_last), \
		&MyPState->prf_buffer[((ring_index) % readahead_buffer_size)] \
	) \
)

which makes this checks contradictory or redundant.
If we really assume that prefetch_set_unused can be called for already unused slot, then we should move this check before
GetPrfSlot:

prefetch_set_unused(uint64 ring_index)
{
	PrefetchRequest *slot;

	if (ring_index < MyPState->ring_last)
		return;					/* Should already be unused */

	slot = GetPrfSlot(ring_index);
	if (slot->status == PRFS_UNUSED)
		return;

[github-actions]

2748 tests run: 2608 passed, 0 failed, 140 skipped (full report)

---

Code coverage* (full report)

• functions: 28.1% (6385 of 22723 functions)
• lines: 47.0% (44996 of 95824 lines)

* collected from Rust tests only

---

_{The comment gets automatically updated with the latest test results
c902a19 at 2024-04-03T15:36:26.997Z :recycle:}