Skip to content

Commit

Permalink
neon_walreader: check after local read that the segment still exists.
Browse files Browse the repository at this point in the history
Otherwise read might receive zeros/garbage if the file is recycled (renamed) for
as a future segment.
  • Loading branch information
arssher committed May 31, 2024
1 parent 98dadf8 commit e6db806
Showing 1 changed file with 20 additions and 2 deletions.
22 changes: 20 additions & 2 deletions pgxn/neon/neon_walreader.c
Original file line number Diff line number Diff line change
Expand Up @@ -184,8 +184,8 @@ NeonWALRead(NeonWALReader *state, char *buf, XLogRecPtr startptr, Size count, Ti
}
else if (state->wre_errno == ENOENT)
{
nwr_log(LOG, "local read failed as segment at %X/%X doesn't exist, attempting remote",
LSN_FORMAT_ARGS(startptr));
nwr_log(LOG, "local read at %X/%X len %zu failed as segment file doesn't exist, attempting remote",
LSN_FORMAT_ARGS(startptr), count);
return NeonWALReadRemote(state, buf, startptr, count, tli);
}
else
Expand Down Expand Up @@ -614,6 +614,7 @@ NeonWALReadLocal(NeonWALReader *state, char *buf, XLogRecPtr startptr, Size coun
uint32 startoff;
int segbytes;
int readbytes;
XLogSegNo lastRemovedSegNo;

startoff = XLogSegmentOffset(recptr, state->segcxt.ws_segsize);

Expand Down Expand Up @@ -689,6 +690,23 @@ NeonWALReadLocal(NeonWALReader *state, char *buf, XLogRecPtr startptr, Size coun
return false;
}

/*
* Recheck that the segment hasn't been removed while we were reading
* it.
*/
lastRemovedSegNo = XLogGetLastRemovedSegno();
if (state->seg.ws_segno <= lastRemovedSegNo)
{
char fname[MAXFNAMELEN];

state->wre_errno = ENOENT;

XLogFileName(fname, tli, state->seg.ws_segno, state->segcxt.ws_segsize);
snprintf(state->err_msg, sizeof(state->err_msg), "WAL segment %s has been removed during the read, lastRemovedSegNo " UINT64_FORMAT,
fname, lastRemovedSegNo);
return false;
}

/* Update state for read */
recptr += readbytes;
nbytes -= readbytes;
Expand Down

1 comment on commit e6db806

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3238 tests run: 3092 passed, 1 failed, 145 skipped (full report)


Failures on Postgres 14

  • test_download_churn[github-actions-selfhosted-100-tokio-epoll-uring-30]: release
# Run all failed tests locally:
scripts/pytest -vv -n $(nproc) -k "test_download_churn[release-pg14-github-actions-selfhosted-100-tokio-epoll-uring-30]"
Flaky tests (3)

Postgres 15

  • test_pageserver_restarts_under_worload: debug

Postgres 14

  • test_pageserver_restarts_under_worload: release
  • test_sharding_split_smoke: debug

Code coverage* (full report)

  • functions: 31.4% (6491 of 20681 functions)
  • lines: 48.4% (50206 of 103783 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
e6db806 at 2024-05-31T11:19:24.025Z :recycle:

Please sign in to comment.