Fix connect to PS on MacOS/X

[knizhnik]

Problem

After [0e4f182] which introduce async connect
Neon is not able to connect to page server.

Summary of changes

Perform sync commit at MacOS/X

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.
• Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
• If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

• Do not forget to reformat commit message to not include the above checklist

[github-actions]

3138 tests run: 3005 passed, 0 failed, 133 skipped (full report)

---

Code coverage* (full report)

• functions: 31.4% (6447 of 20543 functions)
• lines: 48.3% (49949 of 103355 lines)

* collected from Rust tests only

---

_{The comment gets automatically updated with the latest test results
dc09a95 at 2024-05-27T09:58:51.470Z :recycle:}

[kelvich]

Feel free to commit as is to unblock yourself.

But what is the root cause here? async connect should works on macOS as well

[knizhnik]

Feel free to commit as is to unblock yourself.

Review from compute/safekeepr team is required

But what is the root cause here? async connect should works on macOS as well

I wrote my hypothesis is Slack: WaitEventSet is implemented at MacOS using kevent.
Looks like there are some issues with kevent when there are different events registered for the same socket.

Because if I use WaitLatchOrSocket (which creates wait event set and delete at the end) instead of WaitEventSetWait then connection is established but then still blocked in read.

In any case it was not the first time I found bug in MacOS (or may be even FreeBSD) socket implementation. So I am not so surprised.

It will be excellent if somebody managed to make ti work but I do not want spent more time on it.

[kelvich]

I'm less worried about macOS itself, then the fact that this issue could be a manifestation of some bug or incorrect usage in async code.

Looks like there are some issues with kevent when there are different events registered for the same socket.

pgfarm has both macos and freebsd. Async libpq is used logical replication tests, so it is covered by that tests.

[knizhnik]

I'm less worried about macOS itself, then the fact that this issue could be a manifestation of some bug or incorrect usage in async code.

Looks like there are some issues with kevent when there are different events registered for the same socket.

pgfarm has both macos and freebsd. Async libpq is used logical replication tests, so it is covered by that tests.

There are two suspicious places in async connect implementation in libpagestore.c (comparing with fe-connect.c and libpqwalreceiver.c):

1. It is calling PQconnectPoll before WaitEventSetWait
2. wes_write includes WL_SOCKET_READABLE
   But fixing this issues does't solve the problem.

So the only real difference with libpqwalreceiver.c is that last one uses WaitLatchOrSocket which creates temporary wait even set and releases it after operation. And @MMeent implementation precreates two event sets which are reused in several operations. This two event sets refers to the same socket. Which may cause some problems, although I have not founding internet that it can not be done or there are some related bugs.

[knizhnik]

Ok, I managed to make async connect top PS work also at MacOS, doing exactly the same as in libpqwalreceiver.c

[kelvich]

Nice! I vaguely remember having a similar issues with macOS in DMQ. (IIRC in some cases it also might have resulted in stuck connection attempts in linux; rewriting exactly as it is in walreceiver helped with that too).

[kelvich]

(Don't forget to squash your commits into one)

[knizhnik]

I still waiting for approval from somebody from @neondatabase/compute or @neondatabase/safekeepers team.

[petuhovskiy]

I see nothing that prevents merging, you can merge now. @knizhnik