Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Set SO_REUSEADDR on epoll tcp listener sockets #4544

Merged
merged 1 commit into from
Sep 11, 2024
Merged

Set SO_REUSEADDR on epoll tcp listener sockets #4544

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions src/platform/datapath_epoll.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -926,6 +926,29 @@ CxPlatSocketContextInitialize(
goto Exit;
}
}
} else if (SocketType == CXPLAT_SOCKET_TCP_LISTENER) {
//
// Set SO_REUSEADDR on listener sockets to avoid
// TIME_WAIT state on shutdown.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am a bit confused. The listener socket itself can go from listen state to timewait state? I thought a socket had to go through finwait1 and finwait2 to enter timewait state (which implicitly means only sockets who initiate graceful shutdown).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, on linux a listener socket itself can go into timewait. Its a bit weird. But it was extremely easy to reproduce.

Copy link
Contributor

@csujedihy csujedihy Sep 11, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mind sharing how it could be repro'd?

Just create a listener socket and have a connection accepted and kill both the accepted connection and the listener socket itself? Then bind to the same listening port will fail?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup. Have the listener socket accept a connection, and then kill the listener side. The listener side then will not be able to restart due to a bind failure.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am really surprised linux has this weird behavior as it violates RFC793.

image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't the listener socket follow the left side of that tree down to TIME_WAIT?

Copy link
Contributor

@csujedihy csujedihy Sep 11, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The socket that's accepted from the listener socket would follow the left side but the listener socket itself, in my understanding, would not. When you call accept() on a listener socket, you get a new socket (in SynRcvd or ESTAB state) and the listener itself should stay in listening state.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was a bit confusing to me too. libuv actually sets SO_REUSEADDR for both Client initiated sockets and listener sockets unconditionally, but at least for my use case I didn't need it set on the client side, and thats a little more invasive too.

//
Option = TRUE;
Result =
setsockopt(
SocketContext->SocketFd,
SOL_SOCKET,
SO_REUSEADDR,
(const void*)&Option,
sizeof(Option));
if (Result == SOCKET_ERROR) {
Status = errno;
QuicTraceEvent(
DatapathErrorStatus,
"[data][%p] ERROR, %u, %s.",
Binding,
Status,
"setsockopt(SO_REUSEPORT) failed");
goto Exit;
}
}

CxPlatCopyMemory(&MappedAddress, &Binding->LocalAddress, sizeof(MappedAddress));
Expand Down
Loading