Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

client/v3: clear auth token when encounter ErrInvalidAuthToken #12549

Merged
merged 1 commit into from
Dec 30, 2020

Conversation

bbiao
Copy link
Contributor

@bbiao bbiao commented Dec 14, 2020

Old etcdserver which have not apply pr of #12165 will check auth token
even if the request is an Authenticate request.

If the client has a invalid auth token, it will not able to update it's
token, since the Authenticate has a invalid auth token.
This fix clear the auth token when encounter an ErrInvalidAuthToken to
talk with old version etcd servers.
With pr #12165 and #12264 this will fix the problem in #12385

Fix #12385

// call c.Auth.Authenticate with an invalid token will always fail the auth check on the server-side,
// if the server has not apply the patch of pr #12165 (https://github.com/etcd-io/etcd/pull/12165)
// and a rpctypes.ErrInvalidAuthToken will recursively call c.getToken until system run out of resource.
c.authTokenBundle.UpdateAuthToken("")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the fix @bbiao , I think it would work fine but let me make sure it works as expected.

Copy link
Contributor

@mitake mitake left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks a lot for fixing @bbiao ! Defer to @jingyih @gyuho @jpbetz

Copy link
Contributor

@cfc4n cfc4n left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, Thanks.

@@ -74,6 +74,12 @@ func (c *Client) unaryClientInterceptor(logger *zap.Logger, optFuncs ...retryOpt
continue
}
if callOpts.retryAuth && rpctypes.Error(lastErr) == rpctypes.ErrInvalidAuthToken {
// clear auth token befault refreshing it.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit, typo "befault". Also, don't we need a similar update for https://github.com/etcd-io/etcd/blob/master/client/v3/retry_interceptor.go#L242 for a similar call to getToken when ErrInvalidAuthToken? Thanks! /cc @mitake @cfc4n

@spzala
Copy link
Member

spzala commented Dec 24, 2020

@bbiao thanks for addressing my one comment, how about the second comment/question?

@bbiao
Copy link
Contributor Author

bbiao commented Dec 25, 2020

@bbiao thanks for addressing my one comment, how about the second comment/question?

It's ok not to clear auth token when encounter the ErrInvalidAuthToken at https://github.com/etcd-io/etcd/blob/master/client/v3/retry_interceptor.go#L242, since the getToken itself is a unary rpc call, when failed it will retry, and at that moment the unary interceptor will clean the auth token.

But I think it's better to clean the auth token too at https://github.com/etcd-io/etcd/blob/master/client/v3/retry_interceptor.go#L242, this will avoid a never-success rpc call.

@spzala
Copy link
Member

spzala commented Dec 25, 2020

@bbiao thanks for addressing my one comment, how about the second comment/question?

It's ok not to clear auth token when encounter the ErrInvalidAuthToken at https://github.com/etcd-io/etcd/blob/master/client/v3/retry_interceptor.go#L242, since the getToken itself is a unary rpc call, when failed it will retry, and at that moment the unary interceptor will clean the auth token.

But I think it's better to clean the auth token too at https://github.com/etcd-io/etcd/blob/master/client/v3/retry_interceptor.go#L242, this will avoid a never-success rpc call.

@bbiao lgtm, thanks for addressing the comments, can you please squash commits now?

Old etcdserver which have not apply pr of etcd-io#12165 will check auth token
even if the request is an Authenticate request.
If the client has a invalid auth token, it will not able to update it's
token, since the Authenticate has a invalid auth token.
This fix clear the auth token when encounter an ErrInvalidAuthToken to
talk with old version etcd servers.

Fix etcd-io#12385 with etcd-io#12165 and etcd-io#12264
@bbiao
Copy link
Contributor Author

bbiao commented Dec 27, 2020

@spzala commits have been squashed ;-)

@mitake
Copy link
Contributor

mitake commented Dec 28, 2020

Travis is failing because of linux-amd64-integration-4-cpu (the case is TestMemberPromote) I think this is a non deterministic one. Also semaphore failed because of TestIssue6361 but on my local env the case could run successfully on the branch. I'll run semaphore again for checking. But I think the PR can be merged.

@mitake
Copy link
Contributor

mitake commented Dec 28, 2020

@spzala do you know how to rerun semaphore CI? I found that I cannot do that...

@spzala
Copy link
Member

spzala commented Dec 28, 2020

@mitake there used to be a rerun option but I also don't see it. Need to investigate it, for now, I think closing/reopening PR is one quick option. As you said, failure seems not related to the changes.

@spzala spzala closed this Dec 28, 2020
@spzala spzala reopened this Dec 28, 2020
@spzala
Copy link
Member

spzala commented Dec 30, 2020

@mitake @bbiao the CI is green now :)

@spzala spzala merged commit a4570a6 into etcd-io:master Dec 30, 2020
@mitake
Copy link
Contributor

mitake commented Jan 2, 2021

@spzala yeah that's smart, thanks for handling!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

watch stream return "permission denied" if token expired
4 participants