-
Notifications
You must be signed in to change notification settings - Fork 52
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: log DirectPath misconfiguration #2105
Changes from 3 commits
f93bfb7
6454a46
ec9116e
908a54d
752b4a3
f764786
1c5f4bb
b3ce533
cb685dc
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -64,6 +64,8 @@ | |
import java.util.concurrent.Executor; | ||
import java.util.concurrent.ScheduledExecutorService; | ||
import java.util.concurrent.TimeUnit; | ||
import java.util.logging.Level; | ||
import java.util.logging.Logger; | ||
import javax.annotation.Nullable; | ||
import javax.net.ssl.KeyManagerFactory; | ||
import org.threeten.bp.Duration; | ||
|
@@ -82,6 +84,8 @@ | |
*/ | ||
@InternalExtensionOnly | ||
public final class InstantiatingGrpcChannelProvider implements TransportChannelProvider { | ||
private static final Logger LOG = | ||
Logger.getLogger(InstantiatingGrpcChannelProvider.class.getName()); | ||
private static final String DIRECT_PATH_ENV_DISABLE_DIRECT_PATH = | ||
"GOOGLE_CLOUD_DISABLE_DIRECT_PATH"; | ||
private static final String DIRECT_PATH_ENV_ENABLE_XDS = "GOOGLE_CLOUD_ENABLE_DIRECT_PATH_XDS"; | ||
|
@@ -233,6 +237,7 @@ public TransportChannel getTransportChannel() throws IOException { | |
} | ||
|
||
private TransportChannel createChannel() throws IOException { | ||
logDirectPathMisconfig(); | ||
return GrpcTransportChannel.create( | ||
ChannelPool.create( | ||
channelPoolSettings, InstantiatingGrpcChannelProvider.this::createSingleChannel)); | ||
|
@@ -266,6 +271,33 @@ boolean isDirectPathXdsEnabled() { | |
return false; | ||
} | ||
|
||
private void logDirectPathMisconfig() { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. IIUC, this logic is added specifically for log statements. How do we make sure it is in parity with directPath decision making? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Are you referring to the bit that actually handles connecting to DirectPath in gRPC? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The DirectPath decision making is L363:
And I came up with the 4 cases based on it. WDYT? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What I meant was that this logic will be invoked only when directPath is misconfigured. But this is not the logic which decides if directPath is misconfigured. This code snippet could become stale if there is change in logic which is actually deciding when directpath is misconfigured. Just as an e.g. this is the code snippet which decides directPath misconfiguration
Here we check for I guess there is no easy way to enforce it and this is fine for now. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks for your inputs! There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I have the same concern as @singhravidutt, the logic in
This also makes the new code much more testable as SonarCheck is failing due to lacking test coverage of the new code. @mohanli-ml Let me know if it makes sense, if you think it might be too much at this moment, our team can take it as a follow up PR. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks for the suggestion! Moving the DirectPath enabling logic to the builder instead of checking it every time when construct a channel sounds good to me. However, pay attention to the logic here:
In short I feel the logic is already complex and I would suggest to do this with the cleanup of removing DirectPath gRPCLB, which is blocked by CBT and is expected to be done in Q4. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I think that's reasonable. For now, can you please add some unit tests for these logging statement? So that the test coverage for new code meets our least standard and gives us more confidence when we refactor it. For example, one way to test it is to create a FakeLogger and assert the log message captured in the FakeLogger. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done. Thanks! |
||
boolean isDirectPathOptionSet = Boolean.TRUE.equals(attemptDirectPath); | ||
boolean isDirectPathXdsEnvSet = | ||
Boolean.parseBoolean(envProvider.getenv(DIRECT_PATH_ENV_ENABLE_XDS)); | ||
boolean isDirectPathXdsOptionSet = Boolean.TRUE.equals(attemptDirectPathXds); | ||
if (isDirectPathXdsEnvSet || isDirectPathXdsOptionSet) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we already have isDirectPathXdsEnabled() for the same purpose? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done. |
||
// Case 1: does not enable DirectPath | ||
if (!isDirectPathOptionSet) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why not use isDirectPathEnabled()? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done. |
||
LOG.log( | ||
Level.WARNING, "DirectPath is misconfigured. Please set the attemptDirectPath option."); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @singhravidutt this PR is w.r.t R.1 for observability. Would hadoop connector customers be able to see these warnings emitted when running the connector or would this require a change? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, we should be able to surface these logs. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. should we add values of There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In this case user enables DirectPathXds either by env or option, but DirectPath itself is not enabled. I think the log is already clear that the user needs to set There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Recommend:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done. |
||
} else { | ||
// Case 2: credential is not correctly set | ||
if (!isNonDefaultServiceAccountAllowed()) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is it possible to have a There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes it is possible, this is the original fix for this problem, googleapis/gax-java#1250. |
||
LOG.log( | ||
Level.WARNING, | ||
"DirectPath is misconfigured. Please make sure the credential is an instance of" | ||
+ " ComputeEngineCredentials."); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. use fully qualified class name TIL you need GCE credentials for DirectPath; what is a customer is using Workload Identity would that work? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done. GKE workload identity is compatible with DirectPath. |
||
} | ||
// Case 3: not running on GCE | ||
if (!isOnComputeEngine()) { | ||
LOG.log( | ||
Level.WARNING, "DirectPath is misconfigured. Please run in the GCE environment. "); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Suggestion:
You also have whitespace at the end. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done. |
||
} | ||
} | ||
} | ||
} | ||
|
||
private boolean isNonDefaultServiceAccountAllowed() { | ||
if (allowNonDefaultServiceAccount != null && allowNonDefaultServiceAccount) { | ||
return true; | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
createChannel()
could still be called multiple times, I would suggest to log the info in the builderThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can not do this because
logDirectPathMisconfig()
is a non-static function so it can not be used before the construction. Can you suggest more?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we move
logDirectPathMisconfig()
into the Builder class then it should be fine, ideally we should do it but then we would need to move all related logics into it as well. The quickest solution would be add it to the constructor.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.