Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Always use configured host on preconnect #14

Merged
merged 3 commits into from
Aug 20, 2020
Merged

Always use configured host on preconnect #14

merged 3 commits into from
Aug 20, 2020

Conversation

jeffalder
Copy link
Contributor

@jeffalder jeffalder commented Aug 11, 2020 •
edited
Loading

At least two language agents always go back to their original configured host on preconnect. The Java agent will make the second round of preconnect on the redirect host.

This is currently causing issues because calling preconnect on the gov-collector-NNN hosts redirects you back to the collector-NNN shards. This is wrong and will cause LicenseException if they are prevented from allowing anything into the default backend.

This fixes that problem by always calling preconnect on the configured host (e.g., gov-collector.newrelic.com) and then updating the redirect host from there.

Auto app naming will create multiple instances of DataSenderImpl, so those won't conflict. This was tested by me on staging through a ForceRestartException.

Example of bad behavior:

com.newrelic INFO: Sent JSON(preconnect) to: https://gov-collector-001.newrelic.com:443/agent_listener/invoke_raw_method?method=preconnect&license_key=--elided--&marshal_format=json&protocol_version=17, with payload: ...
com.newrelic INFO: Received JSON(preconnect): {"return_value":{"redirect_host":"collector-001.newrelic.com"}}
com.newrelic INFO: Collector redirection to collector-001.newrelic.com:443

Note that yes, this is actually a collector problem, which we are pursuing.

This is after the change. Note that preconnect goes to gov-collector.newrelic.com and after the redirect, we're still pointing to gov-collector-007.

com.newrelic WARN: Received a ForceRestartException: com.newrelic.agent.ForceRestartException: . The agent will attempt to reconnect for data reporting. If this message continues, please contact support@newrelic.com.
com.newrelic INFO: Sent JSON(shutdown) to: https://gov-collector-007.newrelic.com:443/agent_listener/invoke_raw_method?method=shutdown&license_key=--elided--&marshal_format=json&protocol_version=17&run_id=run-token, with payload: ["run-token",1597103260686]
...
com.newrelic INFO: Sent JSON(preconnect) to: https://gov-collector.newrelic.com:443/agent_listener/invoke_raw_method?method=preconnect&license_key=--elided--&marshal_format=json&protocol_version=17, with payload: ...
com.newrelic INFO: Received JSON(preconnect): {"return_value":{"redirect_host":"gov-collector-007.newrelic.com"}}
com.newrelic INFO: Collector redirection to gov-collector-007.newrelic.com:443

jeffalder
jeffalder commented Aug 11, 2020
View reviewed changes
newrelic-agent/src/main/java/com/newrelic/agent/transport/DataSenderImpl.java
@@ -207,7 +209,7 @@ private String parsePreconnectAndReturnHost() throws Exception {
}
token.put("high_security", agentConfig.isHighSecurity());
params.add(token);
Object response = invokeNoRunId(CollectorMethods.PRECONNECT, compressedEncoding, params);
Object response = invokeNoRunId(originalHost, CollectorMethods.PRECONNECT, compressedEncoding, params);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

preconnect should always use the original collector host.

jeffalder
jeffalder commented Aug 11, 2020
View reviewed changes
newrelic-agent/src/main/java/com/newrelic/agent/transport/DataSenderImpl.java
@@ -232,7 +234,7 @@ private String parsePreconnectAndReturnHost() throws Exception {
startupOptions.put(ENV_METADATA, metadata);

params.add(startupOptions);
Object response = invokeNoRunId(CollectorMethods.CONNECT, compressedEncoding, params);
Object response = invokeNoRunId(redirectHost, CollectorMethods.CONNECT, compressedEncoding, params);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

connect should always use the redirect host.

jeffalder
jeffalder commented Aug 11, 2020
View reviewed changes
newrelic-agent/src/main/java/com/newrelic/agent/transport/DataSenderImpl.java
@@ -487,17 +489,17 @@ void setMaxPayloadSizeInBytes(int payloadSizeInBytes) {

private Object invokeRunId(String method, String encoding, Object runId, JSONStreamAware params) throws Exception {
String uri = MessageFormat.format(agentRunIdUriPattern, method, runId.toString());
return invoke(method, encoding, uri, params);
return invoke(redirectHost, method, encoding, uri, params);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything that takes a run token should use the redirect host.

@jeffalder
Copy link
Contributor Author

@tspring LMK if you want me to merge this or you will. Thanks!

breedx-nr
breedx-nr reviewed Aug 18, 2020
View reviewed changes
newrelic-agent/src/test/java/com/newrelic/agent/transport/DataSenderImplConnectCycleTest.java
assertEquals("value", result.get("other"));

// This will assert that preconnect was called with the original host, not the redirect host.
target.connect(startupOptions);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is unclear to me how this would do any assertions around preconnect. Does this test actually check what this comment says it is checking? If so, how does it do that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests use the DataSenderImpl with a customized HttpClientWrapper. HttpClientWrapper is the interface that masks almost all the Apache HTTP client code. On line 80, this test applies the ConnectCycleNoSecuritySuccessClientWrapper wrapper. When a preconnect URL comes through that wrapper, it verifies that the host is no-collector.example.com and sends a redirect host of new_host.example.com.

Without the code change, this code would fail because the second preconnect would use the host new_host.example.com.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, thanks for clarifying...because that's pretty complicated for testing. At least it is tested. Thanks!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah - probably should have done it with mockito.

@breedx-nr breedx-nr merged commit 010b9fb into main Aug 20, 2020
@breedx-nr breedx-nr deleted the preconnect-original branch August 20, 2020 20:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@breedx-nr breedx-nr breedx-nr left review comments

@tspring tspring tspring approved these changes

Assignees
No one assigned
Labels
None yet
Projects
Java Engineering Board
Archived in project
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jeffalder @tspring @breedx-nr