Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent AnonymousKernelCommands from ending up inside serialized KernelEvents #2888

Merged
merged 2 commits into from
Apr 7, 2023

Conversation

shyamnamboodiripad
Copy link
Contributor

AnonymousKernelCommands are not supposed to be serialized across process boundaries and should therefore not be included as part of KernelEvents.

…elEvents.

AnonymousKernelCommands are not supposed to be serialized across process boundaries and should therefore not be included as part of KernelEvents.
@shyamnamboodiripad shyamnamboodiripad enabled auto-merge (squash) April 6, 2023 03:41
@@ -365,6 +366,15 @@ private static SubmissionParser CreateSubmissionParser(string defaultLanguage =
return compositeKernel.SubmissionParser;
}

[Fact]
public async Task DiagnosticsProduced_events_always_point_back_to_the_original_command()
Copy link
Contributor

@jonsequitur jonsequitur Apr 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is intended to be generally true, so not specific to DiagnosticsProduced. The internal command (AnonymousKernelCommand in this case, but it could as easily be one of the public command types) is an implementation detail that the external caller shouldn't have to be aware of.

It might be useful to add another similar test for the split between subkernels because the same should be true for, for example:

#!csharp
123
#!fsharp
456

This should apply to command types including various language service commands.

Copy link
Contributor Author

@shyamnamboodiripad shyamnamboodiripad Apr 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So looks like this invariant is not satisfied for all events in the above case. For example, CodeSubmissionReceived, CompleteCodeSubmissionReceived, ReturnValueProduced events for the repro code you shared above point back to the per-language SubmitCode commands created internally for C# and F# respectively. Only the final CommandSucceded event points back to the original SubmitCode command.

If I change the submission to the following then interestingly, the DiagnosticsProduced event for the erroneous line points to the F#-specific SubmitCode command with code some error. However, the line spans in the DiagnosticsProduced.Diagnostics reflect the spans in the original submission.

#!csharp
123
#!fsharp
some error

This may be reasonable considering that the inner commands reflect the expected command splitting behavior - however, it also seems like internal implementation details of the callee are bleeding through to the caller.

If this is unexpected, I can log a separate bug to a) fix the behavior and b) add tests to ensure that inner commands are never exposed to the caller. Thoughts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jonsequitur Btw the current PR should be ready to merge. The above issue needs more investigation / follow-up and I'd prefer not to hold up the current PR for that. Please approve if you agree :) Thanks!

@@ -160,7 +160,6 @@ private bool CanHandleLocally(KernelCommand command)
switch (command)
{
case AnonymousKernelCommand:
return base.HandleAsync(command, context);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this could lead to a deadlock if the anonymous command targets the proxy. For sure this will throw exception as it will try to forward the command on the sender and will fail at serialization. This looks like a test gap to pinpoint

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes this PR is fixing one specific case where we were serializing AnonymousKernelCommand (inside SubmissionParser.cs). But the larger test gap has not been addressed yet - I have included this in #2884 and plan to address that in a follow up PR.

Copy link
Member

@colombod colombod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

anonymous command should not leave the current host and should be handled locally even by proxy kernel

@shyamnamboodiripad
Copy link
Contributor Author

anonymous command should not leave the current host and should be handled locally even by proxy kernel

@colombod Agreed. This is part of what Jon and I discussed, and I have included the larger problem in this follow up issue - #2884 which I plan to look into soon.

This PR addresses one known case where AnonymousKernelCommand ends up leaving the current host today (see the fix in SubmissionParser.cs line 127 in the current PR). The plan that @jonsequitur and I discussed was to merge this fix now and tackle the larger issue in subsequent PR. I was essentially waiting for him to take a second look and approve the PR today.

Note that I was planning to merge this change today since this is one of the issues that came up when testing the VS side integration. So would appreciate if you could remove the block on the PR :)

@colombod
Copy link
Member

colombod commented Apr 7, 2023

As a tactical solution I would rather add to the case a check to see if the handler is defined or not, if the anonymous command has a handler execute it otherwise skip it (this forces the command to stay in the current host and not leaving the proxy) .

@jonsequitur
Copy link
Contributor

Let's find some time to discuss how we want to do this while removing rather than adding code. There's too much complexity here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants