Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

getting stuck #3283

Open
superichmann opened this issue Oct 20, 2023 · 16 comments
Open

getting stuck #3283

superichmann opened this issue Oct 20, 2023 · 16 comments
Labels
bug Something isn't working waiting-on-feedback

Comments

@superichmann
Copy link

need to run initialize cell 3 times before it hits
notebooks will run infinity, need to restart vscode

can you fix?

@superichmann
Copy link
Author

Error: Failed to load kernel extension "AutoMLMonitorKernelExtension" from assembly C:\Users\x.nuget\packages\microsoft.ml.automl\0.21.0-preview.23406.1\interactive-extensions\dotnet\Microsoft.ML.AutoML.Interactive.dll
Error: Microsoft.DotNet.Interactive.KernelExtensionLoadException: Failure loading Kernel Extension
---> System.MissingMethodException: Method not found: 'Void Microsoft.DotNet.Interactive.Commands.SubmitCode..ctor(System.String, System.String, Microsoft.DotNet.Interactive.Commands.SubmissionType)'.
at Microsoft.ML.AutoML.AutoMLMonitorKernelExtension.LoadExtensionApiAsync(Kernel cSharpKernel)
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
at Microsoft.ML.AutoML.AutoMLMonitorKernelExtension.LoadExtensionApiAsync(Kernel cSharpKernel)
at Microsoft.ML.AutoML.AutoMLMonitorKernelExtension.OnLoadAsync(Kernel kernel)
at Microsoft.DotNet.Interactive.App.PackageDirectoryExtensionLoader.LoadFromAssemblyFile(FileInfo assemblyFile, Kernel kernel, KernelInvocationContext context) in D:\a_work\1\s\src\dotnet-interactive\PackageDirectoryExtensionLoader.cs:line 117
--- End of inner exception stack trace ---

@superichmann
Copy link
Author

oh and clicking stop does not actually stop the execution

@superichmann
Copy link
Author

Error: Microsoft.DotNet.Interactive.NoSuitableKernelException: No kernel found for Microsoft.DotNet.Interactive.Commands.RequestHoverText with target kernel '.NET'.
at Microsoft.DotNet.Interactive.Commands.KernelCommand.InvokeAsync(KernelInvocationContext context) in D:\a_work\1\s\src\Microsoft.DotNet.Interactive\Commands\KernelCommand.cs:line 184
at Microsoft.DotNet.Interactive.Kernel.HandleAsync(KernelCommand command, KernelInvocationContext context) in D:\a_work\1\s\src\Microsoft.DotNet.Interactive\Kernel.cs:line 323
at Microsoft.DotNet.Interactive.CompositeKernel.HandleAsync(KernelCommand command, KernelInvocationContext context) in D:\a_work\1\s\src\Microsoft.DotNet.Interactive\CompositeKernel.cs:line 183
at Microsoft.DotNet.Interactive.KernelCommandPipeline.b__6_0(KernelCommand command, KernelInvocationContext context, KernelPipelineContinuation _) in D:\a_work\1\s\src\Microsoft.DotNet.Interactive\KernelCommandPipeline.cs:line 60
at Microsoft.DotNet.Interactive.KernelCommandPipeline.<>c__DisplayClass6_1.<b__3>d.MoveNext() in D:\a_work\1\s\src\Microsoft.DotNet.Interactive\KernelCommandPipeline.cs:line 75
--- End of stack trace from previous location ---
at Microsoft.DotNet.Interactive.App.KernelExtensions.<>c__DisplayClass3_0.<b__0>d.MoveNext() in D:\a_work\1\s\src\dotnet-interactive\KernelExtensions.cs:line 124
--- End of stack trace from previous location ---
at Microsoft.DotNet.Interactive.KernelCommandPipeline.<>c__DisplayClass6_1.<b__3>d.MoveNext() in D:\a_work\1\s\src\Microsoft.DotNet.Interactive\KernelCommandPipeline.cs:line 75
--- End of stack trace from previous location ---
at Microsoft.DotNet.Interactive.App.KernelExtensionLoader.<>c__DisplayClass0_0.<b__0>d.MoveNext() in D:\a_work\1\s\src\dotnet-interactive\KernelExtensionLoader.cs:line 21
--- End of stack trace from previous location ---
at Microsoft.DotNet.Interactive.KernelCommandPipeline.<>c__DisplayClass6_0.<g__Combine|2>d.MoveNext() in D:\a_work\1\s\src\Microsoft.DotNet.Interactive\KernelCommandPipeline.cs:line 73
--- End of stack trace from previous location ---
at Microsoft.DotNet.Interactive.KernelCommandPipeline.<>c__DisplayClass6_0.<g__Combine|2>d.MoveNext() in D:\a_work\1\s\src\Microsoft.DotNet.Interactive\KernelCommandPipeline.cs:line 73
--- End of stack trace from previous location ---
at Microsoft.DotNet.Interactive.KernelCommandPipeline.SendAsync(KernelCommand command, KernelInvocationContext context) in D:\a_work\1\s\src\Microsoft.DotNet.Interactive\KernelCommandPipeline.cs:line 41
Loading extensions from C:\Users\x\.nuget\packages\microsoft.ml.automl\0.21.0-preview.23406.1\interactive-extensions\dotnet\Microsoft.ML.AutoML.Interactive.dll
Error: Failed to load kernel extension "AutoMLMonitorKernelExtension" from assembly C:\Users\x.nuget\packages\microsoft.ml.automl\0.21.0-preview.23406.1\interactiv

@superichmann
Copy link
Author

so many bugs...

@shyamnamboodiripad
Copy link
Contributor

Error: Failed to load kernel extension "AutoMLMonitorKernelExtension" from assembly C:\Users\x.nuget\packages\microsoft.ml.automl\0.21.0-preview.23406.1\interactive-extensions\dotnet\Microsoft.ML.AutoML.Interactive.dll Error: Microsoft.DotNet.Interactive.KernelExtensionLoadException: Failure loading Kernel Extension ---> System.MissingMethodException: Method not found: 'Void Microsoft.DotNet.Interactive.Commands.SubmitCode..ctor(System.String, System.String, Microsoft.DotNet.Interactive.Commands.SubmissionType)'. at Microsoft.ML.AutoML.AutoMLMonitorKernelExtension.LoadExtensionApiAsync(Kernel cSharpKernel) at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.ML.AutoML.AutoMLMonitorKernelExtension.LoadExtensionApiAsync(Kernel cSharpKernel) at Microsoft.ML.AutoML.AutoMLMonitorKernelExtension.OnLoadAsync(Kernel kernel) at Microsoft.DotNet.Interactive.App.PackageDirectoryExtensionLoader.LoadFromAssemblyFile(FileInfo assemblyFile, Kernel kernel, KernelInvocationContext context) in D:\a_work\1\s\src\dotnet-interactive\PackageDirectoryExtensionLoader.cs:line 117 --- End of inner exception stack trace ---

Looks like this may be happening because you are using an older version of the AutoML package against latest version of polyglot notebooks extension.

Could you try updating to the latest version from nuget.org? This version which was released last week may fix the error https://www.nuget.org/packages/Microsoft.ML.AutoML/0.21.0-preview.23511.1https://www.nuget.org/packages/Microsoft.ML.AutoML/0.21.0-preview.23511.1.

@shyamnamboodiripad
Copy link
Contributor

Error: Failed to load kernel extension "AutoMLMonitorKernelExtension" from assembly C:\Users\x.nuget\packages\microsoft.ml.automl\0.21.0-preview.23406.1\interactive-extensions\dotnet\Microsoft.ML.AutoML.Interactive.dll Error: Microsoft.DotNet.Interactive.KernelExtensionLoadException: Failure loading Kernel Extension ---> System.MissingMethodException: Method not found: 'Void Microsoft.DotNet.Interactive.Commands.SubmitCode..ctor(System.String, System.String, Microsoft.DotNet.Interactive.Commands.SubmissionType)'. at Microsoft.ML.AutoML.AutoMLMonitorKernelExtension.LoadExtensionApiAsync(Kernel cSharpKernel) at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.ML.AutoML.AutoMLMonitorKernelExtension.LoadExtensionApiAsync(Kernel cSharpKernel) at Microsoft.ML.AutoML.AutoMLMonitorKernelExtension.OnLoadAsync(Kernel kernel) at Microsoft.DotNet.Interactive.App.PackageDirectoryExtensionLoader.LoadFromAssemblyFile(FileInfo assemblyFile, Kernel kernel, KernelInvocationContext context) in D:\a_work\1\s\src\dotnet-interactive\PackageDirectoryExtensionLoader.cs:line 117 --- End of inner exception stack trace ---

This is the same issue as #3244 and #2366. We are currently investigating a fix for this.

@cartercanedy
Copy link
Contributor

@shyamnamboodiripad is there already a branch created for this fix in another fork?

@cartercanedy
Copy link
Contributor

Error: Failed to load kernel extension "AutoMLMonitorKernelExtension" from assembly C:\Users\x.nuget\packages\microsoft.ml.automl\0.21.0-preview.23406.1\interactive-extensions\dotnet\Microsoft.ML.AutoML.Interactive.dll Error: Microsoft.DotNet.Interactive.KernelExtensionLoadException: Failure loading Kernel Extension ---> System.MissingMethodException: Method not found: 'Void Microsoft.DotNet.Interactive.Commands.SubmitCode..ctor(System.String, System.String, Microsoft.DotNet.Interactive.Commands.SubmissionType)'. at Microsoft.ML.AutoML.AutoMLMonitorKernelExtension.LoadExtensionApiAsync(Kernel cSharpKernel) at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.ML.AutoML.AutoMLMonitorKernelExtension.LoadExtensionApiAsync(Kernel cSharpKernel) at Microsoft.ML.AutoML.AutoMLMonitorKernelExtension.OnLoadAsync(Kernel kernel) at Microsoft.DotNet.Interactive.App.PackageDirectoryExtensionLoader.LoadFromAssemblyFile(FileInfo assemblyFile, Kernel kernel, KernelInvocationContext context) in D:\a_work\1\s\src\dotnet-interactive\PackageDirectoryExtensionLoader.cs:line 117 --- End of inner exception stack trace ---

Looks like this may be happening because you are using an older version of the AutoML package against latest version of polyglot notebooks extension.

Could you try updating to the latest version from nuget.org? This version which was released last week may fix the error https://www.nuget.org/packages/Microsoft.ML.AutoML/0.21.0-preview.23511.1https://www.nuget.org/packages/Microsoft.ML.AutoML/0.21.0-preview.23511.1.

Just saw this, nevermind

@cartercanedy
Copy link
Contributor

I have noticed that the hanging is still persistent, though

@shyamnamboodiripad
Copy link
Contributor

Could you please clarify the exact repro steps for the hang? Am I correct in assuming that the hang only happens when the AutoML extension is in play? Is there a particular notebook that you are able to repro the hang with?

@cartercanedy
Copy link
Contributor

cartercanedy commented Oct 22, 2023

image

That code submission should've only taken 10 secs (per awaiting that sleeping promise), but it only completes on the first submission, any subsequent submissions of the unaltered code hang.

@shyamnamboodiripad
Copy link
Contributor

Thanks for sharing the repro @cartercanedy. I am also able to repro this with the code you shared below. Sometimes I am able to cancel execution, other times it hangs and yet other times, the cell completes.

sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
await sleep(10000);
const someVar = "Hello, world";

We will take a look - more than half of our team is away this week so it may be next week before we can get to this.

(cc @colombod FYI looks like there are some more problems beyond what we fixed in #3268)

@shyamnamboodiripad
Copy link
Contributor

@superichmann Is the hang you were running into above similar to the one @cartercanedy reported above? Are you also running into the hang in JavaScript? Please share a repro if you think it is different.

@jonsequitur jonsequitur added bug Something isn't working waiting-on-feedback labels Nov 2, 2023
@superichmann
Copy link
Author

superichmann commented Nov 7, 2023

TLDR: FIXED, changed Microsoft Data analysis with version 0.21.0-preview.23511.1

I am keeping this open in case you want to change the infinite run behavior and if you would want to reproduce. feel free to close.

This is exactly what is happening on my side:

  1. I am creating new notebook
  2. I save the notebook as a file (ipynb)
  3. I add this to the first cell:
Console.WriteLine("Initializing...");
#r "nuget: Microsoft.ML,3.0.0-preview.23511.1"
#r "nuget: Microsoft.ML.TimeSeries,3.0.0-preview.23511.1"
#r "nuget: Microsoft.Data.Analysis"
#r "nuget: Microsoft.ML.AutoML,0.21.0-preview.23511.1"
#r "nuget: Microsoft.ML.OneDal,0.21.0-preview.23511.1"
#r "nuget: npgsql"
#r "nuget: Telegram.Bot"
#r "nuget: MathNet.Numerics"
  1. I run the first cell, get this output after 4 seconds:
Installed Packages
MathNet.Numerics, 5.0.0
Microsoft.Data.Analysis, 0.20.1
Microsoft.ML, 3.0.0-preview.23511.1
Microsoft.ML.AutoML, 0.21.0-preview.23511.1
Microsoft.ML.OneDal, 0.21.0-preview.23511.1
Microsoft.ML.TimeSeries, 3.0.0-preview.23511.1
npgsql, 7.0.6
Telegram.Bot, 19.0.0
Loading extensions from `C:\Users\X\.nuget\packages\microsoft.data.analysis\0.20.1\interactive-extensions\dotnet\Microsoft.Data.Analysis.Interactive.dll`
Error: Failed to load kernel extension "DataFrameKernelExtension" from assembly C:\Users\X\.nuget\packages\microsoft.data.analysis\0.20.1\interactive-extensions\dotnet\Microsoft.Data.Analysis.Interactive.dll
Error: Microsoft.DotNet.Interactive.KernelExtensionLoadException: Failure loading Kernel Extension
---> System.MissingMethodException: Method not found: 'Void Microsoft.DotNet.Interactive.Commands.SubmitCode..ctor(System.String, System.String, Microsoft.DotNet.Interactive.Commands.SubmissionType)'.
at Microsoft.Data.Analysis.Interactive.DataFrameKernelExtension.LoadExtensionApiAsync(Kernel cSharpKernel)
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
at Microsoft.Data.Analysis.Interactive.DataFrameKernelExtension.LoadExtensionApiAsync(Kernel cSharpKernel)
at Microsoft.Data.Analysis.Interactive.DataFrameKernelExtension.OnLoadAsync(Kernel kernel)
at Microsoft.DotNet.Interactive.App.PackageDirectoryExtensionLoader.LoadFromAssemblyFile(FileInfo assemblyFile, Kernel kernel, KernelInvocationContext context) in D:\a\_work\1\s\src\dotnet-interactive\PackageDirectoryExtensionLoader.cs:line 117
--- End of inner exception stack trace ---
  1. I run the cell again, it never ends running - continuous infinitely

I have tried restarting vs code and it didn't help

more info:

Version: 1.84.1 (user setup)
Commit: 2b35e1e6d88f1ce073683991d1eff5284a32690f
Date: 2023-11-06T12:35:59.022Z
Electron: 25.9.2
ElectronBuildId: 24603566
Chromium: 114.0.5735.289
Node.js: 18.15.0
V8: 11.4.183.29-electron.0
OS: Windows_NT x64 10.0.19045

Polyglot Notebooks v1.0.4552011 Pre-Release

dotnet --list-sdks
3.1.426 [C:\Program Files\dotnet\sdk]
6.0.202 [C:\Program Files\dotnet\sdk]
5.0.214 [C:\Program Files\dotnet\sdk]
5.0.416 [C:\Program Files\dotnet\sdk]
7.0.416 [C:\Program Files\dotnet\sdk]
8.0.203 [C:\Program Files\dotnet\sdk]
7.0.310 [C:\Program Files\dotnet\sdk]
7.0.403 [C:\Program Files\dotnet\sdk]

if further information is needed from my side please tell me how to obtain it and I will add it here

@princeanire
Copy link

Hi, I encountered this also, when i tried running the following code
while (true)
Console.WriteLine("Please stop!");

clicking the Stop execution button doesn't stop the execution and I have to restart the .NET interactive kernel for it to stop

@jonsequitur
Copy link
Contributor

@princeanire, this is unrelated. The cancel button will not cancel infinite loops, since .NET Core / .NET 5 / .NET 6 did not support an equivalent of .NET Framework's Thread.Abort. We are working on a potential fix using the ControlledExecution API introduced in .NET 7.

Related: #3049.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working waiting-on-feedback
Projects
None yet
Development

No branches or pull requests

5 participants