Debugging VS Code with VS Code is flaky and frustrating

[jrieken]

Sorry, that's it. Cannot say more, the experience is a lottery - it sometimes works flawless, it sometimes doesn't work. That it works only randomly is frustrating but what's worst is that I don't feel empowered to provide any kind of qualified feedback. I cannot file good issues and have to resort to ranting. I believe most have confirmed the broken window theory and just moved back to devtools or never made the switch.

[jrieken]

For some background: I am trying to use the "VS Code" launch config. I am trying to debug tests with the newish test integration. Symptoms are hangs and crashes, sometimes right after starting, sometimes after reloading, sometimes after the Nth debug attempt.

[jrieken]

/cc @eamodio @rebornix @joaomoreno who have expressed positive or negative emotions towards debugging in the recent past

[jrieken]

related to @alexdima's issue: #126826

[rebornix]

I asked in general probably more than twice about how to reliably debug VS Code or tests and also asked for @connor4312 and @roblourens personally for help. Maybe with one more bug to fix the experience can be smooth but I want to share my overall feeling with debugging in VS Code for the last couple of years and workarounds I ended up with

• Debugging single unit/integration tests is flaky
  • At the beginning the breakpoint seldomly hit when debugging single file, the test always finished before the debugger is attached (Debug unit test in VS Code fail to attach randomly #125567). I then started to use await new Promise(c => setTimeout(c, 3000));
  • Recently there are good fixes in js debug so breakpoints will mostly hit but occasionally the process failed probably because of crash crash on debugging VS Code renderer crashes when setting certain breakpoints #122225
• Debug VS Code all processes fail pretty often, I can't call it random as when it fails it's 100%. No clue why is that
  • might be timeout Alex is seeing Connection refused debugging extension host #126826
  • might be crash as above VS Code renderer crashes when setting certain breakpoints #122225
  • sometimes it succeed connecting, but breakpoints never hit (breakpoints should just work for all processes, right?). I saw this happening in a few ppl's machines
  • disconnect doesn't always work, have to right click kills the process
  • At the end, whenever I want to troubleshoot end to end, say notebook, what I do now is (almost work every time)
    • Run OSS, open extension host code, set breakpoints
    • Debug a notebook extension, in the new window use chrome dev tools in the renderer process

In addition to above, with the extensions starting to use bundler, I sometime find breakpoints doesn't hit in ts file but only in js file, it seems random but when that happens my confidence with source map becomes low and by the end of the day, I stopped using source map in both VS Code and browser dev tools.

With that being said, I'm not happy with all the workarounds so often gave the builtin debugger another try, it's getting better though the randomness is still there. I would love to see a clear detailed guidance for troubleshooting (it would be great if it's a command like Toubleshoot Keyboard Shortcuts) and we can file good bug reports.

[bpasero]

Related:

• Breakpoint does not correctly apply on destructured variable expression #127397
• Debug tests: often first click to "Debug Test" does nothing #127396
• Debugger does not step into correct function #127345

[joaomoreno]

Recently talking with @isidorn and @bpasero, we had the idea to have a PM-driven selfhosting effort They would collect information from team members of what the biggest selfhosting pains are and whether and why they are or aren't using VS Code core features. They would then help prioritize work and get it on the plan accordingly. The goal is to maximize selfhosting by putting highlight on areas with most pain.

—-

Specifically on selfhost debugging: I share your sentiments today. While it was great back in January/February, I can no longer say the same. I subscribe to the lottery analogy, which is the worst situation since it makes us silently ditch the feature, which I’ve certainly done recently.

[jrieken]

They would collect information from team members of what the biggest selfhosting pains are

I like that and that's also the idea of the papercut label

[connor4312]

Here's the state of where the mentioned bugs stand for me:

• VS Code renderer crashes when setting certain breakpoints #122225 remains the big one, at least for me. @deepak1556 please let me know if there's anything I can do to help with that. If it's insurmountable I could try to just brute force and try to disable features or behavior in the debugger until the crashes stop, but this is not sustainable. I would assume this is being caused by setting breakpoints via regexes; while these patterns are long, they're also pretty simple.
• Connection refused debugging extension host #126826 is on my to do. I believe this is something going wrong in VS Code itself where the extension host doesn't get launched in debug mode, but it's racey. I want to look at this this week.

  • There is a similar error for the main process and renderer. Electron behaves like Chrome, where only the first "main process" will look at the debugging instructions, so if an OSS window or main process is already open not in debug mode, then the debug ports will not be opened and a connection refused will happen.

    This can be fixed by using a separate user data dir for VS code when launched in debug mode, like we do for Chrome, but this could make life difficult. In Chrome there is a lockfile which we warn about if it's present when launching, but we don't have a similar file in VS Code right now.

  • Update: the linked bug was actually 'expected' since Alex was paused at a point before the extension host was launched, we just needed a larger timeout. I've also added fixes for the error with the renderer, Debugging VS Code with VS Code is flaky and frustrating #127861 (comment)

• Breakpoint does not correctly apply on destructured variable expression #127397 / Debugger does not step into correct function #127345 we'll see about. Ben mentioned devtools does this as well, it may or may not be something we can control/work around.
  • Update: both of these appear to be Chrome issues, the former will be fixed when we update, the latter is also a current issue the happens in Chrome.
• Debug unit test in VS Code fail to attach randomly #125567 there's now code to wait for at least one breakpoint to be verified by js-debug before the test session starts. However, while breakpoints are set in parallel it's possible that this could be insufficient--and there's also a 7-second timeout which may not be sufficient on certain machines. @rebornix are you still seeing this today, and when you are is it the case that either you have multiple breakpoints set, or you see that log in the test output?
  • Update: I've increased this, @rebornix let me know if you see this again.

• In addition to above, with the extensions starting to use bundler, I sometime find breakpoints doesn't hit in ts file but only in js file, it seems random but when that happens my confidence with source map becomes low and by the end of the day, I stopped using source map in both VS Code and browser dev tools

  • This is, as far as I've seen, always a configuration issue in the compiler/bundler or launch config. If you have examples of cases where this happens, please share them and I can take a look.

• I would love to see a clear detailed guidance for troubleshooting (it would be great if it's a command like Toubleshoot Keyboard Shortcuts) and we can file good bug reports.

  • If you have breakpoints that are set but unverified, use the "Debug: Diagnose Breakpoint Problems" command while the debug session is running.
  • Otherwise, add "trace: true" to the launch config and share the resulting log.

I will follow up with VS Code team members to gather feedback.

[connor4312]

With regard to debugging tests specifically, I will use the new test run configurations API to allow running in browsers which should some most of these issues (by using a newer Chromium/V8 version where bugs aren't present.)

[connor4312]

There is a similar error for the main process and renderer. Electron behaves like Chrome, where only the first "main process" will look at the debugging instructions, so if an OSS window or main process is already open not in debug mode, then the debug ports will not be opened and a connection refused will happen.

For this, I did two things:

1. Launching OSS from the configuration will now use a .profile-oss folder in the workspace by default, and
2. There is now a lockfile, so if there's an existing Code instance running from that profile directory but not listening on the debug port, you will be presented with the same dialog we show in the browser case:

Also, for #126826 I noticed that the issue was just that we timed out at all -- it was expected that the ext host didn't launch in that case.

[deepak1556]

@connor4312 for #122225 it would help if there are reliable steps to repro the crash, I will add some logs in the runtime this week to better understand the regex causing the crash.

[connor4312]

Debugging a unit test with a breakpoint set anywhere in the codebase reproduces the issue ~50% of the time for me. You can use Test: Show Output to see the console output and confirm when a crash happens.

[rebornix]

Tried to debug a unit test today and I'm currently seeing from "Test: Show Output"

./scripts/test test/unit/electron/index.js --reporter full-json-stream --grep /^(NotebookTextModel multiple inserts in one edit$)/ --remote-debugging-port=9222 --timeout=0 --waitServer=53242

[16776:0803/203559.179:ERROR:tcp_socket_win.cc(353)] bind() returned an error: Only one usage of each socket address (protocol/network address/port) is normally permitted. (0x2740)
[16776:0803/203559.179:ERROR:devtools_http_handler.cc(298)] Cannot start http server for devtools.

and logs for netstat -ab and apparently it's being used

 [Code - Insiders.exe]
  TCP    127.0.0.1:9222         kubernetes:57079       TIME_WAIT
  TCP    127.0.0.1:9222         kubernetes:57081       TIME_WAIT
  TCP    127.0.0.1:9222         kubernetes:57083       TIME_WAIT
  TCP    127.0.0.1:9222         kubernetes:57084       TIME_WAIT
  TCP    127.0.0.1:9222         kubernetes:57085       TIME_WAIT
  TCP    127.0.0.1:9222         kubernetes:57086       TIME_WAIT
  TCP    127.0.0.1:9222         kubernetes:57087       TIME_WAIT
  TCP    127.0.0.1:9222         kubernetes:57088       TIME_WAIT
  TCP    127.0.0.1:9222         kubernetes:57093       TIME_WAIT
  TCP    127.0.0.1:9222         kubernetes:57094       TIME_WAIT
  TCP    127.0.0.1:9656         0.0.0.0:0              LISTENING

Can we try to detect which process is actually using that port to help us self-troubleshoot what's going on?

[rebornix]

Ok, if you have Remote Container extension installed, it will use 9222 then you can't debug VS Code at all.

Disabling the extension at least gets rid of this error, but now I got an infinite progress of "Running tests", which simply does nothing

./scripts/test test/unit/electron/index.js --reporter full-json-stream --grep /^(NotebookTextModel multiple inserts in one edit$)/ --remote-debugging-port=9222 --timeout=0 --waitServer=64769


DevTools listening on ws://127.0.0.1:9222/devtools/browser/b7ed2d74-b60b-4144-9f9d-00797742016b
[23856:0803/205139.940:INFO:CONSOLE(29729)] "'window.webkitStorageInfo' is deprecated. Please use 'navigator.webkitTemporaryStorage' or 'navigator.webkitPersistentStorage' instead.", source: file:///C:/Users/rebor/code/vscode/node_modules/mocha/mocha.js (29729)
[23856:0803/205140.310:INFO:CONSOLE(33)] "The vm module of Node.js is deprecated in the renderer process and will be removed.", source: electron/js2c/renderer_init.js (33)

[connor4312]

@chrmarti on the Remote Containers issue Peng mentioned ^ Though I also have and use remote containers and don't see that, so more info may be needed

@rebornix I don't repro that,

1. Does that happen even when not in debugging
2. If not, does it eventually run after ~15 seconds?
3. If so, please add trace: true to the "Attach to VS Code" config

[rebornix]

@connor4312

1. running is fine
2. it never finishes, I waited several minutes but didn't see any progress
3. will do that and share the log

[rebornix]

@connor4312 here is the log after waiting for a few minutes

vscode-debugadapter-45eeec47.json.gz

[connor4312]

@rebornix thanks, I think I see what's up, seems to be a subtle change in recent Chrome/Electron versions that you're hitting based on timing. Here's a vsix for a nightly build that has a prospective patch: https://memes.peet.io/img/a5d9867c-9921-41e2-843c-163b8eed82a1.vsix

[connor4312]

Followup from PM: this seems to be an Electron bug; a fs.readdir call in the glob module we use to discover unit tests never fires its callback. Deepack is working on an Electron build with additional logging to diagnose the issue.

[weinand]

When debugging VS Code by using the "VS Code" launch config, I was frequently running into this crash:

When trying to understand why/when this happens, I noticed that breakpoints (or the number of breakpoints) are the trigger. Without breakpoints I was never crashing; with around 10 breakpoints crashes occurred 6 out of ten.

A workaround is to deactivate all breakpoints temporarily (before running the "VS Code" launch config):

and then activating breakpoints individually as soon as the CPU activity has calmed down.

[connor4312]

@weinand I suspect you're hitting this: #122225 If you send a crash report to Deepak, he will be able to confirm if it's the same issue or different.

[deepak1556]

@rebornix for the freeze issue, can you try running with --disable-features=CalculateNativeWinOcclusion. It is very likely the cause of getting the event loop suspended.

[connor4312]

I've now included the aforementioned flag, as well as the flag for the breakpoint crash issue, in the launch config and selfhost test provider by default.

[snalesso]

@rebornix for the freeze issue, can you try running with --disable-features=CalculateNativeWinOcclusion. It is very likely the cause of getting the event loop suspended.

Is there a way to set this flag in a config file instead of editing the shortcut which opens VSC?
Also, why isn't this issue handled? I shouldn't need to do this stuff, I guess ...

[connor4312]

@snalesso the freeze issue is tracked in #123257

[connor4312]

There have been no further problems shared here, outstanding issues are tracked separately. Thus closing.