Debug issue with typescript, nodemon, and external projects

[snotmare]

Type: Bug

Thank you for anyone who is able to offer some insight about what is going on. VS Code debugging has always been finicky, but I've got a situation that I think should work and it isn't.

Issue:
I can set breakpoints in typescript and debug in 2 of my projects (server and common-internal), but not the 3rd (common-external). Why can't I debug common-external, and how do I set it up so I can? VS Code cannot find the source to bind the breakpoint.

I've created a simple repo to reproduce the issue I'm seeing: https://github.com/snotmare/debug-issue

Steps to reproduce:

1. Clone repo
2. Run 3 init tasks (for common-internal, common-external, and server)
3. Run 2 watch tasks (for common-internal and common-external)
4. Run the start server task
5. Run the debugger by attaching to the process
6. Set a breakpoint in server/src/index.ts (works)
7. Set a breakpoint in common-internal/src/utils.ts (works)
8. Set a breakpoint in common-external/src/external-utils.ts (does not work)

Project setup:

• Typescript project with a utils class (common-internal).
• NodeJS server project that imports common-internal and common-external.
• Both of these projects are within the same folder, with a .vscode folder at the same root for task and launch config.

External project setup:

• Typescript project with utils class (common-external).
• This is setup exactly the same as common-internal, just renamed.
• This has its own .vscode folder for task config.

The server project depends on the 2 common projects like this:

"dependencies": {
    "common-internal": "file:../common-internal",
    "common-external": "file:../../common-external/common-external",
    "express": "4.18.2"
},

I believe the issue is related to the folder structure, which looks like this:

root folder
- test-app
- - .vscode
- - common-internal
- - server
- common-external
- - .vscode
- - common-external

The folder structure needs to be this way because I want test-app to be in a git repo while common-external is in a separate git repo.

When I debug the server, why can't I set a breakpoint in common-external and debug it? Thank you for your help!

Ben

VS Code version: Code 1.74.0 (5235c6b, 2022-12-05T16:38:16.075Z)
OS version: Windows_NT x64 10.0.22000
Modes:
Sandboxed: No

System Info

Item	Value
CPUs	12th Gen Intel(R) Core(TM) i7-12800H (20 x 2803)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_renderer: enabled_on video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: disabled_off
Load (avg)	undefined
Memory (System)	31.68GB (18.50GB free)
Process Argv	--crash-reporter-id 3f0908f6-4361-4d5f-960e-2544e5ce4a19
Screen Reader	no
VM	0%

Extensions (11)

Extension	Author (truncated)	Version
ng-template	Ang	15.0.3
vscode-eslint	dba	2.2.6
gitlens	eam	13.1.1
gitlab-workflow	Git	3.57.3
todo-tree	Gru	0.0.220
vscode-sshfs	Kel	1.25.0
dotenv	mik	1.0.1
vsliveshare	ms-	1.0.5788
vsliveshare-audio	ms-	0.1.91
vsliveshare-pack	ms-	0.4.0
project-snippets	reb	0.5.0

A/B Experiments

vsliv368cf:30146710
vsreu685:30147344
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
pythonvspyl392:30443607
vserr242cf:30382550
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vshan820:30294714
vstes263:30335439
vscoreces:30445986
pythondataviewer:30285071
vscod805cf:30301675
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
cmake_vspar411:30581797
vsaa593cf:30376535
pythonvs932:30410667
cppdebug:30492333
vsclangdf:30486550
c4g48928:30535728
dsvsc012:30540252
azure-dev_surveyone:30548225
pyindex848:30577860
nodejswelcome1:30587005
282f8724:30602487
gswce1:30612156
3d0df643:30613357
dbltrim-noruby:30604474
f6dab269:30613381
fim-prod:30623723
vscsbc:30628655

[snotmare]

Update... I was able to make a change in my launch.json file that helped. I overwrote the outFiles section like this...

"outFiles": [
	"${workspaceFolder}/**/*.js",
	"${workspaceFolder}/../common-external/**/*.js",
	"!**/node_modules/**"
]

The trick was to point go up a folder to point to the external project. I found this by accident, so could the documentation on this be improved? It would be helpful to better understand the internal workings.

In my larger project, I made this same change and was also able to debug. However, it's hit and miss when it comes to debugging. Sometimes it works, but mostly it doesn't. I restart vs code, restart my projects, etc. When it doesn't work, it says it can't find the source code even though it's there. I also see the message that "It's taking a while to configure your breakpoints. You can speed this up by updating the 'outFiles' in your launch.json". I've done that similar to the code above.

My real project is a bit of a mono-repo. It contains 4 typescript projects, 5 nodejs servers, and an angular app. I usually have tsc --watch enabled for the typescript projects and use nodemon on the servers.

Is there an upper limit on the number of files that vs code can handle with debugging? Are there some performance metrics I could run to see why the debugger can't find the source files?

[connor4312]

Yea, the outFiles change is what you need. By default, only sourcemaps within the workspace folder (excluding node_modules) are discovered. We do have a docs section on source map discovery: https://code.visualstudio.com/docs/nodejs/nodejs-debugging#_source-map-discovery

Is there an upper limit on the number of files that vs code can handle with debugging? Are there some performance metrics I could run to see why the debugger can't find the source files?

There is no upper limit. I would check to make sure that you didn't accidentally include any big extraneous folders in your outFiles -- and ensure the negation for node_modules is still there.

Logs might help...

/jsDebugLogs

[vscodenpa]

Please collect trace logs using the following instructions:

If you're able to, add "trace": true to your launch.json and reproduce the issue. The location of the log file on your disk will be written to the Debug Console. Share that with us.

⚠️ This log file will not contain source code, but will contain file paths. You can drop it into https://microsoft.github.io/vscode-pwa-analyzer/index.html to see what it contains. If you'd rather not share the log publicly, you can email it to connor@xbox.com

[snotmare]

Thanks for your help Connor! I did find that link explaining the sourcemap files. Perhaps the piece I was missing is where "workspaceFolder" actually points to. I had assumed it was one level higher than it was, but it turns out it points to the folder containing the .vscode folder.

I've run the debugger with the trace and attached that file. This is from the larger project, not the sample repo I originally posted about.

I did notice one thing earlier that helped. The nodejs server that I'm running uses our shared code from the other typescript projects. The package.json for our server contains dependencies like this:

"shared": "file:../shared",
"shared-server": "file:../shared-server",

If I start vs code and simply run the server, the debugger cannot find any breakpoints. However, if I start vs code, start the watch process for our shared typescript projects...

tsc --watch

... and then start the server, I have a much higher chance of vs code finding my breakpoints. I've attached a trace for the one that works as well (second attachment with the "-works" in the file path).

Thank you! I'd love to know more about how this works so I can improve our environment.

vscode-debugadapter-3ecd154d.json.gz
vscode-debugadapter-7613a7c0-works.json.gz

[connor4312]

Does adding an extra negation for !../ts-base/**/node_modules/** resolve the issue?

[snotmare]

Hey, that did help some! It definitely performs better, good call!

I am now seeing my breakpoints find the source a little over half the time. I managed to get a trace for both when it worked (first link) and when it was broke (2nd link).

vscode-debugadapter-e619ebd7.json.gz
vscode-debugadapter-0bc7bab9-broken.json.gz

[connor4312]

Good to know. The glob behavior is something we can improve, automatically applying it if we see a relative pattern navigates outside the workspace folder.

I managed to get a trace for both when it worked (first link) and when it was broke (2nd link).

Are you sure you're connecting to the right process in the broken one? Looking at the scripts loaded by the runtime in the trace log, they're mostly from node_modules/nodemon, while in the working one there's no nodemon sources at all.

[snotmare]

Ok, you're right. I wasn't paying attention to that before, but this is tied to which process you pick. I was always picking the top process (assuming it was the most recent). After paying attention more, I noticed the order of the top 2 processes changes (see screen shot)...

I tested the following scenarios and found these results:

1. Restart server, pick nodemon process (broken)
2. Restart server, pick not nodemon process (works)
3. Leave server running, pick nodemon process (still works)
4. Restart server, pick nodemon process (broken)
5. Leave server running, pick not nodemon process (still broken)

Is there a way to weed out the processes from this list I should not be using for debugging? Maybe something with the presentation option?

If you want any traces with these scenarios, I'll be happy to provide them.

Thanks again for all your help!

[connor4312]

Leave server running, pick not nodemon process (still broken)

I think this one is still unexpected. I've opened issues to track the other two improvements we've talked about.

[snotmare]

Ok, I've run through those scenarios and generated traces. The file labels match these scenarios:

1. Restart server, pick nodemon process (broken)
2. Leave server running, pick not nodemon process (still broken)
3. Restart server, pick not nodemon process (works)
4. Leave server running, pick nodemon process (still works)

vscode-debugadapter-ff37efdf-1-nodemon-broken.json.gz
vscode-debugadapter-79a57277-2-not-nodemon-broken.json.gz
vscode-debugadapter-2b780da5-3-not-nodemon-works.json.gz
vscode-debugadapter-18c76ecb-4-nodemon-works.json.gz

[connor4312]

Leave server running, pick not nodemon process (still broken)

This looks like Nodemon again 😬 you can also see the scripts if you use the Loaded Sources view.

[snotmare]

Ya, that makes sense. It appears that if I debug the wrong process to begin with, I have to restart my server before attaching to the correct process. Now that I know this, I feel better about debugging. Thanks for your help!

[connor4312]

No problem, thanks for your patience and info 🙂 I will close this out since I captured the work in the aforelinked issues.