Too many code lens options for running tests

[sandstrom]

The code lens feature to run tests is great! 🎉

But right now, there are triggers everywhere:

1. Run all the tests
2. Run all the tests in the file
3. Hover a specific test to run it
4. Gutter trigger for a specific test (same thing as 3)
5. Inline trigger for a test (same thing as 3)

To be fair, 1-2 are different, but 3-5 are the same thing.

I find 5 quite annoying, because it makes the test file hard to read and difficult to work with visually (moving blocks of code around, quickly scanning code, etc), with every few lines being broken with this magic extra line, with controls.

Obviously subjective, but since 4 is just a few pixels away and 3 also easy to reach, I prefer both of them to 5.

Two suggestions (either would work)

A. Remove 5, and let people use 3 or 4 to run a test.
B. Make it configurable to remove/hide 5.

[andyw8]

I'm going to move this to the ruby-lsp repo for discussion, since this isn't specific to the Rails addon.

[Earlopain]

I actually really like 5, its how I use this exclusively. I guess I would use something else if it wasn't available but it is what I reach for first.

[st0012]

FWIW, there's a similar request in ruby-lsp-rspec: st0012/ruby-lsp-rspec#28

I personally also use 5 most of the time, and IMO removing it will be a major breaking change that I strongly against. I think it's ok to make it opt-out-able though.

[st0012]

We decided to not take any actions for now because:

• Among the 5 triggers, 1-4 are repeated intentionally by VS Code, not by Ruby LSP. That is, as long as we support the Test Explorer, you'd always get that 4 triggers
• Because 5 (code lens) are displayed automatically if the editor receives the response, it actually takes extra work to disable it
• We also can't make the code lens response optional because the current Test Explorer cases are populated using code lens response. If we stop sending them Test Explorer wouldn't work either.

[sandstrom]

@st0012 All of that sounds reasonable.

Perhaps it'll be made optional in the future, when there is better support in VS Code and other editors.

Other languages

In the python issue below there is a screenshot purportedly showing a way to hide 5.

I've looked around to find the original issue and reasoning, but it points to an internal issue tracker. For what it's worth, the Microsoft VS Code team thought it was a good idea to rely only on the gutter (4), and skip code lens for tests (5).

I think this is the commits removing it: microsoft/vscode-python#16200

Links

For future reference, here are some similar requests for other languages. In case anyone want to open an issue upstream.

• Option to disable "run test | debug test" links microsoft/vscode-python#10898
• Add option to disable "Run test" link (CodeLens) rust-lang/vscode-rust#458
• Option to disable "run test | debug test" links microsoft/vscode-go#1141

[rsanheim]

In case anyone else ends up here googling for a way to disable these popups for ruby-lsp, as they are very annoying when editing large spec files, here is the setting to disable:

 "rubyLsp.enabledFeatures": {
    "codeLens": false,

I'm not sure what else codeLens provides, but so far navigation / indexing all still work with ruby-lsp, and I never run things via vscode currently. So this works for me so far.

edit after 5 minutes 😏 : codeLens provides all the line by line git info in VSCode, so it turns out this is not a great workaround. Could this issue be reopened to provide more granular control around this?

[vinistock]

Currently the test explorer functionality is entirely backed by code lens. The only way to have more granular control is to rethink how we're discovering tests and completely uncoupled the two.

It's also an opportunity for us to experiment with surfacing all tests in the codebase, which we didn't try initially because of performance reasons.

Another facet of this problem is that the Ruby LSP supports multiple test frameworks, both with internal integrations (Minitest and Test Unit) and through add-ons (RSpec).

The solution has to be general to work for any test framework, decouple test discovery from code lens while at the same allowing code lens to be shown and be performant enough to avoid any slow downs in the editor - both during boot (in case the solution involves indexing) and regular operations.

We do want to revisit all of this at some point, but it's a considerable amount of effort that we cannot prioritize right now.

[sandstrom]

@vinistock Understandable!

But would it make sense to keep it open?

People are bound to google this, and you don't have to commit to it just because the issue if left open.