testing_vscode_extensions.org

Testing VSCode Extensions

who -u

Dan Čermák

Agenda

• Testing Extensions
  • Unit testing
  • Manual testing
  • Integration testing
• Q&A

Visual Studio Code

Testing Extensions

Why is it hard?

• UIs cary a lot of state
• testing workflows is uncommon
• 🙈 what you test is not what you see

External Services

• external libraries: mock or LD_PRELOAD
• external service: development environment or staging

Unit testing

• test one functionality of one unit/function
• anything touching the UI needs extensive setup & teardown
• external services should be mocked

Getting started

• documentation has an example setup
• code coverage setup more involved

Extension Settings

• can be read & modified in tests

Events

quickPick.onDidChangeValue(async (val: string) => {
  if (verifyInput(val)) {
    await launchBackgroundTask();
  }
});

→ use fake events when possible

Disposables

• “destructors” in VSCode
• use after() or afterEach()

UI Elements

• only check the interesting parts
• preferably keep UI part as small as possible
• split UI specific parts out of your code

Manual testing

Integration testing

vscode-extension-tester

@@html: <i class=”fab fa-github”></i>@@ redhat-developer/vscode-extension-tester

@@html: <img src=”images/Selenium_Logo.png” height=”64” width=”64” style=”margin-bottom:-20px”/>@@ leverages selenium webdriver

const editor = new TextEditor();
const pkgJsonEditor = await new EditorView().openEditor('package.json');

await pkgJsonEditor.setText('{"foo": [1, 2, 3], "bar": "baz"}');
await pkgJsonEditor.formatDocument();

What to test?

• check your main workflow(s)
• don’t test corner cases & minor regressions

How to test?

• upstream using mocha
• use root hooks for setup
• run steps as individual it()

Catches

• 🐌 slow and resource demanding
• avoid explicit sleeps
• @@html: <i class=”fas fa-low-vision”></i>@@ certain elements invisible by default
• no test coverage

Links

• @@html: <i class=”fab fa-github”></i>@@ SUSE/open-build-service-connector
• @@html: <i class=”fab fa-github”></i>@@ redhat-developer/vscode-extension-tester
• mochajs.org
• basic unit test setup
• code coverage setup
• Kiwi TCMS
• slides: @@html: <i class=”fab fa-github”></i>@@ =dcermak/devconf.cz_2021=

Legal

• VSCode Logo © Microsoft
• Selenium Logo CC-BY-SA 4.0
• reveal.js MIT
• Font Awesome CC-BY-4.0 and SIL OFL 1.1 and MIT
• everything else is my work under CC-BY-4.0

Questions?

Answers!

Demo