Handle proposed extension API incompatibilities

[roblourens]

We've had some difficulties coordinating breaking changes to proposed API between VS Code and an extension. Summarizing the problems and what we're trying to do about them:

I want to be able to land a PR with the API change in VS Code, and at the same time, a PR to adopt it in the extension. And, at extension install time or runtime, we want to be able to ensure that the installed extension is compatible with the current version of vscode

• On one end, it's hard to use the engines.vscode version to handle the case where the extension gets ahead of the VS Code version, because at the time we want to land the extension PR, that version of VS Code has not been published yet, so we can't run extension tests with it
• On the other end, there is no good way to know when the extension is behind the VS Code version because we normally assume that extension API updates are backwards-compatible.

Things we've discussed

• Merging PRs in the extension
  • Manually kick off a build of VS Code to use when landing the extension PR, and vscode-test will download this build
  • Or build VS Code in CI automatically and cache it across builds. It shouldn't take too long to do a bare-minimum OSS build with no tests
  • Either way, we ignore the date part of the engine version (because it will be tomorrow's version)
• Detecting at runtime when the extension is behind
  • A runtime version check for the chat API. We already have this but can't use it currently because it has the same problems as above
  • An API proposal version as an official concept in VS Code- don't load an extension that doesn't support the correct current version of the proposal
• Can we improve this at install-time?
  • @sandy081 suggested locking the extension engine version to a single VS Code version, rather than using it as a "minimum" version
  • Then we can check this in the marketplace and install the proper version
  • This would prevent users from installing the stable extension on VS Code Insiders
• Change how pre-release works to force installing pre-release in Insiders, if it has a pre-release
  • Sandeep suggested making this change for all extensions
  • A downside is that some users may still want to be able to use Insiders without also using pre-release extensions
  • And, an extension might have stopped publishing pre-releases, so it has some but they are very out of date
  • Also see Installing Copilot in Insiders can cause stable Copilot Chat to be installed, not pre-release #213959 for a case where currently it's possible for Insiders users to get the stable Copilot Chat by default
• If a pre-release extension build is delayed, it can leave users unable to use Copilot Chat
  • There's probably no decent solution to this, since the two builds should be independent, but it is not a common issue

[alexdima]

Here is the proposal I mentioned, but spelled out in greater detail:

• the vscode version cannot be used for semantically versioning proposed API. The date portion of the version in VS Code Insiders is not helpful because it is not semantic.
• I suggest experimenting with new keys in the engines section, on a case-by-case basis and not for all proposals, at least not until we validate this works.. We can say that certain proposals are versioned, and they have a runtime _version field and they are also available for targetting via the engines field.
• Copilot Chat could then declare something like:

    "engines": {
        "vscode": "^1.90.0",
        "vscode.chatParticipantAdditions": "^2.0.0",
        "npm": ">=9.0.0",
        "node": ">=18.0.0"
    },

• At runtime, these extra vscode.* engine fields should also be validated, with similar logic to the way the vscode engine field is validated. If the current running instance of VS Code, be it stable, insiders or exploration ships vscode.chatParticipantAdditions version 3 and the extension on disk asks for ^2.0.0, then the requirement can't be satisfied and the extension is not activated.
• Marketplace Extension version checking and updating could also be updated to take these new vscode.* engines into account. @sandy081 to confirm if this is feasible.
• As a proposed API owner, when an API proposal is broken, you bump the major version of the API proposal. This means that when the new Insiders goes out, all extensions requiring the old proposal version will not activate. This might be better than the current situation where they activate and fail in subtle ways. We could even render a message like This extension is not compatible with vscode because its authors need to adopt new vscode API.
• Now that we have semantic rules in place, the problem becomes: how to ship a Copilot Chat extension version at the same time with the Insiders build to avoid having a large time gap where "this extension is not compatible...".
• If we were to keep all proposed versions in a file src/vscode-dts/versions.js or something that can be captured via a glob pattern, we can set up a build which triggers based on this file being changed, and as soon as this file is changed in main, an Insiders build is triggered and not released.
• In parallel, a PR can be prepared in the Copilot Chat extension which adopts the new API, by bumping the version in engines.vscode.chatParticipantAdditions and doing the necessary modifications, but the PR should not be merged until the Insiders is available. The PR could fail to build if the Insiders is not available...
• when Insiders is available, the PR can be merged

[sandy081]

@roblourens Here is the feature request to Provide option to install pre-release of an extension even if latest version is release version

[roblourens]

Kai also suggested the possibility of just building and shipping vscode-copilot inside vscode. In this case, the marketplace extension would just enable that code.

[vscodenpa]

This bug has been fixed in the latest release of VS Code Insiders!

@roblourens, you can help us out by commenting /verified if things are now working as expected.

If things still don't seem right, please ensure you're on version 524d1ec of Insiders (today's or later - you can use Help: About in the command palette to check), and leave a comment letting us know what isn't working as expected.

Happy Coding!