Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Explore enabling type acquisition for web TS Server #172887

Closed
mjbvz opened this issue Aug 3, 2021 · 15 comments · Fixed by #212370
Closed

Explore enabling type acquisition for web TS Server #172887

mjbvz opened this issue Aug 3, 2021 · 15 comments · Fixed by #212370
Assignees
Labels
insiders-released Patch has been released in VS Code Insiders javascript JavaScript support issues typescript Typescript support issues web Issues related to running VSCode in the web
Milestone

Comments

@mjbvz
Copy link
Collaborator

mjbvz commented Aug 3, 2021

Follow up on microsoft/TypeScript#39707

Overview

We now have TSServer running in web environments, but it is currently limited to single file intellisense. It would be nice if we could also offer some IntelliSense for third party packages that are imported in the current file.

If a user is trying out react for example, we should be able to offer IntelliSense for React and ReactDOM:

//@ts-check

import * as React from 'react';
import * as ReactDOM from 'react-dom';

class HelloMessage extends React.Component {
    render() {
        return (
            <div>
                Hello {this.props.name}
            </div>
        );
    }
}

ReactDOM.render(
    <HelloMessage name="Taylor" />,
    document.getElementById('hello-example')
);
@mjbvz
Copy link
Collaborator Author

mjbvz commented Aug 3, 2021

Lots of open questions about how this should behave and how we can implement it:

  • Do we try looking in the user's package.json or at their tsconfig when determining which types to use?

  • Do we handle packages that bundle their own typings?

  • How far should type acquisition go? If one type package references another, do we pull that in too?

  • Would this work differently in JS files vs TS files? Type acquisition works differently for the two on desktop, but on web I think we'd want the behavior to be more similar

  • Could we use a service like unpkg for this? Or could we stand up a more specialized service that just service up a single types file for a given package?

  • Can the type acquisition be async like it is on desktop?

@orta
Copy link
Contributor

orta commented Aug 4, 2021

Could we use a service like unpkg for this? Or could we stand up a more specialized service that just service up a single types file for a given package?

Having NPM provide a CDN like how unpkg works would be a nice solution for this (because I have this same issue in the TS playground)

@orta
Copy link
Contributor

orta commented Aug 24, 2021

I recently re-wrote the ATA for the Playground ( microsoft/TypeScript-Website#1997 ) in what I think is a pretty good conceptual pattern:

  • TypeScript can probably already give you a list of dependencies it knows about from the package.json, and the source files it has read (in the playground I don't have this, so I use ts.preprocessFile to extract that)
  • You make an API call which gets a list of all the files in each module tarball (I used jsDelivr's) from which you can figure out the list of .d.ts files you need to download, or that you need to check DT.
  • If you need to check for @types/module and grab the .d.ts files for that module
  • Loop through all the downloaded .d.ts files and recurse grabbing their modules etc

It's not exactly like that in practice (you need the version number before you can make that API call to jsdelivr) - but that's good enough and you can read the implementation linked above for more details. The implementation of the TypeScript Playground ATA is built for third parties, so you could be a consumer of @typescript/ata when that PR lands.

An ideal state for VS Code which would keep this entirely first-party, is that the npm registry adds two APIs like:

  • https://registry.npmjs.org/npm/:version_or_tag/contents which returns some sort of file list
  • https://registry.npmjs.org/npm/:version_or_tag/file/package.json which returns a CDN'd file from the tarball (in this case the package.json)

Or vscode creates an API which proxies these APIs to jsdelivr (they're real nice folks, I can make an intro if needed) which allows for other implementations behind the scenes in time.

@MartinKolarik
Copy link

Or vscode creates an API which proxies these APIs to jsdelivr (they're real nice folks, I can make an intro if needed)

We are already watching - ping me any time if needed 😄

@orta
Copy link
Contributor

orta commented Oct 13, 2021

The Playground ATA re-write is now available on npm at @typescript/ata and deploying to the web as we speak.

mjbvz referenced this issue in mjbvz/TypeScript Nov 18, 2021
For #45314

This prototypes ATA for the web version of TypeScript. The logic is copied from the `@typescript/ata` project

The current implementation is not complete, mainly because I don't understand how to get the d.ts files into the TS project correctly
@mjbvz
Copy link
Collaborator Author

mjbvz commented Nov 19, 2021

microsoft/TypeScript#46862 Adds a draft implementation of this but I definitely need some help from the TS team to get it into a shippable state

@mjbvz
Copy link
Collaborator Author

mjbvz commented Jan 6, 2022

After discussion, we'd like to move the core auto import implementation to VS Code

@orta I looked into using preProcessFile to get all the imports but it looks like this isn't currently exposed over the TSServer protocol? Is that correct?

After thinking about this more, I am also wondering if using preProcessFile is too low level. We can debounce how often we end up calling preProcessFile to avoid calling it on every key stroke, but it feels like it would make more sense if TS could tell us what auto imports it needs and then VS Code could then go and resolve them

@mjbvz
Copy link
Collaborator Author

mjbvz commented Jan 15, 2022

@orta @amcasey I have a basic prototype now running on the VS Code side and can use this to load global typings such as jquery

The problem I'm running into now though is telling TS that jquery in import 'jquery' maps to a given file the VS Code has opened against the TypeScript server

Here's how the files are currently opened after ATA has completed (I'm using the paths that TS server sees):

^/memfs/sample-folder/file.js — This is the file that the user has opened up

import * as $ from 'jquery'

Then we have a list of jQuery d.ts files that VS Code has opened up with the correct content:

  • ^/ts-typings/node_modules/@types/jquery/JQueryStatic.d.ts
  • ^/ts-typings/node_modules/@types/jquery/JQuery.d.ts
  • ^/ts-typings/node_modules/@types/jquery/misc.d.ts
  • ^/ts-typings/node_modules/@types/jquery/legacy.d.ts
  • ^/ts-typings/node_modules/@types/jquery/index.d.ts

All of these are in-memory files so TS Server cannot read them directly. I believe we also have to keep the different scheme (memfs vs ts-typings) as part of the implementation on the VS Code side

What type of API do you think we need to get typescript to understand that import 'jquery' should resolve to ^/ts-typings/node_modules/@types/jquery/JQuery.d.ts?

@orta
Copy link
Contributor

orta commented Jan 19, 2022

There's a similar tension in doing the full TypeScript compile in the ts-vscode playground - I wonder if a reasonable strategy lays in making the TypeScript web server smarter, and keeping some of the file content inside that. As it exists, there is no way to influence what is available to that TSServer:

    function createWebSystem(args: string[]) {
        Debug.assert(ts.sys === undefined);
        const webHost: WebHost = {
            readFile: webPath => {
                const request = new XMLHttpRequest();
                request.open("GET", webPath, /* asynchronous */ false);
                request.send();
                return request.status === 200 ? request.responseText : undefined;
            },
            fileExists: webPath => {
                const request = new XMLHttpRequest();
                request.open("HEAD", webPath, /* asynchronous */ false);
                request.send();
                return request.status === 200;
            },
            writeMessage,
        };

I think somewhere there needs to be a vfs map which the tsserver prioritises for sys lookups for dirs/files and it might need to be insider the web server. That could be controlled via additional tsserver actions/events.

What type of API do you think we need to get typescript to understand that import 'jquery' should resolve to ^/ts-typings/node_modules/@types/jquery/JQuery.d.ts?

I think that might get tricky with the schema change, I think you'd need to hardcode that somehow into the TS web server code somehow (for examples it is kinda safe that all paths with @types could be treated that way)

@mjbvz
Copy link
Collaborator Author

mjbvz commented Jan 19, 2022

@orta Thanks! Not sure I understand the first part though.

With my current approach, on web all TS files are opened against as in-memory resources. This means that TS knows the file content but should not try to read these files from disk. Are you proposing a different approach where readFile would be needed?

For the second part, who is the best point of contact to discuss this with? If needed I think we could rewrite the paths that VS Code sends to TS Server to make the project system happy, but I'd like to explore a proper solution first

@mjbvz
Copy link
Collaborator Author

mjbvz commented Jan 21, 2022

My current proposal would be a new TS Server api that let a client explicitly pass global typings files to the TypeScript Server. This could be used both to implement ATA, and also for applications like Office Scripts where you have a VS Code extension that wants to augment the global typings in JS files

Some of this discussion also gets into our discussions around virtual file system support for vscode.dev

One idea: could a client for example tell TS Server about a type root directory and then the server would simply treat any files opened under this directory as typings

From a protocol point of view, this would look something like:

  1. Client passes a root directory for typings to TS Server: configure { typeRoots: ['^/ts-typings'] }

  2. Client then opens files under this types root: updateOpen { path: '^/ts-typings/jquery/jquery.d.ts' }

  3. When resolving imports, the server then looks in the types root.

    Since the server only has an in-memory representation of these read files/folders, the server may have to construct its own little virtual file system representation to implement this

@orta
Copy link
Contributor

orta commented Jan 22, 2022

(sorry, been a busy few weeks)

I'm not sure that you need to special case the vfs as only being typings specifically, for the playground ATA we emulated having files in node_modules/@types/jquery/index.d.ts which meant it was picked up by the default behavior in TypeScript

If you have a generic system to put vfs files into the web tsserver's sys then you can put files into the right place and it would automatically be respected by the rest of the tsserver

@mjbvz
Copy link
Collaborator Author

mjbvz commented Jan 25, 2022

Opened microsoft/TypeScript#47600 to continue discussion of generic virtual file system support for TS Server. Using this, I believe we could implement ATA by writing the acquired typings to the correct location in the virtual file system

@ameerthehacker
Copy link

is the intellisense between typescript files also in scope? like will I get intellisense for a hello function declared in a different file when I import it?

@mjbvz mjbvz added this to the December 2023 milestone Dec 4, 2023
@mjbvz mjbvz added typescript Typescript support issues javascript JavaScript support issues web Issues related to running VSCode in the web labels Dec 4, 2023
@mjbvz mjbvz modified the milestones: December / January 2024, February 2024 Jan 22, 2024
@mjbvz mjbvz modified the milestones: February 2024, March 2024 Feb 21, 2024
@mjbvz mjbvz modified the milestones: March 2024, April 2024 Mar 25, 2024
@mjbvz mjbvz modified the milestones: April 2024, May 2024 Apr 22, 2024
mjbvz added a commit to mjbvz/vscode that referenced this issue May 9, 2024
@mjbvz mjbvz closed this as completed in a4643b0 May 9, 2024
@vscodenpa vscodenpa added the unreleased Patch has not yet been released in VS Code Insiders label May 9, 2024
@vscodenpa vscodenpa added insiders-released Patch has been released in VS Code Insiders and removed unreleased Patch has not yet been released in VS Code Insiders labels May 10, 2024
@microsoft microsoft locked and limited conversation to collaborators Jun 23, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
insiders-released Patch has been released in VS Code Insiders javascript JavaScript support issues typescript Typescript support issues web Issues related to running VSCode in the web
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants