SourceGenerators as a replacement for VS extensions?

[msedi]

This is just question for SourceGenerators for some items that were not quite clear from the current design specs.

We currently have a VS extension that is invoked in a single CUDA (*.cu, e.g. FFT.cu) file every time the user saves this files.
Internally on saving, the extension invokes the NVCC compiler, deliver by nvidia and creates a so called PTX file (here: FFT.ptx) which is added to the project if not already available (if already available it will be overriden). It is also marked as embedded resource (related to #49923).

Additionally, we run clang to extract the syntax tree from the CUDA code (from FFT.cu) and created a wrapper for the CUDA code so that it can be invoked from C#. This wrapper also loads the embedded resources (here: FFT.ptx) and does the necessary CUDA things. This wrapper is created or added to the project (here: FFT.designer.cs), marked with "compile". As above if it already there, it will be overriden.

Addtionally, another file for user specific code is created (here: FFT.cs). But only once, it is not overriden when the users saves the CUDA code (FFT.cu).

There reason why this cannot be done during typing is that the process takes a while (some 5-10 seconds) and therefore we have decided only to do it when saving the code.

One problem is that extensions cannot be versioned and if I want to have another v ersion of the extension and need to uninstall it and install another version. So if I want to go back in source code history and in the meantime the CUDA extension has changed and need to know about this, otherwise I would create wrong ptx code. Also debugging extensions is a nightmare.

So in my opinion source generator would be the optimal tool. I can add it via nuget, which versions itself automatically and I can add this solution wide via Directory.Build.props.

My question is can source generator currently do this, and if not will they be able to do it in future versions?

[jasonmalinowski]

Today you'll run into a few problems. A source generator is capable of producing .cs files, so your wrapper could be easily produced. As you observed though we don't support generating embedded resources nor non-.cs files, so for now you wouldn't be able to do that. We're currently working on designs though for that so we absolutely expect those should be doable in future versions.

There reason why this cannot be done during typing is that the process takes a while (some 5-10 seconds)

What parts of the process you describe are the slow ones? At this point we don't have a great plan for generators that take a really long time like this, but that's something further we'd have to work on. Better understanding your use case here would help us accommodate that scenario better.

[msedi]

@jasonmalinowski: I checked again with visual studio. Currently we are using the custom tool and are creating our own IVsSingleFileGenerator. I think if this concept can be extended to also allow generators added to the project as nuget packages this would help. But I think this is nothing for this discussion, right?

[jasonmalinowski]

So I don't imagine you're necessarily registering for "IDE events" as much as having more granular ways to break apart the different parts of things that run. The IDE needs the C# source files for your inner editing loop, but we don't need the embedded resources until you're doing a build, or potentially edit-and-continue.

[jasonmalinowski]

I think if this concept can be extended to also allow generators added to the project as nuget packages this would help. But I think this is nothing for this discussion, right?

I guess we imagine source generators as our replacement, and we'll continue to beef up the design as needed.

[msedi]

@jasonmalinowski. Thanks a lot for the discussion. I will wait until you have more details/improvements on your side ;-) If you need some details of my current situation -> Just ask.

[jasonmalinowski]

@msedi I think the one specific detail would be for your clang processing, how expensive is it? Is the process on the order of a few milliseconds, tens, hundreds, or a second or more? Also, your extension today that reprocesses a CUDA file, when a single file is modified you only have to recompute that file, right? You don't have to reanalyze all files at once?