Ensure watched projects are not built in parallel to avoid file locking issues

[JunTaoLuo]

Fixes #580.

I've experimented with several approaches and this seems the most general. One of the drawbacks is that if there are multiple services that are based on the same project file, re-running build on all of them will now be much slower. However, I'm skeptical of running build once and re-using the results since the files could have changed during the time the project was built the first time. Ideally we would track what file changes have been accounted for and whether builds need to be re-run but I don't think the file watcher has that capability at the momennt.

[jkotalik]

Add a debug log here

[rynowak]

I've experimented with several approaches and this seems the most general.

Did you consider the approach of synthesizing an MSBuild project w/ p2ps and calling build on that? This is something I discussed with @rainersigwald in the past, and it lets MSBuild just do its thing. This way you get to leverage parallelism in the build engine without creating a new build engine 😆 I know the idea of generating a project seems like a hack, but IIRC it's a technique that's used in a few places already.

[jkotalik]

Pending verification that I mentioned off of github.

[jkotalik]

Did you consider the approach of synthesizing an MSBuild project w/ p2ps and calling build on that? This is something I discussed with @rainersigwald in the past, and it lets MSBuild just do its thing. This way you get to leverage parallelism in the build engine without creating a new build engine 😆 I know the idea of generating a project seems like a hack, but IIRC it's a technique that's used in a few places already.

Yeah isn't that what we already do for the initial build? Could we leverage that here?

[rynowak]

Yeah isn't that what we already do for the initial build? Could we leverage that here?

I'm not sure if y'all ever built that idea. I don't think I got past the idea stage with it, but the expert said its the best way to avoid MSBuild stepping on itself while keeping parallelism.

[rainersigwald]

How are y'all building? Shelling out to dotnet build or using the MSBuild API? If the latter, could you do all your build submissions in the same beginbuild/endbuild pair? Then MSBuild should reuse build results so even if there's concurrent requests to build the same project we'll deduplicate them just like we would in a single build with a synthesized project that points to everything.

(Normally I'd dig in a bit deeper but I have ~1 hour of work time left before going on parental leave so not today!)

[rynowak]

(Normally I'd dig in a bit deeper but I have ~1 hour of work time left before going on parental leave so not today!)

And it sounds like not next week either. Congrats!

[JunTaoLuo]

@rynowak @jkotalik just to be clear, we evaluate all services (i.e. restore and read project metadata) using a generated meta-project file, as described above. However, we build and run each service individually, as seen here. nvm I read the code wrongly, it's restoring and building all projects in single msbuild invocations a meta project file.

Conceptually, it's not quite that simple either. Since we run each service in a different process, they each have a different file watcher. When some of them change, we'll need each process to report back to a centralized place where we can then determine which service(s) need to be rebuilt by tracing through all their dependencies and then generate a meta-project file to rebuild. This might be worth exploring in the future, but I'm not sure it's worth redesigning in this PR.

[JunTaoLuo]

It's unlikely I'll make progress on this PR.

[onionhammer]

Has this made it into a release yet? Seems like if I tye run --watch and change a project referenced by two services they often clash and 1 then fails to restart (and the other 'wins')