Fix "The process cannot access the file" error after a failed build

[adamsitnik]

While working on #1662 I've hit the following issue:

• run benchmarks_ci.py against Microbenchmarks.csproj that does not compile
• the script fails
• fix the build, re-run the benchmarks_ci.py script
• get the file in use error

PS C:\Projects\performance> py .\scripts\benchmarks_ci.py -f netcoreapp3.1 --filter *LockUnlock                                                                                                                                                                                 [2021/02/12 10:50:30][INFO] ----------------------------------------------
[2021/02/12 10:50:30][INFO] Initializing logger 2021-02-12 10:50:30.329528
[2021/02/12 10:50:30][INFO] ----------------------------------------------
[2021/02/12 10:50:30][INFO] Installing tools.
[2021/02/12 10:50:30][INFO] ----------------------
[2021/02/12 10:50:30][INFO] Downloading DotNet Cli
[2021/02/12 10:50:30][INFO] ----------------------
[2021/02/12 10:50:30][INFO] DotNet Install Path: 'C:\Projects\performance\tools\dotnet\x64'
[2021/02/12 10:50:30][INFO] Downloading https://dot.net/v1/dotnet-install.ps1
[2021/02/12 10:50:31][INFO] $ pushd "C:\Projects\performance"
[2021/02/12 10:50:31][INFO] $ powershell.exe -NoProfile -ExecutionPolicy Bypass C:\Projects\performance\tools\dotnet\x64\dotnet-install.ps1 -InstallDir C:\Projects\performance\tools\dotnet\x64 -Architecture x64 -Channel release/3.1.3xx
[2021/02/12 10:50:31][INFO] dotnet-install: Note that the intended use of this script is for Continuous Integration (CI) scenarios, where:
[2021/02/12 10:50:31][INFO] dotnet-install: - The SDK needs to be installed without user interaction and without admin rights.
[2021/02/12 10:50:31][INFO] dotnet-install: - The SDK installation doesn't need to persist across multiple CI runs.
[2021/02/12 10:50:31][INFO] dotnet-install: To set up a development environment or to run apps, use installers rather than this script. Visit https://dotnet.microsoft.com/download to get the installer.
[2021/02/12 10:50:31][INFO]
[2021/02/12 10:50:36][INFO] dotnet-install: Downloading primary link https://dotnetcli.azureedge.net/dotnet/Sdk/3.1.303-servicing-015192/dotnet-sdk-3.1.303-servicing-015192-win-x64.zip
[2021/02/12 10:50:39][INFO] dotnet-install: Extracting zip from https://dotnetcli.azureedge.net/dotnet/Sdk/3.1.303-servicing-015192/dotnet-sdk-3.1.303-servicing-015192-win-x64.zip
[2021/02/12 10:50:41][INFO] C:\Projects\performance\tools\dotnet\x64\dotnet-install.ps1 : Exception calling "ExtractToFile" with "3" argument(s): "The process cannot access the file 'C:\Projects\performance\tools\dotnet\x64\dotnet.exe' because it is being used by another process."
[2021/02/12 10:50:41][INFO] At line:1 char:1
[2021/02/12 10:50:41][INFO] + C:\Projects\performance\tools\dotnet\x64\dotnet-install.ps1 -InstallD ...
[2021/02/12 10:50:41][INFO] + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[2021/02/12 10:50:41][INFO]     + CategoryInfo          : NotSpecified: (:) [dotnet-install.ps1], MethodInvocationException
[2021/02/12 10:50:41][INFO]     + FullyQualifiedErrorId : IOException,dotnet-install.ps1
[2021/02/12 10:50:41][INFO]
[2021/02/12 10:50:41][INFO] $ popd
[2021/02/12 10:50:41][INFO] $ pushd "C:\Projects\performance"
[2021/02/12 10:50:41][INFO] $ powershell.exe -NoProfile -ExecutionPolicy Bypass C:\Projects\performance\tools\dotnet\x64\dotnet-install.ps1 -InstallDir C:\Projects\performance\tools\dotnet\x64 -Architecture x64 -Channel release/3.1.3xx
[2021/02/12 10:50:42][INFO] dotnet-install: Note that the intended use of this script is for Continuous Integration (CI) scenarios, where:
[2021/02/12 10:50:42][INFO] dotnet-install: - The SDK needs to be installed without user interaction and without admin rights.
[2021/02/12 10:50:42][INFO] dotnet-install: - The SDK installation doesn't need to persist across multiple CI runs.
[2021/02/12 10:50:42][INFO] dotnet-install: To set up a development environment or to run apps, use installers rather than this script. Visit https://dotnet.microsoft.com/download to get the installer.
[2021/02/12 10:50:42][INFO]
[2021/02/12 10:50:47][INFO] dotnet-install: Downloading primary link https://dotnetcli.azureedge.net/dotnet/Sdk/3.1.303-servicing-015192/dotnet-sdk-3.1.303-servicing-015192-win-x64.zip
[2021/02/12 10:50:49][INFO] dotnet-install: Extracting zip from https://dotnetcli.azureedge.net/dotnet/Sdk/3.1.303-servicing-015192/dotnet-sdk-3.1.303-servicing-015192-win-x64.zip
[2021/02/12 10:50:50][INFO] C:\Projects\performance\tools\dotnet\x64\dotnet-install.ps1 : Exception calling "ExtractToFile" with "3" argument(s): "The process cannot access the file 'C:\Projects\performance\tools\dotnet\x64\dotnet.exe' because it is being used by another process."
[2021/02/12 10:50:50][INFO] At line:1 char:1
[2021/02/12 10:50:50][INFO] + C:\Projects\performance\tools\dotnet\x64\dotnet-install.ps1 -InstallD ...
[2021/02/12 10:50:50][INFO] + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[2021/02/12 10:50:50][INFO]     + CategoryInfo          : NotSpecified: (:) [dotnet-install.ps1], MethodInvocationException
[2021/02/12 10:50:50][INFO]     + FullyQualifiedErrorId : IOException,dotnet-install.ps1
[2021/02/12 10:50:50][INFO]
[2021/02/12 10:50:50][INFO] $ popd
[2021/02/12 10:50:50][ERROR] Process exited with status 1

We had exactly the same problem in BenchmarkDotNet. I was hoping that setting UseSharedCompilation to false is enough, but MSBuild was still spawning some child processes and they were keeping some resources locked.

The only solution that I've found working was to enforce MSBuild to not perform a parallel build and don't start more than a single process.

I've tested it against all TFMs that we support (netcoreapp2.1 netcoreapp3.1 netcoreapp5.0 net6.0 net461) and it works.

[DrewScoggins]

LGTM, except adding a comment.

[billwert]

Agree with @DrewScoggins about a comment.

[adamsitnik]

The scenarios failures looked fishy and initially I thought that they were related to my change:

 Fatal error. Internal CLR error. (0x80131506)
    at System.Diagnostics.Tracing.NativeRuntimeEventSource.ThreadPoolWorkerThreadStart(UInt32, UInt32, UInt16)
    at System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()
    at System.Threading.Thread.StartCallback()

But looking at other PRs like #1665 it seems that this problem is not related to my change. Sample logs

@kouvel have we hit a bug in PortableThreadPool?

[billwert]

Looks like it is dotnet/runtime#48407