The obj folder should be created with Directory.CreateDirectory

[nkolev92]

Originally reported in #7341.

Specifically this is the issue that @bricelam & team reported.

System.IO.DirectoryNotFoundException: Could not find a part of the path '/Users/vsts/agent/2.148.1/work/1/s/artifacts/obj/70d88d93-1df8-47b8-8c56-447722545fd1'.
   at System.IO.FileSystem.CreateDirectory(String fullPath)
   at System.IO.Directory.CreateDirectory(String path)
   at NuGet.Common.DirectoryUtility.CreateSingleSharedDirectory(String path)
   at NuGet.Common.DirectoryUtility.CreateSharedDirectory(String path)
   at NuGet.Commands.NoOpRestoreUtilities.PersistDGSpecFile(DependencyGraphSpec spec, RestoreRequest request, ILogger log)
   at NuGet.Commands.NoOpRestoreUtilities.GetHash(RestoreRequest request)
   at NuGet.Commands.RestoreCommand.EvaluateCacheFile()
   at NuGet.Commands.RestoreCommand.ExecuteAsync(CancellationToken token)
   at NuGet.Commands.RestoreRunner.ExecuteAsync(RestoreSummaryRequest summaryRequest, CancellationToken token)
   at NuGet.Commands.RestoreRunner.ExecuteAndCommitAsync(RestoreSummaryRequest summaryRequest, CancellationToken token)
   at NuGet.Commands.RestoreRunner.CompleteTaskAsync(List`1 restoreTasks)
   at NuGet.Commands.RestoreRunner.RunAsync(IEnumerable`1 restoreRequests, RestoreArgs restoreContext, CancellationToken token)
   at NuGet.Commands.RestoreRunner.RunAsync(RestoreArgs restoreContext, CancellationToken token)
   at NuGet.Build.Tasks.RestoreTask.ExecuteAsync(ILogger log)

The scenario is restore on mac/linux & it repros equivalently on 2.1, 2.2 and 3.0 preview runtimes.

NuGet writes to various directories during restore. NuGet uses the /tmp directory and as it’s a machine-wide directory NuGet has to create a directory that’s shareable across users as multiple users.
We have an implementation, that basically creates a folder as the same level as the wanted one and moves to the intended directory path.

NuGet restore is frequently the first tool to write to the obj folder during build. As such we need to create the obj folder. An oversight during an implementation of a new feature caused this CreateSharedDirectory method to be called for the obj folder. Now changing back to Directory.CreateDirectory fixes this particular problem, but understanding what exactly happened is important for our future investigation of issues in a similar codepath.

The scenario is the following. All the projects in a solution are redirected from root directory of the project obj to /obj/projectName/.
NuGet runs 8 concurrent project restore operations at the same time.
All 8 make a call to create the path/to/solution/root/obj/projectName.
Due to reasons I don’t understand Directory.CreateDirectory method throws with an error such as:
error : Could not find a part of the path '/home/nikolev/nuget.client/artifacts/39b567ca-81be-4b14-a271-a9be767a6888'. [/home/nikolev/nuget.client/src/NuGet.Core/NuGet.DependencyResolver.Core/NuGet.DependencyResolver.Core.csproj]

This only repros when there are at least 6-7 projects being restored. I was never able to repro it on with fewer projects.

I have added logs to the DirectoryUtility, and generated the logs.
https://github.com/NuGet/NuGet.Client/blob/1f9ad07fbbc5e80ab11eee4b4ea125d99dc6b291/src/NuGet.Core/NuGet.Common/PathUtil/DirectoryUtility.cs

The branch with the logs is dev-nkolev92-pathIssues

PathIssueLogs.txt

On the side I'll be working to truly understand the root cause.

I will raise a PR fixing this particular problem.

[nkolev92]

Will close when it's merged to 5.0.1.

[nkolev92]

@rrelyea Do we want to patch #7854 as well?

It's a similar code path, but the impact is lower.

[nkolev92]

Merged with NuGet/NuGet.Client@e32a2ea.

[rrelyea]

This security fix shipped in NuGet 5.0.2
Details:
https://portal.msrc.microsoft.com/en-US/security-guidance/advisory/CVE-2019-0976

[carnil]

@rrelyea, @nkolev92: Is there any information in which version the issue reffered to by CVE-2019-0976 was introduced?

[nkolev92]

@carnil
It was introduced in 5.0.0.

[carnil]

@nkolev92: thanks for the confirmation