Flexible packages folder locations for kpm and runtime

[davidfowl]

Today we find packages based on a few heuristics, see #213 for the issues that arise. KPM and K will be changed to do the following:

• Remove the logic that finds the packages folder by walking up the directory tree. It's undeterministic and flaky. Instead we will look for a packages folder in this order:
• If --packages was passed to either kpm or k then use that as the packages location.
• If KRE_PACKAGES was set, then use that as the packages location.
• global.json will have a way to declare a single path that can be used to find packages

{
  "sources": [...],
  "packages": "packages"
}

If the global.json packages property is set then use that as the packages folder

• If none of the above are specified %userprofile%\.kpm\packages (windows) or $HOME/.kpm/packages (*nix) will be used.

Other things that will change as a result:

• The package layout on disk today is {name}.{version}. Instead, it will be {name}/{version}
• kpm pack should write the packages folder into the global.json so that the behavior is deterministic for a packaged application.

Open questions:

• The problem with changing the location and folder layout means nuget won't be able to read them. We can make nuget also understand this layout (when local). The effect is if you have a csproj based class library that uses nuget packages, those won't see the kpm installed nuget packages (when it's global, it'll be transparent though). NuGet can't use global package references directly because the csproj has the path to the dll.
• NuGet.config has a repositoryPath setting that we could read but the runtime would have to understand the hierarchy of nuget configuration (which may bring in dependencies we don't want).
• CI machines can have an ever growing global packages location. We plan to use KRE_PACKAGES to overcome this but we need to figure out if we recommend that people set the packages option in global.json or not.

/cc @glennc @lodejard @Praburaj @sayedihashimi @lucamorelli @ChengTian @Eilon

[jeffhandley]

I think we should use NuGet's repositoryPath and do any necessary refactoring to be able to get the setting. nuget.exe config repositoryPath -aspath is a good way to get it from the command line today.

I don't think we'd want to support 2 config systems.

[davidfowl]

@jeffhandley I'm not 100% opposed to being able to read that setting from nuget.config. We already read nuget.config for kpm but I don't want the runtime itself to have to understand the nuget.config hierarchy with clears and overrides.

[davidfowl]

Maybe can can just support a nuget.config next to the solution

[jeffhandley]

I expect you'll end up with the same feature requests we got though, for instance allowing settings to be specified once on the machine.

It would be useful to be able to have a NuGet.config on the root drive of my server, and all apps share a repository path configured on the server without having to put the setting into the solution folder.

[davidfowl]

you can do that with an environment variable, see KRE_PACKAGES.

[davidfowl]

it is worth looking into regardless.

[davidfowl]

@jeffhandley what about the folder layout?

[sayedihashimi]

I like the idea of having a central packages folder for the local dev box. Whether it should read from KRE_PACKAGES or nuget.config, I'm not sure.

Maybe can can just support a nuget.config next to the solution

I would be careful relying on any files next to the solution. The solution concept is very loose in msbuild terms. Also a project can be in several solutions and in a lot of cases the .sln file is in some arbitrary location.

The effect is if you have a csproj based class library that uses nuget packages, those won't see the kpm installed nuget packages (when it's global, it'll be transparent though).

For .csproj when a nuget pkg adds a reference it is added in the following way.

<ItemGroup>
  <Reference Include="Microsoft.Web.Infrastructure, Version=1.0.0.0, ...cture=MSIL">
    <Private>True</Private>
    <HintPath>..\packages\Microsoft.Web.Infrastructure.1.0.0.0\lib\net40\Microsoft.Web.Infrastructure.dll</HintPath>
  </Reference>
</ItemGroup>

I think we can make this recognize a local cache as well. We could write the elements as something more like the following.

<PropertyGroup>
  <PackagesDir Condition=" '$(PackagesDir)'=='' ">..\packages\</PackagesDir>
</PropertyGroup>

<ItemGroup>
  <Reference Include="Microsoft.Web.Infrastructure, Version=1.0.0.0, ...cture=MSIL">
    <Private>True</Private>
    <HintPath>$(PackagesDir)Microsoft.Web.Infrastructure.1.0.0.0\lib\net40\Microsoft.Web.Infrastructure.dll</HintPath>
  </Reference>
</ItemGroup>

When a build is kicked off in VS we can set the PackagesDir as a global MSBuild property. For command line builds we can either:

• Work on getting this added to the .sln build
• Instruct users to set PackagesDir as a env var if they want it to use the shared location

[davidfowl]

@sayedihashimi In the past I've tried things like the above and it doesn't work (due to when different parts of the csproj get evaluated in VS). What you really want is a node that will run the same logic to figure out the actual hint path at build time.

We're also using {name}/{version} instead of {name}.{version}

[Eilon]

@GrabYourPitchforks @blowdart because they are interested in this from a servicing perspective.

[davidfowl]

Shouldn't change servicing but sure

[Eilon]

Servicing might want to use a similar solution. Or not. Or maybe.