Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update assembly version from hardcoded to MajorVersion #74157

Merged
merged 23 commits into from
Nov 2, 2022

Conversation

carlossanlop
Copy link
Member

Addresses

Update the assembly version that will be hardcoded to 7.0, to 8.0.

from #72112

@ghost
Copy link

ghost commented Aug 18, 2022

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

Issue Details

Addresses

Update the assembly version that will be hardcoded to 7.0, to 8.0.

from #72112

Author: carlossanlop
Assignees: carlossanlop
Labels:

area-Infrastructure-libraries

Milestone: 8.0.0

@ViktorHofer
Copy link
Member

@carlossanlop tests are failing in the PR because they assert on MajorVersion being 7 instead of 8. Can you please take care of updating the tests and remove such assumptions so that this won't fail against when we brand to NET 9 next year?

@carlossanlop
Copy link
Member Author

Currently looking into this.

@carlossanlop
Copy link
Member Author

The System.Resources.Extensions failure is more complicated. It seems it is not retrieving the version info in plain text, so I'll have to investigate further.

@carlossanlop
Copy link
Member Author

Ah, the tests csproj says I should execute this target from console:

dotnet build -f net7.0-windows /t:GenerateTestResourcesFile System.Resources.Extensions.Tests.csproj

The problem is that if I bump the version number of the command to 8, I get this error:

D:\runtime2\src\libraries\System.Resources.Extensions\tests\System.Resources.Extensions.Tests.csproj(49,5): error MSB6003: The specified task executable "cmd.exe" could not be run. Syste
m.IO.DirectoryNotFoundException: The working directory "D:\runtime2\artifacts\bin\System.Resources.Extensions.Tests\Debug\net8.0-windows\" does not exist.
D:\runtime2\src\libraries\System.Resources.Extensions\tests\System.Resources.Extensions.Tests.csproj(49,5): error MSB6003:    at Microsoft.Build.Tasks.Exec.GetWorkingDirectory()
D:\runtime2\src\libraries\System.Resources.Extensions\tests\System.Resources.Extensions.Tests.csproj(49,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.GetProcessStartInfo(St
ring pathToTool, String commandLineCommands, String responseFileSwitch)
D:\runtime2\src\libraries\System.Resources.Extensions\tests\System.Resources.Extensions.Tests.csproj(49,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.ExecuteTool(String pat
hToTool, String responseFileCommands, String commandLineCommands)
D:\runtime2\src\libraries\System.Resources.Extensions\tests\System.Resources.Extensions.Tests.csproj(49,5): error MSB6003:    at Microsoft.Build.Tasks.Exec.ExecuteTool(String pathToTool,
 String responseFileCommands, String commandLineCommands)
D:\runtime2\src\libraries\System.Resources.Extensions\tests\System.Resources.Extensions.Tests.csproj(49,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.Execute()

And I checked, and that net8.0-windows folder does not exist. I did build the whole repo, and I'm using the branch of this PR. So what's missing? How do I ensure the artifacts of the test project get automatically generated in the folder D:\runtime2\artifacts\bin\System.Resources.Extensions.Tests\Debug\net8.0-windows\?

@ViktorHofer
Copy link
Member

And I checked, and that net8.0-windows folder does not exist. I did build the whole repo, and I'm using the branch of this PR. So what's missing? How do I ensure the artifacts of the test project get automatically generated in the folder D:\runtime2\artifacts\bin\System.Resources.Extensions.Tests\Debug\net8.0-windows?

Bumping the target framework from net7.0 to net8.0 is tracked in #72112. You will want to find a solution without depending on the project already targeting net8.0.

@carlossanlop
Copy link
Member Author

I hardcoded net8.0-windows in TargetFrameworks in the test project. I built one level above (without the specific Target) and it created the bin and obj folders. But now I'm getting this error:

D:\runtime2\src\libraries\System.Resources.Extensions\tests\System.Resources.Extensions.Tests.csproj(49,5): error MSB3073: The command "D:\runtime2\artifacts\bin\testhost\net7.0-windows-
Debug-x64\dotnet exec --runtimeconfig D:\runtime2\artifacts\bin\System.Resources.Extensions.Tests\Debug\net8.0-windows\System.Resources.Extensions.Tests.runtimeconfig.json Microsoft.DotN
et.RemoteExecutor.dll System.Resources.Extensions.Tests System.Resources.Extensions.Tests.TestData WriteResources D:\runtime2\artifacts\bin\System.Resources.Extensions.Tests\Debug\net8.0
-windows\System.Resources.Extensions.Tests.TestDataWriteResources.exception.txt D:\runtime2\src\libraries\System.Resources.Extensions\tests\TestData.resources" exited with code -21474507
51.

This is because the Tests.csproj has this... uh... dependency?.. on the Target:

  <Target Name="GenerateTestResourcesFile">
    <PropertyGroup>
      <_executor>Microsoft.DotNet.RemoteExecutor.dll</_executor>

I tried searching for other places in the whole repo where we mention Microsoft.DotNet.RemoteExecutor.dll, but I only found the generated obj and bin json files, which have that dll listed as a dependency, but its the 7.0 version. Both files seem to have identical content:

D:\runtime2\artifacts\bin\System.Resources.Extensions.Tests\Debug\net7.0-unix\System.Resources.Extensions.Tests.deps.json
D:\runtime2\artifacts\bin\System.Resources.Extensions.Tests\Debug\net7.0-windows\System.Resources.Extensions.Tests.deps.json

  "compilationOptions": {},
  "targets": {
    ".NETCoreApp,Version=v7.0": {
      "System.Resources.Extensions.Tests/8.0.0-dev": {
       ...
        "runtime": {
          "lib/net7.0/Microsoft.DotNet.RemoteExecutor.dll": {

I'll try searching without the dll extension, in csproj files only, see what I can find.

@carlossanlop
Copy link
Member Author

That error number in the output was not helpful, so I executed the command manually, and I think the problem is that the executor DLL is not being found:

D:\runtime2\artifacts\bin\testhost\net7.0-windows-Debug-x64\dotnet exec --runtimeconfig D:\runtime2\artifacts\bin\System.Resources.Extensions.Tests\Debug\net8.0-windows\System.Resources.Extensions.Tests.runtimeconfig.json Microsoft.DotNet.RemoteExecutor.dll System.Resources.Extensions.Tests System.Resources.Extensions.Tests.TestData WriteResources D:\runtime2\artifacts\bin\System.Resources.Extensions.Tests\Debug\net8.0-windows\System.Resources.Extensions.Tests.TestDataWriteResources.exception.txt D:\runtime2\src\libraries\System.Resources.Extensions\tests\TestData.resources
The application to execute does not exist: 'Microsoft.DotNet.RemoteExecutor.dll'

Now I'm trying to figure out how to get the full path for the remote executor. The only thing I can find so far in other test projects is that they have <IncludeRemoteExecutor>true</IncludeRemoteExecutor> and that takes care of it. Need to learn how the DLL is found by the tests.

@carlossanlop
Copy link
Member Author

Making progress... Found the location of the executor, so I hardcoded it in the csproj. Now the command complains that the json is not found:

D:\runtime2\artifacts\bin\testhost\net7.0-windows-Debug-x64\dotnet exec --runtimeconfig D:\runtime2\artifacts\bin\System.Resources.Extensions.Tests\Debug\net7.0-windows\System.Resources.Extensions.Tests.runtimeconfig.json D:\runtime2\artifacts\bin\System.Resources.Extensions.Tests\Debug\net7.0-windows\Microsoft.DotNet.RemoteExecutor.dll System.Resources.Extensions.Tests System.Resources.Extensions.Tests.TestData WriteResources D:\runtime2\artifacts\bin\System.Resources.Extensions.Tests\Debug\net7.0-windows\System.Resources.Extensions.Tests.TestDataWriteResources.exception.txt D:\runtime2\src\libraries\System.Resources.Extensions\tests\TestData.resources
The specified runtimeconfig.json [D:\runtime2\artifacts\bin\System.Resources.Extensions.Tests\Debug\net8.0-windows\System.Resources.Extensions.Tests.runtimeconfig.json] does not exist

I decided to hardcode all those embedded paths to keep using the net7.0-windows files, and the command was successful in generating the new TestData.resources binary file.

I ran all the tests locally and they passed when consuming the new resources file:

  === TEST EXECUTION SUMMARY ===
     System.Resources.Extensions.Tests  Total: 54, Errors: 0, Failed: 0, Skipped: 0, Time: 0.594s
  ----- end 2022-08-25 12:30:39.52 ----- exit code 0 ----------------------------------------------------------
  System.Resources.Extensions -> D:\runtime2\artifacts\bin\System.Resources.Extensions\ref\Debug\net462\System.Resources.Extensions.dll
  TestUtilities -> D:\runtime2\artifacts\bin\TestUtilities\Debug\net462\TestUtilities.dll
  System.Resources.Extensions -> D:\runtime2\artifacts\bin\System.Resources.Extensions\Debug\net462\System.Resources.Extensions.dll
  System.Resources.Extensions.Tests -> D:\runtime2\artifacts\bin\System.Resources.Extensions.Tests\Debug\net462\System.Resources.Extensions.Tests.dll
  ----- start 2022-08-25 12:30:39.82 ===============  To repro directly: =====================================================
  pushd D:\runtime2\artifacts\bin\System.Resources.Extensions.Tests\Debug\net462\
  xunit.console.exe System.Resources.Extensions.Tests.dll -xml testResults.xml -nologo -notrait category=OuterLoop -notrait category=failing
  popd
  ===========================================================================================================
    Discovering: System.Resources.Extensions.Tests (app domain = on [no shadow copy], method display = ClassAndMethod, method display options = None)
    Discovered:  System.Resources.Extensions.Tests (found 25 test cases)
    Starting:    System.Resources.Extensions.Tests (parallel test collections = on, max threads = 12)
    Finished:    System.Resources.Extensions.Tests
  === TEST EXECUTION SUMMARY ===
     System.Resources.Extensions.Tests  Total: 54, Errors: 0, Failed: 0, Skipped: 0, Time: 0.693s
  ----- end 2022-08-25 12:30:41.81 ----- exit code 0 ----------------------------------------------------------

@carlossanlop
Copy link
Member Author

If the CI passes, I'd like to get this merged to unblock whoever depends on these changes, and I'll open an issue to separately fix the Tests.csproj so we don't have to go through this again. It will need a deeper investigation, because I don't know how to address it yet.

@carlossanlop
Copy link
Member Author

I see new failures caused by this PR. I'm investigating:

Affected queues:

  • Build Browser wasm Linux Release LibraryTests
  • Build Browser wasm Windows Release LibraryTests

Affected tests:

  • WasmTestOnBrowser-System.Net.Http.Functional.Tests
  • WasmTestOnBrowser-System.Net.WebSockets.Client.Tests

Job results:
https://dev.azure.com/dnceng/public/_build/results?buildId=1967392&view=logs&j=11c5c840-d20a-5536-428b-70d8cd9c6259&t=73598586-463f-5241-5dc8-b8ffdebf8b89&l=82

Log file:
https://helix.dot.net/api/2019-06-17/jobs/bc65ec7f-88cd-4c94-894f-e7d4786636d7/workitems/WasmTestOnBrowser-System.Net.Http.Functional.Tests/console

Relevant callstack:

Could not load file or assembly 'System.Runtime, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.

   at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module)
   at System.Reflection.Assembly.GetTypes()
   at Microsoft.DotNet.XHarness.CLI.CommandArguments.TypeFromAssemblyArgument`1.GetLoadedTypes()+MoveNext() in /_/src/Microsoft.DotNet.XHarness.CLI/CommandArguments/Arguments/TypeFromAssemblyArgument.cs:line 29
   at Microsoft.DotNet.XHarness.CLI.Commands.WebServer.<>c__DisplayClass0_0.<Start>b__9(TestWebServerOptions options) in /_/src/Microsoft.DotNet.XHarness.CLI/Commands/WebServer.cs:line 60
   at Microsoft.Extensions.Options.OptionsFactory`1.Create(String name)
   at Microsoft.Extensions.Options.OptionsCache`1.<>c__3`1.<GetOrAdd>b__3_0(String name, ValueTuple`2 arg)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd[TArg](TKey key, Func`3 valueFactory, TArg factoryArgument)
   at Microsoft.Extensions.Options.OptionsCache`1.GetOrAdd[TArg](String name, Func`3 createOptions, TArg factoryArgument)
   at Microsoft.DotNet.XHarness.CLI.Commands.WebServer.TestWebServerStartup.Configure(IApplicationBuilder app, IOptionsMonitor`1 optionsAccessor) in /_/src/Microsoft.DotNet.XHarness.CLI/Commands/WebServer.cs:line 126
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Microsoft.AspNetCore.Hosting.ConfigureBuilder.Invoke(Object instance, IApplicationBuilder builder)
   at Microsoft.AspNetCore.Hosting.ConventionBasedStartup.Configure(IApplicationBuilder app)
   at Microsoft.AspNetCore.Hosting.WebHost.BuildApplication()
System.IO.FileNotFoundException: Could not load file or assembly 'System.Runtime, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.

File name: 'System.Runtime, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'
System.IO.FileNotFoundException: Could not load file or assembly 'System.Runtime, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.

@ViktorHofer
Copy link
Member

@carlossanlop kindly asking if you can share an update on the remaining test failures? Did you find the root cause for the above mentioned issue?

@smasher164
Copy link
Member

The build failure

../build/Microsoft.NET.ApiCompat.ValidatePackage.targets(37,5): error CP0004: Assembly with name 'left' does not exist at System.Diagnostics.EventLog.Messages. [/__w/1/s/src/libraries/System.Diagnostics.EventLog/src/System.Diagnostics.EventLog.csproj]

is reporting CP004, which is likely from
https://github.com/dotnet/sdk/blob/bb5c030b3081596060c55d2c4117c1527d4acb6a/src/ApiCompat/Microsoft.DotNet.ApiCompatibility/Rules/AssemblyIdentityMustMatch.cs#L36

The default MetadataInformation will lead to the assembly name "left".

The AssemblyMapper passes that metadata here when it's null: https://github.com/dotnet/sdk/blob/bb5c030b3081596060c55d2c4117c1527d4acb6a/src/ApiCompat/Microsoft.DotNet.ApiCompatibility/Rules/RuleRunner.cs#L45.

If the assembly is already null at that point, then AssemblySymbolLoader probably failed to load it in https://github.com/dotnet/sdk/blob/bb5c030b3081596060c55d2c4117c1527d4acb6a/src/ApiCompat/Microsoft.DotNet.ApiCompatibility/AssemblySymbolLoader.cs? I'm surprised, since all this PR changes is that it prefixes $(MajorVersion).

/cc @ViktorHofer

- The RemoteLoopServer, and NetCoreServer are projects used as aspnetcore
middleware, and loaded by xharness.
- These are built against the live artifacts, same as other projects.
    - But this can be a problem when the libraries in `runtime` are on a
    newer assembly version (say `8.0.0`), but xharness is still built with
    `7.0.0` libraries.
    - In that case, xharness fails to load the middleware:

```
Application startup exception: System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types.

...

Could not load file or assembly 'System.Runtime, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.

   at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module)
   at System.Reflection.Assembly.GetTypes()
   at Microsoft.DotNet.XHarness.CLI.CommandArguments.TypeFromAssemblyArgument`1.GetLoadedTypes()+MoveNext() in /_/src/Microsoft.DotNet.XHarness.CLI/CommandArguments/Arguments/TypeFromAssemblyArgument.cs:line 29
   at Microsoft.DotNet.XHarness.CLI.Commands.WebServer.<>c__DisplayClass0_0.<Start>b__9(TestWebServerOptions options) in /_/src/Microsoft.DotNet.XHarness.CLI/Commands/WebServer.cs:line 60
   at Microsoft.Extensions.Options.OptionsFactory`1.Create(String name)
   at Microsoft.Extensions.Options.OptionsCache`1.<>c__3`1.<GetOrAdd>b__3_0(String name, ValueTuple`2 arg)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd[TArg](TKey key, Func`3 valueFactory, TArg factoryArgument)
   at Microsoft.Extensions.Options.OptionsCache`1.GetOrAdd[TArg](String name, Func`3 createOptions, TArg factoryArgument)
   at Microsoft.DotNet.XHarness.CLI.Commands.WebServer.TestWebServerStartup.Configure(IApplicationBuilder app, IOptionsMonitor`1 optionsAccessor) in /_/src/Microsoft.DotNet.XHarness.CLI/Commands/WebServer.cs:line 126
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Microsoft.AspNetCore.Hosting.ConfigureBuilder.Invoke(Object instance, IApplicationBuilder builder)
   at Microsoft.AspNetCore.Hosting.ConventionBasedStartup.Configure(IApplicationBuilder app)
   at Microsoft.AspNetCore.Hosting.WebHost.BuildApplication()
```

- Build the project in isolation from rest of the repo, so that it is
built with references only from the SDK. The built assembly can then be
deployed for use with xharness, just like before.
.. for runtime tests since the paths are set explicitly in the project
The middleware projects are now being built separately, so they can
target a different TFM. But when `RunAOTCompilation=true` is set, like
on CI/AOT builds, the build breaks with:

```
To build this project, the following workloads must be installed: macos [/__w/1/s/Build.proj]
To install these workloads, run the following command: dotnet workload restore [/__w/1/s/Build.proj]
```

This is described in dotnet#77707 .
And the issue is present in `7.0 rc1` also, which is currently used for
building the repo. To workaround it, we unset some properties which
aren't needed anyway for the middleware projects builds.
@radical radical self-requested a review as a code owner November 1, 2022 06:08
@radical
Copy link
Member

radical commented Nov 1, 2022

/azp run runtime-wasm

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@radical
Copy link
Member

radical commented Nov 1, 2022

The remaining wasm failures in runtime-wasm should be unrelated to this PR, and will be fixed by #77704 . I will confirm that we don't see any other failures here, tomorrow.

@radical
Copy link
Member

radical commented Nov 1, 2022

The wasm/nodejs failures are #77631 .

@radical
Copy link
Member

radical commented Nov 1, 2022

@carlossanlop this is good to go from my pov.

@carlossanlop
Copy link
Member Author

Thank you so much, @radical for your help with this. The rest of the non-wasm failures are also unrelated.

@carlossanlop carlossanlop merged commit f84c1b5 into dotnet:main Nov 2, 2022
@carlossanlop carlossanlop deleted the Net8Changes branch November 2, 2022 20:28
ericstj added a commit that referenced this pull request Nov 4, 2022
carlossanlop pushed a commit that referenced this pull request Nov 7, 2022
rainersigwald added a commit to dotnet/sdk that referenced this pull request Nov 29, 2022
This has dotnet/runtime#74157, which is needed to use compiled-against-8.0 tasks.
@ghost ghost locked as resolved and limited conversation to collaborators Dec 3, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants