XUnitLogChecker: `runtime-libraries-coreclr outerloop` tests are unable to find the tool DLL #96035

carlossanlop · 2023-12-14T18:12:06Z

Error Blob

{
  "ErrorMessage": "XUnitLogChecker.dll does not exist in the expected location:",
  "BuildRetry": false,
  "ErrorPattern": "",
  "ExcludeConsoleLog": false
}

Reproduction Steps

Queue: runtime-libraries-coreclr outerloop
Build example: https://dev.azure.com/dnceng-public/public/_build/results?buildId=498370&view=logs&j=631ed29b-f0dd-5b15-38a0-c6c555556dea&t=b9941189-b5a8-5df3-6b3a-0964fabb3acd
Log example: https://helixre107v0xdcypoyl9e7f.blob.core.windows.net/dotnet-runtime-refs-pull-95898-merge-d360b025a7c14161a1/ComInterfaceGenerator.Tests/1/console.66454013.log?helixlogtype=result
Output example:

Console log: 'ComInterfaceGenerator.Tests' from job d360b025-a7c1-4161-a1a4-19f920345fd3 workitem 57966585-bda4-4f37-8ff1-0af071d4373f (windows.81.amd64.open.rt) executed on machine a0018BJ running Windows-2012ServerR2-6.3.9600-SP0

C:\h\w\A22708CE\w\A5080926\e>taskkill.exe /f /im corerun.exe 
ERROR: The process "corerun.exe" not found.

C:\h\w\A22708CE\w\A5080926\e>call RunTests.cmd --runtime-path C:\h\w\A22708CE\p 
========================= Begin custom configuration settings ==============================
set __TestArchitecture=x64
set __IsXUnitLogCheckerSupported=1
========================== End custom configuration settings ===============================
----- start Wed 12/13/2023 18:40:04.47 ===============  To repro directly: =====================================================
pushd C:\h\w\A22708CE\w\A5080926\e\
"C:\h\w\A22708CE\p\dotnet.exe" exec --runtimeconfig ComInterfaceGenerator.Tests.runtimeconfig.json --depsfile ComInterfaceGenerator.Tests.deps.json xunit.console.dll ComInterfaceGenerator.Tests.dll -xml testResults.xml -nologo -nocolor -trait category=OuterLoop -notrait category=IgnoreForCI -notrait category=failing 
popd
===========================================================================================================

C:\h\w\A22708CE\w\A5080926\e>"C:\h\w\A22708CE\p\dotnet.exe" exec --runtimeconfig ComInterfaceGenerator.Tests.runtimeconfig.json --depsfile ComInterfaceGenerator.Tests.deps.json xunit.console.dll ComInterfaceGenerator.Tests.dll -xml testResults.xml -nologo -nocolor -trait category=OuterLoop -notrait category=IgnoreForCI -notrait category=failing  
  Discovering: ComInterfaceGenerator.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  ComInterfaceGenerator.Tests (found 0 of 45 test cases)
=== TEST EXECUTION SUMMARY ===
   ComInterfaceGenerator.Tests.dll  Total: 0
----- end Wed 12/13/2023 18:40:06.22 ----- exit code 0 ----------------------------------------------------------
----- start ===============  XUnitLogChecker Output =====================================================
XUnitLogChecker.dll does not exist in the expected location: C:\h\w\A22708CE\p\XUnitLogChecker.dll
----- end ===============  XUnitLogChecker Output - exit code 1 ===============
2023-12-13T18:40:06.788Z	INFO   	run.py	run(48)	main	Beginning reading of test results.
2023-12-13T18:40:06.788Z	INFO   	run.py	__init__(42)	read_results	Searching 'C:\h\w\A22708CE\w\A5080926\e' for test results files
2023-12-13T18:40:06.788Z	INFO   	run.py	__init__(48)	read_results	Found results file C:\h\w\A22708CE\w\A5080926\e\testResults.xml with format xunit
2023-12-13T18:40:06.788Z	INFO   	run.py	__init__(42)	read_results	Searching 'C:\h\w\A22708CE\w\A5080926\uploads' for test results files
2023-12-13T18:40:06.788Z	INFO   	run.py	packing_test_reporter(30)	report_results	Packing 0 test reports to 'C:\h\w\A22708CE\w\A5080926\e\__test_report.json'
2023-12-13T18:40:06.788Z	INFO   	run.py	packing_test_reporter(33)	report_results	Packed 1537 bytes
ERROR: The process "corerun.exe" not found.
gen-debug-dump-docs.py: Did not find dumps, skipping dump docs generation.
['ComInterfaceGenerator.Tests' END OF WORK ITEM LOG: Command exited with 1]

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=498370
Error message validated: XUnitLogChecker.dll does not exist in the expected location:
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 12/14/2023 6:12:51 PM UTC

Report

Build	Definition	Test	Pull Request
579272	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
579267	dotnet/runtime	System.Composition.Convention.Tests.WorkItemExecution
578674	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
578671	dotnet/runtime	System.Diagnostics.StackTrace.Tests.WorkItemExecution
578189	dotnet/runtime	System.Diagnostics.DiagnosticSource.Switches.Tests.WorkItemExecution
578178	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
576975	dotnet/runtime	System.ComponentModel.TypeConverter.Tests.WorkItemExecution
576972	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
575556	dotnet/runtime	System.Composition.Convention.Tests.WorkItemExecution
575550	dotnet/runtime	System.DirectoryServices.AccountManagement.Tests.WorkItemExecution
574212	dotnet/runtime	System.Composition.Convention.Tests.WorkItemExecution
574209	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
572522	dotnet/runtime	System.ComponentModel.TypeConverter.Tests.WorkItemExecution
572516	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
571517	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
571515	dotnet/runtime	System.Composition.Hosting.Tests.WorkItemExecution
571093	dotnet/runtime	System.ComponentModel.TypeConverter.Tests.WorkItemExecution
571083	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
570521	dotnet/runtime	System.ComponentModel.TypeConverter.Tests.WorkItemExecution
570517	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
569242	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
569223	dotnet/runtime	System.Diagnostics.EventLog.Tests.WorkItemExecution
567659	dotnet/runtime	System.Composition.Hosting.Tests.WorkItemExecution
567650	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
565842	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
565835	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
565841	dotnet/runtime	System.Diagnostics.DiagnosticSource.Tests.WorkItemExecution
565821	dotnet/runtime	System.Composition.AttributeModel.Tests.WorkItemExecution
565639	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
565131	dotnet/runtime	System.Diagnostics.DiagnosticSource.Tests.WorkItemExecution	#97878
564673	dotnet/runtime	System.Diagnostics.Debug.Tests.WorkItemExecution	#97881
564228	dotnet/runtime	System.Diagnostics.PerformanceCounter.Tests.WorkItemExecution
564213	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
564223	dotnet/runtime	System.Composition.Convention.Tests.WorkItemExecution
564233	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
564235	dotnet/runtime	System.Diagnostics.FileVersionInfo.Tests.WorkItemExecution
564034	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
562431	dotnet/runtime	System.Diagnostics.TraceSource.Config.Tests.WorkItemExecution	#95001
562373	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
562370	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
562367	dotnet/runtime	System.Composition.Hosting.Tests.WorkItemExecution
562368	dotnet/runtime	System.ComponentModel.EventBasedAsync.Tests.WorkItemExecution
562232	dotnet/runtime	System.DirectoryServices.AccountManagement.Tests.WorkItemExecution
561856	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
561837	dotnet/runtime	System.Composition.Hosting.Tests.WorkItemExecution
561835	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
561841	dotnet/runtime	System.ComponentModel.Composition.Registration.Tests.WorkItemExecution
561784	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
561464	dotnet/runtime	Microsoft.Win32.SystemEvents.Tests.WorkItemExecution
561459	dotnet/runtime	System.Composition.AttributeModel.Tests.WorkItemExecution
561468	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
561473	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
561345	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
560075	dotnet/runtime	System.Console.Tests.WorkItemExecution	#97881
559956	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
559946	dotnet/runtime	System.Composition.Convention.Tests.WorkItemExecution
559942	dotnet/runtime	System.Collections.NonGeneric.Tests.WorkItemExecution
559954	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
559702	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
558426	dotnet/runtime	System.DirectoryServices.AccountManagement.Tests.WorkItemExecution
558420	dotnet/runtime	System.Collections.Tests.WorkItemExecution
558261	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
556981	dotnet/runtime	System.Console.Tests.WorkItemExecution	#95001
556803	dotnet/runtime	System.Composition.Convention.Tests.WorkItemExecution
556807	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
556787	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
556804	dotnet/runtime	System.Diagnostics.Process.Tests.WorkItemExecution
556542	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
554882	dotnet/runtime	System.Diagnostics.DiagnosticSource.Switches.Tests.WorkItemExecution
554898	dotnet/runtime	System.Composition.Runtime.Tests.WorkItemExecution
554901	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
554889	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
554895	dotnet/runtime	System.Diagnostics.EventLog.Tests.WorkItemExecution
554677	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
553193	dotnet/runtime	System.Composition.Hosting.Tests.WorkItemExecution
553196	dotnet/runtime	Microsoft.Win32.Registry.AccessControl.Tests.WorkItemExecution
553210	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
553190	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
553048	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
552711	dotnet/runtime	System.Composition.AttributeModel.Tests.WorkItemExecution
552713	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
552709	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
552698	dotnet/runtime	System.ComponentModel.EventBasedAsync.Tests.WorkItemExecution
552621	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
552162	dotnet/runtime	System.Composition.Convention.Tests.WorkItemExecution
552163	dotnet/runtime	System.ComponentModel.Primitives.Tests.WorkItemExecution
552147	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
552156	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
552062	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
551898	dotnet/runtime	System.Diagnostics.Debug.Tests.WorkItemExecution	#97878
550625	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
550618	dotnet/runtime	Microsoft.XmlSerializer.Generator.Tests.WorkItemExecution
550619	dotnet/runtime	System.Composition.Hosting.Tests.WorkItemExecution
550617	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
550482	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
549069	dotnet/runtime	System.Composition.AttributeModel.Tests.WorkItemExecution
549067	dotnet/runtime	System.ComponentModel.Tests.WorkItemExecution
549059	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
549062	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
548952	dotnet/runtime	System.Data.OleDb.Tests.WorkItemExecution	#95001
Displaying 100 of 123 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
2	14	123

The text was updated successfully, but these errors were encountered:

ghost · 2023-12-14T18:12:14Z

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

Error Blob

{
  "ErrorMessage": "XUnitLogChecker.dll does not exist in the expected location:",
  "BuildRetry": false,
  "ErrorPattern": "",
  "ExcludeConsoleLog": false
}

Reproduction Steps

Queue: runtime-libraries-coreclr outerloop
Build example: https://dev.azure.com/dnceng-public/public/_build/results?buildId=498370&view=logs&j=631ed29b-f0dd-5b15-38a0-c6c555556dea&t=b9941189-b5a8-5df3-6b3a-0964fabb3acd
Log example: https://helixre107v0xdcypoyl9e7f.blob.core.windows.net/dotnet-runtime-refs-pull-95898-merge-d360b025a7c14161a1/ComInterfaceGenerator.Tests/1/console.66454013.log?helixlogtype=result
Output example:

Console log: 'ComInterfaceGenerator.Tests' from job d360b025-a7c1-4161-a1a4-19f920345fd3 workitem 57966585-bda4-4f37-8ff1-0af071d4373f (windows.81.amd64.open.rt) executed on machine a0018BJ running Windows-2012ServerR2-6.3.9600-SP0

C:\h\w\A22708CE\w\A5080926\e>taskkill.exe /f /im corerun.exe 
ERROR: The process "corerun.exe" not found.

C:\h\w\A22708CE\w\A5080926\e>call RunTests.cmd --runtime-path C:\h\w\A22708CE\p 
========================= Begin custom configuration settings ==============================
set __TestArchitecture=x64
set __IsXUnitLogCheckerSupported=1
========================== End custom configuration settings ===============================
----- start Wed 12/13/2023 18:40:04.47 ===============  To repro directly: =====================================================
pushd C:\h\w\A22708CE\w\A5080926\e\
"C:\h\w\A22708CE\p\dotnet.exe" exec --runtimeconfig ComInterfaceGenerator.Tests.runtimeconfig.json --depsfile ComInterfaceGenerator.Tests.deps.json xunit.console.dll ComInterfaceGenerator.Tests.dll -xml testResults.xml -nologo -nocolor -trait category=OuterLoop -notrait category=IgnoreForCI -notrait category=failing 
popd
===========================================================================================================

C:\h\w\A22708CE\w\A5080926\e>"C:\h\w\A22708CE\p\dotnet.exe" exec --runtimeconfig ComInterfaceGenerator.Tests.runtimeconfig.json --depsfile ComInterfaceGenerator.Tests.deps.json xunit.console.dll ComInterfaceGenerator.Tests.dll -xml testResults.xml -nologo -nocolor -trait category=OuterLoop -notrait category=IgnoreForCI -notrait category=failing  
  Discovering: ComInterfaceGenerator.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  ComInterfaceGenerator.Tests (found 0 of 45 test cases)
=== TEST EXECUTION SUMMARY ===
   ComInterfaceGenerator.Tests.dll  Total: 0
----- end Wed 12/13/2023 18:40:06.22 ----- exit code 0 ----------------------------------------------------------
----- start ===============  XUnitLogChecker Output =====================================================
XUnitLogChecker.dll does not exist in the expected location: C:\h\w\A22708CE\p\XUnitLogChecker.dll
----- end ===============  XUnitLogChecker Output - exit code 1 ===============
2023-12-13T18:40:06.788Z	INFO   	run.py	run(48)	main	Beginning reading of test results.
2023-12-13T18:40:06.788Z	INFO   	run.py	__init__(42)	read_results	Searching 'C:\h\w\A22708CE\w\A5080926\e' for test results files
2023-12-13T18:40:06.788Z	INFO   	run.py	__init__(48)	read_results	Found results file C:\h\w\A22708CE\w\A5080926\e\testResults.xml with format xunit
2023-12-13T18:40:06.788Z	INFO   	run.py	__init__(42)	read_results	Searching 'C:\h\w\A22708CE\w\A5080926\uploads' for test results files
2023-12-13T18:40:06.788Z	INFO   	run.py	packing_test_reporter(30)	report_results	Packing 0 test reports to 'C:\h\w\A22708CE\w\A5080926\e\__test_report.json'
2023-12-13T18:40:06.788Z	INFO   	run.py	packing_test_reporter(33)	report_results	Packed 1537 bytes
ERROR: The process "corerun.exe" not found.
gen-debug-dump-docs.py: Did not find dumps, skipping dump docs generation.
['ComInterfaceGenerator.Tests' END OF WORK ITEM LOG: Command exited with 1]

Author:	carlossanlop
Assignees:	-
Labels:	`area-Infrastructure-coreclr`, `blocking-clean-ci`, `Known Build Error`
Milestone:	-

ericstj · 2023-12-14T18:42:41Z

Might have to do with how the interop generator test payloads are generated? ComInterfaceGenerator.Tests might be different from other libraries tests.

rzikm · 2024-01-17T19:02:21Z

Has there been any progress on this? This is blocking Known Build Errors on outerloop pipelines because too many test failures are reported in AzDO and legit test failures end up not getting processed by the bot. See discussion on #96481 (comment) for more info.

ghost · 2024-01-17T19:44:13Z

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

Issue Details

Error Blob

{
  "ErrorMessage": "XUnitLogChecker.dll does not exist in the expected location:",
  "BuildRetry": false,
  "ErrorPattern": "",
  "ExcludeConsoleLog": false
}

Reproduction Steps

Queue: runtime-libraries-coreclr outerloop
Build example: https://dev.azure.com/dnceng-public/public/_build/results?buildId=498370&view=logs&j=631ed29b-f0dd-5b15-38a0-c6c555556dea&t=b9941189-b5a8-5df3-6b3a-0964fabb3acd
Log example: https://helixre107v0xdcypoyl9e7f.blob.core.windows.net/dotnet-runtime-refs-pull-95898-merge-d360b025a7c14161a1/ComInterfaceGenerator.Tests/1/console.66454013.log?helixlogtype=result
Output example:

Console log: 'ComInterfaceGenerator.Tests' from job d360b025-a7c1-4161-a1a4-19f920345fd3 workitem 57966585-bda4-4f37-8ff1-0af071d4373f (windows.81.amd64.open.rt) executed on machine a0018BJ running Windows-2012ServerR2-6.3.9600-SP0

C:\h\w\A22708CE\w\A5080926\e>taskkill.exe /f /im corerun.exe 
ERROR: The process "corerun.exe" not found.

C:\h\w\A22708CE\w\A5080926\e>call RunTests.cmd --runtime-path C:\h\w\A22708CE\p 
========================= Begin custom configuration settings ==============================
set __TestArchitecture=x64
set __IsXUnitLogCheckerSupported=1
========================== End custom configuration settings ===============================
----- start Wed 12/13/2023 18:40:04.47 ===============  To repro directly: =====================================================
pushd C:\h\w\A22708CE\w\A5080926\e\
"C:\h\w\A22708CE\p\dotnet.exe" exec --runtimeconfig ComInterfaceGenerator.Tests.runtimeconfig.json --depsfile ComInterfaceGenerator.Tests.deps.json xunit.console.dll ComInterfaceGenerator.Tests.dll -xml testResults.xml -nologo -nocolor -trait category=OuterLoop -notrait category=IgnoreForCI -notrait category=failing 
popd
===========================================================================================================

C:\h\w\A22708CE\w\A5080926\e>"C:\h\w\A22708CE\p\dotnet.exe" exec --runtimeconfig ComInterfaceGenerator.Tests.runtimeconfig.json --depsfile ComInterfaceGenerator.Tests.deps.json xunit.console.dll ComInterfaceGenerator.Tests.dll -xml testResults.xml -nologo -nocolor -trait category=OuterLoop -notrait category=IgnoreForCI -notrait category=failing  
  Discovering: ComInterfaceGenerator.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  ComInterfaceGenerator.Tests (found 0 of 45 test cases)
=== TEST EXECUTION SUMMARY ===
   ComInterfaceGenerator.Tests.dll  Total: 0
----- end Wed 12/13/2023 18:40:06.22 ----- exit code 0 ----------------------------------------------------------
----- start ===============  XUnitLogChecker Output =====================================================
XUnitLogChecker.dll does not exist in the expected location: C:\h\w\A22708CE\p\XUnitLogChecker.dll
----- end ===============  XUnitLogChecker Output - exit code 1 ===============
2023-12-13T18:40:06.788Z	INFO   	run.py	run(48)	main	Beginning reading of test results.
2023-12-13T18:40:06.788Z	INFO   	run.py	__init__(42)	read_results	Searching 'C:\h\w\A22708CE\w\A5080926\e' for test results files
2023-12-13T18:40:06.788Z	INFO   	run.py	__init__(48)	read_results	Found results file C:\h\w\A22708CE\w\A5080926\e\testResults.xml with format xunit
2023-12-13T18:40:06.788Z	INFO   	run.py	__init__(42)	read_results	Searching 'C:\h\w\A22708CE\w\A5080926\uploads' for test results files
2023-12-13T18:40:06.788Z	INFO   	run.py	packing_test_reporter(30)	report_results	Packing 0 test reports to 'C:\h\w\A22708CE\w\A5080926\e\__test_report.json'
2023-12-13T18:40:06.788Z	INFO   	run.py	packing_test_reporter(33)	report_results	Packed 1537 bytes
ERROR: The process "corerun.exe" not found.
gen-debug-dump-docs.py: Did not find dumps, skipping dump docs generation.
['ComInterfaceGenerator.Tests' END OF WORK ITEM LOG: Command exited with 1]

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=498370
Error message validated: XUnitLogChecker.dll does not exist in the expected location:
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 12/14/2023 6:12:51 PM UTC

Report

Build	Definition	Test	Pull Request
530217	dotnet/runtime	System.Diagnostics.FileVersionInfo.Tests.WorkItemExecution	#97099
530040	dotnet/runtime	System.DirectoryServices.AccountManagement.Tests.WorkItemExecution
530021	dotnet/runtime	System.Diagnostics.DiagnosticSource.Switches.Tests.WorkItemExecution
529841	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
528391	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
528387	dotnet/runtime	System.Composition.Hosting.Tests.WorkItemExecution
528222	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
527291	dotnet/runtime	System.ComponentModel.TypeConverter.Tests.WorkItemExecution
527295	dotnet/runtime	System.DirectoryServices.AccountManagement.Tests.WorkItemExecution
527109	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
526657	dotnet/runtime	System.ComponentModel.TypeConverter.Tests.WorkItemExecution
526641	dotnet/runtime	System.DirectoryServices.AccountManagement.Tests.WorkItemExecution
526606	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
526213	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
526190	dotnet/runtime	System.Composition.Hosting.Tests.WorkItemExecution
526138	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
524921	dotnet/runtime	System.Composition.AttributeModel.Tests.WorkItemExecution
524929	dotnet/runtime	System.DirectoryServices.AccountManagement.Tests.WorkItemExecution
524754	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
524010	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution	#96721
523473	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
523460	dotnet/runtime	System.Composition.AttributeModel.Tests.WorkItemExecution
523324	dotnet/runtime	System.Formats.Tar.Tests.WorkItemExecution
522887	dotnet/runtime	System.Diagnostics.Process.Tests.WorkItemExecution	#96792
522478	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution	#96721
521443	dotnet/runtime	System.Composition.Convention.Tests.WorkItemExecution
521438	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
521090	dotnet/runtime	System.Formats.Tar.Tests.WorkItemExecution
520677	dotnet/runtime	System.Diagnostics.Tracing.Tests.WorkItemExecution	#96721
519180	dotnet/runtime	System.Composition.AttributeModel.Tests.WorkItemExecution
519175	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
518995	dotnet/runtime	System.Formats.Tar.Tests.WorkItemExecution
517695	dotnet/runtime	System.Composition.Convention.Tests.WorkItemExecution
517699	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
517525	dotnet/runtime	System.Formats.Tar.Tests.WorkItemExecution
517190	dotnet/runtime	System.Composition.Hosting.Tests.WorkItemExecution
517179	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
517148	dotnet/runtime	System.Formats.Tar.Tests.WorkItemExecution
516787	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
516780	dotnet/runtime	System.Composition.AttributeModel.Tests.WorkItemExecution
516736	dotnet/runtime	System.Formats.Tar.Tests.WorkItemExecution
516249	dotnet/runtime	System.Diagnostics.PerformanceCounter.Tests.WorkItemExecution	#95001
516019	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
516009	dotnet/runtime	System.Composition.Convention.Tests.WorkItemExecution
515863	dotnet/runtime	System.Formats.Tar.Tests.WorkItemExecution
514579	dotnet/runtime	System.Composition.Hosting.Tests.WorkItemExecution
514560	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
514443	dotnet/runtime	System.Formats.Tar.Manual.Tests.WorkItemExecution
513239	dotnet/runtime	System.Diagnostics.Contracts.Tests.WorkItemExecution	#95001
513205	dotnet/runtime	System.Composition.Convention.Tests.WorkItemExecution
513214	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
513048	dotnet/runtime	System.Formats.Tar.Tests.WorkItemExecution
512932	dotnet/runtime	System.Formats.Tar.Tests.WorkItemExecution	#96431
512403	dotnet/runtime	System.Formats.Tar.Tests.WorkItemExecution	#96412
512706	dotnet/runtime	System.Diagnostics.TraceSource.Tests.WorkItemExecution	#96423
512100	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution	#95001
512029	dotnet/runtime	System.DirectoryServices.AccountManagement.Tests.WorkItemExecution
512014	dotnet/runtime	Microsoft.Win32.Primitives.Tests.WorkItemExecution
511981	dotnet/runtime	System.Console.Tests.WorkItemExecution	#95001
511926	dotnet/runtime	System.Diagnostics.TraceSource.Tests.WorkItemExecution	#96393
511888	dotnet/runtime	System.Diagnostics.DiagnosticSource.Tests.WorkItemExecution
511520	dotnet/runtime	Microsoft.Extensions.Hosting.WindowsServices.Tests.WorkItemExecution
511527	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
511483	dotnet/runtime	System.Diagnostics.DiagnosticSource.Tests.WorkItemExecution
511416	dotnet/runtime	System.Formats.Tar.Tests.WorkItemExecution	#96358
511312	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
511297	dotnet/runtime	Microsoft.Extensions.Primitives.Tests.WorkItemExecution
511270	dotnet/runtime	System.Formats.Tar.Tests.WorkItemExecution
510974	dotnet/runtime	Microsoft.Extensions.Logging.Generators.Roslyn3.11.Tests.WorkItemExecution
510958	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
510908	dotnet/runtime	System.Formats.Tar.Tests.WorkItemExecution
510627	dotnet/runtime	System.Diagnostics.DiagnosticSource.Tests.WorkItemExecution	#96358
510448	dotnet/runtime	System.DirectoryServices.Protocols.Tests.WorkItemExecution
510441	dotnet/runtime	System.Composition.Hosting.Tests.WorkItemExecution
510357	dotnet/runtime	System.Formats.Tar.Tests.WorkItemExecution
509967	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
509315	dotnet/runtime	System.Formats.Tar.Tests.WorkItemExecution
509703	dotnet/runtime	System.Formats.Tar.Tests.WorkItemExecution	#96333
509393	dotnet/runtime	System.Diagnostics.Contracts.Tests.WorkItemExecution
509402	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
508991	dotnet/runtime	System.Configuration.ConfigurationManager.Tests.WorkItemExecution
508924	dotnet/runtime	System.Formats.Tar.Tests.WorkItemExecution
508528	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
507929	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
507376	dotnet/runtime	System.Diagnostics.DiagnosticSource.Tests.WorkItemExecution
505319	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
505311	dotnet/runtime	System.Diagnostics.PerformanceCounter.Tests.WorkItemExecution
505179	dotnet/runtime	System.Formats.Tar.Tests.WorkItemExecution
504213	dotnet/runtime	System.DirectoryServices.Tests.WorkItemExecution
504210	dotnet/runtime	System.Diagnostics.StackTrace.Tests.WorkItemExecution
503995	dotnet/runtime	System.Formats.Tar.Tests.WorkItemExecution
503885	dotnet/runtime	System.Diagnostics.StackTrace.Tests.WorkItemExecution	#96157

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
4	25	92

Author:	carlossanlop
Assignees:	-
Labels:	`area-Infrastructure-libraries`, `blocking-clean-ci`, `untriaged`, `Known Build Error`
Milestone:	-

carlossanlop · 2024-01-18T03:35:36Z

I think I know what's going on...

The regular libraries runs build the product (including XUnitLogChecker) in one job, push it to the feed, and then later it is downloaded and consumed by the job that runs the tests.

In outerloop, we build the product in the same job, in a previous step before the test running step. We aren't building XUnitLogChecker in that build step.

I'm now trying to figure out where and how to add XUnitLogChecker so it builds in outerloop jobs too.

ericstj · 2024-02-13T20:34:45Z

Looks like XunitLogChecker is normally build by libs\tests.proj. This is built in both the regular build and in the outerloop build, however during the outerloop build it seems that the condition to inlcude the checker project is false.

We looked into why this is happening - it's because BuildTargetFramework is not set when the outerloop tests build -- which is happening because the -framework swith is not passed to the build script.

So this is false:

runtime/src/libraries/tests.proj

Lines 759 to 763 in b8e1296

    
           <ItemGroup Condition="'$(ArchiveTests)' == 'true' and '$(BuildTargetFramework)' == '$(NetCoreAppCurrent)' and '$(IsXUnitLogCheckerSupported)' == 'true'"> 
        
             <ProjectReference 
        
               Include="$(RepoRoot)src\tests\Common\XUnitLogChecker\XUnitLogChecker.csproj" 
        
               AdditionalProperties="%(AdditionalProperties);Configuration=Release;OutDir=$(XUnitLogCheckerLibrariesOutDir)" /> 
        
           </ItemGroup>

In outerloop no -framework passed:

runtime/eng/pipelines/libraries/outerloop.yml

Line 44 in b8e1296

    
           buildArgs: -s clr+host.native+libs+libs.tests -lc $(_BuildConfig) -hc $(_BuildConfig) -rc Release -testscope outerloop /p:ArchiveTests=true

But in the regular libraries build it will be:

runtime/eng/pipelines/libraries/base-job.yml

Line 74 in b8e1296

- _finalFrameworkArg: -framework ${{ parameters.framework }}

To fix this we should see if we can just update the condition for including the log checker CSProj to not depend on BuildTargetFramework. Maybe TargetFrameworkIdentifier is sufficient (if we were trying to filter to netcore only).

carlossanlop added area-Infrastructure-coreclr blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' Known Build Error Use this to report build issues in the .NET Helix tab labels Dec 14, 2023

ghost added the untriaged New issue has not been triaged by the area owner label Dec 14, 2023

carlossanlop mentioned this issue Dec 14, 2023

Fix calculation of channel bindings hash in managed NTLM implementation #95898

Merged

ivdiazsa mentioned this issue Dec 14, 2023

Add new Pipeline for Running Libs Tests with TestReadyToRun #91229

Merged

JulieLeeMSFT added area-Infrastructure-libraries and removed area-Infrastructure-coreclr labels Jan 17, 2024

JulieLeeMSFT assigned carlossanlop Jan 17, 2024

JulieLeeMSFT added this to the 9.0.0 milestone Jan 17, 2024

ghost removed the untriaged New issue has not been triaged by the area owner label Jan 17, 2024

carlossanlop mentioned this issue Jan 18, 2024

Ensuring XUnitLogChecker is included in libraries outerloop #97135

Merged

ghost added the in-pr There is an active PR which will close this issue when it is merged label Jan 18, 2024

jkoritzinsky mentioned this issue Jan 29, 2024

Enable Windows runs of AddressSanitizer #95758

Merged

rzikm mentioned this issue Jan 30, 2024

[TestFailure] System.Net.Security.Tests.CertificateValidationRemoteServer.DefaultConnect_EndToEnd_Ok #96689

Closed

antonfirsov mentioned this issue Feb 13, 2024

Send connection WINDOW_UPDATE before RTT PING #97881

Merged

carlossanlop closed this as completed in #97135 Feb 27, 2024

ghost removed the in-pr There is an active PR which will close this issue when it is merged label Feb 27, 2024

carlossanlop mentioned this issue Feb 27, 2024

[outerloop] RegexKnownPatternTests.RecreationalRegex_Rectangle either fails with Specified argument was out of the range of valid values. or times out #98962

Closed

github-actions bot locked and limited conversation to collaborators Mar 28, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

XUnitLogChecker: `runtime-libraries-coreclr outerloop` tests are unable to find the tool DLL #96035

XUnitLogChecker: `runtime-libraries-coreclr outerloop` tests are unable to find the tool DLL #96035

carlossanlop commented Dec 14, 2023 •

edited by build-analysis bot

Loading

ghost commented Dec 14, 2023

Error Blob

Reproduction Steps

ericstj commented Dec 14, 2023

rzikm commented Jan 17, 2024

ghost commented Jan 17, 2024

Error Blob

Reproduction Steps

Known issue validation

Report

Summary

carlossanlop commented Jan 18, 2024

ericstj commented Feb 13, 2024

XUnitLogChecker: runtime-libraries-coreclr outerloop tests are unable to find the tool DLL #96035

XUnitLogChecker: runtime-libraries-coreclr outerloop tests are unable to find the tool DLL #96035

Comments

carlossanlop commented Dec 14, 2023 • edited by build-analysis bot Loading

Error Blob

Reproduction Steps

Known issue validation

Report

Summary

ghost commented Dec 14, 2023

Error Blob

Reproduction Steps

ericstj commented Dec 14, 2023

rzikm commented Jan 17, 2024

ghost commented Jan 17, 2024

Error Blob

Reproduction Steps

Known issue validation

Report

Summary

carlossanlop commented Jan 18, 2024

ericstj commented Feb 13, 2024

XUnitLogChecker: `runtime-libraries-coreclr outerloop` tests are unable to find the tool DLL #96035

XUnitLogChecker: `runtime-libraries-coreclr outerloop` tests are unable to find the tool DLL #96035

carlossanlop commented Dec 14, 2023 •

edited by build-analysis bot

Loading