Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hardware intrinsics tests in new style #74886

Merged

Conversation

davidwrighton
Copy link
Member

@davidwrighton davidwrighton commented Aug 31, 2022

Port all of the vector tests to the new xunit style specification

  • Test disables need to use the attribute based form
  • All tests are run on all architectures (This wasn't happening in the old system, but hopefully with test striping the need for this has gone away.)
  • The xunit wrapper generator generated methods which are too big for the merged assembly as designed earlier. I've broken that up into chunks of 100 methods each which seems to have solved that problem.
  • The xunit wrapper generator and runner have issues with running this many tests under GCStress, I've implemented a striping scheme to allow these tests to run across more machines when running under GCStress.

- Except for General\HwiOp tests
- Also I've found that the Vector128<Single>.Dot test is failing due to floating point precision issues. Adding a fuzzy comparison seems to have fixed that, but I don't know why it isn't failing more often today.
- And consolidate them so that they all compile with a single project file
… that we don't create an excessively large single method
@ghost ghost assigned davidwrighton Aug 31, 2022
@build-analysis build-analysis bot mentioned this pull request Nov 4, 2022
2 tasks
- Fix the ActiveIssue with a conditional member implementation in the test source generator
- Tweak down the number of methods per test gen method again. There's still a LOT of them
davidwrighton added a commit to davidwrighton/runtime that referenced this pull request Nov 4, 2022
These fixes were built for PR dotnet#74886; however, as that PR is so utterly large an unreviewable, I've pulled out the test infra changes for separate review

Changes
- Increase the number of trampolines in the llvm aot compilation process to 20,000 from 10,000 (This avoids running out of them in some of the hardware intrinsics tests
- Add a concept of striping tests when running under GC stress
  - To use this new feature, specify <NumberOfStripesToUseInStress>N</NumberOfStripesToUseInStress> within the merged test assembly. If this value is set, then the tests within that merged test assembly will be run across N different work items instead of 1 when running under any form of GC stress based scenario. At this moment the largest supported value of N is 99
- Emit the testresults.xml file as a file which is exported from the tests. This is useful for debugging testresult.xml parsing failures
- Fix the testresults summary generator to never emit an empty CDATA string. If one is present the parser may fail the parse.
- In the XUnitWrapperGenerator fix the implementation of the Outerloop and ActiveIssue when used with a conditional member.
- Add PlatformDetection.IsMonoLLVMAOT, PlatformDetection.IsMonoLLVMFULLAOT, and PlatformDetection.IsMonoInterpreter boolean properties to the PlatformDetection type for use with the ActiveIssue attribute
@davidwrighton
Copy link
Member Author

/azp list

@azure-pipelines
Copy link

CI/CD Pipelines for this repository:

@davidwrighton
Copy link
Member Author

/azp run runtime-androidemulator

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

davidwrighton added a commit that referenced this pull request Nov 8, 2022
These fixes were built for PR #74886; however, as that PR is so utterly large and unreviewable, I've pulled out the test infra changes for separate review

Changes
- Increase the number of trampolines in the llvm aot compilation process to 20,000 from 10,000 (This avoids running out of them in some of the hardware intrinsics tests
- Add a concept of striping tests when running under GC stress
  - To use this new feature, specify <NumberOfStripesToUseInStress>N</NumberOfStripesToUseInStress> within the merged test assembly. If this value is set, then the tests within that merged test assembly will be run across N different work items instead of 1 when running under any form of GC stress based scenario. At this moment the largest supported value of N is 99
- Emit the testresults.xml file as a file which is exported from the tests. This is useful for debugging testresult.xml parsing failures
- Fix the testresults summary generator to never emit an empty CDATA string. If one is present the parser may fail the parse.
- In the XUnitWrapperGenerator fix the implementation of the Outerloop and ActiveIssue when used with a conditional member.
- Add PlatformDetection.IsMonoLLVMAOT, PlatformDetection.IsMonoLLVMFULLAOT, and PlatformDetection.IsMonoInterpreter boolean properties to the PlatformDetection type for use with the ActiveIssue attribute
- Add some documentation about project files for coreclr tests, as well as some documentation on the command line parameters for merged test runner assemblies

namespace JIT.HardwareIntrinsics.Arm
namespace JIT.HardwareIntrinsics.Arm._AdvSimd.Arm64
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just wondering, why the need for the underscore here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its been quite a while, but I think there was a conflict between a namespace name and class name, and I stuffed in an underscore on the namespace to shut it all up.

Copy link
Member

@tannergooding tannergooding left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes LGTM. Fairly mechanical in nature

@davidwrighton
Copy link
Member Author

/azp run runtime-coreclr outerloop

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@davidwrighton davidwrighton merged commit 2095119 into dotnet:main Nov 11, 2022
@ghost ghost locked as resolved and limited conversation to collaborators Dec 12, 2022
@davidwrighton davidwrighton deleted the HardwareIntrinsicsTestsInNewStyle branch April 13, 2023 18:54
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants