Updating the JIT to support marshaling blittable generics.

[tannergooding]

This resolves https://github.com/dotnet/coreclr/issues/1685.
This is a port of dotnet/coreclr#23899.

CC. @jkoritzinsky, @AaronRobinsonMSFT, @jkotas

[tannergooding]

I think the only open question I had was on how to properly test crossgen2 against the appropriate tests: dotnet/coreclr#23899 (comment)

[ViktorHofer]

cc @trylek for the crossgen2 question.

[trylek]

@tannergooding - right now the only CI testing available for Crossgen2 is a single pipeline I created recently, "runtime crossgen2" which runs checked Pri0 tests on the three OS'es / x64. Sadly the results aren't yet completely clean, there are typically something like 60~70 remaining failures. For local testing, it's easiest to use the tool ReadyToRun.SuperIlc - ping me or Jan Vorlicek on IM if you're interested in guidance how to do that.

[tannergooding]

I was able to locally validate that the Crossgen2 changes work as expected.

I believe this should be ready for review/merging (provided CI is stable enough for it to pass now).

[jkotas]

Is there test coverage for the unmanaged -> managed direction (delegates)? I am not able to find it, but it may be just a github search not working.

[tannergooding]

No, I hadn't written any tests for the reverse case. I'll try to get a few written tomorrow.

[tannergooding]

@jkotas, the changes for the reverse case are a bit more complex and require updating the Marshal APIs. This is currently tracked by https://github.com/dotnet/coreclr/issues/25973 and https://github.com/dotnet/coreclr/issues/27754.

I've started updating the relevant checks, but was wondering if you wanted it as part of this PR or if it would be preferred to have a separate PR handling them?

[jkotas]

Yes, it would be better to have those as separate change.

What does the current version of the change does in these cases? Does it fail predictably?

[tannergooding]

What does the current version of the change does in these cases? Does it fail predictably?

Yes. These paths all throw an ArgumentException which states the type must be non-generic today:

• Marshal.GetDelegateForFunctionPointer: https://source.dot.net/#System.Private.CoreLib/shared/System/Runtime/InteropServices/Marshal.cs,873
• Marshal.GetFunctionPointerForDelegate: https://github.com/dotnet/runtime/blob/master/src/coreclr/src/vm/comdelegate.cpp#L1214
• Marshal.PtrToStructure: https://source.dot.net/#System.Private.CoreLib/shared/System/Runtime/InteropServices/Marshal.cs,556

[jkotas]

What about cases like delegate void MyDelegate(Vector256<double> x); + GetFunctionPointerForDelegate ?

[tannergooding]

What about cases like delegate void MyDelegate(Vector256 x); + GetFunctionPointerForDelegate ?

That looks to work today, even without these changes (as does the reverse).

[jkotas]

It does not work in master today. You will get an exception like:

Unhandled exception. System.Runtime.InteropServices.MarshalDirectiveException: Cannot marshal 'parameter #1': Generic types cannot be marshaled.

Note that you need to actually call the delegate. The marshaling stub compilation is done lazily.

It should continue to throw exception, same as the forward direction.

[tannergooding]

The marshaling stub compilation is done lazily.

Ah, I see. I had only validated that the calls succeeded. I can reproduce the above failure without these changes.

With these changes, the marshaling stub works for blittable generics and fails with the following for blocked types (such as Vector128<float>):

Unhandled exception. System.Runtime.InteropServices.MarshalDirectiveException: Cannot marshal 'parameter #1': Non-blittable generic types cannot be marshaled.