Attributes and Analyzer Diagnostics to support custom struct marshalling in the DllImport Source Generator

[jkoritzinsky]

Background and Motivation

As part of the DllImport Source Generator proposal, we need a mechanism to allow users to enable marshalling their struct types from managed code to native code. This issue proposes a design along with an extension we have been using in the DllImportGenerator that is flexible enough to enable us to create a struct interop source generator if we so desire. This design uses the following attributes as well as a convention-based model for the emitted code as described in the linked design doc. The DllImportGenerator will consume the proposed attributes in combination with the DisableRuntimeMarshallingAttribute for its definition of 'does not require marshalling' to allow marshalling of user-defined types.

Proposed API

Related proposal: #46822

namespace System.Runtime.InteropServices
{
    /// <summary>
    /// Indicates the default marshalling when the attributed type is used in source-generated interop.
    /// </summary>
    [AttributeUsage(AttributeTargets.Struct | AttributeTargets.Class)]
    public sealed class NativeMarshallingAttribute : Attribute
    {
        /// <summary>
        /// Initializes a new instance of the <see cref="NativeMarshallingAttribute"/>
        /// </summary>
        /// <param name="nativeType">Type that should be used when marshalling the attributed type. The type must not require marshalling.</param>
        public NativeMarshallingAttribute(Type nativeType) {}
    }

    /// <summary>
    /// Indicates the marshalling to use in source-generated interop.
    /// </summary>
    [AttributeUsage(AttributeTargets.Parameter | AttributeTargets.ReturnValue | AttributeTargets.Field, AllowMultiple = true)]
    public sealed class MarshalUsingAttribute : Attribute
    {
        /// <summary>
        /// Initializes a new instance of the <see cref="MarshalUsingAttribute"/>
        /// </summary>
        public MarshalUsingAttribute() {}

        /// <summary>
        /// Initializes a new instance of the <see cref="MarshalUsingAttribute"/>
        /// </summary>
        /// <param name="nativeType">Type that should be used when marshalling the attributed parameter or field. The type must not require marshalling.</param>
        public MarshalUsingAttribute(Type nativeType) {}

        /// <summary>
        /// Type that should be used when marshalling the attributed parameter, field or, when ElementIndirectionLevel is specified, collection element on the attributed item.
        /// </summary>
        public Type? NativeType { get; }

        /// <summary>
        /// Name of the parameter that contains the number of native collection elements. A value of <see cref="ReturnsCountValue"/> indicates the return value of the p/invoke should be used.
        /// </summary>
        public string CountElementName { get; set; }

        /// <summary>
        /// Number of elements in the native collection.
        /// </summary>
        public int ConstantElementCount { get; set; }

        /// <summary>
        /// The level of indirection in a collection to which this attribute should apply.
        /// </summary>
        public int ElementIndirectionLevel { get; set; }

        /// <summary>
        /// When returned by <see cref="CountElementName" />, indicates the return value of the p/invoke should be used as the count.
        /// </summary>
        public const string ReturnsCountValue = "return-value";
    }

    /// <summary>
    /// Indicates this type is a marshaller type to be used in source-generated interop.
    /// </summary>
    [AttributeUsage(AttributeTargets.Struct)]
    public sealed class CustomTypeMarshallerAttribute : Attribute
    {
        public CustomTypeMarshallerAttribute(Type managedType, CustomTypeMarshallerKind marshallerKind = CustomTypeMarshallerKind.Value)
        {
            ManagedType = managedType;
            MarshallerKind = marshallerKind;
        }

        /// <summary>
       /// The managed type that the attributed type is a marshaller for.
       /// </summary>
        public Type ManagedType { get; }

        /// <summary>
       /// The expected shape of this marshaller type
       /// </summary>
        public CustomTypeMarshallerKind MarshallerKind { get; }

        /// <summary>
        /// The size of the caller-allocated buffer in scenarios where using caller-allocated buffer is support.
        /// </summary>
        public int BufferSize { get; set; }

        /// <summary>
       /// Whether or not the caller-allocated buffer must be stack-allocated.
       /// </summary>
        public bool RequiresStackBuffer { get; set; }

        /// <summary>
        /// This type is used as a placeholder for the first generic parameter when generic parameters cannot be used
        /// to identify the managed type (i.e. when the marshaller type is generic over T and the managed type is T[])
        /// </summary>
        public struct GenericPlaceholder
        {
        }
    }


     /// <summary>
    /// The kind (or shape) of the custom marshaller type.
    /// </summary>
    public enum CustomTypeMarshallerKind
    {
         /// <summary>
        /// This marshaller represents marshalling a single value.
        /// </summary>
        Value,
        /// <summary>
        /// This marshaller represents marshalling a Span-like collection of values
        /// </summary>
        SpanCollection
    }
}

The types used with MarshalUsing/NativeMarshalling or attributed with GenericContiguousCallectionMarshaller are expected to conform to a defined shape. Analyzers will be provided to validate their usage.

Naming:

As noted in #46838 (comment), these are three different attributes with three different forms of 'marshal'. We may want to rework the names to all use the same form of 'marshal'.

Usage Examples

NativeMarshallingAttribute

The [NativeMarshalling] attribute is applied on a struct and points to a struct that does not require marshalling or a struct with a Value property that does not required marshalling (as per the StructMarshalling design doc). This attribute indicates the default marshalling for the type when passed to a method that uses source-generated interop such as the DllImportGenerator source generator.

[NativeMarshalling(typeof(StringContainerNative))]
public struct StringContainer
{
    public string str1;
    public string str2;
}

[CustomTypeMarshaller(typeof(StringContainer))]
public struct StringContainerNative
{
    public IntPtr str1;
    public IntPtr str2;
    public StringContainerNative(StringContainer managed)
    {
        str1 = Marshal.StringToCoTaskMemUTF8(managed.str1);
        str2 = Marshal.StringToCoTaskMemUTF8(managed.str2);
    }
    public StringContainer ToManaged()
    {
        return new StringContainer
        {
            str1 = Marshal.PtrToStringUTF8(str1),
            str2 = Marshal.PtrToStringUTF8(str2)
        };
    }
    public void FreeNative()
    {
        Marshal.FreeCoTaskMem(str1);
        Marshal.FreeCoTaskMem(str2);
    }
}

MarshalUsingAttribute

The [MarshalUsing] attribute on a parameter, return type, or field indicates the marshaller type to use for the parameter, return value, or field in this particular case. This choice overrides any default marshalling rules, including the [NativeMarshalling] attribute.

[GeneratedDllImport("NativeLib", EntryPoint = "get_long_bytes_as_double")]
public static partial double GetLongBytesAsDouble([MarshalUsing(typeof(DoubleToLongMarshaller))] double d);

[CustomTypeMarshaller(typeof(double))]
public struct DoubleToLongMarshaller
{
    public long l;
    public DoubleToLongMarshaller(double d)
    {
        l = MemoryMarshal.Cast<double, long>(MemoryMarshal.CreateSpan(ref d, 1))[0];
    }
    public double ToManaged() => MemoryMarshal.Cast<long, double>(MemoryMarshal.CreateSpan(ref l, 1))[0];
    public long Value
    {
        get => l;
        set => l = value;
    }
}

Example of code that could be generated:

[System.Runtime.CompilerServices.SkipLocalsInitAttribute]
public static partial double GetLongBytesAsDouble(double d)
{
    long __d_gen_native;
    double __retVal;
    //
    // Setup
    //
    global::SharedTypes.DoubleToLongMarshaller __d_gen_native__marshaller = default;
    //
    // Marshal
    //
    __d_gen_native__marshaller = new(d);
    __d_gen_native = __d_gen_native__marshaller.Value;
    //
    // Invoke
    // 
    __retVal = __PInvoke__(__d_gen_native);
    return __retVal;
    //
    // Local P/Invoke
    //
    [System.Runtime.InteropServices.DllImportAttribute("NativeLib", EntryPoint = "get_long_bytes_as_double")]
    extern static unsafe double __PInvoke__(long d);
}

The CountElementName and ConstantElementCount properties can be used to simply specify collection count information without specifying a native type.

[GeneratedDllImport("NativeLib")]
[return: MarshalUsing(ConstantElementCount = 10)]
public static partial byte[] Method(
    [MarshalUsing(CountElementName = "outSize")] out byte[] outBuffer,
    out int outSize);

SpanCollection shape

The [MarshalUsing] or [NativeMarshalling] attribute can be used to point to a type marked [CustomTypeMarshaller(typeof(...), CustomTypeMarshallerKind.SpanCollection)] to support custom marshalling of contiguous collections.

[GeneratedDllImport("NativeLib")]
public static partial void Method([MarshalUsing(typeof(ListMarshaller<int>))] List<int> values);

[CustomTypeMarshaller(typeof(List<>), CustomTypeMarshallerKind.SpanCollection)]
public unsafe ref struct ListMarshaller<T>
{
    private List<T> managedList;
    private readonly int sizeOfNativeElement;
    private IntPtr allocatedMemory;

    public ListMarshaller(int sizeOfNativeElement)
        : this(null, sizeOfNativeElement)
    { }

    public ListMarshaller(List<T> managed, int sizeOfNativeElement)
    {
        allocatedMemory = default;
        this.sizeOfNativeElement = sizeOfNativeElement;
        if (managed is null)
        {
            managedList = null;
            NativeValueStorage = default;
            return;
        }

        managedList = managed;
        int spaceToAllocate = Math.Max(managed.Count * sizeOfNativeElement, 1);
        allocatedMemory = Marshal.AllocCoTaskMem(spaceToAllocate);
        NativeValueStorage = new Span<byte>((void*)allocatedMemory, spaceToAllocate);
    }

    public Span<T> ManagedValues => CollectionsMarshal.AsSpan(managedList);

    public Span<byte> NativeValueStorage { get; private set; }

    public void SetUnmarshalledCollectionLength(int length)
    {
        managedList = new List<T>(length);
        for (int i = 0; i < length; i++)
        {
            managedList.Add(default);
        }
    }

    public ref byte GetPinnableReference() => ref NativeValueStorage.GetPinnableReference();
    public byte* Value
    {
        get => (byte*)Unsafe.AsPointer(ref GetPinnableReference());
        set
        {
            if (value is null)
            {
                managedList = null;
                NativeValueStorage = default;
            }
            else
            {
                allocatedMemory = (IntPtr)value;
                NativeValueStorage = new Span<byte>(value, (managedList?.Count ?? 0) * sizeOfNativeElement);
            }
        }
    }

    public void FreeNative() => Marshal.FreeCoTaskMem(allocatedMemory);
}

Analyzer Diagnostics

An analyzer will be provided to validate the usage of these APIs as per the design are defined in https://github.com/dotnet/runtime/blob/29cc2a2c23ce15469575a0b25c8b8b453a1d975f/src/libraries/System.Runtime.InteropServices/gen/DllImportGenerator/Analyzers/ManualTypeMarshallingAnalyzer.cs

These will focus on ensuring the types specified in MarshalUsingAttribute or NativeMarshallingAttribute and attributed with CustomTypeMarshallerAttribute conform to the expected shapes for use with p/invoke source generator. The generator will also error if the types do not conform to the expected shapes:

// Shape for Value kind
[CustomTypeMarshaller(typeof(ManagedType))]
public struct Marshaller
{
    // Support for marshalling from managed to native.
    // Required if ToManaged() is not defined.
    // May be omitted if marshalling from managed to native is not required.
    public Marshaller(ManagedType managed) {}

    // Optional.
    // Support for marshalling from managed to native using a caller-allocated buffer
    // Requires the BufferSize field to be set on the CustomTypeMarshaller attribute.
   public Marshaller(ManagedType managed, Span<byte> buffer) {}

    // Support for marshalling from native to managed.
    // Required if constructor taking TManaged is not defined.
    // May be omitted if marshalling from native to managed is not required.
    public ManagedType ToManaged() {}

    // Optional.
    // If defined, Value will be passed to native instead of the Marshaller itself
    // If setter is defined, Value will be set when marshalling from native to managed.
    // NativeType must not require marshalling
    public NativeType Value { get; set; }

    // Optional.
    // If defined and Value is defined, will be called before Value property getter and its return value will be pinned
    public ref NativeType GetPinnableReference() {}

    // Optional.
    // Release native resources.
    public void FreeNative() {}
}

// Shape for SpanCollection kind
[CustomTypeMarshaller(typeof(GenericCollection<>), CustomTypeMarshallerKind.SpanCollection)]
public struct GenericContiguousCollectionMarshallerImpl<T>
{
    // Support for marshalling from native to managed.
    // May be omitted if marshalling from native to managed is not required.
    public GenericContiguousCollectionMarshallerImpl(int nativeSizeOfElement);

    // Support for marshalling from managed to native.
    // May be omitted if marshalling from managed to native is not required.
    public GenericContiguousCollectionMarshallerImpl(GenericCollection<T> collection, int nativeSizeOfElement);

    // Optional.
    // Support for marshalling from managed to native using a caller-allocated buffer
    // Requires the BufferSize field to be set on the CustomTypeMarshaller attribute.
    public GenericContiguousCollectionMarshallerImpl(GenericCollection<T> collection, Span<byte> buffer, int nativeSizeOfElement);

    // Required.
    // Points to the memory where the managed values of the collection are stored (in the marshalling case) or should be stored (in the unmarshalling case).
    public Span<CollectionElement> ManagedValues { get; }

    // Set the expected length of the managed collection based on the parameter/return value/field marshalling information.
    // Required for unmarshalling (native to managed) support.
    public void SetUnmarshalledCollectionLength(int length);

    // Required.
    // Points to the memory where the native values of the collection should be stored.
    public unsafe Span<byte> NativeValueStorage { get; }

    // Required.
    // NativeType must not require marshalling
    public NativeType Value { get; set; }
}

Risks

This proposal relies on a definition of 'does not require marshalling'. The DllImportGenerator will require DisableRuntimeMarshallingAttribute (assembly-level) for any user-defined types in p/invokes and emit an error if the attribute is not present. This also means that any usage of the DllImportGenerator will require switching over the entire assembly (rather than individual p/invokes) to use the proposed MarshalUsing or NativeMarshalling attributes.

The successful usage of these attributes also rely on the user defining types that conform to a certain shape, which can be a bit complicated - especially in the SpanCollection case. The UX around writing those types might not be great, as walidation is only provided through analyzer diagnostics and generator errors.

Fixes

Fixes #8719

[jkoritzinsky]

/cc @stephentoub @AaronRobinsonMSFT @elinor-fung @jkotas @tannergooding

[AaronRobinsonMSFT]

@jkoritzinsky We should have a link to how we define blittable here. The official definition has the following section we should confirm we want to continue:

The following complex types are also blittable types

Is supporting both value types and classes what we want in this model? Do we have a justification to support marking classes blittable? Experience tells me that can be very confusing at times and implies things that perhaps shouldn't be permitted any longer. I believe @davidwrighton also had an opinion about this.

[jkoritzinsky]

The definition I'm using for blittable types is the updated definition in the Interop Best Practices documentation: https://docs.microsoft.com/en-us/dotnet/standard/native-interop/best-practices#blittable-types

This definition only includes structs. I've updated the proposed API to only allow the BlittableType attribute on structs.

[jkoritzinsky]

API Review for this will be in the 7.0.0 time frame.

[RussKie]

It feels that we're trying to use "marshal" in as many different combinations of words as possible. We have "native marshalling", "marshal using", "marshal", "marshaller", etc. I feel that for me as a developer this would be a constant source of struggle and frustration to remember which form of word "marshal" I need to use.
It would be great if we could review and try to minimise the required vocabulary.

[bartonjs]

We started discussing this, and ran out of time.

The main points:

• Some of the behavior that's described in the shape of a Marshaller type feels like it could/should be described on an attribute applied to the Marshaller (attribute TBD).
• The collection marshaller and the item marshaller seem to have gotten out of sync on some of the names (when they have the same concept), and they should align one way or the other.

[jkoritzinsky]

Design has been updated to use an attribute instead of constants for some of the information and names have been cleaned up in the proposal in the shape definitions. Marking as api-ready-for-review again

[bartonjs]

Video

• We changed all of the properties in the "shapes" to be methods, and made up names that made sense with that conversion.
• ElementIndirectionLevel => ElementIndirectionDepth for clarity on if 0 is the collection type (yes) or the element type (no).
• We dropped RequiresStackBuffer
• We renamed CustomTypeMarshallerKind.SpanCollection to CustomTypeMarshallerKind.LinearCollection to avoid a misunderstanding of "a collection of spans"
• We discussed adding a CustomTypeMarshallerDirection/CustomTypeMarshallerFeatures flags enum to help be declarative on what sub-shape is desired

namespace System.Runtime.InteropServices
{
    /// <summary>
    /// Indicates the default marshalling when the attributed type is used in source-generated interop.
    /// </summary>
    [AttributeUsage(AttributeTargets.Struct | AttributeTargets.Class | AttributeTargets.Type | AttributeTargets.Delegate)]
    public sealed class NativeMarshallingAttribute : Attribute
    {
        /// <summary>
        /// Initializes a new instance of the <see cref="NativeMarshallingAttribute"/>
        /// </summary>
        /// <param name="nativeType">Type that should be used when marshalling the attributed type. The type must not require marshalling.</param>
        public NativeMarshallingAttribute(Type nativeType) {}
    }


    /// <summary>
    /// Indicates the marshalling to use in source-generated interop.
    /// </summary>
    [AttributeUsage(AttributeTargets.Parameter | AttributeTargets.ReturnValue, AllowMultiple = true)]
    public sealed class MarshalUsingAttribute : Attribute
    {
        /// <summary>
        /// Initializes a new instance of the <see cref="MarshalUsingAttribute"/>
        /// </summary>
        public MarshalUsingAttribute() {}

        /// <summary>
        /// Initializes a new instance of the <see cref="MarshalUsingAttribute"/>
        /// </summary>
        /// <param name="nativeType">Type that should be used when marshalling the attributed parameter or field. The type must not require marshalling.</param>
        public MarshalUsingAttribute(Type nativeType) {}

        /// <summary>
        /// Type that should be used when marshalling the attributed parameter, field or, when ElementIndirectionLevel is specified, collection element on the attributed item.
        /// </summary>
        public Type? NativeType { get; }

        /// <summary>
        /// Name of the parameter that contains the number of native collection elements. A value of <see cref="ReturnsCountValue"/> indicates the return value of the p/invoke should be used.
        /// </summary>
        public string CountElementName { get; set; }

        /// <summary>
        /// Number of elements in the native collection.
        /// </summary>
        public int ConstantElementCount { get; set; }

        /// <summary>
        /// The degree of indirection in a collection to which this attribute should apply.
        /// </summary>
        public int ElementIndirectionDepth { get; set; }

        /// <summary>
        /// When returned by <see cref="CountElementName" />, indicates the return value of the p/invoke should be used as the count.
        /// </summary>
        public const string ReturnsCountValue = "return-value";
    }

    /// <summary>
    /// Indicates this type is a marshaller type to be used in source-generated interop.
    /// </summary>
    [AttributeUsage(AttributeTargets.Struct)]
    public sealed class CustomTypeMarshallerAttribute : Attribute
    {
        public CustomTypeMarshallerAttribute(Type managedType, CustomTypeMarshallerKind marshallerKind = CustomTypeMarshallerKind.Value)
        {
            ManagedType = managedType;
            MarshallerKind = marshallerKind;
        }

        /// <summary>
        /// The managed type that the attributed type is a marshaller for.
        /// </summary>
        public Type ManagedType { get; }

        /// <summary>
        /// The expected shape of this marshaller type
        /// </summary>
        public CustomTypeMarshallerKind MarshallerKind { get; }

        /// <summary>
        /// The size of the caller-allocated buffer in scenarios where using caller-allocated buffer is support.
        /// </summary>
        public int BufferSize { get; set; }

        /// <summary>
        /// This type is used as a placeholder for the first generic parameter when generic parameters cannot be used
        /// to identify the managed type (i.e. when the marshaller type is generic over T and the managed type is T[])
        /// </summary>
        public struct GenericPlaceholder
        {
        }
    }

    /// <summary>
    /// The kind (or shape) of the custom marshaller type.
    /// </summary>
    public enum CustomTypeMarshallerKind
    {
        /// <summary>
        /// This marshaller represents marshalling a single value.
        /// </summary>
        Value,

        /// <summary>
        /// This marshaller represents marshalling a Span-like collection of values
        /// </summary>
        LinearCollection,
    }
}

And for the shapes:

// Shape for Value kind
[CustomTypeMarshaller(typeof(ManagedType))]
public struct Marshaller
{
    // Support for marshalling from managed to native.
    // Required if ToManaged() is not defined.
    // May be omitted if marshalling from managed to native is not required.
    public Marshaller(ManagedType managed) {}

    // Optional.
    // Support for marshalling from managed to native using a caller-allocated buffer
    // Requires the BufferSize field to be set on the CustomTypeMarshaller attribute.
    public Marshaller(ManagedType managed, Span<byte> buffer) {}

    // Support for marshalling from native to managed.
    // Required if constructor taking TManaged is not defined.
    // May be omitted if marshalling from native to managed is not required.
    public ManagedType ToManaged() {}

    // Optional.
    // If defined, the return value will be passed to native instead of the Marshaller itself
    // NativeType must not require marshalling
    public NativeType ToNativeValue();

    // Optional
    // If defined, will be called when marshalling from native to managed.
    // NativeType must not require marshalling, and must match ToNativeValue (if it exists)
    public void FromNativeValue(NativeType nativeValue);

    // Optional.
    // If defined and ToNativeValue or FromNativeValue is defined, will be called before Value property getter and its return value will be pinned
    public ref NativeType GetPinnableReference() {}

    // Optional.
    // Release native resources.
    public void FreeNative() {}
}

[CustomTypeMarshaller(typeof(GenericCollection<>), CustomTypeMarshallerKind.SpanCollection)]
public struct GenericContiguousCollectionMarshallerImpl<T>
{
    // Support for marshalling from native to managed.
    // May be omitted if marshalling from native to managed is not required.
    public GenericContiguousCollectionMarshallerImpl(int nativeSizeOfElement);

    // Support for marshalling from managed to native.
    // May be omitted if marshalling from managed to native is not required.
    public GenericContiguousCollectionMarshallerImpl(GenericCollection<T> collection, int nativeSizeOfElement);

    // Optional.
    // Support for marshalling from managed to native using a caller-allocated buffer
    // Requires the BufferSize field to be set on the CustomTypeMarshaller attribute.
    public GenericContiguousCollectionMarshallerImpl(GenericCollection<T> collection, Span<byte> buffer, int nativeSizeOfElement);


    public Span<ManagedType> GetManagedValuesDestination(int length);
    public ReadOnlySpan<ManagedType> GetManagedValuesSource();

    public ReadOnlySpan<byte> GetNativeValuesSource(int length);
    public Span<byte> GetNativeValuesDestination();

    // Required.
    // NativeType must not require marshalling
    public NativeType ToNativeValue();

    // Required.
    // NativeType must not require marshalling, and must match ToNativeValue (if it exists)
    public void FromNativeValue(NativeType nativeValue);

    // Support for marshalling from native to managed.
    // Required if constructor taking TManaged is not defined.
    // May be omitted if marshalling from native to managed is not required.
    public ManagedType ToManaged() {}

    // Optional.
    // If defined and ToNativeValue or FromNativeValue is defined, will be called before Value property getter and its return value will be pinned
    public ref NativeType GetPinnableReference() {}

    // Optional.
    // Release native resources.
    public void FreeNative() {}
}