Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Assert "rank < MAX_RANK" in runtime/src/coreclr/classlibnative/bcltype/arraynative.cpp Line: 812 #91349

Closed
amanasifkhalid opened this issue Aug 30, 2023 · 11 comments · Fixed by #91651
Closed

Assert "rank < MAX_RANK" in runtime/src/coreclr/classlibnative/bcltype/arraynative.cpp Line: 812 #91349

amanasifkhalid opened this issue Aug 30, 2023 · 11 comments · Fixed by #91651
Assignees
@AndyAyersMS
Labels
area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
Milestone
8.0.0

Comments

@amanasifkhalid
Copy link
Member

amanasifkhalid commented Aug 30, 2023 •
edited
Loading

While testing Orleans with a Checked build of the .NET 8 Preview 7 runtime, I hit the following assert while running Orleans.Serialization.UnitTests on Linux x64: Assert "rank < MAX_RANK" in runtime/src/coreclr/classlibnative/bcltype/arraynative.cpp Line: 812. To reproduce:

  • Replace your .NET 8 Preview 7 installation's runtime with a Checked build
  • Upgrade Orleans to target .NET 8 (using the .NET upgrade-assistant tool, for example)
    • Orleans uses some obsolete APIs, so in order to build successfully, you'll have to ignore some Roslyn errors (SYSLIB0050, SYSLIB0051, CS0672)
  • Set DOTNET_JitRandomEdgeCounts=1
  • Run dotnet test -c Release

For reference, I reproduced this running Ubuntu-22.04 in WSL2.

cc @JulieLeeMSFT @ReubenBond
@dotnet-issue-labeler dotnet-issue-labeler bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Aug 30, 2023
@ghost ghost added the untriaged New issue has not been triaged by the area owner label Aug 30, 2023
@adityamandaleeka
Copy link
Member

cc @AndyAyersMS since this involves the edge count randomization

@amanasifkhalid amanasifkhalid added area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI and removed untriaged New issue has not been triaged by the area owner needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners labels Aug 30, 2023
@ghost
Copy link

ghost commented Aug 30, 2023

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

While testing Orleans with a Checked build of the .NET 8 Preview 7 runtime, I hit the following assert while running Orleans.Serialization.UnitTests on Linux x64: Assert "rank < MAX_RANK" in runtime/src/coreclr/classlibnative/bcltype/arraynative.cpp Line: 812. To reproduce:

  • Replace your .NET 8 Preview 7 installation's runtime with a Checked build
  • Upgrade Orleans to target .NET 8 (using the .NET upgrade-assistant tool, for example)
    • Orleans uses some obsolete APIs, so in order to build successfully, you'll have to ignore some Roslyn errors (SYSLIB0050, SYSLIB0051, CS0672)
  • Set DOTNET_JitRandomEdgeCounts=1
  • Run dotnet test -c Release

For reference, I reproduced this running Ubuntu-22.04 in WSL2.

cc @JulieLeeMSFT @ReubenBond

Author: amanasifkhalid
Assignees: -
Labels:

area-CodeGen-coreclr
Milestone: -

@adityamandaleeka
Copy link
Member

Oh hey @amanasifkhalid, didn't notice that you were the one who filed this 😄

@JulieLeeMSFT JulieLeeMSFT added this to the 8.0.0 milestone Aug 31, 2023
@AndyAyersMS
Copy link
Member

@amanasifkhalid did you upgrade all the orleans projects, or just some subset?

@AndyAyersMS
Copy link
Member

Also did you try a clean dotnet test with the checked jit or did you first try it with the normal jit and then retry (with presumably already built assets) with the checked jit?

I'm asking because when I try installing the checked jit first CSC is failing with exit code 134. Perhaps my jit doesn't match, will look into this too.

@AndyAyersMS
Copy link
Member

If you have your changes still, can you put them on a branch of a fork of orleans that I can pull down, or else package them up as a patch?

@amanasifkhalid
Copy link
Member Author

@AndyAyersMS I upgraded all projects to .NET 8, and only used the Checked JIT to build and run tests, so I started from a clean slate. I didn't hit the CSC error... For consistency, I copied over libclrjit.so, libcoreclr.so, and System.Private.CoreLib.dll from my Checked build to my Preview 7 install.

I've pushed my repro setup here -- setting DOTNET_JitRandomEdgeCounts=1 and running dotnet test -c Release should work out of the box (except for a few Roslyn errors thrown in unrelated tests). Please let me know if it doesn't reproduce. Thanks!

@amanasifkhalid
Copy link
Member Author

Also sorry for the red herring, but I tried reproducing this again, and I can hit the assert without DOTNET_JitRandomEdgeCounts or any other JIT env vars set.

@AndyAyersMS
Copy link
Member

Since the assert comes from the runtime it's likely it will repro with just a checked runtime/SPC and a release JIT. Anyways, let me get a repro and I'll dig into why this happens.

@AndyAyersMS
Copy link
Member

Ok, I can repro:

cd test/Orleans.Serialization.UnitTests
dotnet test -c Release --blame-crash 

[createdump] Gathering state for process 28033 dotnet
[createdump] Crashing thread 6db4 signal 6 (0006)
[createdump] Writing full dump to file /tmp/47ee0b84-c35e-4220-968e-54b98196f229/dotnet_28033_1693953262_crashdump.dmp
[createdump] Written 2773696512 bytes (677172 pages) to core file
[createdump] Target process is alive
[createdump] Dump successfully written in 4705ms

The active test run was aborted. Reason: Test host process crashed : Assert failure(PID 28033 [0x00006d81], Thread: 28084 [0x6db4]): rank < MAX_RANK
    File: /home/andy/repos/runtime0/src/coreclr/classlibnative/bcltype/arraynative.cpp Line: 812
    Image: /usr/share/dotnet/dotnet
waitpid() returned successfully (wstatus 00000000) WEXITSTATUS 0 WTERMSIG 0

@AndyAyersMS
Copy link
Member

Stack trace:

  * frame #0: 0x00007f8271110c7f libc.so.6`wait4 + 95
    frame #1: 0x00007f8270e834ce libcoreclr.so`PROCCreateCrashDump(argv=<unavailable>, errorMessageBuffer=0x0000000000000000, cbErrorMessageBuffer=0) at process.cpp:2278:22
    frame #2: 0x00007f8270e84924 libcoreclr.so`::PROCCreateCrashDumpIfEnabled(signal=<unavailable>, siginfo=<unavailable>) at process.cpp:2492:9
    frame #3: 0x00007f8270e80e78 libcoreclr.so`::PROCAbort(signal=6, siginfo=0x0000000000000000) at process.cpp:2521:5
    frame #4: 0x00007f8270e80d7e libcoreclr.so`::RaiseFailFastException(pExceptionRecord=<unavailable>, pContextRecord=<unavailable>, dwFlags=<unavailable>) at process.cpp:1267:5
    frame #5: 0x00007f8270c8414e libcoreclr.so`FailFastOnAssert() at debug.cpp:60:5
    frame #6: 0x00007f8270c83e88 libcoreclr.so`::_DbgBreakCheck(szFile="/home/andy/repos/runtime0/src/coreclr/classlibnative/bcltype/arraynative.cpp", iLine=812, szExpr="rank < MAX_RANK", fConstrained=<unavailable>) at debug.cpp:288:5
    frame #7: 0x00007f8270c841da libcoreclr.so`_DbgBreakCheckNoThrow(szFile="/home/andy/repos/runtime0/src/coreclr/classlibnative/bcltype/arraynative.cpp", iLine=812, szExpr="rank < MAX_RANK", fConstrained=NO) at debug.cpp:307:18
    frame #8: 0x00007f8270c84523 libcoreclr.so`::DbgAssertDialog(szFile="/home/andy/repos/runtime0/src/coreclr/classlibnative/bcltype/arraynative.cpp", iLine=812, szExpr=<unavailable>) at debug.cpp:0
    frame #9: 0x00007f8270d7748f libcoreclr.so`ArrayNative::CreateInstance(pElementTypeUNSAFE=<unavailable>, rank=32, pLengths=0x00007f4277b55210, pLowerBounds=0x0000000000000000) at arraynative.cpp:812:9

00007F40EEFF9370 00007f8271110c7f [HelperMethodFrame_PROTECTOBJ: 00007f40eeff9370] System.Array.InternalCreate(System.RuntimeType, Int32, Int32*, Int32*)
00007F40EEFF94B0 00007F81F1719FEA System.Array.CreateInstance(System.Type, Int32[])
00007F40EEFF94E0 00007F820120D3CD Orleans.Serialization.Codecs.MultiDimensionalArrayCodec`1[[System.Int32, System.Private.CoreLib]].ReadValue[[Orleans.Serialization.Buffers.ReadOnlySequenceInput, Orleans.Serialization]](Orleans.Serialization.Buffers.Reader`1<Orleans.Serialization.Buffers.ReadOnlySequenceInput> ByRef, Orleans.Serialization.WireProtocol.Field) [/home/andy/repos/orleans/src/Orleans.Serialization/Codecs/MultiDimensionalArrayCodec.cs @ 120]
00007F40EEFF95E0 00007F820070CF77 Orleans.Serialization.Codecs.UntypedCodecWrapper`1[[System.__Canon, System.Private.CoreLib]].ReadValue[[Orleans.Serialization.Buffers.ReadOnlySequenceInput, Orleans.Serialization]](Orleans.Serialization.Buffers.Reader`1<Orleans.Serialization.Buffers.ReadOnlySequenceInput> ByRef, Orleans.Serialization.WireProtocol.Field)
00007F40EEFF9640 00007F820120CFA9 OrleansCodeGen.Orleans.Serialization.UnitTests.Codec_ArrayPoco`1[[System.Int32, System.Private.CoreLib]].Deserialize[[Orleans.Serialization.Buffers.ReadOnlySequenceInput, Orleans.Serialization]](Orleans.Serialization.Buffers.Reader`1<Orleans.Serialization.Buffers.ReadOnlySequenceInput> ByRef, Orleans.Serialization.UnitTests.ArrayPoco`1<Int32>) [/home/andy/repos/orleans/test/Orleans.Serialization.UnitTests/Orleans.CodeGenerator/Orleans.CodeGenerator.OrleansSerializationSourceGenerator/Orleans.Serialization.UnitTests.orleans.g.cs @ 3358]

Assert is

_ASSERTE(rank < MAX_RANK); // Ensures that the stack buffer size allocations below won't overlow

with

#define MAX_RANK 32

and the code is requesting a rank-32 array:

(lldb) dumparray 00007f4277b55200
Name:        System.Int32[]
MethodTable: 00007f81f29f6630
EEClass:     00007f81f29f6580
Size:        152(0x98) bytes
Array:       Rank 1, Number of elements 32, Type Int32

elsewhere we see 32 should be ok:

            if (rank > MAX_RANK)
            {
                ThrowTypeLoadException(pKey, IDS_CLASSLOAD_RANK_TOOLARGE);
            }

so the assert in arraynative.cpp is too aggressive, and should be

_ASSERTE(rank <= MAX_RANK); // Ensures that the stack buffer size allocations below won't overflow

AndyAyersMS added a commit to AndyAyersMS/runtime that referenced this issue Sep 5, 2023
@AndyAyersMS
Fix overly aggressive assert
d260b28 
Closes dotnet#91349
@AndyAyersMS AndyAyersMS mentioned this issue Sep 5, 2023
Merged
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Sep 5, 2023
AndyAyersMS added a commit that referenced this issue Sep 6, 2023
@AndyAyersMS
Fix overly aggressive assert (#91651)
808c523 
Closes #91349
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Sep 6, 2023
@ghost ghost locked as resolved and limited conversation to collaborators Oct 6, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@AndyAyersMS AndyAyersMS

Labels
area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
Projects
None yet
Milestone
8.0.0
Development

Successfully merging a pull request may close this issue.

Fix overly aggressive assert about array rank AndyAyersMS/runtime
4 participants
@adityamandaleeka @AndyAyersMS @amanasifkhalid @JulieLeeMSFT