Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PMI diffs are failing due to assert #51728

Closed
tannergooding opened this issue Apr 23, 2021 · 27 comments · Fixed by #53684 or #56192
Closed

PMI diffs are failing due to assert #51728

tannergooding opened this issue Apr 23, 2021 · 27 comments · Fixed by #53684 or #56192
Assignees
Labels
area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
Milestone

Comments

@tannergooding
Copy link
Member

jit-diff.bat diff --diff --pmi --frameworks currently fails for the following methods:

  • PREPALL type# 1350 method# 20377 System.Text.Encoding::GetCharsWithFallback
  • PREPALL type# 1350 method# 20378 System.Text.Encoding::GetCharsWithFallback
  • PREPALL type# 1350 method# 20472 System.Text.Encoding::GetBytesWithFallback
  • PREPALL type# 1350 method# 20473 System.Text.Encoding::GetBytesWithFallback
  • PREPALL type# 1350 method# 20477 System.Text.Encoding::GetCharCountWithFallback
  • PREPALL type# 1350 method# 20478 System.Text.Encoding::GetCharCountWithFallback

In all cases, an assert in emitOutputRR is firing: https://github.com/dotnet/runtime/blob/main/src/coreclr/jit/emitxarch.cpp#L11738-L11739

                    case INS_add:
                    case INS_sub:
                        assert(id->idGCref() == GCT_BYREF);

#ifdef DEBUG
                        regMaskTP regMask;
                        regMask = genRegMask(reg1) | genRegMask(reg2);

                        // r1/r2 could have been a GCREF as GCREF + int=BYREF
                        //                            or BYREF+/-int=BYREF
                        assert(((regMask & emitThisGCrefRegs) && (ins == INS_add)) ||
                               ((regMask & emitThisByrefRegs) && (ins == INS_add || ins == INS_sub)));
#endif
                        // Mark r1 as holding a byref
                        emitGCregLiveUpd(GCT_BYREF, reg1, dst);
                        break;
@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

@dotnet-issue-labeler dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label Apr 23, 2021
@tannergooding
Copy link
Member Author

This was on 120f374, which is about 12 commits behind latest. I don't see any commits that would have addressed the assert in that delta.

@jeffschwMSFT jeffschwMSFT added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Apr 23, 2021
@JulieLeeMSFT JulieLeeMSFT removed the untriaged New issue has not been triaged by the area owner label Apr 23, 2021
@JulieLeeMSFT JulieLeeMSFT added this to the 6.0.0 milestone Apr 23, 2021
@BruceForstall
Copy link
Member

Same as #10821.

@AndyAyersMS

@AndyAyersMS
Copy link
Member

I don't think there is an easy fix. Also I don't think there is an actual GC issue here.

I could be wrong on both counts.

@tannergooding
Copy link
Member Author

It looks like this arises from cases like having an arg byte* pOriginalBytes and then doing new ReadOnlySpan<byte>(pOriginalBytes, originalByteCount).Slice(bytesConsumedSoFar). The arg is a pointer and we correctly track it becomes a byref within the scope of BB02 (in genCodeForBBlist), but for the purposes of what is eventually asserted, we only look to track byrefs of var locations, not temporaries.

The root issue appears to be that the assert only acknowledges BYREF = GCREF + int, BYREF = BYREF + int, and BYREF = BYREF - int (or the commutative alternatives). It doesn't expect BYREF = int + int, which is what we effectively have here (byte* + int).

JitDump.txt

@tannergooding
Copy link
Member Author

tannergooding commented Apr 27, 2021

Changing _pointer = new ByReference<T>(ref Unsafe.As<byte, T>(ref *(byte*)pointer)); in Span/ROSpan to be _pointer = new ByReference<T>(ref Unsafe.AsRef<T>(pointer)); causes this to repro in a lot more places (even though they should logically be "the same") 😄

@AndyAyersMS
Copy link
Member

GC tracking has an inherent idea that GC refs are not created out of thin air. So int + int -> byref is not ok, it means one of those int was really a byref, and the problem lies upstream somewhere.

IIRC the object allocator phase does the right sort of analysis to motivate potentially retyping the local, if the only defs/uses of the local are byrefs.

@tannergooding
Copy link
Member Author

tannergooding commented Apr 27, 2021

GC tracking has an inherent idea that GC refs are not created out of thin air. So int + int -> byref is not ok, it means one of those int was really a byref

@AndyAyersMS, Sorry, I might have explained myself badly. We have a byte* that is being stored in the byref field of a promoted Span, so its not actually a byref just retyped as one. The node represented however is BYREF = BYREF + int.

; Final local variable assignments
;
;  V00 this         [V00,T03] (  3,  3   )     ref  ->  rcx         this class-hnd
;  V01 arg1         [V01,T04] (  3,  3   )    long  ->  rdx
;  V02 arg2         [V02,T00] (  6,  6   )     int  ->   r8
;  V03 arg3         [V03,T05] (  3,  3   )    long  ->   r9
;  V04 arg4         [V04,T02] (  4,  4   )     int  ->  rax
;  V05 arg5         [V05,T09] (  3,  3   )     int  ->  r10
;  V06 arg6         [V06,T10] (  3,  3   )     int  ->  r11
;* V07 loc0         [V07    ] (  0,  0   )  struct (16) zero-ref    ld-addr-op
;* V08 loc1         [V08    ] (  0,  0   )  struct (16) zero-ref    ld-addr-op
;  V09 OutArgs      [V09    ] (  1,  1   )  lclBlk (48) [rsp+00H]   "OutgoingArgSpace"
;* V10 tmp1         [V10    ] (  0,  0   )  struct (16) zero-ref    "NewObj constructor temp"
;* V11 tmp2         [V11    ] (  0,  0   )  struct (16) zero-ref    "NewObj constructor temp"
;* V12 tmp3         [V12    ] (  0,  0   )  struct (16) zero-ref    "impAppendStmt"
;* V13 tmp4         [V13    ] (  0,  0   )  struct (16) zero-ref    "struct address for call/obj"
;* V14 tmp5         [V14    ] (  0,  0   )  struct ( 8) zero-ref    "NewObj constructor temp"
;* V15 tmp6         [V15    ] (  0,  0   )  struct (16) zero-ref    "NewObj constructor temp"
;* V16 tmp7         [V16    ] (  0,  0   )    long  ->  zero-ref    "Inlining Arg"
;* V17 tmp8         [V17    ] (  0,  0   )   byref  ->  zero-ref    "Inlining Arg"
;* V18 tmp9         [V18    ] (  0,  0   )   byref  ->  zero-ref    "Inlining Arg"
;* V19 tmp10        [V19    ] (  0,  0   )  struct ( 8) zero-ref    "NewObj constructor temp"
;  V20 tmp11        [V20,T06] (  2,  4   )     int  ->  rsi         "Inlining Arg"
;* V21 tmp12        [V21    ] (  0,  0   )  struct ( 8) zero-ref    "NewObj constructor temp"
;* V22 tmp13        [V22    ] (  0,  0   )  struct (16) zero-ref    "NewObj constructor temp"
;* V23 tmp14        [V23    ] (  0,  0   )    long  ->  zero-ref    "Inlining Arg"
;* V24 tmp15        [V24    ] (  0,  0   )   byref  ->  zero-ref    "Inlining Arg"
;* V25 tmp16        [V25    ] (  0,  0   )   byref  ->  zero-ref    "Inlining Arg"
;* V26 tmp17        [V26    ] (  0,  0   )  struct ( 8) zero-ref    "NewObj constructor temp"
;  V27 tmp18        [V27,T07] (  2,  4   )     int  ->  r10         "Inlining Arg"
;  V28 tmp19        [V28,T11] (  2,  2   )   byref  ->  rdx         V07._pointer(offs=0x00) P-INDEP "field V07._pointer (fldOffset=0x0)"
;  V29 tmp20        [V29,T23] (  2,  2   )     int  ->  rsi         V07._length(offs=0x08) P-INDEP "field V07._length (fldOffset=0x8)"
;  V30 tmp21        [V30,T12] (  2,  2   )   byref  ->   r9         V08._pointer(offs=0x00) P-INDEP "field V08._pointer (fldOffset=0x0)"
;  V31 tmp22        [V31,T24] (  2,  2   )     int  ->  r10         V08._length(offs=0x08) P-INDEP "field V08._length (fldOffset=0x8)"
;  V32 tmp23        [V32,T13] (  2,  2   )   byref  ->  rdx         V10._pointer(offs=0x00) P-INDEP "field V10._pointer (fldOffset=0x0)"
;* V33 tmp24        [V33,T29] (  0,  0   )     int  ->  zero-ref    V10._length(offs=0x08) P-INDEP "field V10._length (fldOffset=0x8)"
;  V34 tmp25        [V34,T14] (  2,  2   )   byref  ->   r9         V11._pointer(offs=0x00) P-INDEP "field V11._pointer (fldOffset=0x0)"
;* V35 tmp26        [V35,T30] (  0,  0   )     int  ->  zero-ref    V11._length(offs=0x08) P-INDEP "field V11._length (fldOffset=0x8)"
;  V36 tmp27        [V36,T15] (  2,  2   )   byref  ->  rdx         V12._pointer(offs=0x00) P-INDEP "field V12._pointer (fldOffset=0x0)"
;  V37 tmp28        [V37,T25] (  2,  2   )     int  ->  rsi         V12._length(offs=0x08) P-INDEP "field V12._length (fldOffset=0x8)"
;  V38 tmp29        [V38,T16] (  2,  2   )   byref  ->   r9         V13._pointer(offs=0x00) P-INDEP "field V13._pointer (fldOffset=0x0)"
;  V39 tmp30        [V39,T26] (  2,  2   )     int  ->  r10         V13._length(offs=0x08) P-INDEP "field V13._length (fldOffset=0x8)"
;  V40 tmp31        [V40,T17] (  2,  2   )   byref  ->  rdx         V14._value(offs=0x00) P-INDEP "field V14._value (fldOffset=0x0)"
;  V41 tmp32        [V41,T18] (  2,  2   )   byref  ->  rdx         V15._pointer(offs=0x00) P-INDEP "field V15._pointer (fldOffset=0x0)"
;  V42 tmp33        [V42,T27] (  2,  2   )     int  ->  rsi         V15._length(offs=0x08) P-INDEP "field V15._length (fldOffset=0x8)"
;  V43 tmp34        [V43,T19] (  2,  2   )   byref  ->  rdx         V19._value(offs=0x00) P-INDEP "field V19._value (fldOffset=0x0)"
;  V44 tmp35        [V44,T20] (  2,  2   )   byref  ->   r9         V21._value(offs=0x00) P-INDEP "field V21._value (fldOffset=0x0)"
;  V45 tmp36        [V45,T21] (  2,  2   )   byref  ->   r9         V22._pointer(offs=0x00) P-INDEP "field V22._pointer (fldOffset=0x0)"
;  V46 tmp37        [V46,T28] (  2,  2   )     int  ->  r10         V22._length(offs=0x08) P-INDEP "field V22._length (fldOffset=0x8)"
;  V47 tmp38        [V47,T22] (  2,  2   )   byref  ->   r9         V26._value(offs=0x00) P-INDEP "field V26._value (fldOffset=0x0)"
;  V48 tmp39        [V48    ] (  3,  6   )  struct (16) [rsp+40H]   do-not-enreg[XSFB] must-init addr-exposed "by-value struct argument"
;  V49 tmp40        [V49    ] (  3,  6   )  struct (16) [rsp+30H]   do-not-enreg[XSFB] must-init addr-exposed "by-value struct argument"
;  V50 tmp41        [V50,T01] (  3,  6   )     ref  ->  rcx         "argument with side effect"
;  V51 tmp42        [V51,T08] (  2,  4   )     int  ->   r8         "argument with side effect"
;
; Lcl frame size = 80

That is, we have V01 which is byte* and the first thing that happens is this goes into V40, then V40 goes into V32, and V32 goes into V28.
However, the basic block tracks no byref regs coming out of it:

=============== Generating BB02 [000..000) -> BB06 (cond), preds={BB01} succs={BB03,BB06} flags=0x00000000.20000060: i internal LIR
BB02 IN (7)={V02 V04 V00 V01 V03 V05 V06    } + ByrefExposed + GcHeap
     OUT(7)={V02 V04 V00     V03 V05 V06 V28} + ByrefExposed + GcHeap

Recording Var Locations at start of BB02
  V02(r8)  V04(rax)  V00(rcx)  V01(rdx)  V03(r9)  V05(r10)  V06(r11)
Liveness not changing: 000000000000063D {V00 V01 V02 V03 V04 V05 V06}
							Live regs: 00000000 {} => 00000F07 {rax rcx rdx r8 r9 r10 r11}
							GC regs: 00000000 {} => 00000002 {rcx}
							Byref regs: (unchanged) 00000000 {}

      L_M15816_BB02:

Scope info: begin block BB02, IL range [000..000)
Scope info: open scopes =
   2 (V02 arg2) [000..02E)
   4 (V04 arg4) [000..02E)
   0 (V00 this) [000..02E)
   1 (V01 arg1) [000..02E)
   3 (V03 arg3) [000..02E)
   5 (V05 arg5) [000..02E)
   6 (V06 arg6) [000..02E)
Added IP mapping: NO_MAP STACK_EMPTY (G_M15816_IG02,ins#2,ofs#9) label
Generating: N015 (  1,  1) [000085] ------------        t85 =    LCL_VAR   byref  V01 arg1         u:1 rdx (last use) REG rdx $c0
                                                              /--*  t85    byref
Generating: N017 (  5,  4) [000065] DA----------              *  STORE_LCL_VAR byref  V40 tmp31        d:1 rdx REG rdx
							V01 in reg rdx is becoming dead  [000085]
							Live regs: 00000F07 {rax rcx rdx r8 r9 r10 r11} => 00000F03 {rax rcx r8 r9 r10 r11}
							Live vars: {V00 V01 V02 V03 V04 V05 V06} => {V00 V02 V03 V04 V05 V06}
							V40 in reg rdx is becoming live  [000065]
							Live regs: 00000F03 {rax rcx r8 r9 r10 r11} => 00000F07 {rax rcx rdx r8 r9 r10 r11}
							Live vars: {V00 V02 V03 V04 V05 V06} => {V00 V02 V03 V04 V05 V06 V40}
							Byref regs: 00000000 {} => 00000004 {rdx}
Generating: N019 (  3,  2) [000252] -------N----       t252 =    LCL_VAR   byref  V40 tmp31        u:1 rdx (last use) REG rdx $c0
                                                              /--*  t252   byref
Generating: N021 (  3,  3) [000253] DA----------              *  STORE_LCL_VAR byref  V32 tmp23        d:1 rdx REG rdx
							V40 in reg rdx is becoming dead  [000252]
							Live regs: 00000F07 {rax rcx rdx r8 r9 r10 r11} => 00000F03 {rax rcx r8 r9 r10 r11}
							Live vars: {V00 V02 V03 V04 V05 V06 V40} => {V00 V02 V03 V04 V05 V06}
							Byref regs: 00000004 {rdx} => 00000000 {}
							V32 in reg rdx is becoming live  [000253]
							Live regs: 00000F03 {rax rcx r8 r9 r10 r11} => 00000F07 {rax rcx rdx r8 r9 r10 r11}
							Live vars: {V00 V02 V03 V04 V05 V06} => {V00 V02 V03 V04 V05 V06 V32}
							Byref regs: 00000000 {} => 00000004 {rdx}
Added IP mapping: 0x0008 (G_M15816_IG02,ins#2,ofs#9) label
Generating: N023 (???,???) [000349] ------------                 IL_OFFSET void   IL offset: 0x8 REG NA
Generating: N025 (  1,  1) [000255] ------------       t255 =    LCL_VAR   byref  V32 tmp23        u:1 rdx (last use) REG rdx $c0
                                                              /--*  t255   byref
Generating: N027 (  1,  3) [000256] DA----------              *  STORE_LCL_VAR byref  V28 tmp19        d:1 rdx REG rdx
							V32 in reg rdx is becoming dead  [000255]
							Live regs: 00000F07 {rax rcx rdx r8 r9 r10 r11} => 00000F03 {rax rcx r8 r9 r10 r11}
							Live vars: {V00 V02 V03 V04 V05 V06 V32} => {V00 V02 V03 V04 V05 V06}
							Byref regs: 00000004 {rdx} => 00000000 {}
							V28 in reg rdx is becoming live  [000256]
							Live regs: 00000F03 {rax rcx r8 r9 r10 r11} => 00000F07 {rax rcx rdx r8 r9 r10 r11}
							Live vars: {V00 V02 V03 V04 V05 V06} => {V00 V02 V03 V04 V05 V06 V28}
							Byref regs: 00000000 {} => 00000004 {rdx}
Generating: N029 (  1,  1) [000258] ------------       t258 =    LCL_VAR   int    V02 arg2         u:1 r8 REG r8 $100
                                                              /--*  t258   int
Generating: N031 (  5,  4) [000259] DA----------              *  STORE_LCL_VAR int    V29 tmp20        d:1 rsi REG rsi
IN0003:        mov      esi, r8d
							V29 in reg rsi is becoming live  [000259]
							Live regs: 00000F07 {rax rcx rdx r8 r9 r10 r11} => 00000F47 {rax rcx rdx rsi r8 r9 r10 r11}
							Live vars: {V00 V02 V03 V04 V05 V06 V28} => {V00 V02 V03 V04 V05 V06 V28 V29}
Generating: N033 (  1,  1) [000015] ------------        t15 =    LCL_VAR   int    V05 arg5         u:1 r10 REG r10 $102
Generating: N035 (  1,  1) [000089] ------------        t89 =    LCL_VAR   int    V29 tmp20        u:1 rsi (last use) REG rsi $100
                                                              /--*  t15    int
                                                              +--*  t89    int
Generating: N037 (  3,  3) [000090] N------N-U--              *  GT        void   REG NA $181
							V29 in reg rsi is becoming dead  [000089]
							Live regs: 00000F47 {rax rcx rdx rsi r8 r9 r10 r11} => 00000F07 {rax rcx rdx r8 r9 r10 r11}
							Live vars: {V00 V02 V03 V04 V05 V06 V28 V29} => {V00 V02 V03 V04 V05 V06 V28}
IN0004:        cmp      r10d, esi
Generating: N039 (  5,  5) [000091] ------------              *  JTRUE     void   REG NA
IN0005:        ja       L_M15816_BB06

Scope info: end block BB02, IL range [000..000)
Scope info: open scopes =
   2 (V02 arg2) [000..02E)
   4 (V04 arg4) [000..02E)
   0 (V00 this) [000..02E)
   3 (V03 arg3) [000..02E)
   5 (V05 arg5) [000..02E)
   6 (V06 arg6) [000..02E)

We then move continue in the next basic block with Byref regs still being 0, cuasing an assert when we hit IN0009:

=============== Generating BB03 [000..000) -> BB06 (cond), preds={BB02} succs={BB04,BB06} flags=0x00000000.20000060: i internal LIR
BB03 IN (7)={V02 V04 V00 V03 V05 V06 V28        } + ByrefExposed + GcHeap
     OUT(7)={V02 V04 V00 V03     V06     V36 V37} + ByrefExposed + GcHeap

Recording Var Locations at start of BB03
  V02(r8)  V04(rax)  V00(rcx)  V03(r9)  V05(r10)  V06(r11)  V28(rdx)
Liveness not changing: 0000000000000E2D {V00 V02 V03 V04 V05 V06 V28}
							Live regs: 00000000 {} => 00000F07 {rax rcx rdx r8 r9 r10 r11}
							GC regs: 00000000 {} => 00000002 {rcx}
							Byref regs: 00000000 {} => 00000004 {rdx}

      L_M15816_BB03:

Scope info: begin block BB03, IL range [000..000)
Scope info: open scopes =
   2 (V02 arg2) [000..02E)
   4 (V04 arg4) [000..02E)
   0 (V00 this) [000..02E)
   3 (V03 arg3) [000..02E)
   5 (V05 arg5) [000..02E)
   6 (V06 arg6) [000..02E)
Added IP mapping: NO_MAP STACK_EMPTY (G_M15816_IG02,ins#5,ofs#21) label
Generating: N043 (  3,  2) [000103] ------------       t103 =    LCL_VAR   int    V02 arg2         u:1 r8 REG r8 $100
Generating: N045 (  1,  1) [000104] ------------       t104 =    LCL_VAR   int    V05 arg5         u:1 r10 REG r10 $102
                                                              /--*  t103   int
                                                              +--*  t104   int
Generating: N047 (  5,  4) [000105] ------------       t105 = *  SUB       int    REG rsi $182
IN0006:        mov      esi, r8d
IN0007:        sub      esi, r10d
                                                              /--*  t105   int
Generating: N049 (  5,  4) [000145] DA----------              *  STORE_LCL_VAR int    V20 tmp11        d:1 rsi REG rsi
							V20 in reg rsi is becoming live  [000145]
							Live regs: 00000F07 {rax rcx rdx r8 r9 r10 r11} => 00000F47 {rax rcx rdx rsi r8 r9 r10 r11}
							Live vars: {V00 V02 V03 V04 V05 V06 V28} => {V00 V02 V03 V04 V05 V06 V20 V28}
Generating: N051 (  1,  1) [000097] ------------        t97 =    LCL_VAR   int    V05 arg5         u:1 r10 (last use) REG r10 $102
                                                              /--*  t97    int
Generating: N053 (  2,  3) [000098] ---------U--        t98 = *  CAST      long <- ulong <- uint REG r10 $240
							V05 in reg r10 is becoming dead  [000097]
							Live regs: 00000F47 {rax rcx rdx rsi r8 r9 r10 r11} => 00000B47 {rax rcx rdx rsi r8 r9 r11}
							Live vars: {V00 V02 V03 V04 V05 V06 V20 V28} => {V00 V02 V03 V04 V06 V20 V28}
IN0008:        mov      r10d, r10d
Generating: N055 (  1,  1) [000122] ------------       t122 =    LCL_VAR   byref  V28 tmp19        u:1 rdx (last use) REG rdx $c0
                                                              /--*  t98    long
                                                              +--*  t122   byref
Generating: N057 (  4,  5) [000123] ------------       t123 = *  ADD       byref  REG rdx $280
							V28 in reg rdx is becoming dead  [000122]
							Live regs: 00000B47 {rax rcx rdx rsi r8 r9 r11} => 00000B43 {rax rcx rsi r8 r9 r11}
							Live vars: {V00 V02 V03 V04 V06 V20 V28} => {V00 V02 V03 V04 V06 V20}
							Byref regs: 00000004 {rdx} => 00000000 {}
IN0009:        add      rdx, r10
							Byref regs: 00000000 {} => 00000004 {rdx}
...

@tannergooding
Copy link
Member Author

Updating the logic you added in fgUpdateInlineReturnExpressionPlaceHolder to do:

  // If we end up swapping type we may need to retype the tree:
  if (retType != newType)
  {
      if ((retType == TYP_BYREF) && (tree->OperGet() == GT_IND))
      {
          // - in an RVA static if we've reinterpreted it as a byref;
          assert(newType == TYP_I_IMPL);
          JITDUMP("Updating type of the return GT_IND expression to TYP_BYREF\n");
          inlineCandidate->gtType = TYP_BYREF;
      }
      else
      {
          // - under a call if we changed size of the argument.
          GenTree* putArgType = comp->fgCheckCallArgUpdate(data->parent, inlineCandidate, retType);
          if (putArgType != nullptr)
          {
              inlineCandidate = putArgType;
          }
      }
  }

+ if ((retType == TYP_BYREF) && inlineCandidate->OperIsLocal())
+ {
+     unsigned   lclNum = inlineCandidate->AsLclVarCommon()->GetLclNum();
+     LclVarDsc* lclVar = &comp->lvaTable[lclNum];
+ 
+     if (lclVar->TypeGet() == TYP_I_IMPL)
+     {
+         JITDUMP("Updating type of V%02u to TYP_BYREF\n", lclNum);
+         lclVar->lvType = TYP_BYREF;
+     }
+ }

Looks to workaround the current issue as the arg gets tracked as a byref the entire way through

@tannergooding
Copy link
Member Author

Updating genCodeForBBlist to insert a new label if the set of live GC refs has changed also looks to work:

+ if ((GetEmitter()->emitInitGCrefRegs != gcInfo.gcRegGCrefSetCur) ||
+     (GetEmitter()->emitInitByrefRegs != gcInfo.gcRegByrefSetCur))
+ {
+     JITDUMP("Adding label due to changed set of live GC refs\n");
+     needLabel = true;
+ }

  if (needLabel)
  {
      // Mark a label and update the current set of live GC refs
  
      block->bbEmitCookie = GetEmitter()->emitAddLabel(gcInfo.gcVarPtrSetCur, gcInfo.gcRegGCrefSetCur,
                                                       gcInfo.gcRegByrefSetCur, false DEBUG_ARG(block->bbNum));
  }

This changes it as follows:

  Mapped BB01 to G_M15816_IG02
  Label: IG02, GCvars=0000000000000000 {}, gcrefRegs=00000002 {rcx}, byrefRegs=00000000 {}

+ Mapped BB03 to G_M15816_IG03
+ Label: IG03, GCvars=0000000000000000 {}, gcrefRegs=00000002 {rcx}, byrefRegs=00000004 {rdx}

+ Mapped BB05 to G_M15816_IG04
+ Label: IG04, GCvars=0000000000000000 {}, gcrefRegs=00000002 {rcx}, byrefRegs=00000204 {rdx r9}

- Mapped BB06 to G_M15816_IG04
- Label: IG04, GCvars=0000000000000000 {}, gcrefRegs=00000000 {}, byrefRegs=00000000 {}
+ Mapped BB06 to G_M15816_IG06
+ Label: IG06, GCvars=0000000000000000 {}, gcrefRegs=00000000 {}, byrefRegs=00000000 {}

@tannergooding
Copy link
Member Author

tannergooding commented Apr 28, 2021

I think this "loophole" has something to do with emitThisByrefReg not getting updated correctly somewhere, but it isn't clear to me where that is right now

From what I can tell, we set it (and emitInitByrefReg) whenever we start a new (non-inline) label and otherwise, when actual instructions are emitted via emitGCregLiveUpd

We don't actually emit an instruction for new Span(pointer, x) because the byref field is promoted where it and the pointer are assigned the same register. This means the liveness checks that come into play for G_M15816_IG02 (which was created for BB01) only consider the pointer arg (when its not a byref). It doesn't consider the the assignments to the byref field that happen in BB02 (which were elided because the registers are the same).

So when we get to BB03, the "liveness" set has changed, but its not been tracked and so inserting a new label updates emitThisByrefReg to be "correct"

@tannergooding
Copy link
Member Author

tannergooding commented Apr 28, 2021

So it seems like, in the process of BB02 being full elided, we are missing one or more emitGCregLiveUpd calls for the elided moves, since its going from pointer->byref (which has no instruction, but which still has a GC liveness impact). I'd imagine this means we are also missing emitGCregLiveUpd calls for the inverse byref->pointer case (such as from Unsafe.AsPointer), in the case where they are assigned the same register.

We wouldn't be broken here in any scenario where the byref->pointer or pointer->byref produces an actual mov instruction, since that will end up hitting a path that calls emitGCregLiveUpd.

@tannergooding
Copy link
Member Author

or more explicitly, if tgtReg == srcReg but the actual types of tgt and src are not both TYP_BYREF, then we need to still call emitGCregLiveUpd to ensure liveness is correct

@AndyAyersMS
Copy link
Member

Thanks for drilling in. Yes, we may need to update gc info for "elided moves". In past compilers we sometimes had no-encode pseudo ops for things like this.

Do the moves get elided by emitter / codegen peepholes? Or is it from LSRA preferencing?

@tannergooding
Copy link
Member Author

Do the moves get elided by emitter / codegen peepholes?

In this particular case, by emitter/codegen peepholes such as: https://github.com/dotnet/runtime/blob/main/src/coreclr/jit/codegenxarch.cpp#L4512-L4516

We do this fairly frequently for SIMD nodes and never through a centralized helper. You can search for op1Reg != targetReg and targetReg != op1Reg in https://github.com/dotnet/runtime/blob/main/src/coreclr/jit/hwintrinsiccodegenxarch.cpp#L1176 and https://github.com/dotnet/runtime/blob/main/src/coreclr/jit/emitxarch.cpp#L1176.

Its unclear how common this is for general purpose moves, given we have some late stage peepholes and scattered calls to the emitter otherwise. It's likewise unclear if there is anything needed in LSRA preferencing to handle this.

Given we have the following (and other) assert(s) trying to catch same register moves, I'd guess all code paths have this issue in some form or another: https://github.com/dotnet/runtime/blob/main/src/coreclr/jit/emitxarch.cpp#L4168-L4174

@AndyAyersMS
Copy link
Member

I'd have to look more closely to be sure, but what you are describing mirrors some concerns I had working on a reg-reg shuffle elimination (which we dropped in favor of better RA preferencing): dotnet/coreclr#21959 (comment)

So yes even if we skip emitting an instruction we need to process the liveness update if it can impact GC.

@tannergooding
Copy link
Member Author

tannergooding commented Apr 28, 2021

@AndyAyersMS, a quick test confirms that retaining the same register moves does fix the issue.

What is the preferred way to handle this and to help prevent these issues in the future? It sounds like we want to add a new INS_GCregLiveUpd pseudo instruction or to specially handle mov where tgtReg == srcReg (and where EA_SIZE == EA_PTRSIZE), and to ensure that all moves go through this helper.

Another thought is we might be able to take advantage of the various blocks knowing their initial gcRegByrefSetCur, by asserting that it matches emitThisByrefRegs before any instructions for that block are emitted. We might be able to piggyback this information on the inline labels, so it doesn't impact that actual tracked liveness?

The priority of this, given it should only impact PTR->BYREF or BYREF->PTR moves (and so it shouldn't impact "correctness"), isn't clear. However, it would impact the ability for #51599 to be resolved by treating the APIs as intrinsic (since that makes it even easier for the JIT to elide those moves).

It is possible that most of the issues would be handled simply in GT_STORE_LCL_VAR, but further investigation is needed.

@tannergooding
Copy link
Member Author

@AndyAyersMS, I spent some time last night and searched through codegen/emit for all of the move elimination and tried to centralize it: https://github.com/dotnet/runtime/compare/main...tannergooding:track-moves?expand=1

There is at least one spot I've missed since crossgen still hits '((regMask & emitThisGCrefRegs) && (ins == INS_add)) || ((regMask & emitThisByrefRegs) && (ins == INS_add || ins == INS_sub)), but it should be fairly close.

@AndyAyersMS
Copy link
Member

When I looked into this recently I thought we might need a different kind of fix: #10821 (comment) . Not clear to me yet if what you've found supersedes this, or there is indeed some other path where we can hit this assert.

Emitting GC updates for movs, while technically reasonable, is at best a bit of a crutch. Given the volume of your proposed changes I'm not sure it is a direction we should follow. Arguably we should have a similar assert for movs like we have for adds and never (or almost never, there may be one or two cases we handle) allow a mov that creates a byref.

@tannergooding
Copy link
Member Author

tannergooding commented Apr 30, 2021

never allow a mov that creates a byref.

How would that be handled when cases like the following generate just ldarg.1 and therefore create a byref from a pointer via what is effectively an assignment?

public unsafe ref int M(int* x) {
    return ref *x;
}

@AndyAyersMS
Copy link
Member

We'd have to assume in a case like that the callers all get it right, and that treating x as a byref is ok.

@tannergooding
Copy link
Member Author

tannergooding commented May 1, 2021

We'd have to assume in a case like that the callers all get it right, and that treating x as a byref is ok.

@AndyAyersMS, sorry I don't think I'm following here.

We can have a byref created from a pointer in many ways:

  • ref *x
  • Unsafe.AsRef(x): ref T Unsafe.AsRef(void*)
  • new Span<T>(x, length): .ctor(void*, int)
  • new ReadOnlySpan<T>(x, length): .ctor(void*, int)
  • etc

In all of these cases, provided x is actually a pointer (or a pinned ref), the operation should be 100% safe and valid IL.

and in these cases we end up with logic that effectively converts a pointer to a byref, such as:

Generating: N015 (  1,  1) [000085] ------------        t85 =    LCL_VAR   byref  V01 arg1         u:1 rdx (last use) REG rdx $c0
                                                              /--*  t85    byref
Generating: N017 (  5,  4) [000065] DA----------              *  STORE_LCL_VAR byref  V40 tmp31        d:1 rdx REG rdx

arg1 was not a byref, it is a pointer and inspecting the underlying local shows that, and so it correctly should not be annotated as one in the liveness tracking.

  • It being listed as a byref here probably needs to be fixed, it should probably be shown as t85 = LCL_VAR long V01 arg1 u:1 rdx (last use) REG rdx $c0 since arg1 is byte* in this scenario (not differentiating long from native int from byte* is also annoying, but unrelated)

tmp31 however is a byref and so when this assignment happens, we should be updating the liveness tracking to indicate rdx now contains a byref.

Since arg1 was "last use", the register allocator assigns it to be kept in rdx.

However, when we get to codegen we have logic that basically says:

if (tgtReg == srcReg)
{
    // Do Nothing
}
else
{
    emitMove(tgtReg, srcReg);
}

Since we only do liveness updates for emitted instructions and since we opt to not emit an instruction here, we lose rdx becoming a "live" byref.

And so, when we get to the assert in emitOutputRR:

assert(((regMask & emitThisGCrefRegs) && (ins == INS_add)) ||
       ((regMask & emitThisByrefRegs) && (ins == INS_add || ins == INS_sub)));

We fail, because we dropped the liveness update.

The correct fix, as far as I can see, is to not drop the liveness update in emit. There are probably several ways we could do that, one of which is to not elide in codegen, but instead to elide in emit when we try to output the instruction.

@ghost ghost added in-pr There is an active PR which will close this issue when it is merged and removed in-pr There is an active PR which will close this issue when it is merged labels May 12, 2021
@BruceForstall
Copy link
Member

@tannergooding you started working on this with #52661, but that was just part 1 for a subsequent part 2 that would fix this issue (right?). Are you working on part 2? (Can I assign this to you?)

@tannergooding
Copy link
Member Author

tannergooding commented Jun 2, 2021

for a subsequent part 2 that would fix this issue (right?)

That's the intent, yes.

Are you working on part 2? (Can I assign this to you?)

I started the work here but got stuck on a few test failures and haven't had time to finish diagnosing yet. https://github.com/tannergooding/runtime/tree/byref-liveness is the current branch, and its relatively small overall. It basically just stops eliding same register moves and marks those descriptors as being zero size, allowing them to be carried through emit (without actually emitting anything) and ultimately emit the liveness update.

@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Jun 3, 2021
@ghost ghost closed this as completed in #53684 Jun 9, 2021
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Jun 9, 2021
@tannergooding
Copy link
Member Author

Reopening since the PR had to be reverted

@BruceForstall
Copy link
Member

This assert is planned to be disabled in #55861

@BruceForstall
Copy link
Member

Well, the change that was planning to disable this assert (along with other changes) was aborted. So we maybe should just disable the assert (and #54007?) as a separate change.

BruceForstall added a commit to BruceForstall/runtime that referenced this issue Jul 22, 2021
The emitter has asserts that an xarch RMW inc/dec/add/sub of a byref
must have an incoming gcref/byref to be legal. This is (no longer)
true due to extensive use of Span and Unsafe constructs, where we
often see lclheap or other integer typed values cast to byref. Also,
the emitter only updates its GC info when an instruction is generated.
When one of these casts from integer to byref ends up getting the same
register allocated, and its instruction is thus omitted, the emitter
doesn't get the appropriate gcref update (this problem is being
attempted to be solved elsewhere).

For now, disable these asserts.

Re-enable the tests disabled in dotnet#54207

Fixes dotnet#51728, dotnet#54007
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Jul 22, 2021
BruceForstall added a commit that referenced this issue Jul 23, 2021
The emitter has asserts that an xarch RMW inc/dec/add/sub of a byref
must have an incoming gcref/byref to be legal. This is (no longer)
true due to extensive use of Span and Unsafe constructs, where we
often see lclheap or other integer typed values cast to byref. Also,
the emitter only updates its GC info when an instruction is generated.
When one of these casts from integer to byref ends up getting the same
register allocated, and its instruction is thus omitted, the emitter
doesn't get the appropriate gcref update (this problem is being
attempted to be solved elsewhere).

For now, disable these asserts.

Re-enable the tests disabled in #54207

Fixes #51728, #54007
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Jul 23, 2021
@ghost ghost locked as resolved and limited conversation to collaborators Aug 22, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
Projects
None yet
5 participants