[z9332]orchagent crash observed when tunnel termination is not supported (TH3)

[srideepDell]

Description:

Below logs is seen with errors and syncd sends shutdown notification to orchagent which creates crash of orchagent. This issue is seen only when BCM SAI is changed to 3.7.4, issue not seen with 3.7.3

Platform :

DellEMC-Z9332f-O32 with T1 config loaded

Steps to reproduce:

1. Install latest master build on z9332
2. System is configured with T1 config
3. OA crash is seen with sai errors.

drivshell>bsv
bsv
BRCM SAI ver: [3.7.4.2], OCP SAI ver: [1.6.0], SDK ver: [6.5.16]

syslogs with errors

Oct 2 16:20:43.711422 sonic ERR syncd#syncd: [none] brcm_sai_create_tunnel_term_table_entry:1811 P2MP tunnel terminator not supported on this platform.
Oct 2 16:20:43.711496 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_TERM_TABLE_ENTRY_ATTR_VR_ID: oid:0x3000000000024
Oct 2 16:20:43.711528 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_TERM_TABLE_ENTRY_ATTR_TYPE: SAI_TUNNEL_TERM_TABLE_ENTRY_TYPE_P2MP
Oct 2 16:20:43.711554 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_TERM_TABLE_ENTRY_ATTR_TUNNEL_TYPE: SAI_TUNNEL_TYPE_IPINIP
Oct 2 16:20:43.711579 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_TERM_TABLE_ENTRY_ATTR_ACTION_TUNNEL_ID: oid:0x2a00000000055a
Oct 2 16:20:43.711604 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_TERM_TABLE_ENTRY_ATTR_DST_IP: 10.0.0.0
Oct 2 16:20:43.711627 sonic ERR syncd#syncd: :- processQuadEvent: failed to execute api: create, key: SAI_OBJECT_TYPE_TUNNEL_TERM_TABLE_ENTRY:oid:0x2b00000000055b, status: SAI_STATUS_NOT_SUPPORTED
Oct 2 16:20:43.711663 sonic ERR syncd#syncd: :- run: Runtime error: :- processQuadEvent: failed to execute api: create, key: SAI_OBJECT_TYPE_TUNNEL_TERM_TABLE_ENTRY:oid:0x2b00000000055b, status: SAI_STATUS_NOT_SUPPORTED
Oct 2 16:20:43.711663 sonic NOTICE syncd#syncd: :- sendShutdownRequest: sending switch_shutdown_request notification to OA for switch: oid:0x21000000000000
Oct 2 16:20:43.711787 sonic NOTICE syncd#syncd: :- sendShutdownRequestAfterException: notification send successfull

drivshell>Attach
Attach
Attach: Unit 0 (BCM56980_B0): attached (current unit)

drivshell>bsv
bsv
BRCM SAI ver: [3.7.4.2], OCP SAI ver: [1.6.0], SDK ver: [6.5.16]

Syslog

syslog.log

[srideepDell]

Issue seen with 436 Image which has latest SDK and SA I versions

root@sonic:#
root@sonic:# bcmcmd "bsv"
bsv
BRCM SAI ver: [4.2.1.3], OCP SAI ver: [1.6.5], SDK ver: [sdk-6.5.19]
drivshell>
root@sonic:~#

SONiC Software Version: SONiC.master.436-d03de95e
Distribution: Debian 10.6
Kernel: 4.19.0-9-2-amd64
Build commit: d03de95
Build date: Wed Oct 7 17:20:56 UTC 2020
Built by: johnar@jenkins-worker-11

Platform: x86_64-dellemc_z9332f_d1508-r0
HwSKU: DellEMC-Z9332f-O32
ASIC: broadcom
Serial Number: TH0J5HDGCET009740036A00
Uptime: 05:03:35 up 2 min, 1 user, load average: 0.53, 0.49, 0.21

Oct 12 05:01:30.858087 sonic NOTICE swss#portsyncd: :- main: PortInitDone
Oct 12 05:01:30.889213 sonic WARNING syncd#syncd: [0] SAI_API_HOSTIF:_brcm_sai_set_cpu_queue_shaper:11846 Set CPU Queue 0 shaping: cir 600, cbs 307
Oct 12 05:01:30.911396 sonic WARNING syncd#syncd: [0] SAI_API_HOSTIF:_brcm_sai_set_cpu_queue_shaper:11846 Set CPU Queue 1 shaping: cir 6000, cbs 3072
Oct 12 05:01:30.911975 sonic ERR syncd#syncd: [0] SAI_API_TUNNEL:brcm_sai_create_tunnel_term_table_entry:1848 P2MP tunnel terminator not supported on this platform.
Oct 12 05:01:30.912012 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_TERM_TABLE_ENTRY_ATTR_VR_ID: oid:0x3000000000024
Oct 12 05:01:30.912040 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_TERM_TABLE_ENTRY_ATTR_TYPE: SAI_TUNNEL_TERM_TABLE_ENTRY_TYPE_P2MP
Oct 12 05:01:30.912068 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_TERM_TABLE_ENTRY_ATTR_TUNNEL_TYPE: SAI_TUNNEL_TYPE_IPINIP
Oct 12 05:01:30.912095 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_TERM_TABLE_ENTRY_ATTR_ACTION_TUNNEL_ID: oid:0x2a00000000054e
Oct 12 05:01:30.912122 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_TERM_TABLE_ENTRY_ATTR_DST_IP: 10.0.0.0
Oct 12 05:01:30.912169 sonic ERR syncd#syncd: :- processQuadEvent: failed to execute api: create, key: SAI_OBJECT_TYPE_TUNNEL_TERM_TABLE_ENTRY:oid:0x2b00000000054f, status: SAI_STATUS_NOT_SUPPORTED
Oct 12 05:01:30.912223 sonic ERR syncd#syncd: :- run: Runtime error: :- processQuadEvent: failed to execute api: create, key: SAI_OBJECT_TYPE_TUNNEL_TERM_TABLE_ENTRY:oid:0x2b00000000054f, status: SAI_STATUS_NOT_SUPPORTED
Oct 12 05:01:30.912374 sonic NOTICE syncd#syncd: :- sendShutdownRequest: sending switch_shutdown_request notification to OA for switch: oid:0x21000000000000
Oct 12 05:01:30.913094 sonic NOTICE syncd#syncd: :- sendShutdownRequestAfterException: notification send successfull

[gechiang]

@srideepDell Please use an image that has the following PR in place to see if you still encounter this issue...
#8395