- Subnet Decapsulation with SONiC
| Rev | Date | Author | Change Description |
|---|---|---|---|
| 0.1 | 03/30/2024 | Longxiang Lyu | Initial version |
This document describes the subnet decapsulation feature on T0 SONiC that allows Netscan to probe VLAN subnet IP addresses.
| Term | Meaning |
|---|---|
| VLAN | virtual local area network |
| DIP | destination IP |
| SIP | source IP |
| Netscan | Azure service to detect network path failures |
In Azure, Netscan probes the network paths/devices by sending IPinIP traffic. The IPinIP packet crafted by the Netscan sender has the outer DIP equals the destination device Loopback address, and the inner DIP equals the IP address of the Netscan sender. When the IPinIP packet is routed to/received by the destination device, they will be decapsulated and the inner packet will be routed back to the Netscan sender. As of today, Netscan uses this IP-decap based probing to detect route blackholes in the Azure network. The limitation is that Netscan is only able to probe the networking switches without the capability to detect any route blackholes for host nodes, especially VLAN subnet IPs. Due to the fact that the host nodes don’t have native IP-decap functionality, it is more appropriate to implement the IP-decap functionality on T0 SONiC as SONiC supports IPinIP decapsulation, and T0 SONiC will respond to the Netscan probes on behalf of the host nodes to decapsulate the Netscan IPinIP probe packets with DIP as any VLAN subnet IPs. In this design, subnet decap is introduced to enhance SONiC with the capability to generate the decap rules for the VLAN subnet so IPinIP packets from Netscan with DIP as either VLAN subnet IPs could be decapsulated and forwarded back to the Netscan sender to allow Netscan to have the awareness of any possible route blackholes to those destinations.
High level requirements:
- T0 SONiC shall allow Netscan to probe with IPinIP packets with DIP as any local VLAN subnet IP by adding IP decap rules for the VLAN subnet.
| Component | Expected Value |
|---|---|
| Tunnel | N/A |
| Tunnel Decap Terms | N/A |
To support the Netscan probing over VLAN subnet, SONiC needs to generate decapsulation rules to decapsulate IPinIP packets with DIP as any IP address in VLAN subnet. The decapsulation rules will be generated based on the configured VLAN subnet. In this design, we propose the subnet decap feature that has workflow to enable SONiC to add/remove those decapsulation rules based on the configured VLAN subnets on the T0 SONiC.
The tunnels in this design will be generated with the following attributes:
| Attribute | Value | Note |
|---|---|---|
| name | IPINIP_VLAN or IPINIP_V6_VLAN | One IPv4 tunnel, one IPv6 tunnel |
| tunnel type | IPinIP | |
| decap ECN mode | copy_from_outer or standard | |
| decap TTL mode | pipe | |
| decap DSCP mode | uniform |
The decapsulation termination entry will be created with the following attributes:
| Attribute | Value | Note |
|---|---|---|
| term entry type | MP2MP | multi-point to multi-point |
| dest IP | the VLAN subnet | |
| dest IP mask | the vlan subnet mask | |
| source IP | Netscan privately-owned subnet | IPinIP packets that have source IP in the subnet can be safely assumed to be Netscan traffic instead of customer traffic |
| source IP mask | Netscan privately-owned subnet mask |
### SUBNET_DECAP
; Stores subnet based decapsulation configurations
key = SUBNET_DECAP|subnet_type ; SUBNET_DECAP|vlan
status = "enable"/"disable" ; status of subnet based decapsulation
src_ip = source IP prefix ; source IP prefix used for decap terms of IPv4 tunnel
src_ip_v6 = source IP prefix ; source IPv6 prefix used for decap terms of IPv6 tunnel
### TUNNEL_DECAP_TABLE
; Stores a list of decap tunnels
key = TUNNEL_DECAP_TABLE:tunnel_name ; tunnel name as key
tunnel_type = "IPINIP" ; tunnel type
dscp_mode = "uniform"/"pipe"
ecn_mode = "copy_from_outer"/"standard"
ttl_mode = "uniform"/"pipe"
encap_ecn_mode = "standard"
### TUNNEL_DECAP_TERM_TABLE
; Stores a list of decap terms.
key = TUNNEL_DECAP_TERM_TABLE:tunnel_name:dst_ip ; tunnel name:dst IP prefix as key
term_type = "P2P"/"P2MP"/"MP2MP" ; tunnel decap term type
src_ip = source IP prefix
### TUNNEL_DECAP_TABLE
; Stores a list of created decap tunnels
key = TUNNEL_DECAP_TABLE:tunnel_name ; tunnel name as key
tunnel_type = "IPINIP" ; tunnel type
dscp_mode = "uniform"/"pipe"
ecn_mode = "copy_from_outer"/"standard"
ttl_mode = "uniform"/"pipe"
encap_ecn_mode = "standard"
### TUNNEL_DECAP_TERM_TABLE
; Stores a list of created decap terms.
key = TUNNEL_DECAP_TERM_TABLE:tunnel_name:dst_ip ; tunnel name:dst IP prefix as key
term_type = "P2P"/"P2MP"/"MP2MP" ; tunnel decap term type
src_ip = source IP prefix
The following orchestration agents shall be modified: TunnelDecapOrch:
-
TunnelDecapOrch shall subscribe to TUNNEL_DECAP_TABLE and create/remove the decap tunnels.
-
TunnelDecapOrch shall subscribe to TUNNEL_DECAP_TERM_TABLE and create/remove the decap term entries.
-
TunnelDecapOrch shall subscribe to SUBNET_DECAP and handle the tunnel decapsulation termination entry source IP changes.
The VLAN subnet decap workflow is presented in the above figure. The extra tunnels and decap rules are templated-out and pushed to APPL_DB by swssconfig service if VLAN subnet decap is enabled. TunnelDecapOrch subscribes to both TUNNEL_DECAP_TABLE and TUNNEL_DECAP_TERM_TABLE and processes the request to program the decap rules to SYNCD accordingly.
For Dual-ToR, both ToRs are configured with the same VLAN setup, so they all have the same decap rules. As the downstream traffic from T1s are ECMPed to either ToR, the IPinIP packets to VLAN subnet IPs from Netscan received by either ToR could be decapsulated and forwarded back to the Netscan sender.
TunnelDecapOrch subscribes to the SUBNET_DECAP table and reacts to subnet decap configuration change. Currently, only source IP and source IPv6 prefix change are supported and TunnelDecapOrch will change the decapsulation termination entry source IP according to the configuration change.
The following picture describes the workflow:
TBD
TBD
TBD