diff --git a/docs/ACL-test-plan.md b/docs/ACL-test-plan.md new file mode 100644 index 0000000000..68329a2871 --- /dev/null +++ b/docs/ACL-test-plan.md @@ -0,0 +1,426 @@ +## [DRAFT, UNDER DEVELOPMENT] + + +- [Overview](#overview) + - [Scope](#scope) + - [Related **DUT** CLI commands](#related-dut-cli-commands) +- [Setup configuration](#setup-configuration) + - [Scripts for generating ACL configuration on SONIC](#Scripts-for-generating-ACL-configuration-on-SONIC) + - [Ansible scripts to setup and run ACL test](#Ansible scripts to setup and run ACL test) + - [acl_testbed.yml](#acl-testbed-yml) + - [Setup of DUT switch](#Setup-of-DUT-switch) + - [J2 templates](#j2-templates) +- [PTF Test](#ptf-test) + - [Input files for PTF test](#input-files-for-ptf-test) + - [Traffic validation in PTF](#traffic-validation-in-ptf) +- [Test cases](#test-cases) +- [TODO](#todo) +- [Open Questions](#open-questions) + +## Overview +The purpose is to test functionality of ACL rules on the SONIC switch DUT with and without LAGs configured, closely resembling production environment. +The test assumes all necessary configuration, including ACL and LAG configuration, BGP routes, are already pre-configured on the SONIC switch before test runs. + +### Scope +The test is targeting a running SONIC system with fully functioning configuration. +The purpose of the test is not to test specific SAI API, but functional testing of ACL on SONIC system, making sure that traffic flows correctly, according to BGP routes advertised by BGP peers of SONIC switch, and the LAG configuration. + +NOTE: ACL+LAG test will be able to run **only** in the testbed specifically created for LAG. + +### Related **DUT** CLI commands +Manual ACL configuration can be done using swssconfig utility in swss container. + + swssconfig + +## Test structure +### Setup configuration +ACL configuration should be created on the DUT before running the test. Configuration could be deployed using ansible sonic test playbook with the tag acltb_configure. + +#### Scripts for generating ACL configuration on SONIC + +There will be two j2 template files for the ACL test configuration: acltb_test_table.j2 and acltb_test_rules.j2. They will be used by Ansible playbook to generate json files and apply them on the switch. + +#### Ansible scripts to setup and run ACL test + +##### acl_testbed.yml + +acl_testbed.yml when run with ***different tags*** will + +Tag ***acltb_configure*** will generate acl json files for the ACL test out of the corresponding j2 files and apply them on the switch. +Tag ***acltb_test*** will run ACL test (or ACL+LAG) test. +Tag ***acltb_cleanup*** will clear the test ACL configuration from the switch. + +ACL test consists of a number of subtests, and each of them will include the following steps: + +1. Run lognanalyzer 'init' phase +2. Run ACL Sub Test +3. Run loganalyzer 'analyze' phase + +ACL subtests will be implemented in the PTF (acl_testbed_test.py). Every subtest wibb be implemented in a separate class. + +#### Setup of DUT switch +Setup of SONIC DUT will be done by Ansible script. During setup Ansible will copy json file containing configuration for ACL to the swss container on the DUT. swssconfig utility will be used to push configuration to the SONiC DB. Data will be consumed by orchagent. + +JSON Sample: + +table.json + + [ + { + "ACL_TABLE:Drop_IP": { + "policy_desc" : "Drop_IP_Traffic", + "type" : "L3", + "ports" : "Ethernet0" + }, + "OP": "SET" + } + ] + +rule.json + + [ + { + "ACL_RULE_TABLE:Drop_IP:TheDrop": { + "priority" : "55", + "ETHER_TYPE" : "0x0800", + "PACKET_ACTION" : "DROP" + }, + "OP": "SET" + } + ] + +**NOTE** +Tables and rules configuration will reside in two different jsons and table configuration will be applied before rules to ensure correct objects creation order in SAI. + +##### J2 templates +acltb_test_table.j2 will configure single table bound to all switch ports. + + [ + { + "ACL_TABLE:ACL_Testbed_Test_Table": { + "policy_desc" : "Thistable_contains_rules_needed_for_the_testbed_regression_tests", + "type" : "L3", + "ports" : "{% list_of_ingress_ports %}", + }, + "OP": "SET" + } + ] + +acltb_test_rules.j2 will contain ACL rules needed for the test + +ACL Rules: + +**RulesN..N+1000-:** Any rules, action: forward (placeholder) +**Rule#1:** match src ip 10.0.0.2, action: drop +**Rule#2:** match dst ip TBD, action: drop +**Rule#3:** match l4_src_port 0x1235, action: drop +**Rule#4:** match l4_dst_port 0x1235, action: drop +**Rule#5:** match ether type 0x1234, action: forward +**Rule#6:** match ip protocol 0x7E, action: drop +**Rule#7:** match tcp flags 0xFF/RST, action: drop +**Rule#8:** match ip type TBD, action: drop +**Rules#9.1-9.8:** match source ports range [[0x1240..0x1249], [0x1250..0x1259]...], action: drop (8 rules with different port ranges) +**Rule#10.1-10.8:** match destination ports range [[0x1240..0x1249], [0x1250..0x1259]...], action: drop (8 rules with different port ranges) +**Rules#11-12:** check priority: match some src ip 10.0.0.3, action: drop + match src ip 10.0.0.3 (higher prio), action: forward + +/if needed additionally match src ip/ + +## PTF Test + +### Input files for PTF test + +PTF test will generate traffic between ports and make sure it passes according to the configured ACL rules. Depending on the testbed topology and the existing configuration (e.g. ECMP, LAGS, etc) packets may arrive to different ports. Therefore ports connection information will be generated from the minigraph and supplied to the PTF script. + +### Traffic validation in PTF +Depending on the test PTF test will verify the packet arrived or dropped. + +## Test cases + +Each test case will be additionally validated by the loganalizer and counters reading utility. + +### Generic packet +Packet with the values below should not trigger any "drop" rule. +
+###[ Ethernet ]###
+  dst = [auto]
+  src = [auto]
+  type = 0x800
+###[ IP ]###
+    version = 4  
+    ttl =   
+    proto = tcp  
+    chksum = None  
+    src = 10.0.0.1  
+    dst = [get_from_route_info]
+###[ TCP ]###  
+    sport = 4660 (0x1234)  
+    dport = http (80)  
+    flags = S  
+
+ +### Test case \#0 - Resources consuming test + +#### Test objective + +Verify whether ACL engine resources are being freed on rule/range/counter/table delete. + +#### Test steps + +- Clear ACL configuration. +- Reapply ACL configuration. +- Verify there are no errors in the log + +### Test case \#1 - Verify source IP match + +#### Test objective + +Verify match source IP address works. + +#### Packet to trigger the rule #1 +
+...
+###[ IP ]###
+    version = 4  
+    ttl =   
+    proto = tcp  
+    chksum = None  
+    src = 10.0.0.2
+    dst = [get_from_route_info]
+...
+
+ +#### Test steps + +- PTF host will send packet specifying particular source IP address in the packet. +- When packet reaches SONIC DUT, it should be dropped by the rule #1. PTF docker should not receive this packet. +- Counter for the rule #1 should increment + + +### Test case \#2 - Verify destination IP match + +#### Test objective + +Verify match destination IP address works. + +#### Packet to trigger the rule #2 +
+...
+###[ IP ]###
+    version = 4  
+    ttl =   
+    proto = tcp  
+    chksum = None  
+    src = 10.0.0.1
+    dst = [get_from_route_info]
+...
+
+ +#### Test steps + +- PTF host will send packet specifying particular destination IP address in the packet. +- When packet reaches SONIC DUT, it should be dropped by the rule #2. PTF docker should not receive this packet. +- Counter for the rule #2 should increment + + +### Test case \#3 - Verify L4 source port match + +#### Test objective + +Verify match L4 source port works. + +#### Packet to trigger the rule #3 +
+...
+###[ TCP ]###  
+    sport = 4661 (0x1235)
+    dport = 80
+    flags = S
+...
+
+ +#### Test steps + +- PTF host will send packet with the specific L4 source port in the packet. +- When packet reaches SONIC DUT, it should be dropped by the rule #3. PTF docker should not receive this packet. +- Counter for the rule #3 should increment + +### Test case \#4 - Verify L4 destination port match + +#### Test objective + +Verify match L4 source port works. + +#### Packet to trigger the rule #4 +
+...
+###[ TCP ]###  
+    sport = 4660 (0x1234)
+    dport = 4661 (0x1235)
+    flags = S
+...
+
+ +#### Test steps + +- PTF host will send packet with the specific L4 destination port in the packet. +- When packet reaches SONIC DUT, it should be dropped by the rule #4. PTF docker should not receive this packet. +- Counter for the rule #4 should increment + +### Test case \#5 - Verify ether type match + +#### Test objective + +Verify match packet ether type works. + +#### Packet to trigger the rule #5 +
+###[ Ethernet ]###
+  dst = [auto]
+  src = [auto]
+  type = 0x1234
+...
+
+ +#### Test steps + +- PTF host will send packet with the specific ether type in the packet. +- When packet reaches SONIC DUT, it should be dropped because non-IP ethertype. But will be forwarded by the rule #5. PTF docker should receive this packet. +- Counter for the rule #5 should increment + +***NOTE*** Ether type used in this test should be "exotic" enough to exclude possible interference with the other tests traffic. + +### Test case \#6 - Verify ip protocol match + +#### Test objective + +Verify match ip protocol works. + +#### Packet to trigger the rule #6 +
+...
+###[ IP ]###
+    version = 4  
+    ttl =   
+    proto = 0x7E
+    chksum = None  
+    src = 10.0.0.1  
+    dst = [get_from_route_info]
+
+... +#### Test steps + +- PTF host will send packet with the specific ip protocol field in the packet. +- When packet reaches SONIC DUT, it should be dropped by the rule #6. PTF docker should not receive this packet. +- Counter for the rule #6 should increment + +***NOTE*** IP protocol used in this test should be "exotic" enough to exclude possible interference with the other tests traffic. For example 0x7E (Combat Radio Transport Protocol) + +### Test case \#7 - Verify TCP flags match + +#### Test objective + +Verify match TCP flags works. + +#### Packet to trigger the rule #7 +
+...
+###[ TCP ]###  
+    sport = 4660 (0x1234)
+    dport = 80
+    flags = RS
+...
+
+#### Test steps + +- PTF host will send TCP packet with the specific flags in the packet. +- When packet reaches SONIC DUT, it should be dropped by the rule #7. PTF docker should not receive this packet. +- Counter for the rule #7 should increment. + +### Test case \#8 - Verify ip type match + +#### Test objective + +Verify match ip protocol works. + +#### Test steps + +- PTF host will send packet with the specific ip protocol field in the packet. +- When packet reaches SONIC DUT, it should be dropped by the rule #8. PTF docker should not receive this packet. +- Counter for the rule #8 should increment + +***TODO*** Think about IP protocol to use for the test. Maybe add another match criteria (source ip?) + +### Test case \#9 - Verify source port range match + +#### Test objective + +Verify match source port range works. + +#### Packet to trigger the rule #9 +
+...
+###[ TCP ]###  
+    sport = 0x1236..0x1240
+    dport = 80
+    flags = S
+...
+
+#### Test steps + +- PTF host will send TCP packet with the specific source port in the packet. +- When packet reaches SONIC DUT, it should be dropped by the rule #9. PTF docker should not receive this packet. +- Counter for the rule #9 should increment + +### Test case \#10 - Verify destination port range match + +#### Test objective + +Verify match destination port range works. + +#### Packet to trigger the rule #9 +
+...
+###[ TCP ]###  
+    sport = 0x1234
+    dport = 0x1236..0x1240
+    flags = S
+...
+
+#### Test steps + +- PTF host will send TCP packet with the specific destination port in the packet. +- When packet reaches SONIC DUT, it should be dropped by the rule #10. PTF docker should not receive this packet. +- Counter for the rule #10 should increment + +### Test case \#11 - Verify rules priority + +#### Test objective + +Verify rules priority works. + +#### Test steps + +- PTF host will send TCP packet with the specific source ip in the packet. +- When packet reaches SONIC DUT, it will not be dropped by the rule #11 because rule #12 with the same matching criteria allows packet to pass. +- PTF docker verefies packet arrived. +- Counter for the rule #12 should increment + +### Test case \#12 - False rule triggering check + +#### Test objective + +Verify rules are not triggered by mistake. +This test should be executed the last. + +#### Test steps + +- Send several "Generic packets" +- Verify all rules counters value is equal to number of packets used for each rule in all tests. + +### Other possible tests +- match combinations + +## TODO +- ACL+LAG test configuration and testcases (separate ansible tag) + +## Open Questions diff --git a/docs/BGP-GR-helper-mode-test-plan.md b/docs/BGP-GR-helper-mode-test-plan.md new file mode 100644 index 0000000000..3945f13e1f --- /dev/null +++ b/docs/BGP-GR-helper-mode-test-plan.md @@ -0,0 +1,90 @@ +- [Overview](#overview) + - [Scope](#scope) + - [Testbed](#testbed) +- [Setup configuration](#setup-configuration) + - [Arista VM configuration](#arista-vm-configuration) + - [Ansible scripts to setup and run test](#ansible-scripts-to-setup-and-run-test) + - [everflow_testbed.yml](#everflow-testbed-yml) +- [PTF Test](#ptf-test) + - [Input files for PTF test](#input-files-for-ptf-test) + - [Traffic validation in PTF](#traffic-validation-in-ptf) +- [Test cases](#test-cases) +- [TODO](#todo) +- [Open Questions](#open-questions) + +## Overview +The purpose is to test a functionality of BGP GR mode on the SONIC switch DUT, closely resembling production environment. +The test assumes all necessary configuration is already pre-configured on the SONIC switch before test runs. + +### Scope +The test is targeting a running SONIC system with fully functioning configuration. +The purpose of the test is not to test specific API, but functional testing of BGP GR helper mode on SONIC system, making sure that traffic flows correctly, according to BGP routes advertised by BGP peers of SONIC switch. + +### Testbed +The test will run on the following testbeds: +- t1 +- t1-lag + +## Setup configuration + +#### Arista VM configuration + +Test assumes that BGP GR is enabled and preconfigured on Arista VMs. BGP GR timer value should be more than time required for VM reboot. + +#### Ansible scripts to setup and run test + +##### bgp_gr_helper.yml + +bgp_gr_helper.yml when run with tag "bgp_gr_helper" will do the following: + +1. Randomly choose VM. +2. Run test. + +BGP GR helper test consists of a number of subtests, and each of them will include the following steps: + +1. Run lognanalyzer 'init' phase +2. Run BGP GR helper Sub Test +3. Run loganalyzer 'analyze' phase + +## PTF Test + +To run traffic FIB PTF test will be reused. + +## Test cases + +Each test case will be additionally validated by the loganalizer utility. + +### Test case \#1 - BGP GR helper mode. + +#### Test objective + +Verify that routes are preserved during neighbor graceful restart. + +#### Test steps + +- Randomly choose VM for the test. +- Reboot VM. +- Verify BGP timeout (at least 115 seconds routes should stay in fib). +- Verify all routes are preserved (no reinstallation after BGP open message from the neighbor). +- Verify that BGP session with the VM established. + +### Test case \#2 - BGP GR helper mode routes change. + +#### Test objective + +Verify that traffic run without changes during neighbor graceful restart. + +#### Test steps + +- Randomly choose VM for the test. +- Change VM startup config (advertised routes should be different). +- Reboot VM. +- Verify that preserved routes are removed when VM back. +- Verify that new routes are installed when VM back. +- Restore VM startup config. + +## TODO + +## Open Questions +- Should tests run for neighbors behind physical interfaces only or behind LAGs as well? +- On which topologies test should run? diff --git a/docs/BGP-MP-test-plan.md b/docs/BGP-MP-test-plan.md new file mode 100644 index 0000000000..4003ff0db6 --- /dev/null +++ b/docs/BGP-MP-test-plan.md @@ -0,0 +1,55 @@ +# BGP-MP test plan + +* [Overview](#Overview) + * [Scope](#Scope) + * [Testbed](#Testbed) +* [Setup configuration](#Setup%20configuration) + * [Ansible scripts to setup and run test](#Ansible%20scripts%20to%20setup%20and%20run%20test) + * [bgp_mp.yml](#bgp_mp.yml) + * [Setup of DUT switch](#Setup%20of%20DUT%20switch) +* [Test](#Test) +* [Test cases](#Test%20cases) +* [TODO](#TODO) +* [Open questions](#Open%20questions) + +## Overview +The purpose is to test functionality of BGP-MP on the SONIC switch DUT, closely resembling production environment. The test assumes all necessary configurations are already pre-configured on the SONIC switch before test runs. + +### Scope +The test is targeting a running SONIC system with fully functioning configuration. The purpose of the test is not to test specific API, but functional testing of BGP-MP on SONIC system. + +### Testbed +The test will run on the following testbeds: +* t0 + +## Setup configuration +IPv4 BGP neighborship will be configured between DUT and exabgp and each neighbor will redistribute IPv6 routes to each other. +### Ansible scripts to setup and run test +#### bgp_mp.yml +bgp_mp.yml when run with tag “bgp_mp” will do the following: +1. Generate and apply exabgp configuration. +2. Run test. +3. Clean up dynamic and temporary exabgp configuration. + +## Test +On PTF host, exabgp tool will be used to configure bgp peer and redistribute IPv6 routes via IPv4 BGP session. + +## Test cases +### Test case # 1 – BGP-MP IPv6 routes over IPv4 session +#### Test objective +Verify that IPv6 routes are correctly redistributed over IPv4 BGP session. +#### Test steps +* Generate IPv4 BGP peer configuration for exabgp instance. +* Generate IPv6 routes, to be announced via IPv4 session, for exabgp instance. +* Run exabgp instance. +* Verify that IPv4 BGP neighborship is established. +* Redistribute IPv6 routes using exabgp. +* Verify that IPv6 routes are correctly redistributed to the DUT. +* Redistribute IPv6 routes from the DUT to exabgp. +* Verify that IPv6 routes are correctly redistributed to the exabgp. +* Set default configuration. + +## TODO + +## Open questions +* Should be some traffic test cases performed as part of this test? \ No newline at end of file diff --git a/docs/CRM-test-plan.md b/docs/CRM-test-plan.md new file mode 100644 index 0000000000..2fc9d9080f --- /dev/null +++ b/docs/CRM-test-plan.md @@ -0,0 +1,221 @@ +# CRM test plan + +* [Overview](#Overview) + * [Scope](#Scope) + * [Testbed](#Testbed) +* [Setup configuration](#Setup%20configuration) + * [Ansible scripts to setup and run test](#Ansible%20scripts%20to%20setup%20and%20run%20test) + * [crm.yml](#crm.yml) +* [Test](#Test) +* [Test cases](#Test%20cases) +* [TODO](#TODO) +* [Open questions](#Open%20questions) + +## Overview +The purpose is to test functionality of CRM on the SONIC switch DUT, closely resembling production environment. + +### Scope +The test is targeting a running SONIC system with fully functioning configuration. The purpose of the test is not to test specific API, but functional testing of CRM on SONIC system. + +### Testbed +The test will run on the all testbeds. + +## Setup configuration +No setup pre-configuration is required, test will configure and clean-up all the configuration. +### Ansible scripts to setup and run test +#### crm.yml +crm.yml when run with tag “crm” will do the following for each CRM resource: +1. Apply required configuration. +2. Verify "used" and "free" counters. +3. Verify "EXCEEDED" and "CLEAR" messages using all types of thresholds. +4. Restore configuration. + +## Test + +## Test cases + +### Test case # 1 – IPv4 route +#### Test objective +Verify "IPv4 route" CRM resource. +#### Test steps +* Set polling interval to 1 minute. +* Configure 1 route and observe that counters were updated as expected. +* Remove 1 route and observe that counters were updated as expected. +* Perform the following steps for all threshold types ("percentage", "used", "free"): + * Set low and high thresholds according to current usage and type. + * Verify that "EXCEEDED" message is logged (using log analyzer). + * Set low and high thresholds to default values. + * Verify that "CLEAR" message is logged (using log analyzer). +* Restore default configuration. + +### Test case # 2 – IPv6 route +#### Test objective +Verify "IPv6 route" CRM resource. +#### Test steps +* Set polling interval to 1 minute. +* Configure 1 route and observe that counters were updated as expected. +* Remove 1 route and observe that counters were updated as expected. +* Perform the following steps for all threshold types ("percentage", "used", "free"): + * Set low and high thresholds according to current usage and type. + * Verify that "EXCEEDED" message is logged (using log analyzer). + * Set low and high thresholds to default values. + * Verify that "CLEAR" message is logged (using log analyzer). +* Restore default configuration. + +### Test case # 3 – IPv4 nexthop +#### Test objective +Verify "IPv4 nexthop" CRM resource. +#### Test steps +* Set polling interval to 1 minute. +* Add 1 nexthop and observe that counters were updated as expected. +* Remove 1 nexthop and observe that counters were updated as expected. +* Perform the following steps for all threshold types ("percentage", "used", "free"): + * Set low and high thresholds according to current usage and type. + * Verify that "EXCEEDED" message is logged (using log analyzer). + * Set low and high thresholds to default values. + * Verify that "CLEAR" message is logged (using log analyzer). +* Restore default configuration. + +### Test case # 4 – IPv6 nexthop +#### Test objective +Verify "IPv6 nexthop" CRM resource. +#### Test steps +* Set polling interval to 1 minute. +* Add 1 nexthop and observe that counters were updated as expected. +* Remove 1 nexthop and observe that counters were updated as expected. +* Perform the following steps for all threshold types ("percentage", "used", "free"): + * Set low and high thresholds according to current usage and type. + * Verify that "EXCEEDED" message is logged (using log analyzer). + * Set low and high thresholds to default values. + * Verify that "CLEAR" message is logged (using log analyzer). +* Restore default configuration. + +### Test case # 5 – IPv4 neighbor +#### Test objective +Verify "IPv4 neighbor" CRM resource. +#### Test steps +* Set polling interval to 1 minute. +* Configure 1 neighbor and observe that counters were updated as expected. +* Remove 1 neighbor and observe that counters were updated as expected. +* Perform the following steps for all threshold types ("percentage", "used", "free"): + * Set low and high thresholds according to current usage and type. + * Verify that "EXCEEDED" message is logged (using log analyzer). + * Set low and high thresholds to default values. + * Verify that "CLEAR" message is logged (using log analyzer). +* Restore default configuration. + +### Test case # 6 – IPv6 neighbor +#### Test objective +Verify "IPv6 neighbor" CRM resource. +#### Test steps +* Set polling interval to 1 minute. +* Configure 1 neighbor and observe that counters were updated as expected. +* Remove 1 neighbor and observe that counters were updated as expected. +* Perform the following steps for all threshold types ("percentage", "used", "free"): + * Set low and high thresholds according to current usage and type. + * Verify that "EXCEEDED" message is logged (using log analyzer). + * Set low and high thresholds to default values. + * Verify that "CLEAR" message is logged (using log analyzer). +* Restore default configuration. + +### Test case # 7 – Nexthop group object +#### Test objective +Verify "nexthop group object" CRM resource. +#### Test steps +* Set polling interval to 1 minute. +* Configure 1 ECMP route and observe that counters were updated as expected. +* Remove 1 ECMP route and observe that counters were updated as expected. +* Perform the following steps for all threshold types ("percentage", "used", "free"): + * Set low and high thresholds according to current usage and type. + * Verify that "EXCEEDED" message is logged (using log analyzer). + * Set low and high thresholds to default values. + * Verify that "CLEAR" message is logged (using log analyzer). +* Restore default configuration. + +### Test case # 8 – Nexthop group member +#### Test objective +Verify "nexthop group member" CRM resource. +#### Test steps +* Set polling interval to 1 minute. +* Configure 1 ECMP route and observe that counters were updated as expected. +* Remove 1 ECMP route and observe that counters were updated as expected. +* Perform the following steps for all threshold types ("percentage", "used", "free"): + * Set low and high thresholds according to current usage and type. + * Verify that "EXCEEDED" message is logged (using log analyzer). + * Set low and high thresholds to default values. + * Verify that "CLEAR" message is logged (using log analyzer). +* Restore default configuration. + +### Test case # 9 – FDB entry +#### Test objective +Verify "FDB entry" CRM resource. +#### Test steps +* Set polling interval to 1 minute. +* Configure 1 FDB entry and observe that counters were updated as expected. +* Remove 1 FDB entry and observe that counters were updated as expected. +* Perform the following steps for all threshold types ("percentage", "used", "free"): + * Set low and high thresholds according to current usage and type. + * Verify that "EXCEEDED" message is logged (using log analyzer). + * Set low and high thresholds to default values. + * Verify that "CLEAR" message is logged (using log analyzer). +* Restore default configuration. + +### Test case # 10 – ACL group +#### Test objective +Verify "ACL group" CRM resource. +#### Test steps +* Set polling interval to 1 minute. +* Configure 1 ACL and observe that counters were updated as expected. +* Remove 1 ACL and observe that counters were updated as expected. +* Perform the following steps for all threshold types ("percentage", "used", "free"): + * Set low and high thresholds according to current usage and type. + * Verify that "EXCEEDED" message is logged (using log analyzer). + * Set low and high thresholds to default values. + * Verify that "CLEAR" message is logged (using log analyzer). +* Restore default configuration. + +### Test case # 11 – ACL table +#### Test objective +Verify "ACL table" CRM resource. +#### Test steps +* Set polling interval to 1 minute. +* Configure 1 ACL and observe that counters were updated as expected. +* Remove 1 ACL and observe that counters were updated as expected. +* Perform the following steps for all threshold types ("percentage", "used", "free"): + * Set low and high thresholds according to current usage and type. + * Verify that "EXCEEDED" message is logged (using log analyzer). + * Set low and high thresholds to default values. + * Verify that "CLEAR" message is logged (using log analyzer). +* Restore default configuration. + +### Test case # 12 – ACL entry +#### Test objective +Verify "ACL entry" CRM resource. +#### Test steps +* Set polling interval to 1 minute. +* Configure 1 ACL rule and observe that counters were updated as expected. +* Remove 1 ACL rule and observe that counters were updated as expected. +* Perform the following steps for all threshold types ("percentage", "used", "free"): + * Set low and high thresholds according to current usage and type. + * Verify that "EXCEEDED" message is logged (using log analyzer). + * Set low and high thresholds to default values. + * Verify that "CLEAR" message is logged (using log analyzer). +* Restore default configuration. + +### Test case # 13 – ACL counter +#### Test objective +Verify "ACL entry" CRM resource. +#### Test steps +* Set polling interval to 1 minute. +* Configure 1 ACL rule and observe that counters were updated as expected. +* Remove 1 ACL rule and observe that counters were updated as expected. +* Perform the following steps for all threshold types ("percentage", "used", "free"): + * Set low and high thresholds according to current usage and type. + * Verify that "EXCEEDED" message is logged (using log analyzer). + * Set low and high thresholds to default values. + * Verify that "CLEAR" message is logged (using log analyzer). +* Restore default configuration. + +## TODO + +## Open questions diff --git a/docs/Everflow-test-plan.md b/docs/Everflow-test-plan.md new file mode 100644 index 0000000000..16183ffeee --- /dev/null +++ b/docs/Everflow-test-plan.md @@ -0,0 +1,351 @@ +- [Overview](#overview) + - [Scope](#scope) + - [Related **DUT** CLI commands](#related-dut-cli-commands) +- [Setup configuration](#setup-configuration) + - [Scripts for generating configuration on SONIC](#scripts-for-generating-configuration-on-SONIC) + - [Ansible scripts to setup and run test](#ansible-scripts-to-setup-and-run-test) + - [everflow_testbed.yml](#everflow-testbed-yml) + - [Setup of DUT switch](#Setup-of-DUT-switch) + - [J2 templates](#j2-templates) +- [PTF Test](#ptf-test) + - [Input files for PTF test](#input-files-for-ptf-test) + - [Traffic validation in PTF](#traffic-validation-in-ptf) +- [Test cases](#test-cases) +- [TODO](#todo) +- [Open Questions](#open-questions) + +##Overview +The purpose is to test functionality of Everflow on the SONIC switch DUT with and without LAGs configured, closely resembling production environment. +The test assumes all necessary configuration, including Everflow session and ACL rules, LAG configuration and BGP routes, are already pre-configured on the SONIC switch before test runs. + +###Scope +The test is targeting a running SONIC system with fully functioning configuration. +The purpose of the test is not to test specific SAI API, but functional testing of Everflow on SONIC system, making sure that traffic flows correctly, according to BGP routes advertised by BGP peers of SONIC switch, and the LAG configuration. + +NOTE: Everflow+LAG test will be able to run **only** in the testbed specifically created for LAG. + +###Related **DUT** CLI commands +Manual Everflow configuration can be done using swssconfig utility in swss container. + + swssconfig + +##Test structure +###Setup configuration +Everflow configuration should be created on the DUT before running the test. Configuration could be deployed using ansible sonic test playbook with the tag **everflow_tb**. + +#### Scripts for generating configuration on SONIC + +There will be three j2 template files for the Everflow test configuration: everflow_tb_test_session.j2, everflow_tb_test_acl_table.j2 and everflow_tb_test_acl_rue.j2. They will be used by Ansible playbook to generate json files and apply them on the switch. + +#### Ansible scripts to setup and run test + +##### everflow_testbed.yml + +everflow_testbed.yml when run with tag "everflow_tb" will to the following: + +1. Generate JSON files and apply them on the switch. +2. Run test. +3. Clean up dynamic configuration and temporary configuration on the DUT. + +Everflow test consists of a number of subtests, and each of them will include the following steps: + +1. Run lognanalyzer 'init' phase +2. Run Everflow Sub Test +3. Run loganalyzer 'analyze' phase + +Everflow subtests will be implemented in the PTF (everflow_testbed_test.py). Every subtest will be implemented in a separate class. + +#### Setup of DUT switch +Setup of SONIC DUT will be done by Ansible script. During setup Ansible will copy JSON file containing configuration for Everflow to the swss container on the DUT. swssconfig utility will be used to push configuration to the SONiC DB. Data will be consumed by orchagent. + +JSON Sample: + +everflow_session.json +``` +[ + { + "MIRROR_SESSION_TABLE:session_1": { + "src_ip": "1.1.1.1", + "dst_ip": "2.2.2.2", + "gre_type": "0x6558", + "dscp": "8", + "ttl": "64", + "queue": "0" + }, + "OP": "SET" + } +] +``` + +everfow_acl_table.json +``` +[ + { + "ACL_TABLE:acl_table_mirror": { + "policy_desc" : "Everflow_ACL_table", + "type" : "MIRROR", + "ports" : "Ethernet0, Ethernet4, Ethernet8, Ethernet12, Ethernet16, Ethernet20, Ethernet24, Ethernet28, Ethernet32, Ethernet36, Ethernet40, Ethernet44, Ethernet48, Ethernet52, Ethernet56, Ethernet60, Ethernet64, Ethernet68, Ethernet72, Ethernet76, Ethernet80, Ethernet84, Ethernet88, Ethernet92, Ethernet96, Ethernet100, Ethernet104, Ethernet108, Ethernet112, Ethernet116, Ethernet120, Ethernet124, Ethernet128" + }, + "OP": "SET" + } +] +``` +everflow_acl_rule_persistent.json +``` +[ + { + "ACL_RULE_TABLE:acl_table_mirror:Rule01": { + "policy_desc" : "Mirror_packet_with_tcp_flag_fin", + "priority" : "50", + "tcp_flags" : "0x01/0xff", + "mirror_action" : "session_1" + }, + "OP": "SET" + }, + { + "ACL_RULE_TABLE:acl_table_mirror:Rule02": { + "policy_desc" : "Mirror_packet_with_tcp_flag_syn_and_dscp", + "priority" : "50", + "tcp_flags" : "0x02/0xff", + "dscp" : "1" + "mirror_action" : "session_1" + }, + "OP": "SET" + }, + { + "ACL_RULE_TABLE:acl_table_mirror:Rule03": { + "policy_desc" : "Mirror_packet_with_tcp_flag_rst", + "priority" : "50", + "tcp_flags" : "0x04/0xff", + "mirror_action" : "session_1" + }, + "OP": "SET" + }, + { + "ACL_RULE_TABLE:acl_table_mirror:Rule04": { + "policy_desc" : "Mirror_packet_with_specific_tcp_port", + "priority" : "50", + "ip_protocol" : "0x06", + "l4_src_port" : "1101", + "mirror_action" : "session_1" + }, + "OP": "SET" + }, + +] +``` +everflow_acl_rule_dynamic.json +``` +[ + { + "ACL_RULE_TABLE:acl_table_mirror:RuleDynamic01": { + "policy_desc" : "Mirror_packet_with_specific_src_ip", + "priority" : "50", + "src_ip" : "10.0.0.0/32", + "mirror_action" : "session_1" + }, + "OP": "SET" + }, + { + "ACL_RULE_TABLE:acl_table_mirror:RuleDynamic02": { + "policy_desc" : "Mirror_packet_with_specific_dst_ip", + "priority" : "50", + "dst_ip" : "10.0.0.5/32", + "mirror_action" : "session_1" + }, + "OP": "SET" + }, + { + "ACL_RULE_TABLE:acl_table_mirror:RuleDynamic03": { + "policy_desc" : "Mirror_packet_with_specific_src_and_dst_ip", + "priority" : "50", + "src_ip" : "10.0.0.0/32", + "dst_ip" : "10.0.0.5/32", + "mirror_action" : "session_1" + }, + "OP": "SET" + } +] +``` +##PTF Test + +### Input files for PTF test + +PTF test will generate traffic between ports and make sure it mirrored according to the configured Everflow session and ACL rules. Depending on the testbed topology and the existing configuration (e.g. ECMP, LAGS, etc) packets may arrive to different ports. Therefore ports connection information will be generated from the minigraph and supplied to the PTF script. + +### Traffic validation in PTF +Depending on the test PTF test will verify the packet arrived or dropped. + +##Test cases + +Each test case will be additionally validated by the loganalizer utility. + +Each test case will add dynamic Everflow ACL rules at the beginning and remove them at the end. + +Each test case will run traffic for persistent and dynamic Everflow ACL rules. + +Each test case will analyze Everflow packet header and payload (if mirrored packet is equal to original). + +### Test case \#1 - Resolved route + +#### Test objective + +Verify that session with resolved route has active state. + +#### Test steps + +- Create route that matches session destination IP with unresolved next hop. +- Resolve route next hop. +- Verify that session state in APP DB changed to active. +- Send packets that matches each Everflow ACL rule. +- Verify that packet mirrored to appropriate port. +- Analyze mirrored packet header. +- Verify that mirrored packet payload is equal to sent packet. +- Verify that counters value of each Everflow ACL rule is correct. + +### Test case \#2 - Longer prefix route with resolved next hop + +#### Test objective + +Verify that session destination port and MAC address are changed after best match route insertion. + +#### Test steps + +- Create route that matches session destination IP with unresolved next hop. +- Resolve route next hop. +- Verify that session state in APP DB changed to active. +- Send packets that matches each Everflow ACL rule. +- Verify that packets mirrored to appropriate port. +- Analyze mirrored packet header. +- Verify that mirrored packet payload is equal to sent packet. +- Create best match route that matches session destination IP with unresolved next hop. +- Send packets that matches each Everflow ACL rule. +- Verify that packets are mirrored to the same port. +- Resolve best match route next hop (neighbor should be on different port). +- Verify that session state in APP DB is active. +- Send packets that matches each Everflow ACL rule. +- Verify that packets are mirrored and destination port changed accordingly. + +### Test case \#3 - Remove longer prefix route. + +#### Test objective + +Verify that session destination port and MAC address are changed after best match route removal. + +#### Test steps + +- Create route that matches session destination IP with unresolved next hop. +- Resolve route next hop. +- Verify that session state in APP DB changed to active. +- Send packets that matches each Everflow ACL rule. +- Verify that packets mirrored to appropriate port. +- Analyze mirrored packet header. +- Verify that mirrored packet payload is equal to sent packet. +- Create best match route that matches session destination IP with unresolved next hop. +- Resolve best match route next hop (neighbor should be on different port). +- Send packets that matches each Everflow ACL rule. +- Verify that packets are mirrored and destination port changed accordingly. +- Remove best match route. +- Send packets that matches each Everflow ACL rule. +- Verify that packets are mirrored and destination port changed accordingly. + +### Test case \#4 - Change neighbor MAC address. + +#### Test objective + +Verify that session destination MAC address is changed after neighbor MAC address update. + +#### Test steps + +- Create route that matches session destination IP with unresolved next hop. +- Resolve route next hop. +- Verify that session state in APP DB changed to active. +- Send packets that matches each Everflow ACL rule. +- Verify that packets mirrored to appropriate port. +- Analyze mirrored packet header. +- Verify that mirrored packet payload is equal to sent packet. +- Change neighbor MAC address. +- Send packets that matches each Everflow ACL rule. +- Verify that DST MAC address in mirrored packet header is changed accordingly. + +### Test case \#5 - Resolved ECMP route. + +#### Test objective + +Verify that session with resolved ECMP route has active state. + +#### Test steps + +- Create ECMP route that matches session destination IP with two unresolved next hops. +- Resolve route next hops. +- Verify that session state in APP DB is changed to active. +- Send packets that matches each Everflow ACL rule. +- Verify that packets mirrored to appropriate port. +- Analyze mirrored packets header. +- Verify that mirrored packets payload is equal to sent packet. + +### Test case \#6 - ECMP route change (add next hop). + +#### Test objective + +Verify that insertion of additional next hop to ECMP group doesn't affects session DST MAC and port. + +#### Test steps + +- Create ECMP route that matches session destination IP with two unresolved next hops. +- Resolve route next hops. +- Verify that session state in APP DB is changed to active. +- Send packets that matches each Everflow ACL rule. +- Verify that packets mirrored to appropriate port. +- Analyze mirrored packet header. +- Verify that mirrored packet payload is equal to sent packet. +- Add resolved next hop to ECMP route. +- Send packets that matches each Everflow rule. +- Verify that packets are mirrored to the same port. +- Verify that mirrored packets have the same DST MAC. + +### Test case \#7 - ECMP route change (remove next hop used by session). + +#### Test objective + +Verify that removal of next hop that is not used by session doesn't cause DST port and MAC change. + +#### Test steps + +- Create ECMP route that matches session destination IP with two unresolved next hops. +- Resolve route next hops. +- Verify that session state in APP DB is changed to active. +- Send packets that matches each Everflow rule. +- Verify that packets mirrored to appropriate port. +- Analyze mirrored packet header. +- Verify that mirrored packets payload is equal to sent packets. +- Remove next hop that is not used by session. +- Send packets that matches each Everflow rule. +- Verify that packets are mirrored to the same port. +- Verify that mirrored packets have the same DST MAC. + +### Test case \#8 - ECMP route change (remove next hop not used by session). + +#### Test objective + +Verify that after removal of next hop that was used by session from ECMP route session state is active. + +#### Test steps + +- Create ECMP route that matches session destination IP with two unresolved next hops. +- Resolve route next hops. +- Verify that session state in APP DB is changed to active. +- Send packets that matches each Everflow ACL rule. +- Verify that packets mirrored to appropriate port. +- Analyze mirrored packet header. +- Verify that mirrored packets payload is equal to sent packets. +- Remove next hop that is used by session. +- Send packets that matches each Everflow ACL rule. +- Verify that packets are mirrored and destination port changed accordingly. + +### Other possible tests + +## TODO +- Everflow+LAG test configuration and testcases (separate ansible tag) + +## Open Questions \ No newline at end of file diff --git a/docs/IPv4-Decapsulation-test.md b/docs/IPv4-Decapsulation-test.md new file mode 100644 index 0000000000..fa1fc22448 --- /dev/null +++ b/docs/IPv4-Decapsulation-test.md @@ -0,0 +1,187 @@ +**Related documents** + +* [Tests readme](https://github.com/Azure/sonic-mgmt/blob/master/ansible/README.test.md) +* [Test system readme](https://github.com/Azure/sonic-mgmt/blob/master/ansible/README.testbed.md) +* [swss-schema](https://github.com/Azure/sonic-swss/blob/master/doc/swss-schema.md) + +##Overview +*** + +This test case is aimed at testing the DUTs ability to do de-capsulation of IP encapsulated packets, and verify that each decapsulated packet is with the right properties in each field, and forward with the corresponding underlay destination IP to the correct route. +The test assumes all routes and decapsulation are set prior by to test, so no configuration is required to be done by the test itself, and the test will correspond only to the right IPs that is configured in the test. + +The validation to the routes and the decapsulation is done by sending packets with the corresponding IPs both, in the overlay and underlay. If the test fails, it is the testing responsibility to provide all the debug information which can be retrieved in order to shade more light to the failure for offline debugging as much as possible. + +###Scope +--------- +The scope is this test plan is only the Ansible test, including the PTF test and the necessary configuration. + +###Scale / Performance +------------------- +N/A + + +###Related **DUT** CLI commands +---------------------------- +No CLI commands will be needed for this test. + +###Related DUT configuration files +----------------------------------- +The decap_conf.j2 will create JSON file that will configure the decap rule for the **Decap IP** + +###Related SAI APIs +---------------- +N/A + +### D.U.T Requirement +-------------------- +The D.U.T must connect with 32 different ports to the PTF docker. +And the test must run after BGP configuration so that the default route will be configured + +##Test structure +=============== + +###Setup configuration +------------------- +The setup tests assume to have single SONiC (DUT) connected to a switch connected to a server running 32 Arista VMs. + +There will be 32 BGP peers connected to the switch. The peers will advertise the default route and update the switch. + +PTF host needs to be connected to a port through which it will send packets to the switch and needs to have a connection via ports through which the switch will send forward received packet back to the host for validation. + +###Configuration scripts +--------------------- +The peers and SONIC (DUT) will be deployed by an Ansible script. As part of the deployment, the script will generate the routes and decapsulation commands. +The decapsulation rule will be generated by J2 script that will output JSON file with **Decap IP** that will configure through the SWSS config tool. +#### J2 script +
[
+         {
+                 "TUNNEL_DECAP_TABLE:NETBOUNCER" : {
+                         "tunnel_type":"IPINIP",
+                         "dst_ip":"{{ansible_lo['ipv4']['address']}}",
+                         "dscp_mode":"pipe",
+                         "ecn_mode":"copy_from_outer",
+                         "ttl_mode":"pipe"
+                 },
+                 "OP": "SET"
+         }
+]
+ +#### Example for SWSS JSON file output +
[
+ 	{
+ 		"TUNNEL_DECAP_TABLE:NETBOUNCER" : {
+ 			"tunnel_type":"IPINIP",
+ 			"dst_ip":"10.0.0.1",
+ 			"dscp_mode":"pipe",
+ 			"ecn_mode":"copy_from_outer",
+ 			"ttl_mode":"pipe"
+ 		},
+ 		"OP": "SET"
+ 	}
+]
+ + +Test cases +---------- + +The test assumes there is a mechanism to validate logs on the DUT where we should be able to analyze /var/log/syslog for error and warning messages related to the current test. +In case such messages are detected, the test is considered failed. + +The test should have standardized summary section in the following format: + +*TEST SUMMARY:* + +*TEST*: OK/FAIL + +*LOGS*: OK/FAIL + + + +### Test case \#1 + +#### Test objective + +The objective is to validate decapsulation ability and each route has been added to the switch and is functioning properly with the decapsulated packet. + +#### Test configurations + - IP decap IPv4 that will be taken from loopback IP: _**Decap IP** + - default IPv4 routes that will be configured through the BGP session as ECMP routes. + - unicase IPv4 routes that will configure throught the BGP session as TOR routes. + +#### Test description +1. The test will use **host IP** that fall into the default route and for the TOR routes. +2. The test will use different outer and inner TTL value combinations for different TTL modes + +TTL mode | ttl_outer | ttl_inner | Decap & forwarded? | Expected TTL +---------|-----------|-----------|-------------------|-------------- +pipe | 2 ~ 64 | 2 ~ 64 | Yes | ttl_inner - 1 +pipe | 64 | 2 | Yes | 1 +pipe | 1 | 64 | Yes | 63 +uniform | 2 ~ 64 | 2 ~ 64 | Yes | ttl_outer - 1 +uniform | 64 | 2 | Yes | 63 +uniform | 2 | 64 | Yes | 1 +3. From the PTF docker, craft and sent through all the ports a double encapsulated IP packets as follows: + * `outer IP header [S:2.2.2.0,D:`**Decap IP**`]` + * `inner IP header [S:1.1.1.1,D:`**host IP**`]` + * `ECN and DSCP will be set to values picked from allowed range in a round robin way, for the overlay and the underlay.` + * `ECN_outer != ECN_inner1` + * `DSCP_outer != DSCP_inner1` + * `TTL_outer != TTL_outer1` + * `all other fields will be PTF default for simple_tcp_packet function` +4. Verify the Sonic does not see the encapsulated packet. the IP-in-IP packet should not go to CPU, the packet should not be seen on the DUT. + +5. Confirm that the packet that comes back to PTF Docker decapsulated from one of the expected ports. and the L3 header fields will look like this : + * `IP header [S:1.1.1.1,D:`**host port0 IP**`]` + * `TTL = TTL_inner - 1 when ttl_mode=pipe` + * `TTL = TTL_outer - 1 when ttl_mode=uniform` + * `ECN = ECN_outer` + * `DSCP = DSCP_outer ` + +6. repeat steps 1-4 32 times, so each port will send 2 packets one for unicast route and one ecmp route. + +### Test case \#2 +####Test objective + +The objective is to validate decapsulation ability with triple encapsulated packet and each route has been added to the switch and is functioning properly with the decapsulated packet. + +#### Test configurations + - IP decap IPv4 that will be taken from loopback IP: **Decap IP** + - ECMP route IPv4 that will be configured through the BGP session as ECMP routes. + - unicase IPv4 routes that will configure throught the BGP session as TOR routes. + +####Test description +1. The test will use **host IP** that is one of the routes preconfigured in the BGP session (ECMP and Unicast). +2. The test will use different outer and inner TTL value combinations for different TTL modes + +TTL mode | ttl_outer | ttl_inner | Decap & forwarded? | Expected TTL +---------|-----------|-----------|-------------------|-------------- +pipe | 2 ~ 64 | 2 ~ 64 | Yes | ttl_inner - 1 +pipe | 64 | 2 | Yes | 1 +pipe | 1 | 64 | Yes | 63 +uniform | 2 ~ 64 | 2 ~ 64 | Yes | ttl_outer - 1 +uniform | 64 | 2 | Yes | 63 +uniform | 2 | 64 | Yes | 1 +3. From the PTF docker, craft and sent through all the ports a triple encapsulated IP packets as follows: + + * `outer IP header [S:2.2.2.0,D:`**Decap IP**`]` + * `inner1 IP header [S:1.1.1.1,D:`**host IP**`]` + * `inner2 IP header [S:4.4.4.4,D:4.4.4.3]` + * `ECN and DSCP will be set to values picked from allowed range in a round robin way, for the overlay and the underlay.` + * `ECN_outer != ECN_inner1` + * `DSCP_outer != DSCP_inner1` + * `TTL_outer != TTL_inner1` + * `all other fields will be PTF default for simple_tcp_packet function` + +4. Verify the Sonic does not see the encapsulated packet. the IP-in-IP packet should not go to CPU, the packet should not be seen on the DUT. + +5. Confirm that the packet that comes back to PTF Docker decapsulated from one of the expected ports. and the L3 header fields will look like this: + * outer IP headers + - `IPs [S:1.1.1.1,D:`**host IP**`]` + - `TTL = TTL_inner - 1 when ttl_mode=pipe` + - `TTL = TTL_outer - 1 when ttl_mode=uniform` + - `ECN = ECN_outer` + - `DSCP = DSCP_outer ` + * inner IP headers : PTF default for simple_tcp_packet function + +6. repeat steps 1-4 32 times, so each port will send 2 packets one for unicast route and one ecmp route. diff --git a/docs/QoS-configuration-in-Config-DB.-ECN-WRED-configuration-utility-test-plan.md b/docs/QoS-configuration-in-Config-DB.-ECN-WRED-configuration-utility-test-plan.md new file mode 100644 index 0000000000..483736cec1 --- /dev/null +++ b/docs/QoS-configuration-in-Config-DB.-ECN-WRED-configuration-utility-test-plan.md @@ -0,0 +1,137 @@ +## Overview +The purpose is to test the functionality of ECN Configuration on the SONIC Switch. +The test does not require any links on the Switch. + +Test cases cover initial ECN WRED configuration and + +## Related tools and CLI utilities +Utility to be used for QoS configuration (including ECN WRED) validation: `sonic-cfggen`. + + sonic-cfggen -j qos.json --print-config +Utility to test ECN WRED configuration update: `ecnconfig`. + + ecnconfig -p AZURE_LOSSY -gmax 516096 + +Utility to explore DB: `redis-cli`. + + redis-cli -n 4 keys * + +## Manual tests +To be performed after changes touching QoS ECN components: + +- Orchagent: qosorch.cpp, qosorch.h, orchdaemon.cpp +- sonic-cfggen utility +- Common table classes: ProducerTable, ConsumerTable +- sonic-py-swsssdk library +- first-time init script rc.load +- manipulations with hwsku +- adding configuration for new platforms/hwskus +- ecnconfig utility + +## Automated tests +To be performed on testbed or stand-alone switch in scope of regression tests. + +### Test cases +Test cases marked with (M) - manual - recommended for manual testing and (A) if it makes sense to automate the test and run it in regression. (MA) - suitable for both. + +#### Test case #1(M): Check configuration file(s) available on Switch +Build SONiC, install to Switch, boot and check if `qos.json` files in folders + + /usr/share/sonic/device/// +Check for related or updated platforms. + +#### Test case #2(M): Check QoS configuration reaches DB + +Ssh to Switch and check whether QoS configuration was added to the Config DB. + + redis-cli -n 4 keys \* | grep QUEUE + +Expected records like: + + 43) "QUEUE|Ethernet0,Ethernet4.... + +#### Test case #3(MA): Check configuration applied + +Ssh to Switch and check whether QoS configuration applied. + + redis-cli -n 1 keys \* | grep QUEUE + +Expected records like: + + 1034) "ASIC_STATE:SAI_OBJECT_TYPE_QUEUE:oid:0x150000000003bf" + +#### Test case #4(M): Ecnconfig utility test. List mode +Ssh to Switch and execute: + + ecnconfig -l + +Expected output: + + Profile: AZURE_LOSSLESS + -------------------- ------- + red_max_threshold 516096 + wred_green_enable true + ecn ecn_all + green_min_threshold 184320 + red_min_threshold 516096 + wred_yellow_enable true + yellow_min_threshold 516096 + green_max_threshold 184320 + yellow_max_threshold 516096 + -------------------- ------- + ... + +Compare to values in init_cfg.json: + + sonic-cfggen -j /etc/sonic/init_cfg.json --print-data | grep -A20 \"AZURE_LOSSLESS\" + +#### Test case #5(M): Ecnconfig utility test. Set mode +Ssh to Switch and execute: + + ecnconfig -p AZURE_LOSSY -gmax 184320 -rmax 516096 + +No error messages expected. + +#### Test case #6(MA): Ecnconfig utility test. Set function +Ssh to Switch and execute: + + ecnconfig -p AZURE_LOSSY -rmin 491520 + +Check ASIC DB records. + + redis-cli -n 1 keys \* | grep "ASIC_STATE:SAI_OBJECT_TYPE_WRED" + +Expected output example: + + root@arc-switch1028:/home/admin# redis-cli -n 1 hgetall "ASIC_STATE:SAI_OBJECT_TYPE_WRED:oid:0x130000000005cc" + ... + 13) "SAI_WRED_ATTR_RED_MIN_THRESHOLD" + 14) "491520" + ... + +#### Test case #7(M): Ecnconfig utility test. Integration with `show` and `config` +Ssh to Switch and run the following commands: + + show ecn + +Expected output: like in test case #4 + + config ecn --help + +Expected output: utility usage help screen + + config ecn -profile AZURE_LOSSY -rmin 430080 + +Check value applied like in test case #6 + +#### Test case #8(M): Ecnconfig utility test. Negative tests +Ssh to Switch and run the following commands: + + ecnconfig -l -p + ecnconfig -l -p LOSS + ecnconfig -p LOSS + ecnconfig -p LOSS -gmax + config ecn -gmax 45 + show ecn -p + +All should fail with the appropriate error messages. diff --git a/docs/VLAN-trunk-test-plan.md b/docs/VLAN-trunk-test-plan.md new file mode 100644 index 0000000000..0cfe39ff31 --- /dev/null +++ b/docs/VLAN-trunk-test-plan.md @@ -0,0 +1,260 @@ +## Overview +The purpose is to test VLAN functions on the SONiC switch. + +### Scope +The tests will include: + +1. Functionalities of VLAN ports. +2. VLAN interfaces routing. +3. IP2me traffic on VLAN interfaces. + +The test will trying to cover all functionalities of VLAN ports including Ethernet ports and LAGs. And will make sure the IP traffic and IP2me traffic is working well. + +### Functionalities of VLAN ports + +A VLAN port will include three attributes: + +* PVID: Ingress untagged packets will be tagged with PVID, and PVID will always in Permit VLAN IDs. + +* Permit VLAN IDs: Which VLAN ID of ingress and egress packets is allowed in the port. + +* tagged VLAN IDs: Determine which VLAN IDs egress packets will be tagged. + + For the VLAN trunk feature, the tagged VLAN IDs are limited to Permit VLAN IDs besides PVID, e.g., if PVID is 100, Permit VLAN IDs are 100,200,300, tagged VLAN IDs are 200,300, in other words, untagged VLAN ID is 100. + +The detail actions of VLAN ports: + + +| Packet Tagged or Untagged | Direction | Action | +| :------------------------ | :-------- | :--------------------------------------- | +| Untagged | Ingress | Tags the packet with the PVID tag. | +| | Egress | If VLAN ID of the packet is equal with untagged VLAN ID, untag and send out the packet. Besides, if VLAN ID is in Permit VLAN IDs, send out the packet with tag. | +| Tagged | Ingress | If VLAN ID of the packet is not in Permit VLAN IDs, drop the packet. | +| | Egress | If VLAN ID of the packet is equal with untagged VLAN ID, untag and send out the packet. Besides, if VLAN ID is in Permit VLAN IDs, send out the packet with tag. | + +## TEST structure + +1. The tests assume fanout switch support QinQ (stacked VLAN), so that stacked VLAN packets can passthrough fanout switch and can be tested on DUT with inner VLAN. + + ``` + | testbed server | <------------------> | Fanout switch | <------------------> | DUT | + stacked vlan pkt single vlan pkt + outer vlan: 1681 vlan: 100 + inner vlan: 100 + ``` + +2. Tests will be based on *t0* testbed type. The IP address of every LAGs on the DUT will be flushed to make all LAGs act as L2 ports. New test IP addresses will be configured on VLAN interfaces. + +3. VMs are only used to do LACP negotiation for LAGs; PTF is used to send packet and verify VLAN functionalities. + +4. The test contains three files: + + vlan_info.j2: Define VLAN ports info and VLAN interface info for vlan.yml and vlan_test.py + + vlan.yml: Configure DUT for the test according to vlan_info.j2 + + vlan_test.py: Do the PTF test according to vlan_info.j2 + +5. vlan_info.j2 will choose several Ethernet ports and LAG ports from minigraph of current topology and generate VLAN port info and VLAN interface info for PTF python script to do test. + + ```jinja2 + {% set vlan_id_list = [ 100, 200 ] %} + vlan_ports_list: + {% for lag_number in range(2) %} + - dev: '{{ minigraph_portchannels.keys()[lag_number|int] }}' + port_index: '{{ minigraph_port_indices[minigraph_portchannels[minigraph_portchannels.keys()[lag_number|int]].members[0]] }}' + pvid: '{{ vlan_id_list[(lag_number|int)%2] }}' + permit_vlanid: + {% for vlan in vlan_id_list %} + '{{ vlan }}': + peer_ip: '192.168.{{ vlan }}.{{ minigraph_port_indices[minigraph_portchannels[minigraph_portchannels.keys()[lag_number|int]].members[0]] }}' + remote_ip: '{{vlan}}.1.1.{{ minigraph_port_indices[minigraph_portchannels[minigraph_portchannels.keys()[lag_number|int]].members[0]] }}' + {% endfor %} + {% endfor %} + {% for port_number in range(2) %} + - dev: '{{ minigraph_ports.keys()[port_number|int]}}' + port_index: '{{ minigraph_port_indices[minigraph_ports.keys()[port_number|int]]}}' + pvid: '{{ ((port_number|int)%2+1)*100}}' + permit_vlanid: + {% for vlan in vlan_id_list %} + '{{ vlan }}': + peer_ip: '192.168.{{ vlan }}.{{ minigraph_port_indices[minigraph_ports.keys()[port_number|int]] }}' + remote_ip: '{{vlan}}.1.1.{{ minigraph_port_indices[minigraph_ports.keys()[port_number|int]] }}' + {% endfor %} + {% endfor %} + + vlan_intf_list: + {% for vlan in vlan_id_list %} + - vlan_id: '{{ (vlan|int) }}' + ip: '192.168.{{ vlan }}.1/24' + {% endfor %} + ``` + + and generate vlan.yml. Below is for an example: + + ```yaml + vlan_ports_list: + - dev: 'PortChannel03' + port_index: '30' + pvid: '100' + permit_vlanid: + '100': + peer_ip: '192.168.100.30' + remote_ip: '100.1.1.30' + '200': + peer_ip: '192.168.200.30' + remote_ip: '200.1.1.30' + - dev: 'PortChannel02' + port_index: '29' + pvid: '200' + permit_vlanid: + '100': + peer_ip: '192.168.100.29' + remote_ip: '100.1.1.29' + '200': + peer_ip: '192.168.200.29' + remote_ip: '200.1.1.29' + - dev: 'Ethernet8' + port_index: '8' + pvid: '100' + permit_vlanid: + '100': + peer_ip: '192.168.100.8' + remote_ip: '100.1.1.8' + '200': + peer_ip: '192.168.200.8' + remote_ip: '200.1.1.8' + - dev: 'Ethernet9' + port_index: '9' + pvid: '200' + permit_vlanid: + '100': + peer_ip: '192.168.100.9' + remote_ip: '100.1.1.9' + '200': + peer_ip: '192.168.200.9' + remote_ip: '200.1.1.9' + + vlan_intf_list: + - vlan_id: '100' + ip: '192.168.100.1/24' + - vlan_id: '200' + ip: '192.168.200.1/24' + ``` + + +## TEST case + +All the test cases will try to send packets from all the VLAN ports defined in vlan.yml, and try to verify packets from all expected VLAN ports. + +### Test case #1 + +#### Test objective + +To verify untagged packets received and be sent out with tag or without tag determined by egress port PVID. + +#### Test description + +``` +Test example: + |(untag:100/permit:100,200)->pkt(untagged) +pkt(untagged)->(pvid:100/permit:100,200)|DUT| + |(untag:200/permit:100,200)->pkt(tagged:100) +``` + +1. PTF send untagged packets(destination MAC unknown). +2. Verify packets can be received from other ports which permit PVID on ingress port. And packets will egress untagged if PVID on ingress port is same with egress ports , or packets will egress tagged with ingress port PVID. + +### Test case #2 + +#### Test objective + +To verify if tagged packets received in Permit VLAN IDs and be sent out with tag or without tag determined by egress port PVID. + +#### Test description + +``` +Test example: + |(untag:100/permit:100,200)->pkt(untagged) +pkt(tagged:100)->(pvid:100/permit:100,200)|DUT| + |(untag:200/permit:100,200)->pkt(tagged:100) + + |(untag:100/permit:100,200)->pkt(tagged:200) +pkt(tagged:200)->(pvid:100/permit:100,200)|DUT| + |(untag:200/permit:100,200)->pkt(untagged) +``` + +1. PTF send tagged packets(destination MAC unknown), which VLAN ID is in Permit VLAN IDs of ingress port. +2. Verify packets can be received from other ports which permit PVID on ingress port. And packets will egress untagged if PVID on ingress port is same with egress ports , or packets will egress tagged with ingress port PVID. + +### Test case #3 + +#### Test objective + +To verify if tagged packets received not in Permit VLAN IDs, the packets will be dropped + +#### Test description + +``` +Test example: + |(untag:100/permit:100,200)->no pkt egress +pkt(tagged:4095)->(pvid:100/permit:100,200)|DUT| + |(untag:200/permit:100,200)->no pkt egress +``` + +1. PTF send tagged packets(destination MAC unknown), which VLAN ID is not in Permit VLAN IDs of ingress port. +2. Verify no packets received from other ports. + +### Test case #4 +#### Test objective + +To verify the VLAN interface routing is working. + +#### Test description + +``` +Test example: +Vlan100: 192.168.100.1/24 +Vlan200: 192.168.200.1/24 + +192.168.100.30->192.168.200.30 (for directly-connected routing) +pkt(tagged:100)->(pvid:200/permit:100,200)|DUT|(untag:100/permit:100,200)->pkt(tagged:200) +pkt(untagged)->(pvid:100/permit:100,200)|DUT|(untag:100/permit:100,200)->pkt(tagged:200) +pkt(untagged)->(pvid:100/permit:100,200)|DUT|(untag:200/permit:100,200)->pkt(untagged) + +1.1.1.30->2.1.1.30 (for indirectly-connected routing) +pkt(tagged:100)->(pvid:200/permit:100,200)|DUT|(untag:100/permit:100,200)->pkt(tagged:200) +pkt(untagged)->(pvid:100/permit:100,200)|DUT|(untag:100/permit:100,200)->pkt(tagged:200) +pkt(untagged)->(pvid:100/permit:100,200)|DUT|(untag:200/permit:100,200)->pkt(untagged) +``` + +1. PTF send IP packets over VLAN interfaces. +2. Verify packets can be receive on the egress port. + +### Test case #5 +#### Test objective + +To verify the IP traffic to VLAN interface self is working. + +#### Test description + +``` +Test example: +Vlan100: 192.168.100.1/24 +Vlan200: 192.168.200.1/24 + +192.168.100.30->192.168.100.1 +pkt(untagged)-> + (pvid:100/untag:100/permit:100,200)|DUT| +pkt(untagged)<- + +192.168.100.30->192.168.100.1 +pkt(tagged:100)-> + (pvid:200/untag:200/permit:100,200)|DUT| +pkt(tagged:100)<- +``` + +1. PTF send ICMP request packet to VLAN interfaces. +2. Verify ICMP reply packets can be received from ingress port. + +