[test plans] Moving test plans from sonic-wiki repo to sonic-mgmt repo

docs/BGP-GR-helper-mode-test-plan.md

@@ -0,0 +1,90 @@
+- [Overview](#overview)
+    - [Scope](#scope)
+    - [Testbed](#testbed)
+- [Setup configuration](#setup-configuration)
+    - [Arista VM configuration](#arista-vm-configuration)
+    - [Ansible scripts to setup and run test](#ansible-scripts-to-setup-and-run-test)
+        - [everflow_testbed.yml](#everflow-testbed-yml)
+- [PTF Test](#ptf-test)
+    - [Input files for PTF test](#input-files-for-ptf-test)
+    - [Traffic validation in PTF](#traffic-validation-in-ptf)
+- [Test cases](#test-cases)
+- [TODO](#todo)
+- [Open Questions](#open-questions)
+
+## Overview
+The purpose is to test a functionality of BGP GR mode on the SONIC switch DUT, closely resembling production environment.
+The test assumes all necessary configuration is already pre-configured on the SONIC switch before test runs.
+
+### Scope
+The test is targeting a running SONIC system with fully functioning configuration.
+The purpose of the test is not to test specific API, but functional testing of BGP GR helper mode on SONIC system, making sure that traffic flows correctly, according to BGP routes advertised by BGP peers of SONIC switch.
+
+### Testbed
+The test will run on the following testbeds:
+- t1
+- t1-lag
+
+## Setup configuration
+
+#### Arista VM configuration
+
+Test assumes that BGP GR is enabled and preconfigured on Arista VMs. BGP GR timer value should be more than time required for VM reboot.
+
+#### Ansible scripts to setup and run test
+
+##### bgp_gr_helper.yml
+
+bgp_gr_helper.yml when run with tag "bgp_gr_helper" will do the following:
+
+1. Randomly choose VM.
+2. Run test.
+
+BGP GR helper test consists of a number of subtests, and each of them will include the following steps:
+
+1. Run lognanalyzer 'init' phase
+2. Run BGP GR helper Sub Test
+3. Run loganalyzer 'analyze' phase
+
+## PTF Test
+
+To run traffic FIB PTF test will be reused.
+
+## Test cases
+
+Each test case will be additionally validated by the loganalizer utility.
+
+### Test case \#1 - BGP GR helper mode.
+
+#### Test objective
+
+Verify that routes are preserved during neighbor graceful restart.
+
+#### Test steps
+
+- Randomly choose VM for the test.
+- Reboot VM.
+- Verify BGP timeout (at least 115 seconds routes should stay in fib).
+- Verify all routes are preserved (no reinstallation after BGP open message from the neighbor).
+- Verify that BGP session with the VM established.
+
+### Test case \#2 - BGP GR helper mode routes change.
+
+#### Test objective
+
+Verify that traffic run without changes during neighbor graceful restart.
+
+#### Test steps
+
+- Randomly choose VM for the test.
+- Change VM startup config (advertised routes should be different). 
+- Reboot VM.
+- Verify that preserved routes are removed when VM back.
+- Verify that new routes are installed when VM back.
+- Restore VM startup config.
+
+## TODO
+
+## Open Questions
+- Should tests run for neighbors behind physical interfaces only or behind LAGs as well?
+- On which topologies test should run?

docs/BGP-MP-test-plan.md

@@ -0,0 +1,55 @@
+# BGP-MP test plan
+
+* [Overview](#Overview)
+   * [Scope](#Scope)
+   * [Testbed](#Testbed)
+* [Setup configuration](#Setup%20configuration)
+   * [Ansible scripts to setup and run test](#Ansible%20scripts%20to%20setup%20and%20run%20test)
+     * [bgp_mp.yml](#bgp_mp.yml)
+   * [Setup of DUT switch](#Setup%20of%20DUT%20switch)
+* [Test](#Test)
+* [Test cases](#Test%20cases)
+* [TODO](#TODO)
+* [Open questions](#Open%20questions)
+
+## Overview
+The purpose is to test functionality of BGP-MP on the SONIC switch DUT, closely resembling production environment. The test assumes all necessary configurations are already pre-configured on the SONIC switch before test runs.
+
+### Scope
+The test is targeting a running SONIC system with fully functioning configuration. The purpose of the test is not to test specific API, but functional testing of BGP-MP on SONIC system.
+
+### Testbed
+The test will run on the following testbeds:
+* t0
+
+## Setup configuration
+IPv4 BGP neighborship will be configured between DUT and exabgp and each neighbor will redistribute IPv6 routes to each other.
+### Ansible scripts to setup and run test
+#### bgp_mp.yml
+bgp_mp.yml when run with tag “bgp_mp” will do the following:
+1. Generate and apply exabgp configuration.
+2. Run test.
+3. Clean up dynamic and temporary exabgp configuration.
+
+## Test
+On PTF host, exabgp tool will be used to configure bgp peer and redistribute IPv6 routes via IPv4 BGP session.
+
+## Test cases
+### Test case # 1 – BGP-MP IPv6 routes over IPv4 session
+#### Test objective
+Verify that IPv6 routes are correctly redistributed over IPv4 BGP session.
+#### Test steps
+* Generate IPv4 BGP peer configuration for exabgp instance.
+* Generate IPv6 routes, to be announced via IPv4 session, for exabgp instance.
+* Run exabgp instance.
+* Verify that IPv4 BGP neighborship is established.
+* Redistribute IPv6 routes using exabgp.
+* Verify that IPv6 routes are correctly redistributed to the DUT.
+* Redistribute IPv6 routes from the DUT to exabgp.
+* Verify that IPv6 routes are correctly redistributed to the exabgp.
+* Set default configuration.
+
+## TODO
+
+## Open questions
+* Should be some traffic test cases performed as part of this test?

docs/CRM-test-plan.md

@@ -0,0 +1,221 @@
+# CRM test plan
+
+* [Overview](#Overview)
+   * [Scope](#Scope)
+   * [Testbed](#Testbed)
+* [Setup configuration](#Setup%20configuration)
+   * [Ansible scripts to setup and run test](#Ansible%20scripts%20to%20setup%20and%20run%20test)
+     * [crm.yml](#crm.yml)
+* [Test](#Test)
+* [Test cases](#Test%20cases)
+* [TODO](#TODO)
+* [Open questions](#Open%20questions)
+
+## Overview
+The purpose is to test functionality of CRM on the SONIC switch DUT, closely resembling production environment.
+
+### Scope
+The test is targeting a running SONIC system with fully functioning configuration. The purpose of the test is not to test specific API, but functional testing of CRM on SONIC system.
+
+### Testbed
+The test will run on the all testbeds.
+
+## Setup configuration
+No setup pre-configuration is required, test will configure and clean-up all the configuration.
+### Ansible scripts to setup and run test
+#### crm.yml
+crm.yml when run with tag “crm” will do the following for each CRM resource:
+1. Apply required configuration.
+2. Verify "used" and "free" counters.
+3. Verify "EXCEEDED" and "CLEAR" messages using all  types of thresholds.
+4. Restore configuration.
+
+## Test
+
+## Test cases
+
+### Test case # 1 – IPv4 route
+#### Test objective
+Verify "IPv4 route" CRM resource.
+#### Test steps
+* Set polling interval to 1 minute.
+* Configure 1 route and observe that counters were updated as expected.
+* Remove 1 route and observe that counters were updated as expected.
+* Perform the following steps for all threshold types ("percentage", "used", "free"):
+	* Set low and high thresholds according to current usage and type.
+	* Verify that "EXCEEDED" message is logged (using log analyzer).
+	* Set low and high thresholds to default values.
+	* Verify that "CLEAR" message is logged (using log analyzer).
+* Restore default configuration.
+
+### Test case # 2 – IPv6 route
+#### Test objective
+Verify "IPv6 route" CRM resource.
+#### Test steps
+* Set polling interval to 1 minute.
+* Configure 1 route and observe that counters were updated as expected.
+* Remove 1 route and observe that counters were updated as expected.
+* Perform the following steps for all threshold types ("percentage", "used", "free"):
+	* Set low and high thresholds according to current usage and type.
+	* Verify that "EXCEEDED" message is logged (using log analyzer).
+	* Set low and high thresholds to default values.
+	* Verify that "CLEAR" message is logged (using log analyzer).
+* Restore default configuration.
+
+### Test case # 3 – IPv4 nexthop
+#### Test objective
+Verify "IPv4 nexthop" CRM resource.
+#### Test steps
+* Set polling interval to 1 minute.
+* Add 1 nexthop and observe that counters were updated as expected.
+* Remove 1 nexthop and observe that counters were updated as expected.
+* Perform the following steps for all threshold types ("percentage", "used", "free"):
+	* Set low and high thresholds according to current usage and type.
+	* Verify that "EXCEEDED" message is logged (using log analyzer).
+	* Set low and high thresholds to default values.
+	* Verify that "CLEAR" message is logged (using log analyzer).
+* Restore default configuration.
+
+### Test case # 4 – IPv6 nexthop
+#### Test objective
+Verify "IPv6 nexthop" CRM resource.
+#### Test steps
+* Set polling interval to 1 minute.
+* Add 1 nexthop and observe that counters were updated as expected.
+* Remove 1 nexthop and observe that counters were updated as expected.
+* Perform the following steps for all threshold types ("percentage", "used", "free"):
+	* Set low and high thresholds according to current usage and type.
+	* Verify that "EXCEEDED" message is logged (using log analyzer).
+	* Set low and high thresholds to default values.
+	* Verify that "CLEAR" message is logged (using log analyzer).
+* Restore default configuration.
+
+### Test case # 5 – IPv4 neighbor
+#### Test objective
+Verify "IPv4 neighbor" CRM resource.
+#### Test steps
+* Set polling interval to 1 minute.
+* Configure 1 neighbor and observe that counters were updated as expected.
+* Remove 1 neighbor and observe that counters were updated as expected.
+* Perform the following steps for all threshold types ("percentage", "used", "free"):
+	* Set low and high thresholds according to current usage and type.
+	* Verify that "EXCEEDED" message is logged (using log analyzer).
+	* Set low and high thresholds to default values.
+	* Verify that "CLEAR" message is logged (using log analyzer).
+* Restore default configuration.
+
+### Test case # 6 – IPv6 neighbor
+#### Test objective
+Verify "IPv6 neighbor" CRM resource.
+#### Test steps
+* Set polling interval to 1 minute.
+* Configure 1 neighbor and observe that counters were updated as expected.
+* Remove 1 neighbor and observe that counters were updated as expected.
+* Perform the following steps for all threshold types ("percentage", "used", "free"):
+	* Set low and high thresholds according to current usage and type.
+	* Verify that "EXCEEDED" message is logged (using log analyzer).
+	* Set low and high thresholds to default values.
+	* Verify that "CLEAR" message is logged (using log analyzer).
+* Restore default configuration.
+
+### Test case # 7 – Nexthop group object
+#### Test objective
+Verify "nexthop group object" CRM resource.
+#### Test steps
+* Set polling interval to 1 minute.
+* Configure 1 ECMP route and observe that counters were updated as expected.
+* Remove 1 ECMP route and observe that counters were updated as expected.
+* Perform the following steps for all threshold types ("percentage", "used", "free"):
+	* Set low and high thresholds according to current usage and type.
+	* Verify that "EXCEEDED" message is logged (using log analyzer).
+	* Set low and high thresholds to default values.
+	* Verify that "CLEAR" message is logged (using log analyzer).
+* Restore default configuration.
+
+### Test case # 8 – Nexthop group member
+#### Test objective
+Verify "nexthop group member" CRM resource.
+#### Test steps
+* Set polling interval to 1 minute.
+* Configure 1 ECMP route and observe that counters were updated as expected.
+* Remove 1 ECMP route and observe that counters were updated as expected.
+* Perform the following steps for all threshold types ("percentage", "used", "free"):
+	* Set low and high thresholds according to current usage and type.
+	* Verify that "EXCEEDED" message is logged (using log analyzer).
+	* Set low and high thresholds to default values.
+	* Verify that "CLEAR" message is logged (using log analyzer).
+* Restore default configuration.
+
+### Test case # 9 – FDB entry
+#### Test objective
+Verify "FDB entry" CRM resource.
+#### Test steps
+* Set polling interval to 1 minute.
+* Configure 1 FDB entry and observe that counters were updated as expected.
+* Remove 1 FDB entry and observe that counters were updated as expected.
+* Perform the following steps for all threshold types ("percentage", "used", "free"):
+	* Set low and high thresholds according to current usage and type.
+	* Verify that "EXCEEDED" message is logged (using log analyzer).
+	* Set low and high thresholds to default values.
+	* Verify that "CLEAR" message is logged (using log analyzer).
+* Restore default configuration.
+
+### Test case # 10 – ACL group
+#### Test objective
+Verify "ACL group" CRM resource.
+#### Test steps
+* Set polling interval to 1 minute.
+* Configure 1 ACL and observe that counters were updated as expected.
+* Remove 1 ACL and observe that counters were updated as expected.
+* Perform the following steps for all threshold types ("percentage", "used", "free"):
+	* Set low and high thresholds according to current usage and type.
+	* Verify that "EXCEEDED" message is logged (using log analyzer).
+	* Set low and high thresholds to default values.
+	* Verify that "CLEAR" message is logged (using log analyzer).
+* Restore default configuration.
+
+### Test case # 11 – ACL table
+#### Test objective
+Verify "ACL table" CRM resource.
+#### Test steps
+* Set polling interval to 1 minute.
+* Configure 1 ACL and observe that counters were updated as expected.
+* Remove 1 ACL and observe that counters were updated as expected.
+* Perform the following steps for all threshold types ("percentage", "used", "free"):
+	* Set low and high thresholds according to current usage and type.
+	* Verify that "EXCEEDED" message is logged (using log analyzer).
+	* Set low and high thresholds to default values.
+	* Verify that "CLEAR" message is logged (using log analyzer).
+* Restore default configuration.
+
+### Test case # 12 – ACL entry
+#### Test objective
+Verify "ACL entry" CRM resource.
+#### Test steps
+* Set polling interval to 1 minute.
+* Configure 1 ACL rule and observe that counters were updated as expected.
+* Remove 1 ACL rule and observe that counters were updated as expected.
+* Perform the following steps for all threshold types ("percentage", "used", "free"):
+	* Set low and high thresholds according to current usage and type.
+	* Verify that "EXCEEDED" message is logged (using log analyzer).
+	* Set low and high thresholds to default values.
+	* Verify that "CLEAR" message is logged (using log analyzer).
+* Restore default configuration.
+
+### Test case # 13 – ACL counter
+#### Test objective
+Verify "ACL entry" CRM resource.
+#### Test steps
+* Set polling interval to 1 minute.
+* Configure 1 ACL rule and observe that counters were updated as expected.
+* Remove 1 ACL rule and observe that counters were updated as expected.
+* Perform the following steps for all threshold types ("percentage", "used", "free"):
+	* Set low and high thresholds according to current usage and type.
+	* Verify that "EXCEEDED" message is logged (using log analyzer).
+	* Set low and high thresholds to default values.
+	* Verify that "CLEAR" message is logged (using log analyzer).
+* Restore default configuration.
+
+## TODO
+
+## Open questions