[config]: Filter table from input config while config validation\DPB

[praveen-li]

Changes:
-- Filter table, which does not impact Back-End, from input config
while config validation or while Dynamic Port Breakout(DPB).
-- Ask for confirmation from user while DPB, only when deleted ports
exists in extra tables[i.e in tables without YANG].
-- Test for filter Table.
-- Filter BREAKOUT_CFG table while DPB, because back-end does not process
BREAKOUT_CFG table.

Signed-off-by: Praveen Chaudhary pchaudhary@linkedin.com

- What I did
-- Filter table, which does not impact Back-End, from input config
while config validation or while Dynamic Port Breakout(DPB).
-- Ask for confirmation from user while DPB, only when deleted ports
exists in extra tables[i.e in tables without YANG].
-- Test for filter Table.
-- Filter BREAKOUT_CFG table while DPB, because back-end does not process
BREAKOUT_CFG table.

- How I did it
Added a function which filter predefined tables from config. There predefined tables do not impact back-end, so better to filter them.

Change in config/main.py to ask for user permission when any table exists in config for which YANG models are not written. Note: Now, We ask user confirmation only if delete PORTs exist in these extra tables.

Added test code.

- How to verify it

Test First Approach: Wrote the test first to Make Sure, It fails without fix:

   def test_extra_brk_cfg_tables_cases(self):
        # make sure no prompt with LOCK
        curConfig = dict(configDbJson)
        brk_cfg = { "BREAKOUT_CFG": {
                    "Ethernet0": {
                        "brkout_mode": "1x100G[40G]"
                    }
                  }
                }
        self.updateConfig(curConfig, brk_cfg)
        prevLen = len(curConfig)
        cm = self.config_mgmt_dpb(curConfig)
        # Assert LOCK is removed from Input Config
>       assert (len(cm.configdbJsonIn) ==  prevLen-1) and \
            'BREAKOUT_CFG' not in cm.configdbJsonIn.keys()
E       AssertionError: assert (6 == (6 - 1))
E        +  where 6 = len({'ACL_TABLE': {'NO-NSW-PACL-TEST': {'policy_desc': 'NO-NSW-PACL-TEST', 'ports': ['Ethernet11'], 'stage': 'INGRESS', 't...66', ...}, 'Ethernet10': {'lanes': '75', 'speed': '25000'}, 'Ethernet11': {'lanes': '76', 'speed': '25000'}, ...}, ...})
E        +    where {'ACL_TABLE': {'NO-NSW-PACL-TEST': {'policy_desc': 'NO-NSW-PACL-TEST', 'ports': ['Ethernet11'], 'stage': 'INGRESS', 't...66', ...}, 'Ethernet10': {'lanes': '75', 'speed': '25000'}, 'Ethernet11': {'lanes': '76', 'speed': '25000'}, ...}, ...} = <config_mgmt.ConfigMgmtDPB instance at 0x7f702c1829e0>.configdbJsonIn


Now After Fix, it build fine.

make[1]: Leaving directory '/home/pchaudha/srcCode/azure_myforks/sonic-buildimage'
pchaudha@server05:/home/pchaudha/srcCode/azure_myforks/sonic-buildimage$ date
Fri Jul 31 12:48:57 PDT 2020
pchaudha@server05:/home/pchaudha/srcCode/azure_myforks/sonic-buildimage$ ls -l target/python-debs/python-sonic-utilities_1.2-1_all.deb
-rw-r--r-- 1 pchaudha pchaudha 182780 Jul 31 12:48 target/python-debs/python-sonic-utilities_1.2-1_all.deb
pchaudha@server05:/home/pchaudha/srcCode/azure_myforks/sonic-buildimage$

- Previous command output (if the output of a command-line utility has changed)

- New command output (if the output of a command-line utility has changed)

[praveen-li]

force push due to rebasing.

[lguohan]

retest this please

[praveen-li]

@lguohan @jleveque @daall
Kindly review when possible, this changes has its own tests, so we are good here. Thx

[praveen-li]

Gentle Reminder for review.

[praveen-li]

retest this please

[praveen-li]

@jleveque : kindly review needed to fix one not needed check in DPB.

[jleveque]

Retest this please

[jleveque]

/AzurePipelines run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[praveen-li]

Force pushed due to rebasing.
Fixing a test as well.

[allas-nvidia]

Can someone merge it, please
And it should be in 202012.
Thanks

[zhenggen-xu]

We should have 3 different test cases:
1> Tables that we should ignore specifically even with/without Yang, and no warning. (we probably don't need this based on my latest comments: sonic-net/sonic-buildimage#6801 (comment) ), and I don't see other cases.
2> Tables that does not have Yang models but not depending on port to be deleted. By default , it should continue with warning.
3> Tables that does not have Yang models but depending on the port to be deleted. It should ask user to confirm.

[zhenggen-xu]

Based on sonic-net/sonic-buildimage#6801 (comment) , I would suggest we don't do anything specific for this table.