Display large number of FDB entries fast from state_db and appl_db

[JiangboHe]

This is a copy of pull request #968, we found and fixed it in 201911 branch. It need also be merged to master.

- What I did
An option is provided for showing large number of fdb tables faster.
The time cost to display change from 50 seconds to 2 seconds for 8000 fdb tables or 20 seconds to 1 seconds for 5000 fdb tables.

- How I did it
The key search and judge of fdb tables in asic table is too complex. It will cost 20 seconds for 5000 fdb tables or 50 seconds for 8000 fdb tables. It is not good for users or script running. The option is to show fdb tables from state_db.

- How to verify it
Learn thounds of fdb tables and show them with fdbshow

- Previous command output (if the output of a command-line utility has changed)

- New command output (if the output of a command-line utility has changed)

[JiangboHe]

retest this please

[JiangboHe]

retest this please

[prsunny]

retest this please

[JiangboHe]

Seem it is not the code of this pull request to cause the error result.

Test Result (失败)
test_drop_counters.TestDropCounters.test_deviceCapabilitiesTablePopulated
test_drop_counters.TestDropCounters.test_flexCounterGroupInitialized
test_drop_counters.TestDropCounters.test_createAndRemoveDropCounterBasic
test_drop_counters.TestDropCounters.test_createAndRemoveDropCounterReversed
test_drop_counters.TestDropCounters.test_createCounterWithInvalidCounterType
test_drop_counters.TestDropCounters.test_createCounterWithInvalidDropReason
test_drop_counters.TestDropCounters.test_addReasonToInitializedCounter
test_drop_counters.TestDropCounters.test_removeReasonFromInitializedCounter
test_drop_counters.TestDropCounters.test_removeAllDropReasons
test_drop_counters.TestDropCounters.test_addDropReasonMultipleTimes
test_drop_counters.TestDropCounters.test_addInvalidDropReason
test_drop_counters.TestDropCounters.test_removeDropReasonMultipleTimes
test_drop_counters.TestDropCounters.test_removeNonexistentDropReason
test_drop_counters.TestDropCounters.test_removeInvalidDropReason
test_drop_counters.TestDropCounters.test_createAndDeleteMultipleCounters

11:55:27 /usr/local/lib/python3.8/dist-packages/six.py:702:
11:55:27 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
11:55:27 conftest.py:1503: in dvs
11:55:27 dvs = DockerVirtualSwitch(name, imgname, keeptb, fakeplatform, log_path, max_cpu, forcedvs)
11:55:27 conftest.py:329: in init
11:55:27 server = VirtualServer(self.ctn_sw.name, self.ctn_sw_pid, i)
11:55:27 conftest.py:165: in init
11:55:27 ensure_system(f"ip link add {self.nsname[0:12]} type veth peer name {self.pifname}")

[prsunny]

retest this please

[lguohan]

retest this please

[prsunny]

@JiangboHe , could you please add unit test to cover this?

[JiangboHe]

@prsunny OK, I will add some unit test later.

[liat-grozovik]

seems like the fix for 201911 was closed and the intention was first to take to master and then cherry pick it.
@prsunny can you please confirm?
@JiangboHe do you have plans to provide unit test so the PR can be merged?

[JiangboHe]

@liat-grozovik I try to provide the UT next month.

[liat-grozovik]

@JiangboHe kindly reminder. we would like this fix to be in master and 202012 . will you be able to provide so it can be merged?

[liat-grozovik]

/AzurePipelines run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[JiangboHe]

@liat-grozovik Sorry to reply late. There was some building problem in my enviroment. I am trying to fix it and finish the test case. Thanks

[liat-grozovik]

@JiangboHe where do we stands with this PR?

[JiangboHe]

@liat-grozovik There is some problem with my network in master branch building. I am trying to fix it.

[liat-grozovik]

I noticed some issues with master in the last day as well. It might be general one 202012 is working 201911 I think also not working but not sure in 100% Get Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: JiangboHe ***@***.***> Sent: Friday, March 19, 2021 10:42:40 AM To: Azure/sonic-utilities ***@***.***> Cc: Liat Grozovik ***@***.***>; Mention ***@***.***> Subject: Re: [Azure/sonic-utilities] Display large number of FDB entries fast from state_db and appl_db (#1273) @liat-grozovik<https://github.com/liat-grozovik> There is some problem with my network in master branch building. I am trying to fix it. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#1273 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AKTABAYNFQVB3PCFEPTPYODTEMFABANCNFSM4UCABE6A>.

[liat-grozovik]

@JiangboHe could you please fix conflicts and then we can re run tests to approve the fix?

[moshemos]

@prsunny can you please review the failure and update the status

[moshemos]

@JiangboHe can you update with the UT?

[dgsudharsan]

@JiangboHe Can you please provide an update?

[Yuval-Mellanox]

@zhangyanzhao Let's triage this issue.