[macsec]: Override dp_poll of ptf by MACsec's #5490

Pterosaur · 2022-04-08T09:03:23Z

Description of PR

Summary:
Override dp_poll by MACsec dp_poll so that these dataplane testcases can be tested under MACsec environment

Wait for PR: sonic-net/sonic-buildimage#10507

Type of change

Bug fix
Testbed and Framework(new/improvement)
Test case(new/improvement)

Back port request

201911
202012

Approach

What is the motivation for this PR?

Existing dataplane tests cannot be tested under MACsec environment due to the traffic under MACsec link is encrypted.

How did you do it?

I write a macsec_dp_poll to override the original ptf dp_poll, and for each MACsec packets, the macsec_dp_poll will decrypt them and pass the clear packets to the upper layer.

How did you verify/test it?

Check Azp and try FIB test under MACsec environment.

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

* Add check neighbors health for SONiC neighbor devices Signed-off-by: Ze Gan <ganze718@gmail.com> * Add MACsec test Signed-off-by: Ze Gan <ganze718@gmail.com> * Skip to move bp port to docker if neighbor devices are sonic Signed-off-by: Ze Gan <ganze718@gmail.com> * AddPolish code Signed-off-by: Ze Gan <ganze718@gmail.com> * Add data plane server to neighbor Signed-off-by: Ze Gan <ganze718@gmail.com> * Polish dataplane test Signed-off-by: Ze Gan <ganze718@gmail.com> * Add ptf service in nSONiC eighbor devices Signed-off-by: Ze Gan <ganze718@gmail.com> * Add MACsec pkt checking in neighbor devices Signed-off-by: Ze Gan <ganze718@gmail.com> * polish ansible task name Signed-off-by: Ze Gan <ganze718@gmail.com> * [MACsec test]Add test neighbor to neighbor Signed-off-by: Ze Gan <ganze718@gmail.com> * Fix test Signed-off-by: Ze Gan <ganze718@gmail.com> * Fix warning Signed-off-by: Ze Gan <ganze718@gmail.com> * Add macsec to azp Signed-off-by: Ze Gan <ganze718@gmail.com> * test Signed-off-by: Ze Gan <ganze718@gmail.com> * fixtest Signed-off-by: Ze Gan <ganze718@gmail.com> * fix test Signed-off-by: Ze Gan <ganze718@gmail.com> * Different priority to peers Signed-off-by: Ze Gan <ganze718@gmail.com> * Add enable_macsec_feature before setup Signed-off-by: Ze Gan <ganze718@gmail.com> * Polish test Signed-off-by: Ze Gan <ganze718@gmail.com> * Add xpn and send_sci Signed-off-by: Ze Gan <ganze718@gmail.com> * remove unused import Signed-off-by: Ze Gan <ganze718@gmail.com> * increase timeout in sonictest-sonic-t0 Signed-off-by: Ze Gan <ganze718@gmail.com> * Modify timeout Signed-off-by: Ze Gan <ganze718@gmail.com> * Remove useless function Signed-off-by: Ze Gan <ganze718@gmail.com> * Polish code Signed-off-by: Ze Gan <ganze718@gmail.com> * Polish code Signed-off-by: Ze Gan <ganze718@gmail.com> * Add fault handling test Signed-off-by: Ze Gan <ganze718@gmail.com> * decrease cpu count of neighbor devices Signed-off-by: Ze Gan <ganze718@gmail.com> * Wait until mka session recovering from flap Signed-off-by: Ze Gan <ganze718@gmail.com>

Signed-off-by: Ze Gan <ganze718@gmail.com>

Why I did it Existing dataplane tests cannot be tested under MACsec environment due to the traffic under MACsec link is encrypted. So, I will override the dp_poll of ptf to MACsec dp_poll to decrypt the MACsec packets on injected ports (PR: sonic-net/sonic-mgmt#5490). MACsec decryption library depends on scapy 2.4.5. How I did it Upgrade scapy library to 2.4.5 by pip. How to verify it Check the scapy version in docker-ptf by python -c "import scapy; print(scapy.__version__)" 2.4.5 Signed-off-by: Ze Gan <ganze718@gmail.com>

Signed-off-by: Ze Gan <ganze718@gmail.com>

lgtm-com · 2022-05-12T15:42:55Z

This pull request introduces 1 alert when merging 27eff8253c1da43bd9300210e5d11bf2dbe7a1b4 into 1a1f6e5 - view on LGTM.com

new alerts:

1 for Syntax error

Signed-off-by: Ze Gan <ganze718@gmail.com>

lgtm-com · 2022-05-13T15:39:57Z

This pull request introduces 1 alert and fixes 1 when merging 9ff4ac87960337726f784a85419ebae2e056a5ed into ead6a03 - view on LGTM.com

new alerts:

1 for Unused import

fixed alerts:

1 for Unused import

lgtm-com · 2022-05-14T06:07:02Z

This pull request introduces 1 alert and fixes 1 when merging db98ffe72a79ee4fe1b73a22b16f063f3cf25784 into 7d3e46e - view on LGTM.com

new alerts:

1 for Unused import

fixed alerts:

1 for Unused import

Signed-off-by: Ze Gan <ganze718@gmail.com>

lgtm-com · 2022-05-16T09:10:46Z

This pull request introduces 1 alert and fixes 1 when merging 7351e3f into 57e5b17 - view on LGTM.com

new alerts:

1 for Unused import

fixed alerts:

1 for Unused import

jimmyzhai · 2022-05-24T07:38:55Z

ansible/roles/test/files/ptftests/fib.py

@@ -58,22 +60,22 @@ def __init__(self, file_path):
            for line in f.readlines():
                if pattern.match(line): continue
                entry = line.split(' ', 1)
-                prefix = ip_network(unicode(entry[0]))
+                prefix = ip_network(unicode(entry[0]) if sys.version_info[0] == 2 else entry[0])


Is there another better way to making unicode string for python2 and 3 ?

According to Xin's suggestion, we can leverage the library six for portable text string at here.

Signed-off-by: Ze Gan <ganze718@gmail.com>

This reverts commit 774d323.

Signed-off-by: Ze Gan <ganze718@gmail.com>

lgtm-com · 2022-05-25T04:18:24Z

This pull request introduces 1 alert when merging cdc633452677865384b04c03c6822f01ddf34671 into c1ecb7e - view on LGTM.com

new alerts:

1 for Syntax error

Signed-off-by: Ze Gan <ganze718@gmail.com>

jimmyzhai · 2022-05-27T07:55:53Z

ansible/roles/test/files/ptftests/macsec.py

+            if ptf.dataplane.match_exp_pkt(exp_pkt, pkt):
+                return ret
+            else:
+                continue


Potential loop forever? Then using while timeout > 0 in line 44?

ansible/roles/test/files/ptftests/macsec.py

jimmyzhai · 2022-05-27T08:05:03Z

ansible/roles/test/files/ptftests/macsec.py

+        MACSEC_INFOS = pickle.load(f)
+        if MACSEC_INFOS:
+            __origin_dp_poll = ptf.testutils.dp_poll
+            ptf.testutils.dp_poll = __macsec_dp_poll


Is it possible to use macsec_helper.macsec_dp_poll ? Then not to define __macsec_dp_poll.

Signed-off-by: Ze Gan <ganze718@gmail.com>

abdosi · 2022-07-26T20:49:00Z

@Pterosaur looks like this change is not backward compatible. It forces user to upgrade PTF docker for running fib test. Can we make it if we want to test fib under macsec environment then only trigger this change.

Pterosaur · 2022-07-27T00:41:20Z

@Pterosaur looks like this change is not backward compatible. It forces user to upgrade PTF docker for running fib test. Can we make it if we want to test fib under macsec environment then only trigger this change.

I will take a look.

Pterosaur added 8 commits March 2, 2022 15:46

Merge branch 'master' of https://github.com/Azure/sonic-mgmt

b0b1f8f

Merge branch 'master' of https://github.com/Azure/sonic-mgmt

b488b7b

Merge branch 'master' of https://github.com/Azure/sonic-mgmt

7112bb6

Merge branch 'master' of https://github.com/Azure/sonic-mgmt

622f1bc

Merge branch 'master' of https://github.com/Azure/sonic-mgmt

26bea1c

Merge branch 'master' of https://github.com/Azure/sonic-mgmt

5273f6b

Dataplane macsec dp_poll hook

e610961

Signed-off-by: Ze Gan <ganze718@gmail.com>

Pterosaur changed the title ~~Macsec verify hook~~ [macsec]: Override dp_poll by MACsec dp_poll Apr 8, 2022

Pterosaur changed the title ~~[macsec]: Override dp_poll by MACsec dp_poll~~ [macsec]: Override dp_poll of ptf by MACsec's Apr 8, 2022

Pterosaur mentioned this pull request Apr 8, 2022

[docker-ptf]: Upgrade scapy to 2.4.5 in docker-ptf sonic-net/sonic-buildimage#10507

Merged

6 tasks

Force reload MACsec information

5837af1

Signed-off-by: Ze Gan <ganze718@gmail.com>

Pterosaur force-pushed the macsec_verify_hook branch 2 times, most recently from 2c0ffa1 to e0431d1 Compare April 11, 2022 07:42

Fix match pkt

4dc25c4

Signed-off-by: Ze Gan <ganze718@gmail.com>

Pterosaur force-pushed the macsec_verify_hook branch from e0431d1 to 4dc25c4 Compare April 11, 2022 08:14

Merge branch 'master' into macsec_verify_hook

73486ce

Pterosaur force-pushed the macsec_verify_hook branch from 79c6a0f to 73486ce Compare May 12, 2022 15:15

Convert fib test to python3

1f37a22

Signed-off-by: Ze Gan <ganze718@gmail.com>

Pterosaur force-pushed the macsec_verify_hook branch from 27eff82 to 1f37a22 Compare May 12, 2022 23:47

Pterosaur force-pushed the macsec_verify_hook branch 2 times, most recently from 413d968 to db98ffe Compare May 14, 2022 05:45

Move fib to python3

f82a69f

Signed-off-by: Ze Gan <ganze718@gmail.com>

Pterosaur force-pushed the macsec_verify_hook branch from db98ffe to f82a69f Compare May 16, 2022 08:22

Merge branch 'master' into macsec_verify_hook

7351e3f

Pterosaur marked this pull request as ready for review May 20, 2022 04:01

Pterosaur requested a review from a team as a code owner May 20, 2022 04:01

Pterosaur requested a review from jimmyzhai May 20, 2022 04:02

jimmyzhai reviewed May 24, 2022

View reviewed changes

Pterosaur force-pushed the macsec_verify_hook branch 2 times, most recently from 0c2a547 to 060adbe Compare May 24, 2022 10:06

Pterosaur requested a review from jimmyzhai May 24, 2022 12:16

Pterosaur force-pushed the macsec_verify_hook branch from 060adbe to 7ac68b9 Compare May 24, 2022 12:48

migrate 2 to 3

774d323

Signed-off-by: Ze Gan <ganze718@gmail.com>

Pterosaur force-pushed the macsec_verify_hook branch from 7ac68b9 to 774d323 Compare May 24, 2022 13:05

Pterosaur added 2 commits May 24, 2022 13:13

Revert "migrate 2 to 3"

93faba2

This reverts commit 774d323.

polish code

25cbd7a

Signed-off-by: Ze Gan <ganze718@gmail.com>

Polish code

900ac47

Signed-off-by: Ze Gan <ganze718@gmail.com>

Pterosaur force-pushed the macsec_verify_hook branch 2 times, most recently from 1527417 to 5eadeaf Compare May 27, 2022 00:58

Move hash_test to py3

b292cb7

Signed-off-by: Ze Gan <ganze718@gmail.com>

Pterosaur force-pushed the macsec_verify_hook branch from 5eadeaf to b292cb7 Compare May 27, 2022 04:21

jimmyzhai reviewed May 27, 2022

View reviewed changes

ansible/roles/test/files/ptftests/macsec.py Outdated Show resolved Hide resolved

jimmyzhai reviewed May 27, 2022

View reviewed changes

Merge branch 'master' into macsec_verify_hook

ed14786

Signed-off-by: Ze Gan <ganze718@gmail.com>

Pterosaur force-pushed the macsec_verify_hook branch from 2beca00 to ed14786 Compare May 27, 2022 08:23

fix comments

d641667

Signed-off-by: Ze Gan <ganze718@gmail.com>

jimmyzhai approved these changes May 30, 2022

View reviewed changes

Pterosaur merged commit 8c7010c into sonic-net:master May 30, 2022

AntonHryshchuk mentioned this pull request Jun 27, 2022

[MACsec] degradation in ptf tests runtime and logic #5876

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[macsec]: Override dp_poll of ptf by MACsec's #5490

[macsec]: Override dp_poll of ptf by MACsec's #5490

Pterosaur commented Apr 8, 2022 •

edited

Loading

lgtm-com bot commented May 12, 2022

lgtm-com bot commented May 13, 2022

lgtm-com bot commented May 14, 2022

lgtm-com bot commented May 16, 2022

jimmyzhai May 24, 2022

Pterosaur May 25, 2022

lgtm-com bot commented May 25, 2022

jimmyzhai May 27, 2022 •

edited

Loading

jimmyzhai May 27, 2022

abdosi commented Jul 26, 2022

Pterosaur commented Jul 27, 2022

[macsec]: Override dp_poll of ptf by MACsec's #5490

[macsec]: Override dp_poll of ptf by MACsec's #5490

Conversation

Pterosaur commented Apr 8, 2022 • edited Loading

Description of PR

Type of change

Back port request

Approach

What is the motivation for this PR?

How did you do it?

How did you verify/test it?

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

lgtm-com bot commented May 12, 2022

lgtm-com bot commented May 13, 2022

lgtm-com bot commented May 14, 2022

lgtm-com bot commented May 16, 2022

jimmyzhai May 24, 2022

Choose a reason for hiding this comment

Pterosaur May 25, 2022

Choose a reason for hiding this comment

lgtm-com bot commented May 25, 2022

jimmyzhai May 27, 2022 • edited Loading

Choose a reason for hiding this comment

jimmyzhai May 27, 2022

Choose a reason for hiding this comment

abdosi commented Jul 26, 2022

Pterosaur commented Jul 27, 2022

Pterosaur commented Apr 8, 2022 •

edited

Loading

jimmyzhai May 27, 2022 •

edited

Loading