[202012] Fix race condition caused by strand wrap method #104

[zjswhhh]

3f7a6f2 Fix race condition caused by strand wrap method (#104)

Description of PR

Summary:
Fixes # (issue)
This PR is to fix race condition introduced by wrap method of boost::asio::io_service::strand:

In the following case:

async_op_1(..., s.wrap(a));
async_op_2(..., s.wrap(b));

the completion of the first async operation will perform s.dispatch(a), and the second will perform s.dispatch(b), but the order in which those are performed is unspecified. That is, you cannot state whether one happens-before the other. Therefore, none of the above conditions are met and no ordering guarantee is made.

sign-off: Jing Zhang zhangjing@microsoft.com

Type of change

• Bug fix
• New feature
• Doc/Design
• Unit test

Approach

What is the motivation for this PR?

To fix the bug that linkgmrd cleans mux metrics table after setting mux state to orchagent.

How did you do it?

1. Use boost::asio::post() to replace wrap.
2. Add unit tests to cover this race condition scenario:
   • Remove override of setMuxState, postMetricsEvent in FakeDbinterface so the real implementation can be executed.
   • Override setMuxState, postMetricsEvent in FakeMuxPort as LinkManagerStateMachine tests use fake mux ports, so existing tests won't be broken.
   • Initiate thread pool in MuxManager test to better mimic, and run 1000 times setMuxState & postMetricsEvent to reproduce.

How did you verify/test it?

Unit tests. Verified:

1. With wrap, issue was reproduced 14/1000 times.

[  FAILED  ] 1 test, listed below:
[  FAILED  ] MuxManagerTest.DbInterfaceRaceConditionCheck
test/MuxManagerTest.cpp:1014: Failure
Value of: mDbInterfacePtr->mDbInterfaceRaceConditionCheckFailure
  Actual: true
Expected: false
...

2. With boost::asio::post(), issue was not reproducible.

Any platform specific information?

Documentation

[lguohan]

lgtm failed, is that expected?

[zjswhhh]

lgtm failed, is that expected?

Not really, it doesn't seem to be caused by PR changes.

[zjswhhh]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).