make provider reconnectable

[jaym]

Makes providers reconnectable. Keeps track of all connect requests to a provider. If it crashes, and is restarted, all the connect requests are replayed. Disconnected connections are also recreated because they can be a parent connect, and its non-existence can cause a child to fail to connect

[github-actions]

Test Results

3 098 tests  +1   3 097 ✅ +1   1m 18s ⏱️ -4s
  370 suites ±0       1 💤 ±0 
   28 files   ±0       0 ❌ ±0 

Results for commit 29fa5a8. ± Comparison against base commit 483ba82.

♻️ This comment has been updated with latest results.

[imilchev]

I think this is going to be leaking connections for long running instance (serve mode). If we always reconnect all connections, they won't be closed properly. The scanner will only close connections that were just scanned.

I think this may be solved easily. If you remove the connectRequest when Disconnect is called. We call Disconnect when we "gracefully" shut down the connection. If that happens, it should be safe to drop the connectRequest too for this connection. Parent - child connections are tracked already so it's safe to assume we won't call Disconnect for a parent before the child is scanned

[jaym]

I've updated things so that we keep track of which connections rely on which other connections. This information is used to reconnect. If we disconnect a connection and something else relies on it, we keep around the reconnection information until the thing that relies on it is disconnected

[imilchev]

One general question that I have is what happens in the following case:

1. I connect to the provider with a root asset
2. I connect to the provider with child asset of that root
3. I scan the root
4. Root connection is closed
5. Provider crashes
6. We trigger a restart + reconnect

After step 6 is done what connection will be open for the restarted provider?

[imilchev]

could you add some more comments what we store in these properties? It's not immediately clear for me when I see it, so it may help us in case we need to change this in the future

[imilchev]

it would also help to describe in text how this is intended to work and put it in a comment:

• when do we add connections to the graph?
• when do we remove connections from the graph?

[jaym]

Sure, will add comments here

[jaym]

One general question that I have is what happens in the following case:

1. I connect to the provider with a root asset

2. I connect to the provider with child asset of that root

3. I scan the root

4. Root connection is closed

5. Provider crashes

6. We trigger a restart + reconnect

After step 6 is done what connection will be open for the restarted provider?

So assuming there is another asset that needs to be scanned, and it has a connection that depends on the root, first the root connection is recreated, and then the next assets connection. mondoohq/cnspec#1412 has the change to try to make sure the providers are up before starting an asset scan.

The root connection is then disconnected automatically once all the connections that depend on it are disconnected