Tasks progression frozen on gvmd / gsad side, while actually on-going or even finished on ospd / ospd-openvas side

[wisukind]

Greenbone Vulnerability Manager 21.4.3dev1git-dbe1d6ed1-gvmd-21.04
GIT revision dbe1d6e-gvmd-21.04
Manager DB revision 242
OSP Server for openvas: 21.4.3.dev1
OSP: 21.4.4.dev1
OSPd OpenVAS: 21.4.4.dev1
OpenVAS 21.4.3dev1git-e5624005-openvas-21.04
GIT revision git-e5624005-openvas-21.04
gvm-libs 21.4.3dev1~git-adaaaaa8-gvm-libs-21.04

OS:

Linux ov-slave-monterrey 4.15.0-20-generic #21-Ubuntu SMP Tue Apr 24 06:16:15 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

In some circumstances (often after a short network communication failure between gvmd & scanner), gvmd freeze tasks progression and is unable to refresh status; despite the fact that ospd-openvas is up and tasks are ongoing or set as finished on openvas side. Moreover, in case we want to refresh the task status by stopping / resuming it; the following happens:

1. Gvmd will start a new scan on the scanner side (note it's a NEW scan, so it will run concurrently with the initial one, which may be still running !!?)
2. After a few seconds on gvmd side; scan status will change to Interrupted state
3. Both scans continue to run on ospd side (even the latest one, which appears now as "Interrupted")

Overrall that's always the same story; gvmd is not reliably maintaining the communication with ospd and loose tracks of scans tasks too easily. It should be able to resume tasks where it left, and also to get up to date scan status from the scanner directly, even after the connection has dropped for some time. For old timers, please note this issue was ALREADY signaled under GVM 11. It was closed as being too old when GVM 20+ was released.

Expected behaviour: Gvmd shouldn't start a new scan when the connection is back, but instead resuming the ongoing / finished one (and putting the state as accordingly). And gvmd shouldn't put the resumed task as Interrupted.

Note: Syncing scan IDs between GVM and ospd-openvas would also greatly increase the debugging capability and the overrall investigations. This request was also already submitted back in GVM 11 !

[jjnicola]

I want to add that @wisukind is experimenting this issue with a master-sensor setup, via TLS connection. @wisukind please corrects me if I am wrong.

[wisukind]

Yes, I confirm. It's a master slave setup. Thanks for clarifying.

[hooNger2]

I can confirm this problem. Currently, no scans are possible.

[jjnicola]

Hi @wisukind ! So, we moved to gvmd side now :)
Some days ago, I created the PR #1679. I think this issue is related and the PR can help here. Feel free to try it and give some feedback. And as always, thank you very much for reporting!
Best regards

[wisukind]

@jnicolas, I have upgraded to the latest revision, and I confirm I no longer see this bug. It may be a little too early to close this bug though, since this problem is erratic and I don't have tested it enough, but this is encouraging !

[wisukind]

Hello, unfortunately I just had the same problem again with latest build. It seems to happens less frequently, though. But during the night I had 2 scans changed to Interrupted without reasons. :-(

[wisukind]

Hello,

Bug still occuring with 21.4.3 final version. Much less frequent though, but still occuring on some tasks. Thanks

[MitchDrage]

I believe I'm facing the same issue, however I'm running the community containers.

Greenbone Vulnerability Manager 22.4.2
Greenbone Security Assistant 22.04.0
OSPd OpenVAS: 22.4.6

greenbone-community-edition-gvmd-1                 | event task:MESSAGE:2023-05-02 11h25.49 AEST:330: Status of task <redacted> (9d3fb795-88b8-40a8-b759-69d36462cbff) has changed to Requested
greenbone-community-edition-gvmd-1                 | event task:MESSAGE:2023-05-02 11h25.49 AEST:330: Task <redacted> (9d3fb795-88b8-40a8-b759-69d36462cbff) has been requested to start by admin
greenbone-community-edition-gvmd-1                 | event task:MESSAGE:2023-05-02 11h26.01 AEST:362: Status of task <redacted> (9d3fb795-88b8-40a8-b759-69d36462cbff) has changed to Stopped
greenbone-community-edition-ospd-openvas-1         | OSPD[7] 2023-05-02 01:26:10,773: INFO: (ospd.command.command) Scan 4316b248-eef6-40f4-8ad6-7306541b53c5 added to the queue in position 1.
greenbone-community-edition-ospd-openvas-1         | OSPD[7] 2023-05-02 01:26:18,430: INFO: (ospd.ospd) Currently 1 queued scans.
greenbone-community-edition-ospd-openvas-1         | OSPD[7] 2023-05-02 01:26:18,629: INFO: (ospd.ospd) Starting scan 4316b248-eef6-40f4-8ad6-7306541b53c5.
greenbone-community-edition-mqtt-broker-1          | 1682990801: New connection from 172.18.0.7:58866 on port 1883.
greenbone-community-edition-mqtt-broker-1          | 1682990801: New client connected from 172.18.0.7:58866 as 5e82308a-0eca-41db-9a79-e73eedae9b3f (p5, c1, k0).

Is there any more-detailed debugging I can enable to look at why there is a disconnect between GVMD and OSPD?