Add Heartbeat #4

jeanp413 · 2022-06-29T17:00:55Z

Move heartbeat to gitpod-desktop from gitpod-remote

How to test:

Run this extension
Connect to a workspace in this prev env (use SSH gateway for now) https://jp-heartbeat-test.preview.gitpod-dev.com/workspaces (Add function:sendHeartBeat scope gitpod#11021)
- make sure you have the debug VSCode Desktop window focus
Install this gitpod-remote vsix gitpod-remote-ssh-0.0.42.zip (related PR Remove heartbeat in gitpod-remote openvscode-server#378)
- Upload .zip file to workspace, unzip it
- Right-click on .vsix file and install it
Check __gitpod.cancelGitpodRemoteHeartbeat command executed is logged in the gitpod-remote logs
Open Gitpod with this PR Remove heartbeat in gitpod-remote openvscode-server#378
Exec commands below to see if heartbeat sent

cd dev/gpctl
go build
./gpctl workspaces last-heartbeat <instance_id>

Check only one close heartbeat is send

~~Not sure how to trigger ssh reconnection~~ 🤔

Maybe use sudo killall sshd, but window closed after sshd terminate, not sure if it works

jeanp413 · 2022-06-29T21:41:20Z

cc @akosyakov

akosyakov · 2022-06-30T06:45:30Z

Not sure how to trigger ssh reconnection 🤔

I think you don't need to care here. The test case will be:

observing that heartbeating is happening while window is opened and workspace is running
if workspace is not running heartbeating should be stopped
if window is closed close heartbeat should be send immediately, it should be preset in analytics as well

akosyakov

It looks like a right direction, but we don't respect workspace lifecycle yet.

src/extension.ts

src/heartbeat.ts

src/remoteConnector.ts

src/heartbeat.ts

src/remoteConnector.ts

src/heartbeat.ts

src/remoteConnector.ts

akosyakov · 2022-06-30T06:46:59Z

I opened gitpod-io/gitpod#11021 since I needed to rerun werft job.

src/remoteConnector.ts

src/heartbeat.ts

src/remoteConnector.ts

akosyakov · 2022-07-04T08:08:27Z

It looks great, only tracking of workspace running state is missing that we don't send events if workspace is not running.

src/heartbeat.ts

src/remoteConnector.ts

akosyakov · 2022-07-06T07:25:18Z

It looks almost there, some tweak are needed for heatbeating code. @mustard-mh @andreafalzetti since I'm away till next Monday, could you help to finish with testing after another round of changes? 🙏

src/remoteConnector.ts

akosyakov · 2022-07-11T06:16:26Z

I think logic looks well. Could someone help with testing and giving another look at it? 🙏 cc @mustard-mh @andreafalzetti

mustard-mh · 2022-07-11T09:01:53Z

Tested, and it works well.

But extension requests auth twice when I'm testing. I'll update How to Test section later

gitpod /workspace/gitpod/dev/gpctl (jp/heartbeat-test) $ ./gpctl workspaces last-heartbeat 7fe830ee-af57-49bb-8c6d-91ad81cfc705
2022-07-11T08:54:24.876441322Z
gitpod /workspace/gitpod/dev/gpctl (jp/heartbeat-test) $ ./gpctl workspaces last-heartbeat 7fe830ee-af57-49bb-8c6d-91ad81cfc705
2022-07-11T08:56:18.843874055Z
gitpod /workspace/gitpod/dev/gpctl (jp/heartbeat-test) $ ./gpctl workspaces last-heartbeat 7fe830ee-af57-49bb-8c6d-91ad81cfc705
2022-07-11T08:56:18.843874055Z
gitpod /workspace/gitpod/dev/gpctl (jp/heartbeat-test) $ ./gpctl workspaces last-heartbeat 7fe830ee-af57-49bb-8c6d-91ad81cfc705
2022-07-11T08:56:48.681459343Z

mustard-mh · 2022-07-11T09:03:28Z

After exec

sudo killall sshd

The test VSCode Desktop window closed, don't know if reconnect works or not, since next time I open it, server maybe another one...

src/internalApi.ts

mustard-mh · 2022-07-11T10:13:26Z

src/remoteConnector.ts

+		// Check for gitpod remote extension version to avoid sending heartbeat in both extensions at the same time
+		const gitpodRemoteVersion = await getGitpodRemoteVersion();
+		if (gitpodRemoteVersion && semver.gte(gitpodRemoteVersion, '0.0.40')) {
+			const session = await this.getGitpodSession(connectionInfo.gitpodHost);


Should we use same session above?

Nope, vscode takes care of returning the same session if the scopes are the same, also this code will execute in a new window/after reload when it's connecting to a remote workspace, and the previous code only gets executed when opening a callback uri

mustard-mh

code LGTM

akosyakov

🚀

jeanp413 · 2022-07-12T07:49:00Z

Could you do another test 🙏 I updated the test steps
Changed it so gitpod-remote exposes a command to cancel the heartbeat and if this command returns successful then we start heartbeat on gitpod-desktop, this is safer as it could be possible to end up with many version combinations of both extensions as users could not always update both extensions at the same time
Also added a new telemetry event vscode_desktop_heartbeat_state so we can track if all new connections always activate the heartbeat in gitpod-desktop, If we have 100% rate and all events come from latest extension versions then we can remove the heartbeat code in gitpod-remote and also this telemetry event

mustard-mh · 2022-07-12T08:02:36Z

We can disable computer's network to test reconnect. 😄

akosyakov · 2022-07-12T09:18:57Z

Also added a new telemetry event vscode_desktop_heartbeat_state so we can track if all new connections always activate the heartbeat in gitpod-desktop, If we have 100% rate and all events come from latest extension versions then we can remove the heartbeat code in gitpod-remote and also this telemetry event

sounds good, please update the tracking plan as usual 🙏

akosyakov · 2022-07-12T14:41:34Z

src/remoteConnector.ts

+		await this.context.globalState.update(`${RemoteConnector.SSH_DEST_KEY}${sshDestStr}`, { ...connectionInfo, isFirstConnection: false });
+
+		// Check for gitpod remote extension version to avoid sending heartbeat in both extensions at the same time
+		const isGitpodRemoteHeartbeatCancelled = await cancelGitpodRemoteHeartbeat();


I'm not sure about it. We see that remote-ssh-extension fails to install right now very often. If we do it then heartbeat is missing.

andreafalzetti · 2022-07-12T18:38:33Z

I am having issues to connect to the workspace. Any idea? 🤔

[Info  - 18:41:40.325] gitpod.gitpod-desktop/0.0.40 (21.5.0 darwin arm64) vscode/1.69.0 (Visual Studio Code)
[Info  - 18:41:40.456] Reading sessions from keychain...
[Info  - 18:41:40.458] Getting sessions for function:accessCodeSyncStorage,function:getLoggedInUser,resource:default...
[Trace  - 18:41:40.471] Token acquired from secret storage.
[Info  - 18:41:40.471] Got stored sessions!
[Trace  - 18:41:40.471] Read the following session from the keychain with the following scopes: function:getLoggedInUser function:getOwnerToken function:getWorkspace resource:default
[Info  - 18:41:40.471] Got 1 verified sessions.
[Info  - 18:41:40.471] Got 0 sessions for function:accessCodeSyncStorage,function:getLoggedInUser,resource:default...
[Info  - 18:41:51.320] Opening Gitpod workspace
vscode://gitpod.gitpod-desktop/workspace/gitpod-experiments-a?%7B%22instanceId%22%3A%22ec8fc343-ae4d-4e44-b912-7ff4db88089b%22%2C%22workspaceId%22%3A%22andreafalze-gitpodexper-q2nellczhu0%22%2C%22gitpodHost%22%3A%22https%3A%2F%2Fjp-heartbeat-test.preview.gitpod-dev.com%22%7D
[Info  - 18:41:51.339] Getting sessions for function:getLoggedInUser,function:getOwnerToken,function:getWorkspace,resource:default...
[Info  - 18:41:51.339] Got 1 sessions for function:getLoggedInUser,function:getOwnerToken,function:getWorkspace,resource:default...
[Error  - 18:42:13.899] SSH test connection error:
Error: Timed out while waiting for handshake
    at Timeout._onTimeout (/Users/andrea/dev/gitpod/gitpod-vscode-desktop/node_modules/ssh2/lib/client.js:1014:23)
    at listOnTimeout (node:internal/timers:557:17)
    at processTimers (node:internal/timers:500:7)

jeanp413 · 2022-07-13T17:08:02Z

You need to create a new prev env, the one I posted got merged

akosyakov reviewed Jun 30, 2022

View reviewed changes

src/remoteConnector.ts Outdated Show resolved Hide resolved

jeanp413 added 6 commits July 1, 2022 01:02

WIP: heartbeat

8e790bc

Fixes

3f4f41d

Fix

de87296

TODO: Clean up this

02ddc51

Fix

84dd919

💄

0dd2b15

jeanp413 force-pushed the jp/heartbeat branch from fdaf148 to 0dd2b15 Compare July 1, 2022 06:02

jeanp413 added 2 commits July 2, 2022 12:32

Feedback

c67351b

💄

4bd0a46

akosyakov reviewed Jul 4, 2022

View reviewed changes

src/heartbeat.ts Outdated Show resolved Hide resolved

akosyakov reviewed Jul 4, 2022

View reviewed changes

src/remoteConnector.ts Outdated Show resolved Hide resolved

Check if workspace is running before sending heartbeat

4740630