add activity watching to kernels #1827

minrk · 2016-10-11T14:25:14Z

adds to kernel REST model:

last_activity: datetime-aware timestamp
execution_state: from last recorded status message

related: jupyterlab/jupyterlab#1070

next PR: shutdown idle kernels

minrk · 2016-10-11T15:59:00Z

need to update a few tests to take the added timestamps into account

parente · 2016-10-11T21:05:59Z

I just spotted this in the hackpad. Does it align with the activity tracking added to the kernel gateway a while back?

https://github.com/jupyter/kernel_gateway/blob/master/kernel_gateway/jupyter_websocket/swagger.yaml#L298

parente · 2016-10-11T21:06:30Z

notebook/services/api/api.yaml

+      last_activity:
+        type: string
+        description: |
+          ISO 8601 timestamp for the last-seen activity on this kernel.


Should this explicitly be UTC?

They are explicitly timezone-aware, and UTC as an implementation detail. I'll add that detail, though.

minrk · 2016-10-12T14:58:43Z

@parente I forgot about the KG activity API! This is significantly less than that:

it only tracks a single activity stamp, not categories
it's two added fields (last_activity, execution_state) in the kernels API, rather than a new API endpoint

Do you think we should pull in the KG API wholesale, rather than this simpler tracker? What have you used the more detailed information for?

parente · 2016-10-12T15:39:35Z

Do you think we should pull in the KG API wholesale, rather than this simpler tracker? What have you used the more detailed information for?

I think the extra detail was added shotgun style because we didn't have our hands around everything that operators would want to know to make decisions about monitoring and shutting down kernels. At a minimum it'd be good to have the field names and types that are common match at least, even if the API is not 100% the same.

The original issue is here: jupyter-server/kernel_gateway#96

@rwhorman probably has more detail on what's used or not at the moment.

yuvipanda · 2016-10-24T18:57:39Z

\o/ Can we use this from the hub too, thus taking out last activity tracking from the proxy?

rgbkrk · 2016-10-24T22:18:33Z

notebook/services/kernels/kernelmanager.py

+            "name": kernel.kernel_name,
+            "last_activity": kernel.last_activity,
+            "execution_state": kernel.execution_state,
+        }


rgbkrk · 2016-10-24T22:19:07Z

This is so excellent to see.

yuvipanda · 2016-11-10T18:43:30Z

jupyterhub/jupyterhub#857 is related, talks about adopting a standardized format for exposing white-box metrics (such as last-activity!).

parente · 2016-11-22T18:29:24Z

Haven't heard back from @rwhorman on the comment above.

@jtyberg @lbustelo do either of you guys know if the activity tracking API proposed here vs the one with way more properties in the kernel gateway will satisfy existing use cases?

enigmata · 2016-11-23T12:32:28Z

@parente Nothing conclusive, as we're still experimenting and adjusting our model.

takluyver · 2017-01-13T13:41:33Z

Heads up, this has a merge conflict.

minrk · 2017-01-13T14:54:05Z

Thanks, rebased.

yuvipanda · 2017-01-13T15:09:28Z

Hopefully this will make it into 4.4?

…

On Fri, Jan 13, 2017 at 6:54 AM, Min RK ***@***.***> wrote: Thanks, rebased. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1827 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAB23rAW9fMxFseaT75VW-A6x5fotPLgks5rR5APgaJpZM4KTsKR> .

-- Yuvi Panda T http://yuvi.in/blog

takluyver · 2017-01-13T16:38:35Z

I'm hoping we might go straight to 5.0, because I'm getting tired of telling people that the new clipboard functionality (#1286) is not available yet.

takluyver · 2017-01-20T14:44:55Z

What's the status of this? I'm trying to push towards a 5.0 release. Is this going to be mergeable fairly soon, or should we hold it for 5.1?

adds to kernel REST model: - last_activity: datetime-aware timestamp - execution_state: from last recorded status message

instead of +00:00

add /api/status endpoint for retrieving current status includes - started: start time of the server - last_activity: latest activity across all endpoints - connections: number of current connections - kernels: number of current kernels

because travis is slow

minrk · 2017-01-20T22:52:58Z

@takluyver fixing it now. I think it's in good shape, I just need to update the new tests to deal with the token changes.

ivanov · 2017-01-21T01:57:22Z

The js/tree group for py3.5.1 build was stuck on Travis for 2 hours and 45 minutes without starting, so I stopped it and restarted it just now.

minrk · 2017-01-21T02:56:36Z

Thanks, @ivanov! Looks all green now.

I'm happy with it as-is now, but further reviews and ideas are always welcome.

blink1073 · 2017-01-24T11:31:17Z

🎉

minrk mentioned this pull request Oct 11, 2016

How should sessions be managed? jupyterlab/jupyterlab#1070

Closed

parente reviewed Oct 11, 2016

View reviewed changes

minrk mentioned this pull request Oct 24, 2016

can we find the Jupyter terminals which is running for long ? #1834

Open

minrk added this to the 5.0 milestone Oct 24, 2016

rgbkrk reviewed Oct 24, 2016

View reviewed changes

yuvipanda mentioned this pull request Nov 8, 2016

Read last activity timestamps from single user servers, not proxy jupyterhub/jupyterhub#850

Closed

minrk force-pushed the kernel-activity branch 2 times, most recently from 5d80a46 to 4826fe9 Compare November 18, 2016 10:59

minrk force-pushed the kernel-activity branch 2 times, most recently from 48e7d07 to 26ec1bc Compare November 23, 2016 15:28

minrk force-pushed the kernel-activity branch from 26ec1bc to e0a0d28 Compare January 13, 2017 14:53

minrk added 5 commits January 20, 2017 12:48

add activity watching to kernels

b472c55

adds to kernel REST model: - last_activity: datetime-aware timestamp - execution_state: from last recorded status message

track active kernel connections

d45cb2a

use Z isoformat in UTC timestamps

9f55a85

instead of +00:00

add connections to sessions API tests

bec3963

move tz utils to top-level private module ._tz

8b76de8

minrk added 2 commits January 20, 2017 12:48

track REST API activity

32a3533

add /api/status endpoint for retrieving current status includes - started: start time of the server - last_activity: latest activity across all endpoints - connections: number of current connections - kernels: number of current kernels

give the server a second to notice that the websocket closed

75a0d71

because travis is slow

add token auth to kernel websocket test

2a5f8d2

minrk force-pushed the kernel-activity branch from e0a0d28 to 2a5f8d2 Compare January 20, 2017 23:04

use self.request in status test

dc873f1

blink1073 approved these changes Jan 21, 2017

View reviewed changes

rgbkrk approved these changes Jan 21, 2017

View reviewed changes

parente approved these changes Jan 21, 2017

View reviewed changes

takluyver merged commit f5072cf into jupyter:master Jan 24, 2017

minrk mentioned this pull request Jan 31, 2017

track last activity of kernels #1241

Closed

gnestor added this to Merged PRs in 5.0 Feb 4, 2017

parente mentioned this pull request Feb 19, 2017

Add ability to cull idle kernels after specified period jupyter-server/kernel_gateway#226

Closed

minrk deleted the kernel-activity branch February 20, 2017 13:55

kevin-bates mentioned this pull request Feb 22, 2017

Cull idle kernels #2215

Merged

parente mentioned this pull request Sep 7, 2017

Shift some features from kernel gateway into notebook server? jupyter-server/kernel_gateway#259

Open

parente mentioned this pull request Oct 16, 2017

Question: Does jupyter expose any kind of 'activity' data? #2939

Closed

github-actions bot added the status:resolved-locked label Apr 8, 2021

github-actions bot locked as resolved and limited conversation to collaborators Apr 8, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add activity watching to kernels #1827

add activity watching to kernels #1827

minrk commented Oct 11, 2016

minrk commented Oct 11, 2016

parente commented Oct 11, 2016

parente Oct 11, 2016

minrk Oct 12, 2016

minrk commented Oct 12, 2016

parente commented Oct 12, 2016

yuvipanda commented Oct 24, 2016

rgbkrk Oct 24, 2016

rgbkrk commented Oct 24, 2016

yuvipanda commented Nov 10, 2016 •

edited

Loading

parente commented Nov 22, 2016

enigmata commented Nov 23, 2016

takluyver commented Jan 13, 2017

minrk commented Jan 13, 2017

yuvipanda commented Jan 13, 2017 via email

takluyver commented Jan 13, 2017

takluyver commented Jan 20, 2017

minrk commented Jan 20, 2017

ivanov commented Jan 21, 2017

minrk commented Jan 21, 2017

blink1073 commented Jan 24, 2017

add activity watching to kernels #1827

add activity watching to kernels #1827

Conversation

minrk commented Oct 11, 2016

minrk commented Oct 11, 2016

parente commented Oct 11, 2016

parente Oct 11, 2016

Choose a reason for hiding this comment

minrk Oct 12, 2016

Choose a reason for hiding this comment

minrk commented Oct 12, 2016

parente commented Oct 12, 2016

yuvipanda commented Oct 24, 2016

rgbkrk Oct 24, 2016

Choose a reason for hiding this comment

rgbkrk commented Oct 24, 2016

yuvipanda commented Nov 10, 2016 • edited Loading

parente commented Nov 22, 2016

enigmata commented Nov 23, 2016

takluyver commented Jan 13, 2017

minrk commented Jan 13, 2017

yuvipanda commented Jan 13, 2017 via email

takluyver commented Jan 13, 2017

takluyver commented Jan 20, 2017

minrk commented Jan 20, 2017

ivanov commented Jan 21, 2017

minrk commented Jan 21, 2017

blink1073 commented Jan 24, 2017

yuvipanda commented Nov 10, 2016 •

edited

Loading