-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Diffex improvements: final steps to be done #235
Comments
@maniarathi here's the remaining diffex work. I tagged all our tiledb experts and will defer to the team to see who gets the privilege of doing the work :D. |
Thanks @blrnw3! 👋 |
@blrnw3 Thank you! |
@atolopko-czi @maniarathi - could you review the revised plan, and add owners/tasks/etc? Please note the parallelism possible. Much of the remaining work can occur in parallel. |
@atolopko-czi @metakuni @tihuan , I have made a suggestion on how to split up the work. Let me know if this looks OK to you. |
Related: item one is complete (bugs reported, temporary work-arounds in place), so the work for items 2, 3, 4, and 5 are unblocked. |
For item 8 analytics support, do we know what we want to track already or still need to define that? Thank you! |
CC: @maniarathi - I believe you added this. Is there a spec? @tihuan - if not, I suggest we keep it simple:
|
Sounds great thank you! I'm thinking maybe if those can be tracked in BE it'll be more reliable. Unless some of the info is only available in FE (besides the time to complete)? |
@bkmartinjr @tihuan This was adding the Plausible tags so that our UXR team can also observe the usage of differential expression. |
Thanks @maniarathi ! Plausible tags to record the metrics Bruce mentioned above or something else? Plausible also allows sending events from the server for reliable tracking 🙆♂️ |
I recommend client-side events as this allows the capture of failures and end-to-end timing as well. We will be blind to many of those without doing this on the client-side. |
Revised plan to pull out separate work-streams.
NOTE: dependencies are explicitly declared (eg, item 2 is dependent on item 1, and items 2,3,4, and 5 can be done in parallel).
Resolve possibly blocking tiledb bugs:
Differential expression performance improvements #234 (comment). Test case reported to TileDB, and fix expected in 0.13.2. Work-around is currently in place.
CXG Schema Update - goal: peformance tune the CXG X schema
{"sm.query.dense.reader": "legacy"}
to the TileDB context used for reading/writing dense arrays (has no effect on sparse read/write) OBSOLETE with tiledb update.Diffex algo tuning in Explorer - goal: faster diffex compute in explorer backend.
update deployment instance type and CDN config
Integrate new diffex postings list encoding into explorer f/e and b/e.
System and performance test
Deploy diffexp limit update:
Analytics support
Remove tiledb 0.13.1 work-arounds
Ben's original notes for context
Apologies that I couldn't finish off this work. Here are the remaining steps as I see them
Steps to do, in order
{"sm.query.dense.reader": "legacy"}
Cooldown story: add feature-level analytics support so we can gather metrics on diffex usage
The text was updated successfully, but these errors were encountered: