Sketches Schema UI view #1015
Merged
Sketches Schema UI view #1015
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
To complete this:
|
skrawcz
added
UI
This is hacky UI code. It does not handle "run ids". 1. Adds to pandas & pyspark schema capture using `h_schema.py`. 2. Adds SchemaView and related types to try to show a table... So it's either we've set something up with too many things, or this is just a factor of using typescript... Basically I dont like how many things I had to add to do this.. Otherwise it's non-obvious to me what the pattern should be to handle comparing runs -- this UI change for the schema table doesn't take that into account -- it should probably use Generic Table but I couldn't wrap my head around it. Otherwise open question whether we reuse the h_schema pyarrow stuff, or just roll our own again...
This will make it look like the others, make it expandable, etc...
elijahbenizzy
force-pushed
the
schema_viz
branch
from
d34c9dc
to
ecc9337
Compare
This allows us to hide attributes now that we have multiple
|
Fixed up schema view: Added expand/contract now that we have multiple: |
We now have three categories: 1. Result summaries -- we can output one and if not it will be unsupported in the UI 2. Schema -- we can output one or none 3. Additional -- as many as we want We use a single dispatch function for each, and it makes the code a lot cleaner. We no longer put stuff in lists unless its an additional result. Note they can also supply their own name, if not, we will generate a unique attribute name. In the long term we'll likely want to stop using single dispatch as we need to register multiple with roles (and single dispatch requires one function per role). For now this is clean enough and easy to work with, however.
skrawcz
commented
Jul 13, 2024
Comment on lines
150
to
189
| // dataTypeDisplay={(item: string) => { | ||
| // return ( | ||
| // <RunLink | ||
| // projectId={props.projectId} | ||
| // runId={parseInt(item) as number} | ||
| // setHighlightedRun={() => void 0} | ||
| // highlightedRun={null} | ||
| // ></RunLink> | ||
| // ); | ||
| // }} |
skrawcz
commented
Jul 13, 2024
Comment on lines
92
to
108
| { | ||
| "observability_type": "primitive", | ||
| "observability_value": { | ||
| "type": str(str), | ||
| "value": o_value["cost_explain"], | ||
| }, | ||
| "observability_schema_version": "0.0.1", | ||
| "name": "Cost Explain", | ||
| }, | ||
| { | ||
| "observability_type": "primitive", | ||
| "observability_value": { | ||
| "type": str(str), | ||
| "value": o_value["extended_explain"], | ||
| }, | ||
| "observability_schema_version": "0.0.1", | ||
| "name": "Extended Explain", |
There was a problem hiding this comment.
I don't see these added to additional...
There was a problem hiding this comment.
Oh I just stuck in the o_value there cause I figured it was easier. Doesn't allow us to compare, though, so it might be nice to have. Will add it to additional
elijahbenizzy
force-pushed
the
schema_viz
branch
2 times, most recently
from
1c142f9
to
32b1ef1
Compare
This just treats all primitives as strings, which can show the view. This works with strings and will actually just look fine otherwise. It uses the ReactDiffViewer component which is simple, and we use elsewhere in the app. We can probably tune this a bit (the interaction is a little jumpy), but for now this is OK.
We had captured all of them. Now we capture individual data as well, which allows for easy comparison. It's duplicated, so we use an lru_tools cache (which should cache based on the pyspark dataframe ID)
elijahbenizzy
force-pushed
the
schema_viz
branch
from
71ed230
to
161b028
Compare
elijahbenizzy
approved these changes
Jul 16, 2024
Comment on lines
92
to
108
| { | ||
| "observability_type": "primitive", | ||
| "observability_value": { | ||
| "type": str(str), | ||
| "value": o_value["cost_explain"], | ||
| }, | ||
| "observability_schema_version": "0.0.1", | ||
| "name": "Cost Explain", | ||
| }, | ||
| { | ||
| "observability_type": "primitive", | ||
| "observability_value": { | ||
| "type": str(str), | ||
| "value": o_value["extended_explain"], | ||
| }, | ||
| "observability_schema_version": "0.0.1", | ||
| "name": "Extended Explain", |
There was a problem hiding this comment.
Oh I just stuck in the o_value there cause I figured it was easier. Doesn't allow us to compare, though, so it might be nice to have. Will add it to additional
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Original description follows. This has been fixed up to work correctly.
This is hacky UI code. It does not handle "run ids".
h_schema.py.So it's either we've set something up with too many things, or this is just a factor of using typescript... Basically I dont like how many things I had to add to do this..
Otherwise it's non-obvious to me what the pattern should be to handle comparing runs -- this UI change for the schema table doesn't take that into account -- it should probably use Generic Table but I couldn't wrap my head around it.
Changes
How I tested this
Notes
h_schemasince I kind of already have some of this... we should align on what the "schema" view actually takes in too.Checklist