Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[expressions] caching #180440

Merged
merged 29 commits into from
May 20, 2024
Merged

[expressions] caching #180440

merged 29 commits into from
May 20, 2024

Conversation

ppisljar
Copy link
Member

@ppisljar ppisljar commented Apr 10, 2024

Summary

adds caching support to expressions. Functions can set allowCaching property to true to indicate they can/should be cached.

When an expression is executed with allowCaching parameter set to true the expression engine will cache results of such functions and reuse them in later executions of expression if:

  • input did not change
  • arguments did not change
  • search context did not change
  • cache did not time out

fix #177548

Checklist

Delete any items that are not applicable to this PR.

@ppisljar
Copy link
Member Author

/ci

@ppisljar ppisljar added release_note:skip Skip the PR/issue when compiling release notes backport:skip This commit does not require backporting v8.15.0 labels Apr 15, 2024
@ppisljar
Copy link
Member Author

/ci

@ppisljar
Copy link
Member Author

/ci

@ppisljar
Copy link
Member Author

/ci

@ppisljar ppisljar marked this pull request as ready for review April 29, 2024 12:30
@ppisljar ppisljar requested review from a team as code owners April 29, 2024 12:30
@botelastic botelastic bot added the Feature:ExpressionLanguage Interpreter expression language (aka canvas pipeline) label Apr 29, 2024
@ppisljar ppisljar added the Team:Visualizations Visualization editors, elastic-charts and infrastructure label Apr 29, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-visualizations (Team:Visualizations)

@ppisljar ppisljar changed the title Expr/caching [expressions] caching Apr 29, 2024
@stratoula stratoula self-requested a review April 30, 2024 09:10
@stratoula
Copy link
Contributor

What a lovely PR improving the experience of our apps 😍

I was mostly focused on the ES|QL experience which was works exactly as I am expecting. I also tested the dataview mode and everything seems to work great except from annotations.

Here I tried to add an annotations layer from the dashboard inline editing.

image

@dej611 @mbondyra I think it would be great both of you to test this PR to ensure that everything works as expected!

@stratoula
Copy link
Contributor

@ppisljar I can't reproduce it now 🤔 Annotations work great in my new tests!

image

Copy link
Contributor

@dej611 dej611 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested thoroughly and found only one issue with absolute shifting in formula.

This is what I get in this PR:
Screenshot 2024-05-07 at 10 30 30

This is what it should show instead:
Screenshot 2024-05-07 at 10 30 19

@stratoula
Copy link
Contributor

one bug only is very good news 🎉

@markov00 markov00 self-requested a review May 8, 2024 12:17
src/plugins/expressions/common/execution/execution.ts Outdated Show resolved Hide resolved
src/plugins/expressions/common/executor/executor.ts Outdated Show resolved Hide resolved
src/plugins/expressions/public/types/index.ts Show resolved Hide resolved
src/plugins/expressions/public/loader.ts Outdated Show resolved Hide resolved
x-pack/plugins/lens/public/utils.ts Show resolved Hide resolved
src/plugins/expressions/common/execution/execution.ts Outdated Show resolved Hide resolved
Copy link
Member

@lukasolson lukasolson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm noticing an odd behavior:

  1. Add a delay to the query
  2. Let the query complete, then change the colors (they change just fine without re-issuing the query)
  3. Refresh the query
  4. Before the query completes, try to change the colors (you get "no results found")

I tried to reproduce on main and I couldn't, so I'm pretty sure this has to do with this PR. It looks like the query is being cancelled for some reason.

Untitled.mov

@ppisljar
Copy link
Member Author

good catch @lukasolson ! I updated the code with a check to make sure we only cache the last result from the observable and we are not caching partial results.

Copy link
Contributor

@dej611 dej611 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested out again and cannot find issues so far 👍

Copy link
Member

@lukasolson lukasolson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
expressions 1756 1763 +7

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
lens 1.5MB 1.5MB +940.0B

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
expressions 5 6 +1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
data 419.0KB 419.0KB +70.0B
expressions 98.6KB 99.5KB +858.0B
total +928.0B
Unknown metric groups

API count

id before after diff
expressions 2217 2233 +16

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@markov00 markov00 self-requested a review May 20, 2024 12:39
Copy link
Member

@markov00 markov00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changes looks good to me, great work here!

@ppisljar ppisljar merged commit 79b8bab into elastic:main May 20, 2024
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting Feature:ExpressionLanguage Interpreter expression language (aka canvas pipeline) performance release_note:skip Skip the PR/issue when compiling release notes Team:Visualizations Visualization editors, elastic-charts and infrastructure v8.15.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Lens] Do not run the fetch request when the user changes chart configuration
8 participants