[O11y][MySQL] Migrate `[Logs MySQL] Overview` dashboard visualizations to lens in kibana 8.8.0 #5818

rajvi-patel-22 · 2023-04-07T10:07:44Z

Enhancement

What does this PR do?

Migrate visualizations to lens for MySQL integration package
Statistics for MySQL Lens migration:

MySQL dashboard	Before Migration		After Migration
	Lens	Visualization	Lens	Visualization
[Logs MySQL] Overview	0	5	5	0
	0	5	5	0

Checklist

I have added an entry to my package's changelog.yml file.
I have verified that panels are populated with data.
I have verified that panels are not distorted after being migrated to lens.
I have updated screenshots of dashboard.
I have verified that data count are matching and panel aggregations are same as before.

Author's Checklist

Migrated panels should be removed from visualization folder.
Migrated visualizations are populating in current Kibana version 8.6.0 itself.

Related issues

Relates Lens Migration of OBS integrations #5221

Note:

The Kibana files and before migration dashboard image is not synced. Please find below screenshot to compare before and after migrated panels.

elasticmachine · 2023-04-07T10:42:43Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2023-07-18T12:36:28.751+0000
Duration: 23 min 10 sec

Test stats 🧪

Test	Results
Failed	0
Passed	57
Skipped	0
Total	57

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

elasticmachine · 2023-04-07T10:42:54Z

🌐 Coverage report

Name	Metrics % (`covered/total`)	Diff
Packages	100.0% (`2/2`)	💚
Files	100.0% (`2/2`)	💚
Classes	100.0% (`2/2`)	💚
Methods	92.857% (`26/28`)	👍 26.19
Lines	94.34% (`150/159`)	👎 -5.66
Conditionals	100.0% (`0/0`)	💚

agithomas

Can u update the query time duration unit and convert to sec /min/ hour unit?

agithomas · 2023-04-08T13:30:25Z

@lalit-satapathy , the version change is a steep jump to 8.6.0 for MySQL. Are there concerns?

@rajvi-elastic , can you please share the details what led to this steep jump?

rajvi-patel-22 · 2023-04-10T05:13:25Z

@rajvi-elastic , can you please share the details what led to this steep jump?

@agithomas, Below are the 3 panels which uses conditional operations (ifelse) which is supported in 8.6 and later versions.

[Metrics MySQL] Database Overview	Minimum Kibana Version	Functionality
Buffer Pool Utilization	8.6.0	Adds conditional operations in the Lens formula
Buffer Pool Efficiency	8.6.0	Adds conditional operations in the Lens formula
Network Traffic	8.6.0	Adds conditional operations in the Lens formula

rajvi-patel-22 · 2023-04-10T05:23:19Z

Can u update the query time duration unit and convert to sec /min/ hour unit?

@agithomas, According to the field description, the values are in nanoseconds. Should we convert it to milliseconds or seconds?

agithomas · 2023-04-10T16:25:17Z

@agithomas, According to the field description, the values are in nanoseconds. Should we convert it to milliseconds or seconds?

Yes, please proceed with representing time in human readable format.

agithomas · 2023-04-10T16:27:59Z

@rajvi-elastic , can you please share the details what led to this steep jump?

@agithomas, Below are the 3 panels which uses conditional operations (ifelse) which is supported in 8.6 and later versions.

[Metrics MySQL] Database Overview Minimum Kibana Version Functionality
Buffer Pool Utilization 8.6.0 Adds conditional operations in the Lens formula
Buffer Pool Efficiency 8.6.0 Adds conditional operations in the Lens formula
Network Traffic 8.6.0 Adds conditional operations in the Lens formula

@lalit-satapathy , Please note this dependency. Please let us know if there exist any concerns.

rajvi-patel-22 · 2023-04-11T05:50:29Z

Yes, please proceed with representing time in human readable format.

@agithomas, lens doesn't support human readable format for duration. However, we can normalize the value to millisecond/second using division function. But in that case there would be restrictions when value is too small. In such cases, normalized value might be near to zero and doesn't show accurate results.

agithomas · 2023-04-11T07:00:58Z

@agithomas, lens doesn't support human readable format for duration. However, we can normalize the value to millisecond/second using division function.

Please consider milli-seconds, consider the usecase here of query time.
Also, please see if you can add decimals of 2 numbers to handle when duration is less than 0ms
Please add the unit (milli second) or (ms) in the column title.

rajvi-patel-22 · 2023-04-11T13:35:20Z

@agithomas, lens doesn't support human readable format for duration. However, we can normalize the value to millisecond/second using division function.

Please consider milli-seconds, consider the usecase here of query time. Also, please see if you can add decimals of 2 numbers to handle when duration is less than 0ms Please add the unit (milli second) or (ms) in the column title.

@agithomas, PR has been updated with suggested changes.

lalit-satapathy · 2023-04-12T03:39:51Z

@lalit-satapathy , the version change is a steep jump to 8.6.0 for MySQL. Are there concerns?

Need to wait for large version changes like this below, as we are discussing with others:

 - kibana.version: "^7.14.0 || ^8.0.0"
 + kibana.version: "^8.6.0"

botelastic · 2023-05-16T14:01:13Z

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

botelastic · 2023-06-15T14:53:25Z

Hi! This PR has been stale for a while and we're going to close it as part of our cleanup procedure. We appreciate your contribution and would like to apologize if we have not been able to review it, due to the current heavy load of the team. Feel free to re-open this PR if you think it should stay open and is worth rebasing. Thank you for your contribution!

lalit-satapathy · 2023-07-17T11:33:30Z

@lalit-satapathy , the version change is a steep jump to 8.6.0 for MySQL. Are there concerns?

Need to wait for large version changes like this below, as we are discussing with others:
 - kibana.version: "^7.14.0 || ^8.0.0"
 + kibana.version: "^8.6.0"

We should go ahead with version change for Lens, as TSDB will soon migrate this package to 8.8

lalit-satapathy · 2023-07-17T11:34:54Z

FYI mysql package is already at 8.8.0 kibana version, so this can be un-gated.

rajvi-patel-22 · 2023-07-17T11:45:17Z

FYI mysql package is already at 8.8.0 kibana version, so this can be un-gated.

Yes @lalit-satapathy, I have updated the MySQL PRs (#5818 & #5819).

agithomas

LGTM!

harnish-elastic

LGTM!

elasticmachine · 2023-07-19T10:25:21Z

Package mysql - 1.13.0 containing this change is available at https://epr.elastic.co/search?package=mysql

…s to lens in kibana 8.8.0 (#5818) * Migrate Logs Overview dashboard visualizations to lens * Update changelog * Convert query time to ms * Update screenshot and export dashboard from 8.8.0 * Export dashboard from 8.7.0

Migrate Logs Overview dashboard visualizations to lens

94595ba

rajvi-patel-22 requested a review from a team as a code owner April 7, 2023 10:07

Update changelog

c35eff1

rajvi-patel-22 mentioned this pull request Apr 7, 2023

[O11y][MySQL] Lens Migration #5709

Closed

6 tasks

kush-elastic added enhancement New feature or request Integration:mysql MySQL v8.6.0 Team:Service-Integrations Label for the Service Integrations team labels Apr 7, 2023

agithomas reviewed Apr 8, 2023

View reviewed changes

Convert query time to ms

597dc85

rajvi-patel-22 requested a review from agithomas April 11, 2023 13:35

agithomas mentioned this pull request Apr 25, 2023

Lens Migration of OBS integrations #5221

Closed

36 tasks

This was referenced Apr 25, 2023

[O11y][HAProxy] Lens Migration #5584

Closed

[O11y][Kafka] Lens Migration #5591

Closed

botelastic bot added the Stalled label May 16, 2023

botelastic bot closed this Jun 15, 2023

rajvi-patel-22 reopened this Jun 29, 2023

rajvi-patel-22 requested a review from a team as a code owner June 29, 2023 07:28

botelastic bot removed the Stalled label Jun 29, 2023

rajvi-patel-22 self-assigned this Jul 10, 2023

rajvi-patel-22 added 2 commits July 14, 2023 16:31

Resolve merge conflicts

0fb26bf

Update screenshot and export dashboard from 8.8.0

0439d80

rajvi-patel-22 changed the title ~~[O11y][MySQL] Migrate [Logs MySQL] Overview dashboard visualizations to lens in kibana 8.6.0~~ [O11y][MySQL] Migrate [Logs MySQL] Overview dashboard visualizations to lens in kibana 8.8.0 Jul 18, 2023

Export dashboard from 8.7.0

f757f4d

rajvi-patel-22 mentioned this pull request Jul 18, 2023

[O11y][MySQL] Migrate [Metrics MySQL] Database Overview dashboard visualizations to lens in kibana 8.8.0 #5819

Merged

7 tasks

agithomas approved these changes Jul 19, 2023

View reviewed changes

harnish-elastic approved these changes Jul 19, 2023

View reviewed changes

rajvi-patel-22 merged commit fff5b94 into elastic:main Jul 19, 2023
1 check passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[O11y][MySQL] Migrate `[Logs MySQL] Overview` dashboard visualizations to lens in kibana 8.8.0 #5818

[O11y][MySQL] Migrate `[Logs MySQL] Overview` dashboard visualizations to lens in kibana 8.8.0 #5818

rajvi-patel-22 commented Apr 7, 2023 •

edited

Loading

elasticmachine commented Apr 7, 2023 •

edited

Loading

Build stats

Test stats 🧪

elasticmachine commented Apr 7, 2023 •

edited

Loading

agithomas left a comment

agithomas commented Apr 8, 2023

rajvi-patel-22 commented Apr 10, 2023 •

edited

Loading

rajvi-patel-22 commented Apr 10, 2023

agithomas commented Apr 10, 2023

agithomas commented Apr 10, 2023

rajvi-patel-22 commented Apr 11, 2023

agithomas commented Apr 11, 2023

rajvi-patel-22 commented Apr 11, 2023

lalit-satapathy commented Apr 12, 2023

botelastic bot commented May 16, 2023

botelastic bot commented Jun 15, 2023

lalit-satapathy commented Jul 17, 2023 •

edited

Loading

lalit-satapathy commented Jul 17, 2023

rajvi-patel-22 commented Jul 17, 2023 •

edited

Loading

agithomas left a comment

harnish-elastic left a comment

elasticmachine commented Jul 19, 2023

[O11y][MySQL] Migrate [Logs MySQL] Overview dashboard visualizations to lens in kibana 8.8.0 #5818

[O11y][MySQL] Migrate [Logs MySQL] Overview dashboard visualizations to lens in kibana 8.8.0 #5818

Conversation

rajvi-patel-22 commented Apr 7, 2023 • edited Loading

What does this PR do?

Checklist

Author's Checklist

Related issues

elasticmachine commented Apr 7, 2023 • edited Loading

💚 Build Succeeded

Build stats

Test stats 🧪

🤖 GitHub comments

elasticmachine commented Apr 7, 2023 • edited Loading

🌐 Coverage report

agithomas left a comment

Choose a reason for hiding this comment

agithomas commented Apr 8, 2023

rajvi-patel-22 commented Apr 10, 2023 • edited Loading

rajvi-patel-22 commented Apr 10, 2023

agithomas commented Apr 10, 2023

agithomas commented Apr 10, 2023

rajvi-patel-22 commented Apr 11, 2023

agithomas commented Apr 11, 2023

rajvi-patel-22 commented Apr 11, 2023

lalit-satapathy commented Apr 12, 2023

botelastic bot commented May 16, 2023

botelastic bot commented Jun 15, 2023

lalit-satapathy commented Jul 17, 2023 • edited Loading

lalit-satapathy commented Jul 17, 2023

rajvi-patel-22 commented Jul 17, 2023 • edited Loading

agithomas left a comment

Choose a reason for hiding this comment

harnish-elastic left a comment

Choose a reason for hiding this comment

elasticmachine commented Jul 19, 2023

[O11y][MySQL] Migrate `[Logs MySQL] Overview` dashboard visualizations to lens in kibana 8.8.0 #5818

[O11y][MySQL] Migrate `[Logs MySQL] Overview` dashboard visualizations to lens in kibana 8.8.0 #5818

rajvi-patel-22 commented Apr 7, 2023 •

edited

Loading

elasticmachine commented Apr 7, 2023 •

edited

Loading

elasticmachine commented Apr 7, 2023 •

edited

Loading

rajvi-patel-22 commented Apr 10, 2023 •

edited

Loading

lalit-satapathy commented Jul 17, 2023 •

edited

Loading

rajvi-patel-22 commented Jul 17, 2023 •

edited

Loading