add metrics for indexer and sidecar #642

rinx · 2020-08-25T08:34:38Z

Description:

Added new metrics for indexer and sidecar.

Related Issue:

#343

How Has This Been Tested?:

nothing.

Environment:

Go Version: 1.14.4
Docker Version: 19.03.8
Kubernetes Version: 1.18.2
NGT Version: 1.12.0

Types of changes:

Bug fix [type/bug]
New feature [type/feature]
Add tests [type/test]
Security related changes [type/security]
Add documents [type/documentation]
Refactoring [type/refactoring]
Update dependencies [type/dependency]
Update benchmarks and performances [type/bench]
Update CI [type/ci]

Changes to Core Features:

Have you added an explanation of what your changes do and why you'd like us to include them?
Have you written new tests for your core changes, as applicable?
Have you successfully ran tests with your changes locally?

Checklist:

I have read the CONTRIBUTING document.
I have checked open Pull Requests for the similar feature or fixes?
I have added tests and benchmarks to cover my changes.
I have ensured all new and existing tests passed.
I have commented my code, particularly in hard-to-understand areas
I have updated the documentation accordingly.

pull-assistant · 2020-08-25T08:34:41Z

Best reviewed: commit by commit

Optimal code review plan

✨ Add indexer metrics

✨ add sidecar metrics

🚚 rename

✅ fix tests

🔧 add RoughMillisecondsDistribution

Powered by Pull Assistant. Last update 317fc83 ... 778656f. Read the comment docs.

vdaas-ci · 2020-08-25T08:34:58Z

[CHATOPS:HELP] ChatOps commands.

🙆‍♀️ /approve - approve
💌 /changelog - add changelog comment
🍱 /format - format codes and add licenses
✅ /gen-test - generate test codes
🏷️ /label - add labels
⏪ /rebase - rebase master

codecov · 2020-08-25T08:43:45Z

Codecov Report

Merging #642 into master will decrease coverage by 0.23%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master     #642      +/-   ##
==========================================
- Coverage   15.48%   15.25%   -0.24%     
==========================================
  Files         412      411       -1     
  Lines       21598    21578      -20     
==========================================
- Hits         3344     3291      -53     
- Misses      18000    18036      +36     
+ Partials      254      251       -3

Impacted Files	Coverage Δ
internal/observability/metrics/metrics.go	`0.00% <ø> (ø)`
pkg/agent/sidecar/service/observer/observer.go	`0.00% <0.00%> (ø)`
pkg/agent/sidecar/service/observer/option.go	`9.80% <0.00%> (-1.83%)`	⬇️
pkg/agent/sidecar/service/storage/storage.go	`0.00% <0.00%> (ø)`
pkg/agent/sidecar/usecase/sidecar/sidecar.go	`0.00% <0.00%> (ø)`
pkg/manager/index/service/indexer.go	`0.00% <ø> (ø)`
pkg/manager/index/usecase/indexer.go	`0.00% <0.00%> (ø)`
internal/net/net.go
internal/worker/worker.go	`85.45% <0.00%> (+0.90%)`	⬆️
internal/worker/queue.go	`100.00% <0.00%> (+1.44%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 782e84b...778656f. Read the comment docs.

github-actions · 2020-08-25T09:03:45Z

internal/observability/metrics/manager/index/index.go

+
+func (i *indexerMetrics) View() []*metrics.View {
+	return []*metrics.View{
+		&metrics.View{


[golangci] _{reported by reviewdog 🐶}
File is not gofmt-ed with -s (gofmt)

github-actions · 2020-08-25T09:03:45Z

pkg/agent/sidecar/service/observer/observer.go

@@ -392,8 +394,27 @@ func (o *observer) requestBackup(ctx context.Context) error {
 	return nil
 }

-func (o *observer) backup(ctx context.Context) error {
+func (o *observer) backup(ctx context.Context) (err error) {


[golangci] _{reported by reviewdog 🐶}
Function 'backup' has too many statements (42 > 40) (funlen)

github-actions · 2020-08-26T01:23:40Z

pkg/agent/sidecar/service/observer/observer.go

@@ -392,8 +394,27 @@ func (o *observer) requestBackup(ctx context.Context) error {
 	return nil
 }

-func (o *observer) backup(ctx context.Context) error {
+func (o *observer) backup(ctx context.Context) (err error) {


[golangci] _{reported by reviewdog 🐶}
cognitive complexity 53 of func (*observer).backup is high (> 30) (gocognit)

github-actions · 2020-08-26T01:23:40Z

internal/observability/metrics/metrics.go

@@ -40,7 +40,90 @@ var (
 	LastValue    = view.LastValue
 	Sum          = view.Sum

-	DefaultMillisecondsDistribution = Distribution(0.01, 0.05, 0.1, 0.3, 0.6, 0.8, 1, 2, 3, 4, 5, 6, 8, 10, 13, 16, 20, 25, 30, 40, 50, 65, 80, 100, 130, 160, 200, 250, 300, 400, 500, 650, 800, 1000, 2000, 5000, 10000, 20000, 50000, 100000)
+	DefaultMillisecondsDistribution = Distribution(


[golangci] _{reported by reviewdog 🐶}
DefaultMillisecondsDistribution is a global variable (gochecknoglobals)

github-actions · 2020-08-26T01:23:41Z

internal/observability/metrics/metrics.go

+		50000,
+		100000,
+	)
+	RoughMillisecondsDistribution = Distribution(


[golangci] _{reported by reviewdog 🐶}
RoughMillisecondsDistribution is a global variable (gochecknoglobals)

github-actions · 2020-08-26T01:23:41Z

internal/observability/metrics/metrics.go

@@ -40,7 +40,90 @@ var (
 	LastValue    = view.LastValue
 	Sum          = view.Sum

-	DefaultMillisecondsDistribution = Distribution(0.01, 0.05, 0.1, 0.3, 0.6, 0.8, 1, 2, 3, 4, 5, 6, 8, 10, 13, 16, 20, 25, 30, 40, 50, 65, 80, 100, 130, 160, 200, 250, 300, 400, 500, 650, 800, 1000, 2000, 5000, 10000, 20000, 50000, 100000)
+	DefaultMillisecondsDistribution = Distribution(
+		0.01,


[golangci] _{reported by reviewdog 🐶}
mnd: Magic number: 0.01, in detected (gomnd)

github-actions · 2020-08-26T01:23:41Z

internal/observability/metrics/agent/sidecar/sidecar.go

+	}
+
+	return []*metrics.View{
+		&metrics.View{


[golangci] _{reported by reviewdog 🐶}
File is not gofmt-ed with -s (gofmt)

rinx · 2020-09-01T05:42:59Z

/rebase
/format

vdaas-ci · 2020-09-01T05:43:35Z

[REBASE] Rebase triggered by rinx for branch: feature/observability/add-metrics-for-indexer-and-sidecar

vdaas-ci · 2020-09-01T05:44:35Z

[FORMAT] Updating license headers and formatting go codes triggered by rinx.

pkg/agent/sidecar/service/observer/hook.go

kpango · 2020-09-02T09:40:26Z

/rebase
/format
/approve

vdaas-ci · 2020-09-02T09:41:03Z

[REBASE] Rebase triggered by kpango for branch: feature/observability/add-metrics-for-indexer-and-sidecar

Signed-off-by: Rintaro Okamura <rintaro.okamura@gmail.com>

vdaas-ci · 2020-09-02T09:42:06Z

[FORMAT] Updating license headers and formatting go codes triggered by kpango.

vdaas-ci

[APPROVED] This PR is approved by kpango.

github-actions · 2020-09-02T09:47:07Z

internal/observability/metrics/agent/sidecar/sidecar.go

+	ms []metrics.MeasurementWithTags
+}
+
+type MetricsHook interface {


[golangci] _{reported by reviewdog 🐶}
exported type MetricsHook should have comment or be unexported (golint)

github-actions · 2020-09-02T09:47:08Z

internal/observability/metrics/agent/sidecar/sidecar.go

+	observer.Hook
+}
+
+func New() (MetricsHook, error) {


[golangci] _{reported by reviewdog 🐶}
exported function New should have comment or be unexported (golint)

github-actions · 2020-09-02T09:47:08Z

internal/observability/metrics/agent/sidecar/sidecar.go

+	sm.ms = append(
+		sm.ms,
+		metrics.MeasurementWithTags{
+			Measurement: sm.uploadTotal.M(1),


[golangci] _{reported by reviewdog 🐶}
mnd: Magic number: 1, in detected (gomnd)

github-actions bot added area/agent/sidecar area/internal area/manager/index size/L type/feature New feature priority/medium labels Aug 25, 2020

github-actions bot added the team/set SET team label Aug 25, 2020

github-actions bot reviewed Aug 25, 2020

View reviewed changes

github-actions bot added size/XL and removed size/L labels Aug 26, 2020

github-actions bot reviewed Aug 26, 2020

View reviewed changes

rinx marked this pull request as ready for review August 26, 2020 06:38

vdaas-ci force-pushed the feature/observability/add-metrics-for-indexer-and-sidecar branch from e1c34ee to 6bfcf6b Compare September 1, 2020 05:43

kpango reviewed Sep 2, 2020

View reviewed changes

pkg/agent/sidecar/service/observer/hook.go Show resolved Hide resolved

rinx added 5 commits September 2, 2020 09:41

✨ Add indexer metrics

317fc83

Signed-off-by: Rintaro Okamura <rintaro.okamura@gmail.com>

✨ add sidecar metrics

19077fd

Signed-off-by: Rintaro Okamura <rintaro.okamura@gmail.com>

🚚 rename

4cd09f5

Signed-off-by: Rintaro Okamura <rintaro.okamura@gmail.com>

✅ fix tests

b3d8b7b

Signed-off-by: Rintaro Okamura <rintaro.okamura@gmail.com>

🔧 add RoughMillisecondsDistribution

778656f

Signed-off-by: Rintaro Okamura <rintaro.okamura@gmail.com>

vdaas-ci force-pushed the feature/observability/add-metrics-for-indexer-and-sidecar branch from 6bfcf6b to 778656f Compare September 2, 2020 09:41

vdaas-ci approved these changes Sep 2, 2020

View reviewed changes

github-actions bot reviewed Sep 2, 2020

View reviewed changes

kpango merged commit b8f7ced into master Sep 2, 2020

kpango deleted the feature/observability/add-metrics-for-indexer-and-sidecar branch September 2, 2020 09:48

vdaas-ci mentioned this pull request Sep 2, 2020

[patch] update deps #660

Merged

18 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add metrics for indexer and sidecar #642

add metrics for indexer and sidecar #642

rinx commented Aug 25, 2020

pull-assistant bot commented Aug 25, 2020 •

edited

Loading

vdaas-ci commented Aug 25, 2020

codecov bot commented Aug 25, 2020 •

edited

Loading

github-actions bot Aug 25, 2020

github-actions bot Aug 25, 2020

github-actions bot Aug 26, 2020

github-actions bot Aug 26, 2020

github-actions bot Aug 26, 2020

github-actions bot Aug 26, 2020

github-actions bot Aug 26, 2020

rinx commented Sep 1, 2020

vdaas-ci commented Sep 1, 2020

vdaas-ci commented Sep 1, 2020

kpango commented Sep 2, 2020

vdaas-ci commented Sep 2, 2020

vdaas-ci commented Sep 2, 2020

vdaas-ci left a comment

github-actions bot Sep 2, 2020

github-actions bot Sep 2, 2020

github-actions bot Sep 2, 2020

add metrics for indexer and sidecar #642

add metrics for indexer and sidecar #642

Conversation

rinx commented Aug 25, 2020

Description:

Related Issue:

How Has This Been Tested?:

Environment:

Types of changes:

Changes to Core Features:

Checklist:

pull-assistant bot commented Aug 25, 2020 • edited Loading

vdaas-ci commented Aug 25, 2020

codecov bot commented Aug 25, 2020 • edited Loading

Codecov Report

github-actions bot Aug 25, 2020

Choose a reason for hiding this comment

github-actions bot Aug 25, 2020

Choose a reason for hiding this comment

github-actions bot Aug 26, 2020

Choose a reason for hiding this comment

github-actions bot Aug 26, 2020

Choose a reason for hiding this comment

github-actions bot Aug 26, 2020

Choose a reason for hiding this comment

github-actions bot Aug 26, 2020

Choose a reason for hiding this comment

github-actions bot Aug 26, 2020

Choose a reason for hiding this comment

rinx commented Sep 1, 2020

vdaas-ci commented Sep 1, 2020

vdaas-ci commented Sep 1, 2020

kpango commented Sep 2, 2020

vdaas-ci commented Sep 2, 2020

vdaas-ci commented Sep 2, 2020

vdaas-ci left a comment

Choose a reason for hiding this comment

github-actions bot Sep 2, 2020

Choose a reason for hiding this comment

github-actions bot Sep 2, 2020

Choose a reason for hiding this comment

github-actions bot Sep 2, 2020

Choose a reason for hiding this comment

pull-assistant bot commented Aug 25, 2020 •

edited

Loading

codecov bot commented Aug 25, 2020 •

edited

Loading