UPLOAD-1582 Example metrics for uploads per data_stream #447
Conversation
whytheplatypus
changed the title
upload-server/cmd/cli/hooks.go
Outdated
| if !ok { | ||
| val, ok = resp.ChangeFileInfo.MetaData[key] | ||
| if !ok { | ||
| // no op.. should be something else |
There was a problem hiding this comment.
Assuming this would throw some kind of "invalid metadata" error.
|
Testing example: |
There was a problem hiding this comment.
I was able to fire this up and launch the application just fine. Looks like upload is working as expected. I found the demo-client-node-js tool in /upload-server/example/demo-client-node-js to be really helpful here.
The metrics were getting reported correctly on the localhost:8080/metrics endpoint. It looks like and things were mostly showing up on the dashboard but I did run in to a few things that might require some changes on the dashboard:
- Active Uploads was not correctly configured in grafana and was not showing any changes at all. No metric was set for this dashboard tile. Might want to set this to
dex_server_active_uploadswith time series view. - Uploads might be better as a time series graph. The bar graph over time gives some weird, extrapolated metrics that don't really represent well what the graph is showing.
- HTTP Responses is working fine, but, maybe something different than the pie chart. The graph could get flooded with something like tons of patch requests for small chunks and you can't even see the rest of the graph unless you hide the patch responses. This may be better as straight numbers or maybe bars? Because the values can vary so differently, the percentage difference is too big when one thing is heavily inflated.
Other than that, the metrics seem to be getting into the system as expected, just need to tweak the display/output.
| "title": "HTTP Responses", | ||
| "type": "piechart" | ||
| }, | ||
| { |
There was a problem hiding this comment.
This seems to be the start of the Active Uploads config. Needs an appropriate target/expression to be used for displaying the results.
| } | ||
| ], | ||
| "title": "Uploads", | ||
| "type": "barchart" |
There was a problem hiding this comment.
timeseries might work well here. It was easier to visualize the change in upload counts over time with that.
can now run things like `podman-compose -f docker-compose.yml -f docker-compose.azurite.yml up -d` to enable local azure testing
upload-server/docker-compose.yml
Outdated
| - 9090:9090 | ||
| restart: unless-stopped | ||
| volumes: | ||
| - ./config/local/prometheus:/etc/prometheus |
There was a problem hiding this comment.
There's nothing in the ./config/local folders. I think the things in ./configs/local need to be moved in to there or change the target in the docker-compose configuration on this line.
This is causing prometheus/grafana to get some errors on startup and not load the dex dashboards.
upload-server/docker-compose.yml
Outdated
| - GF_SECURITY_ADMIN_USER=admin | ||
| - GF_SECURITY_ADMIN_PASSWORD=grafana | ||
| volumes: | ||
| - ./config/local/grafana/provisioning:/etc/grafana/provisioning |
There was a problem hiding this comment.
Same problem as above. needs to be either ./configs/local or move the stuff from ./config/local to this folder or change the target here in the docker-compose.
Dashboards won't load up with this configuration.
|
Foundation work for displaying useful metrics based on the Manifest sent with a file. In this case completed uploads per data_stream.
example promethus query:
increase(files_uploaded_for_dextesting_count[1m])metrics: