Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Regression in Full History CSV Percentiles #1873

Merged
merged 8 commits into from
Sep 4, 2021

Conversation

TaylorSMarks
Copy link
Contributor

The docstring for _stats_history_data_rows says it should be outputting *current* stats (emphasis not added by me). In the original commit from December 3rd, 2019, it was properly using the get_current_response_time_percentile function.

On August 18th, 2020, the code was all reformatted and it looks like a regression was accidentally introduced when they switched over to using get_response_time_percentile instead.

This PR simply fixes that regression, switching back to using get_current_response_time_percentile.

The bug is easiest to observe if you have a spikey/bursty test shape. During bursts, latency increases. During quieter times, latency goes back down. This is properly reflected in the Locust Web UI's graph tab, but the full history CSV file doesn't show that - instead there's only a slight decline in latency between bursts, because it's showing the 95th percentile since the test began rather than just for the past 10 seconds.

I suspect the bug went just over a year without being noticed because it's unusual to either (1) use a custom load shape for a test and (2) to closely examine the full history CSV file.

Fixes a regression that I believe was introduced in August 18th, 2020. Before then, the full history CSV used current percentiles (starting from December 3rd, 2019).
@cyberw
Copy link
Collaborator

cyberw commented Sep 4, 2021

Interesting. Can you add a test case? (to make sure it doesnt regress again, and to further explain the issue)

@TaylorSMarks
Copy link
Contributor Author

@cyberw - I’ve fixed the formatting issues that flake8 was complaining about and updated an existing unit test to cover this.

With my changes, the unit test passes. Without my changes, the unit tests will fail because the 95th percentile will never decrease.

@cyberw cyberw merged commit bdf5943 into locustio:master Sep 4, 2021
@cyberw
Copy link
Collaborator

cyberw commented Sep 4, 2021

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants