Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
✨ Better perf using the cursor. (#416)
To support pagination, the hub sets the X-Total header to tell the caller (The UI really) the total number available to be displayed as the total number of pages. To accomplish this, the hub executes the query 2 times. First to get the count(*), again to fetch the data. This doubles the latency and overhead in the hub for each request. Unfortunately, the sq.Rows does not provide the count. Better performance iterating the _cursor_ (sql.Rows) instead of running the query twice. This also eliminates the race condition between the count (query) and fetch query. Adds a new _cursor_ object used to consistently: - execute the query - iterate the sql.Rows - scan - paginate - count The count is capped at 50,000 because for tables (like incident) the total can be tens-hundreds of millions. Iterating the rows to determine the count becomes expensive (cpu/time) with those numbers. The X-Total header is only used to support/improve pagination. After a certain point, extremely large numbers become meaningless since users will never page though them. After a discussion with @mturley, the hub will set X-Total with a leading `>` when capped. Example: `X-Total: >50000`. The UI will display accordingly. 50,000 may/may-not be the appropriate number. PoC using the largest table `Incident` and one of the reports: RuleReport. --------- Signed-off-by: Jeff Ortel <jortel@redhat.com>
- Loading branch information