Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

optimize total values for inscriptions endpoint #52

Closed
rafaelcr opened this issue May 2, 2023 · 2 comments
Closed

optimize total values for inscriptions endpoint #52

rafaelcr opened this issue May 2, 2023 · 2 comments
Assignees
Labels
Milestone

Comments

@rafaelcr
Copy link
Collaborator

rafaelcr commented May 2, 2023

When calling /ordinals/inscriptions, the "total" JSON key returns the number of results for the given filter set. When this number is very large, the SQL query will take a very long time to finish because Postgres has to count every single row before returning the total value.

To optimize this, we should keep a pre-calculated count of common filters such as mime_type for the most popular types. We currently only do this kind of caching for unfiltered queries (and it made a huge difference in performance).

@rafaelcr rafaelcr added the feature request New feature or request label May 2, 2023
@rafaelcr rafaelcr self-assigned this May 2, 2023
@rafaelcr rafaelcr added postgres and removed feature request New feature or request labels May 2, 2023
@rafaelcr
Copy link
Collaborator Author

rafaelcr commented May 2, 2023

A fix for this issue will also help with errors such as

2023-04-30 19:30:07.220 GMT [1207312] ERROR:  could not write to file "base/pgsql_tmp/pgsql_tmp1207312.0": No space left on device

which occur when postgres needs to allocate huge amounts of temporary space in order to count affected rows

@rafaelcr
Copy link
Collaborator Author

rafaelcr commented May 5, 2023

Closed by #55

@rafaelcr rafaelcr closed this as completed May 5, 2023
@smcclellan smcclellan added this to the Q2-2023 milestone May 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Archived in project
Archived in project
Development

No branches or pull requests

2 participants