Track total hits up to 10,000 by default #37466
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit changes the default for the `track_total_hits` option of the search request to `10,000`. This means that by default search requests will accurately track the total hit count up to `10,000` documents, requests that match more than this value will set the `"total.relation"` to `"gte"` (e.g. greater than or equals) and the `"total.value"` to `10,000` in the search response. Scroll queries are not impacted, they will continue to count the total hits accurately. The default is set back to `true` (accurate hit count) if `rest_total_hits_as_int` is set in the search request. I choose `10,000` as the default because that's also the number we use to limit pagination. This means that users will be able to know how far they can jump (up to 10,000) even if the total number of hits is not accurate. Closes elastic#33028
jimczi
added
>enhancement
>breaking
:Search/Search
|
Pinging @elastic/es-search |
…re an accurate count internally
jpountz
approved these changes
Jan 15, 2019
server/src/main/java/org/elasticsearch/common/io/stream/StreamInput.java
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/common/io/stream/StreamOutput.java
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/search/query/EarlyTerminatingCollector.java
Show resolved
Hide resolved
|
run the gradle build tests 1 |
|
@elasticmachine run gradle build tests 1 |
|
@elasticmachine run elasticsearch-ci/2 |
adhulipa
reviewed
Feb 11, 2019
| // no matter what the value of track_total_hits is | ||
| return SearchContext.TRACK_TOTAL_HITS_ACCURATE; | ||
| } | ||
| return request.source() == null ? SearchContext.DEFAULT_TRACK_TOTAL_HITS_UP_TO : request.source().trackTotalHitsUpTo() == null ? |
There was a problem hiding this comment.
is the request.source() == null? check necessary on line 704? Seems like it's redundant given the check on line 700
p.s: I'm a curious engineer looking to learn more about elasticsearch's internals. I'm reading the diffs in my spare time to learn how experts in the field are building real-world distributed systems. Thanks for taking the time to answer my questions!
There was a problem hiding this comment.
The check is on request.scroll which is different than request.source so it is necessary.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit changes the default for the
track_total_hitsoption of the search requestto
10,000. This means that by default search requests will accurately track the total hit countup to
10,000documents, requests that match more than this value will set the"total.relation"to
"gte"(e.g. greater than or equals) and the"total.value"to10,000in the search response.Scroll queries are not impacted, they will continue to count the total hits accurately.
The default is set back to
true(accurate hit count) ifrest_total_hits_as_intis set in the search request.I choose
10,000as the default because that's also the number we use to limit pagination. This means thatusers will be able to know how far they can jump (up to 10,000) even if the total number of hits is not accurate.
Closes #33028