Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Airmail 20240628 #5178

Closed
wants to merge 5 commits into from
Closed

Airmail 20240628 #5178

wants to merge 5 commits into from

Commits on Jun 28, 2024

  1. Updating tantivy 

    - to deal with the \0 sstable ordering bug
- to include @PSeitz version of the columnar multivalue sparse index fix.
    @fulmicoton
    fulmicoton committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    6668248 View commit details
    Browse the repository at this point in the history

  2. Adds a circuit breaker layer. 

    The piece that estimates whether the next request is likely to fail is extremely simplistic for the moment.
It simply counter the number of errors (not taking in account successes) that happened in a given time window.

The reason is that for the moment, we want to use it for persist requests when the WAL is full.
On airmail, the aggressive retry logic of the client was causing a massive grpc storm on the faulty indexer node,
taking all of its CPU and preventing it from getting out of that state.

In this case, the error estimation logic is very simple, a full WAL guarantees that no further persist request will be successful for a little while.
    @fulmicoton
    fulmicoton committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    e5b4f24 View commit details
    Browse the repository at this point in the history

  3. Using the shard throughput information in the scheduling logic.

    @fulmicoton
    fulmicoton committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    0dc4135 View commit details
    Browse the repository at this point in the history

  4. Adding flags to disable docmapper validation and shard load in contro… 

    …l plane.
    @fulmicoton
    fulmicoton committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    be41cb6 View commit details
    Browse the repository at this point in the history

  5. rustfmt

    @fulmicoton
    fulmicoton committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    360c6b7 View commit details
    Browse the repository at this point in the history