Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add new x-pack endpoints to track the progress of a search asynchronously #49931

Merged
merged 81 commits into from
Mar 10, 2020
Merged

Add new x-pack endpoints to track the progress of a search asynchronously #49931

merged 81 commits into from
Mar 10, 2020

Commits on Dec 6, 2019

  1. Add new x-pack endpoints to asynchronously track the progress of a se… 

    …arch
    @jimczi
    jimczi committed Dec 6, 2019
    Configuration menu
    Copy the full SHA
    750a81d View commit details
    Browse the repository at this point in the history

  2. notify partial reduce even on top-docs query

    @jimczi
    jimczi committed Dec 6, 2019
    Configuration menu
    Copy the full SHA
    c28766c View commit details
    Browse the repository at this point in the history

  3. fix npe

    @jimczi
    jimczi committed Dec 6, 2019
    Configuration menu
    Copy the full SHA
    4bfafb4 View commit details
    Browse the repository at this point in the history

Commits on Dec 11, 2019

  1. use a single object named response in the xcontent serialization of t… 

    …he async search response
    @jimczi
    jimczi committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    3ace355 View commit details
    Browse the repository at this point in the history

  2. delete frozen search response automatically

    @jimczi
    jimczi committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    5d000ca View commit details
    Browse the repository at this point in the history

Commits on Dec 12, 2019

  1. add integration tests

    @jimczi
    jimczi committed Dec 12, 2019
    Configuration menu
    Copy the full SHA
    1cd2a9a View commit details
    Browse the repository at this point in the history

  2. simplify schema for the .async-search index

    @jimczi
    jimczi committed Dec 12, 2019
    Configuration menu
    Copy the full SHA
    52085bb View commit details
    Browse the repository at this point in the history

  3. switch to true random uuuids

    @jimczi
    jimczi committed Dec 12, 2019
    Configuration menu
    Copy the full SHA
    da7329d View commit details
    Browse the repository at this point in the history

Commits on Dec 13, 2019

  1. iter

    @jimczi
    jimczi committed Dec 13, 2019
    Configuration menu
    Copy the full SHA
    20c24a0 View commit details
    Browse the repository at this point in the history

  2. replace waitUntil with assertBusy

    @jimczi
    jimczi committed Dec 13, 2019
    Configuration menu
    Copy the full SHA
    438a7b0 View commit details
    Browse the repository at this point in the history

Commits on Dec 16, 2019

  1. add a security layer that restricts the usage of the async search API… 

    …s to the user that submitted the initial rquest
    @jimczi
    jimczi committed Dec 16, 2019
    Configuration menu
    Copy the full SHA
    5a907b9 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into async_search

    @jimczi
    jimczi committed Dec 16, 2019
    Configuration menu
    Copy the full SHA
    a1eac14 View commit details
    Browse the repository at this point in the history

  3. fix checkstyle

    @jimczi
    jimczi committed Dec 16, 2019
    Configuration menu
    Copy the full SHA
    2180dde View commit details
    Browse the repository at this point in the history

  4. fix double clean

    @jimczi
    jimczi committed Dec 16, 2019
    Configuration menu
    Copy the full SHA
    be0ba05 View commit details
    Browse the repository at this point in the history

Commits on Dec 17, 2019

  1. iter

    @jimczi
    jimczi committed Dec 17, 2019
    Configuration menu
    Copy the full SHA
    3c1ffdd View commit details
    Browse the repository at this point in the history

Commits on Dec 18, 2019

  1. Merge branch 'master' into async_search

    @jimczi
    jimczi committed Dec 18, 2019
    Configuration menu
    Copy the full SHA
    8434d4d View commit details
    Browse the repository at this point in the history

  2. add more logging

    @jimczi
    jimczi committed Dec 18, 2019
    Configuration menu
    Copy the full SHA
    b1d2414 View commit details
    Browse the repository at this point in the history

  3. replace routing with preference in get requests

    @jimczi
    jimczi committed Dec 18, 2019
    Configuration menu
    Copy the full SHA
    542fff2 View commit details
    Browse the repository at this point in the history

  4. iter 

    This commit ensures that the rollover index is created before launching an async search.
It also changes the design to not store any document if the submit action returns a final response.
    @jimczi
    jimczi committed Dec 18, 2019
    Configuration menu
    Copy the full SHA
    177e303 View commit details
    Browse the repository at this point in the history

  5. validate the index name when receiving a search id

    @jimczi
    jimczi committed Dec 18, 2019
    Configuration menu
    Copy the full SHA
    baa498d View commit details
    Browse the repository at this point in the history

  6. Merge branch 'master' into async_search

    @jimczi
    jimczi committed Dec 18, 2019
    Configuration menu
    Copy the full SHA
    10a0d66 View commit details
    Browse the repository at this point in the history

Commits on Dec 19, 2019

  1. add authentication test

    @jimczi
    jimczi committed Dec 19, 2019
    Configuration menu
    Copy the full SHA
    fe5c150 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into async_search

    @jimczi
    jimczi committed Dec 19, 2019
    Configuration menu
    Copy the full SHA
    ec8cdf2 View commit details
    Browse the repository at this point in the history

  3. fix block plugin to throw exceptions in createWeight rather than toQuery

    @jimczi
    jimczi committed Dec 19, 2019
    Configuration menu
    Copy the full SHA
    b4fa0de View commit details
    Browse the repository at this point in the history

Commits on Dec 20, 2019

  1. remove clean_on_completion option after the initial submit

    @jimczi
    jimczi committed Dec 20, 2019
    Configuration menu
    Copy the full SHA
    17c0602 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into async_search

    @jimczi
    jimczi committed Dec 20, 2019
    Configuration menu
    Copy the full SHA
    8e5229c View commit details
    Browse the repository at this point in the history

  3. move shard stats in a _shards section for partial response

    @jimczi
    jimczi committed Dec 20, 2019
    Configuration menu
    Copy the full SHA
    93189d3 View commit details
    Browse the repository at this point in the history

Commits on Dec 22, 2019

  1. address review

    @jimczi
    jimczi committed Dec 22, 2019
    Configuration menu
    Copy the full SHA
    d03b940 View commit details
    Browse the repository at this point in the history

Commits on Jan 6, 2020

  1. Merge branch 'master' into async_search

    @jimczi
    jimczi committed Jan 6, 2020
    Configuration menu
    Copy the full SHA
    96a29d9 View commit details
    Browse the repository at this point in the history

  2. address review

    @jimczi
    jimczi committed Jan 6, 2020
    Configuration menu
    Copy the full SHA
    7ffb1dd View commit details
    Browse the repository at this point in the history

Commits on Jan 8, 2020

  1. unused import

    @jimczi
    jimczi committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    9930ccd View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into async_search

    @jimczi
    jimczi committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    28fd67c View commit details
    Browse the repository at this point in the history

  3. line len

    @jimczi
    jimczi committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    9d98706 View commit details
    Browse the repository at this point in the history

Commits on Jan 15, 2020

  1. Merge branch 'master' into async_search

    @jimczi
    jimczi committed Jan 15, 2020
    Configuration menu
    Copy the full SHA
    6c5288a View commit details
    Browse the repository at this point in the history

  2. address review

    @jimczi
    jimczi committed Jan 15, 2020
    Configuration menu
    Copy the full SHA
    9056b87 View commit details
    Browse the repository at this point in the history

  3. add more tests

    @jimczi
    jimczi committed Jan 15, 2020
    Configuration menu
    Copy the full SHA
    8a69a21 View commit details
    Browse the repository at this point in the history

  4. remove unrelated change

    @jimczi
    jimczi committed Jan 15, 2020
    Configuration menu
    Copy the full SHA
    78c2ca4 View commit details
    Browse the repository at this point in the history

  5. fix nocommit

    @jimczi
    jimczi committed Jan 15, 2020
    Configuration menu
    Copy the full SHA
    5eb7148 View commit details
    Browse the repository at this point in the history

Commits on Jan 22, 2020

  1. add more statistics to the search progress listener (number of skippe… 

    …d shards, remote clusters statistics)
    @jimczi
    jimczi committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    fb3364f View commit details
    Browse the repository at this point in the history

  2. Replace generic thread pool execution with asynchronous listener that… 

    … waits for completion in the task
    @jimczi
    jimczi committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    1e102fb View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into async_search

    @jimczi
    jimczi committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    139d56a View commit details
    Browse the repository at this point in the history

  4. fix checkstyle

    @jimczi
    jimczi committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    dc6daa8 View commit details
    Browse the repository at this point in the history

  5. fix wrong assert

    @jimczi
    jimczi committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    5e410d3 View commit details
    Browse the repository at this point in the history

Commits on Jan 24, 2020

  1. Expose the logic to cancel task when the rest channel is closed 

    This commit moves the logic that cancels search requests when the rest channel is closed
to a generic client that can be used by other APIs. This will be useful for any rest action
that wants to cancel the execution of a task if the underlying rest channel is closed by the
client before completion.

Relates elastic#49931
Relates elastic#50990
Relates elastic#50990
    @jimczi
    jimczi committed Jan 24, 2020
    Configuration menu
    Copy the full SHA
    eb7ba8a View commit details
    Browse the repository at this point in the history

  2. address review

    @jimczi
    jimczi committed Jan 24, 2020
    Configuration menu
    Copy the full SHA
    6ae7903 View commit details
    Browse the repository at this point in the history

  3. plug the automatic cancel of search if the rest channel is closed whe… 

    …n submitting the initial async search request
    @jimczi
    jimczi committed Jan 24, 2020
    Configuration menu
    Copy the full SHA
    159ed81 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into async_search

    @jimczi
    jimczi committed Jan 24, 2020
    Configuration menu
    Copy the full SHA
    a529db0 View commit details
    Browse the repository at this point in the history

Commits on Jan 29, 2020

  1. handle expiration time on a per request basis

    @jimczi
    jimczi committed Jan 29, 2020
    Configuration menu
    Copy the full SHA
    19d89f9 View commit details
    Browse the repository at this point in the history

Commits on Jan 31, 2020

  1. use a single replica for the hidden index

    @jimczi
    jimczi committed Jan 31, 2020
    Configuration menu
    Copy the full SHA
    373af3a View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into async_search

    @jimczi
    jimczi committed Jan 31, 2020
    Configuration menu
    Copy the full SHA
    75601ca View commit details
    Browse the repository at this point in the history

Commits on Feb 4, 2020

  1. Merge branch 'master' into async_search

    @jimczi
    jimczi committed Feb 4, 2020
    Configuration menu
    Copy the full SHA
    936cc97 View commit details
    Browse the repository at this point in the history

  2. iter

    @jimczi
    jimczi committed Feb 4, 2020
    Configuration menu
    Copy the full SHA
    4237122 View commit details
    Browse the repository at this point in the history

  3. do not set id twice

    @jimczi
    jimczi committed Feb 4, 2020
    Configuration menu
    Copy the full SHA
    4b53aad View commit details
    Browse the repository at this point in the history

  4. change expectation in test now that we use one replica by default

    @jimczi
    jimczi committed Feb 4, 2020
    Configuration menu
    Copy the full SHA
    434b491 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'master' into async_search

    @jimczi
    jimczi committed Feb 4, 2020
    Configuration menu
    Copy the full SHA
    15de81a View commit details
    Browse the repository at this point in the history

  6. fix x-pack user to allow restricted indicesé

    @jimczi
    jimczi committed Feb 4, 2020
    Configuration menu
    Copy the full SHA
    0f86649 View commit details
    Browse the repository at this point in the history

  7. cleanup

    @jimczi
    jimczi committed Feb 4, 2020
    Configuration menu
    Copy the full SHA
    3b9b89e View commit details
    Browse the repository at this point in the history

  8. iter

    @jimczi
    jimczi committed Feb 4, 2020
    Configuration menu
    Copy the full SHA
    b52a85a View commit details
    Browse the repository at this point in the history

  9. Allow x-pack user to manage async-search indices 

    The x-pack user has all privileges on all indices except security indices and restricted indices.
This change adds the authorization for restricted indices in order to allow the system user to manage
these indices internally. This will allow the async-search restricted indices to be accessed only
with the x-pack user.
    @jimczi
    jimczi committed Feb 4, 2020
    Configuration menu
    Copy the full SHA
    6a1c98d View commit details
    Browse the repository at this point in the history

Commits on Feb 5, 2020

  1. Merge branch 'master' into async_search

    @jimczi
    jimczi committed Feb 5, 2020
    Configuration menu
    Copy the full SHA
    c068667 View commit details
    Browse the repository at this point in the history

  2. another big iter

    @jimczi
    jimczi committed Feb 5, 2020
    Configuration menu
    Copy the full SHA
    1df46f8 View commit details
    Browse the repository at this point in the history

Commits on Feb 6, 2020

  1. Merge branch 'master' into async_search

    @jimczi
    jimczi committed Feb 6, 2020
    Configuration menu
    Copy the full SHA
    aa85e2d View commit details
    Browse the repository at this point in the history

  2. small fix after merging master

    @jimczi
    jimczi committed Feb 6, 2020
    Configuration menu
    Copy the full SHA
    05d9638 View commit details
    Browse the repository at this point in the history

  3. checkstyle

    @jimczi
    jimczi committed Feb 6, 2020
    Configuration menu
    Copy the full SHA
    e0e2072 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into async_search

    @jimczi
    jimczi committed Feb 6, 2020
    Configuration menu
    Copy the full SHA
    8bce32f View commit details
    Browse the repository at this point in the history

  5. restore test

    @jimczi
    jimczi committed Feb 6, 2020
    Configuration menu
    Copy the full SHA
    33f921e View commit details
    Browse the repository at this point in the history

  6. fix more test

    @jimczi
    jimczi committed Feb 6, 2020
    Configuration menu
    Copy the full SHA
    b903d0e View commit details
    Browse the repository at this point in the history

  7. iter

    @jimczi
    jimczi committed Feb 6, 2020
    Configuration menu
    Copy the full SHA
    7612748 View commit details
    Browse the repository at this point in the history

Commits on Mar 5, 2020

  1. Merge branch 'master' into async_search

    @jimczi
    jimczi committed Mar 5, 2020
    Configuration menu
    Copy the full SHA
    2721348 View commit details
    Browse the repository at this point in the history

  2. address review and plug the new async_search origin

    @jimczi
    jimczi committed Mar 5, 2020
    Configuration menu
    Copy the full SHA
    e9b7add View commit details
    Browse the repository at this point in the history

  3. unused import

    @jimczi
    jimczi committed Mar 5, 2020
    Configuration menu
    Copy the full SHA
    3e8ba58 View commit details
    Browse the repository at this point in the history

  4. ensure that the task is unregistered before returning the final respo… 

    …nse to the user
    @jimczi
    jimczi committed Mar 5, 2020
    Configuration menu
    Copy the full SHA
    cd6f3f2 View commit details
    Browse the repository at this point in the history

  5. checkstyle

    @jimczi
    jimczi committed Mar 5, 2020
    Configuration menu
    Copy the full SHA
    20aa0fe View commit details
    Browse the repository at this point in the history

Commits on Mar 9, 2020

  1. Merge branch 'master' into async_search

    @jimczi
    jimczi committed Mar 9, 2020
    Configuration menu
    Copy the full SHA
    05e2209 View commit details
    Browse the repository at this point in the history

  2. address review

    @jimczi
    jimczi committed Mar 9, 2020
    1 Configuration menu
    Copy the full SHA
    415c910 View commit details
    Browse the repository at this point in the history

Commits on Mar 10, 2020

  1. unused import

    @jimczi
    jimczi committed Mar 10, 2020
    Configuration menu
    Copy the full SHA
    91699d5 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into async_search

    @jimczi
    jimczi committed Mar 10, 2020
    Configuration menu
    Copy the full SHA
    20d3f66 View commit details
    Browse the repository at this point in the history

  3. fix rest API reference to the outdated 304 response status

    @jimczi
    jimczi committed Mar 10, 2020
    Configuration menu
    Copy the full SHA
    e3bea16 View commit details
    Browse the repository at this point in the history

  4. remove last_version parameter

    @jimczi
    jimczi committed Mar 10, 2020
    Configuration menu
    Copy the full SHA
    8727385 View commit details
    Browse the repository at this point in the history

  5. rephrase rest option after review

    @jimczi
    jimczi committed Mar 10, 2020
    Configuration menu
    Copy the full SHA
    315bb49 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'master' into async_search

    @jimczi
    jimczi committed Mar 10, 2020
    Configuration menu
    Copy the full SHA
    806ae8f View commit details
    Browse the repository at this point in the history