shouldStoreResult is not set correctly

[mrshwah]

Elasticsearch version (bin/elasticsearch --version): 6.5.3

Plugins installed: []

JVM version: 1.8

OS version (uname -a if on a Unix-like system): Darwin Kernel Version 18.6.0

Description of the problem including expected versus actual behavior:

When trying to make an updateByQuery call with the Java RestHighLevelClient, shouldStoreResult is always false, even if you explicitly set the request object to true. The updateByQuery tasks are not stored when they have finished or failed and when you try to retrieve them, and exception is thrown.

Steps to reproduce:

1. Create an UpdateByQueryRequest and set shouldStoreResult:

UpdateByQueryRequest request = UpdateByQueryRequest("index").setShouldStoreResult(true).setScript(new Script(ScriptType.STORED, null, scriptId, params));

2. Call updateByQueryAsync:

restClient.updateByQueryAsync(request, RequestOptions.DEFAULT, listener);

3. Get the list of tasks, and then try to retrieve the task by id in Kibana:

ListTasksRequest request = new ListTasksRequest().setActions("*byquery").setDetailed(true);
ListTasksResponse response = esRestClient.tasks().list(request, RequestOptions.DEFAULT);

// kibana
GET _tasks/:taskId

Provide logs (if relevant):
Kibana error when you try to fetch

{
  "error" : {
    "root_cause" : [
      {
        "type" : "resource_not_found_exception",
        "reason" : "task [24XlcEQtRfa1FI5rkpioGw:65213] isn't running and hasn't stored its results"
      }
    ],
    "type" : "resource_not_found_exception",
    "reason" : "task [24XlcEQtRfa1FI5rkpioGw:65213] isn't running and hasn't stored its results"
  },
  "status" : 404
}

[elasticmachine]

Pinging @elastic/es-core-features

[hub-cap]

This looks like a bug. A lot of the older classes may not have proper toXContent support, so this is a spot where we probably need to beef up the toXContent for this class. It is not sending enough info over the wire.

[jbonn360]

Hi guys, I can work on this

[shardulsonar]

Hello everyone,
I am new to open source software development,
@jbonn360 Can continue working on this issue, If he discontinues then I can pick up.

I had a doubt,

In the description, the issue is mentioned as Elasticsearch version (bin/elasticsearch --version): 6.5.3, does this mean that we have to fix this issue in the branch 6.5 or in it needs to be fixed in current master branch ?

[hub-cap]

Hi @shardulsonar and welcome,

Feel free to look for other issues to take with good first issue label. The high level rest client has a lot of these.

So the affects version is just the version it was found in. I checked out the code in master and it is still present. But generally speaking the fixes are going to be for the version in master, which is an unreleased major version 8 series, and then also one major version below that, so 7.x, which is the next major versions 7's next unreleased minor branch.

[jbonn360]

Hey guys, I've had a look at this and I don't think it's a bug. Correct me if I'm wrong, but the Task Management API documentation says that it "Returns information about the tasks currently executing in the cluster."

I think that by the time you get to execute GET _tasks/:taskId in Kibana, the task would have already completed and so it wouldn't be running. Does this make sense?

[mrshwah]

If you kick off an update by query task in Kibana with waitForCompletion set to false, you get a task ID. Even when the task is complete, you are able to use that ID to hit that endpoint and receive information about the task. This is very useful information if you have longer running tasks, because then you can know if it completed successfully on all your documents.

Also, it doesn't make any sense to include a method that sets shouldSaveResult to true, and not have the result be saved.

[mrshwah]

By the way, I found a work around for this. You can use the low level rest client instead, and the task results will be stored.

[jbonn360]

@nchouard Hmm ok, I'll look into it further then

[jbonn360]

@nchouard You're right, I took another look at this and it looks like the exception when fetching the task is thrown because the high level client sets wait_for_completion to true and therefore no task result is created.

@hub-cap Should I add support for wait_for_completion being false to HLRC's UpdateByQuery API? Similar to how it was added to the Reindex API in #35202?

[jbonn360]

@hub-cap Any advice, please?

[hub-cap]

Sorry for missing this @jbonn360. Yes, you can add it the same way as the reindex API. Thank you!

[Kasula11]

Is this issue is fixed or need to be worked on?

[Priya-Raut]

Is this issue still available?

[tumile]

@jbonn360 Hi, are you still working on this? I want to give it a try!

[benatwork99]

Would love to see this go in - I think it's just what we need for an issue we're facing.

@nchouard

By the way, I found a work around for this. You can use the low level rest client instead, and the task results will be stored.

If you have the code handy, could you expand on this please?