[CI] SmokeTestWatcherWithSecurityIT.* failing #29877

elasticmachine · 2018-03-14T12:18:00Z

Original comment by @ywelsch:

The tests in SmokeTestWatcherWithSecurityIT have been failing daily for the last few weeks.

testSearchInputWithInsufficientPrivileges:
LINK REDACTED
testIndexActionHasPermissions:
LINK REDACTED

Output:

12:09:37 [2018-03-14T12:08:40,747][ERROR][o.e.x.w.a.i.IndexActionFactory] [node-0] failed to execute action [my_watch/index]
12:09:37 org.elasticsearch.ElasticsearchSecurityException: action [indices:data/write/bulk[s]] is unauthorized for user [watcher_manager]
12:09:37 	at org.elasticsearch.xpack.core.security.support.Exceptions.authorizationError(Exceptions.java:42) ~[?:?]
12:09:37 	at org.elasticsearch.xpack.security.authz.AuthorizationService.denialException(AuthorizationService.java:595) ~[?:?]
12:09:37 	at org.elasticsearch.xpack.security.authz.AuthorizationService.denial(AuthorizationService.java:565) ~[?:?]
12:09:37 	at org.elasticsearch.xpack.security.authz.AuthorizationService.authorize(AuthorizationService.java:310) ~[?:?]
12:09:37 	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$authorizeRequest$4(SecurityActionFilter.java:176) ~[?:?]
12:09:37 	at org.elasticsearch.xpack.security.authz.AuthorizationUtils$AsyncAuthorizer.maybeRun(AuthorizationUtils.java:185) ~[?:?]
12:09:37 	at org.elasticsearch.xpack.security.authz.AuthorizationUtils$AsyncAuthorizer.setRunAsRoles(AuthorizationUtils.java:179) ~[?:?]
12:09:37 	at org.elasticsearch.xpack.security.authz.AuthorizationUtils$AsyncAuthorizer.authorize(AuthorizationUtils.java:167) ~[?:?]
....

The text was updated successfully, but these errors were encountered:

elasticmachine · 2018-04-11T11:31:22Z

Original comment by @albertzaharovits:

I have came upon several such failures:
LINK REDACTED
LINK REDACTED
LINK REDACTED
which don't reproduce locally.

./gradlew :x-pack-elasticsearch:qa:smoke-test-watcher-with-security:integTestRunner \
  -Dtests.seed=E4F4AFFF2B7ABA44 \
  -Dtests.class=org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityIT \
  -Dtests.method="testIndexActionInsufficientPrivileges" \
  -Dtests.security.manager=true \
  -Dtests.locale=ar-IQ \
  -Dtests.timezone=America/Indiana/Indianapolis

The error is as before:

10:18:12 org.elasticsearch.ElasticsearchSecurityException: action [indices:data/write/bulk[s]] is unauthorized for user [watcher_manager]
10:18:12 	at org.elasticsearch.xpack.core.security.support.Exceptions.authorizationError(Exceptions.java:42) ~[?:?]
10:18:12 	at org.elasticsearch.xpack.security.authz.AuthorizationService.denialException(AuthorizationService.java:586) ~[?:?]
10:18:12 	at org.elasticsearch.xpack.security.authz.AuthorizationService.denial(AuthorizationService.java:564) ~[?:?]
10:18:12 	at org.elasticsearch.xpack.security.authz.AuthorizationService.authorize(AuthorizationService.java:309) ~[?:?]
10:18:12 	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$authorizeRequest$4(SecurityActionFilter.java:183) ~[?:?]
10:18:12 	at org.elasticsearch.xpack.security.authz.AuthorizationUtils$AsyncAuthorizer.maybeRun(AuthorizationUtils.java:185) ~[?:?]
10:18:12 	at org.elasticsearch.xpack.security.authz.AuthorizationUtils$AsyncAuthorizer.setRunAsRoles(AuthorizationUtils.java:179) ~[?:?]
10:18:12 	at org.elasticsearch.xpack.security.authz.AuthorizationUtils$AsyncAuthorizer.authorize(AuthorizationUtils.java:167) ~[?:?]
10:18:12 	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.authorizeRequest(SecurityActionFilter.java:195) ~[?:?]
10:18:12 	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$3(SecurityActionFilter.java:173) ~[?:?]
10:18:12 	at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:60) ~[elasticsearch-7.0.0-alpha1-SNAPSHOT.jar:7.0.0-alpha1-SNAPSHOT]
10:18:12 	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$authenticateAsync$2(AuthenticationService.java:184) ~[?:?]
10:18:12 	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$lookForExistingAuthentication$4(AuthenticationService.java:217) ~[?:?]
10:18:12 	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lookForExistingAuthentication(AuthenticationService.java:228) ~[?:?]
10:18:12 	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.authenticateAsync(AuthenticationService.java:182) ~[?:?]
10:18:12 	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.access$000(AuthenticationService.java:143) ~[?:?]

But I suspect there is a race condition with the .security index not being available:

10:18:12 ERROR   10.4s | SmokeTestWatcherWithSecurityIT.testIndexActionInsufficientPrivileges <<< FAILURES!
10:18:12    > Throwable LINK REDACTED: java.lang.AssertionError: 
10:18:12    > Expected: is "started"
10:18:12    >      but: was "stopped"
10:18:12    > 	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
10:18:12    > 	at org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityIT.lambda$startWatcher$0(SmokeTestWatcherWithSecurityIT.java:74)
10:18:12   2> NOTE: leaving temporary files on disk at: /var/lib/jenkins/workspace/elastic+x-pack-elasticsearch+master+periodic/elasticsearch-extra/x-pack-elasticsearch/qa/smoke-test-watcher-with-security/build/testrun/integTestRunner/J0/temp/org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityIT_E4F4AFFF2B7ABA44-001
10:18:12    > 	at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:734)
10:18:12   2> NOTE: test params are: codec=Asserting(Lucene70), sim=RandomSimilarity(queryNorm=false): {}, locale=ar-IQ, timezone=America/Indiana/Indianapolis
10:18:12   2> NOTE: Linux 4.4.0-1048-aws amd64/Oracle Corporation 1.8.0_162 (64-bit)/cpus=4,threads=1,free=377815112,total=514850816
10:18:12   2> NOTE: All tests run in this JVM: [SmokeTestWatcherWithSecurityClientYamlTestSuiteIT, SmokeTestWatcherWithSecurityIT]
10:18:12    > 	at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:708)
10:18:12    > 	at org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityIT.startWatcher(SmokeTestWatcherWithSecurityIT.java:63)
10:18:12    > 	at java.lang.Thread.run(Thread.java:748)
10:18:12    > 	Suppressed: java.lang.AssertionError: 
10:18:12    > Expected: is "started"
10:18:12    >      but: was "stopping"
10:18:12    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
10:18:12    > 		at org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityIT.lambda$startWatcher$0(SmokeTestWatcherWithSecurityIT.java:74)
10:18:12    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:722)
10:18:12    > 		... 38 more
10:18:12    > 	Suppressed: java.lang.AssertionError: 
10:18:12    > Expected: is "started"
10:18:12    >      but: was "stopping"
10:18:12    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
10:18:12    > 		at org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityIT.lambda$startWatcher$0(SmokeTestWatcherWithSecurityIT.java:74)
10:18:12    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:722)
10:18:12    > 		... 38 more
10:18:12    > 	Suppressed: java.lang.AssertionError: 
10:18:12    > Expected: is "started"
10:18:12    >      but: was "stopping"
10:18:12    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
10:18:12    > 		at org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityIT.lambda$startWatcher$0(SmokeTestWatcherWithSecurityIT.java:74)
10:18:12    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:722)
10:18:12    > 		... 38 more
10:18:12    > 	Suppressed: java.lang.AssertionError: 
10:18:12    > Expected: is "started"
10:18:12    >      but: was "stopping"
10:18:12    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
10:18:12    > 		at org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityIT.lambda$startWatcher$0(SmokeTestWatcherWithSecurityIT.java:74)
10:18:12    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:722)
10:18:12    > 		... 38 more
10:18:12    > 	Suppressed: java.lang.AssertionError: 
10:18:12    > Expected: is "started"
10:18:12    >      but: was "stopping"
10:18:12    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
10:18:12    > 		at org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityIT.lambda$startWatcher$0(SmokeTestWatcherWithSecurityIT.java:74)
10:18:12    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:722)
10:18:12    > 		... 38 more
10:18:12    > 	Suppressed: java.lang.AssertionError: 
10:18:12    > Expected: is "started"
10:18:12    >      but: was "stopping"
10:18:12    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
10:18:12    > 		at org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityIT.lambda$startWatcher$0(SmokeTestWatcherWithSecurityIT.java:74)
10:18:12    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:722)
10:18:12    > 		... 38 more
10:18:12    > 	Suppressed: java.lang.AssertionError: 
10:18:12    > Expected: is "started"
10:18:12    >      but: was "stopping"
10:18:12    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
10:18:12    > 		at org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityIT.lambda$startWatcher$0(SmokeTestWatcherWithSecurityIT.java:74)
10:18:12    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:722)
10:18:12    > 		... 38 more
10:18:12    > 	Suppressed: java.lang.AssertionError: 
10:18:12    > Expected: is "started"
10:18:12    >      but: was "stopping"
10:18:12    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
10:18:12    > 		at org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityIT.lambda$startWatcher$0(SmokeTestWatcherWithSecurityIT.java:74)
10:18:12    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:722)
10:18:12    > 		... 38 more
10:18:12    > 	Suppressed: java.lang.AssertionError: 
10:18:12    > Expected: is "started"
10:18:12    >      but: was "stopping"
10:18:12    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
10:18:12    > 		at org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityIT.lambda$startWatcher$0(SmokeTestWatcherWithSecurityIT.java:74)
10:18:12    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:722)
10:18:12    > 		... 38 more
10:18:12    > 	Suppressed: java.lang.AssertionError: 
10:18:12    > Expected: is "started"
10:18:12    >      but: was "stopped"
10:18:12    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
10:18:12    > 		at org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityIT.lambda$startWatcher$0(SmokeTestWatcherWithSecurityIT.java:74)
10:18:12    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:722)
10:18:12    > 		... 38 more
10:18:12    > 	Suppressed: java.lang.AssertionError: 
10:18:12    > Expected: is "started"
10:18:12    >      but: was "stopped"
10:18:12    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
10:18:12    > 		at org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityIT.lambda$startWatcher$0(SmokeTestWatcherWithSecurityIT.java:74)
10:18:12    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:722)
10:18:12    > 		... 38 more
10:18:12    > 	Suppressed: java.lang.AssertionError: 
10:18:12    > Expected: is "started"
10:18:12    >      but: was "stopped"
10:18:12    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
10:18:12    > 		at org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityIT.lambda$startWatcher$0(SmokeTestWatcherWithSecurityIT.java:74)
10:18:12    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:722)
10:18:12    > 		... 38 more
10:18:12    > 	Suppressed: java.lang.AssertionError: 
10:18:12    > Expected: is "started"
10:18:12    >      but: was "stopped"
10:18:12    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
10:18:12    > 		at org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityIT.lambda$startWatcher$0(SmokeTestWatcherWithSecurityIT.java:74)
10:18:12    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:722)
10:18:12    > 		... 38 moreThrowable LINK REDACTED: org.elasticsearch.client.ResponseException: method [DELETE], host [http://[::1]:34429], URI [_xpack/watcher/watch/my_watch], status line [HTTP/1.1 404 Not Found]
10:18:12    > {"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index","resource.type":"index_expression","resource.id":".watches","index_uuid":"_na_","index":".watches"}],"type":"index_not_found_exception","reason":"no such index","resource.type":"index_expression","resource.id":".watches","index_uuid":"_na_","index":".watches"},"status":404}
10:18:12    > 	at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:705)
10:18:12    > 	at org.elasticsearch.client.RestClient.performRequest(RestClient.java:235)
10:18:12    > 	at org.elasticsearch.client.RestClient.performRequest(RestClient.java:198)
10:18:12    > 	at org.elasticsearch.client.RestClient.performRequest(RestClient.java:160)
10:18:12    > 	at org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityIT.stopWatcher(SmokeTestWatcherWithSecurityIT.java:83)
10:18:12    > 	at java.lang.Thread.run(Thread.java:748)
10:18:12    > Caused by: org.elasticsearch.client.ResponseException: method [DELETE], host [http://[::1]:34429], URI [_xpack/watcher/watch/my_watch], status line [HTTP/1.1 404 Not Found]
10:18:12    > {"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index","resource.type":"index_expression","resource.id":".watches","index_uuid":"_na_","index":".watches"}],"type":"index_not_found_exception","reason":"no such index","resource.type":"index_expression","resource.id":".watches","index_uuid":"_na_","index":".watches"},"status":404}

elasticmachine · 2018-04-13T12:37:30Z

Original comment by @javanna:

Another failure similar to the last one posted by Albert:
LINK REDACTED

elasticmachine · 2018-04-13T12:43:03Z

Original comment by @javanna:

And another occurrence: LINK REDACTED

Ensure that Watcher is correctly started and stopped between tests for SmokeTestWatcherWithSecurityIT and SmokeTestWatcherWithSecurityClientYamlTestSuiteIT. The change here is to throw an `AssertionError` instead of `break;` to allow the `assertBusy()` to continue to busy wait until the desired state is reached. closes elastic#33291 closes elastic#29877

#35271) Ensure that Watcher is correctly started and stopped between tests for SmokeTestWatcherWithSecurityIT, SmokeTestWatcherWithSecurityClientYamlTestSuiteIT, SmokeTestWatcherTestSuiteIT, WatcherRestIT, XDocsClientYamlTestSuiteIT, and XPackRestIT The change here is to throw an `AssertionError` instead of `break;` to allow the `assertBusy()` to continue to busy wait until the desired state is reached. closes #33291, closes #29877, closes #34462, closes #30705, closes #34448

elastic#35271) Ensure that Watcher is correctly started and stopped between tests for SmokeTestWatcherWithSecurityIT, SmokeTestWatcherWithSecurityClientYamlTestSuiteIT, SmokeTestWatcherTestSuiteIT, WatcherRestIT, XDocsClientYamlTestSuiteIT, and XPackRestIT The change here is to throw an `AssertionError` instead of `break;` to allow the `assertBusy()` to continue to busy wait until the desired state is reached. closes elastic#33291, closes elastic#29877, closes elastic#34462, closes elastic#30705, closes elastic#34448

jakelandis · 2018-11-28T19:39:40Z

Re-opening. Another occurrence on 6.5

REPRODUCE WITH: ./gradlew :x-pack:qa:smoke-test-watcher-with-security:integTestRunner -Dtests.seed=CEA725D4AE7A0067 -Dtests.class=org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityClientYamlTestSuiteIT -Dtests.method="test {yaml=watcher/put_watch/70_put_watch_with_index_action_using_id/Test put watch api with index action using _id field}" -Dtests.security.manager=true -Dtests.locale=ar-LY -Dtests.timezone=Europe/Tirane -Dcompiler.java=11 -Druntime.java=8FIPS -Djavax.net.ssl.keyStorePassword=password -Djavax.net.ssl.trustStorePassword=password

Test muted via #35361 (comment)

https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+6.5+matrix-java-periodic/ES_BUILD_JAVA=java11,ES_RUNTIME_JAVA=java8fips,nodes=virtual&&linux/68/consoleFull

09:27:36 [2018-11-28T15:25:57,471][ERROR][o.e.x.w.a.i.IndexActionFactory] [node-0] failed to execute action [RkLZTFgslpcRTYyrlzIN/index]
09:27:36 org.elasticsearch.ElasticsearchSecurityException: action [indices:data/write/bulk[s]] is unauthorized for user [watcher_manager]
09:27:36 	at org.elasticsearch.xpack.core.security.support.Exceptions.authorizationError(Exceptions.java:30) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authz.AuthorizationService.denialException(AuthorizationService.java:579) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authz.AuthorizationService.denial(AuthorizationService.java:554) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authz.AuthorizationService.authorize(AuthorizationService.java:298) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$authorizeRequest$4(SecurityActionFilter.java:169) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authz.AuthorizationUtils$AsyncAuthorizer.maybeRun(AuthorizationUtils.java:173) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authz.AuthorizationUtils$AsyncAuthorizer.setRunAsRoles(AuthorizationUtils.java:167) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authz.AuthorizationUtils$AsyncAuthorizer.authorize(AuthorizationUtils.java:155) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.authorizeRequest(SecurityActionFilter.java:181) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$3(SecurityActionFilter.java:159) ~[?:?]
09:27:36 	at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:60) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$authenticateAsync$2(AuthenticationService.java:176) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$lookForExistingAuthentication$4(AuthenticationService.java:209) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lookForExistingAuthentication(AuthenticationService.java:220) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.authenticateAsync(AuthenticationService.java:174) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.access$000(AuthenticationService.java:134) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:104) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.applyInternal(SecurityActionFilter.java:158) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.apply(SecurityActionFilter.java:111) ~[?:?]
09:27:36 	at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:165) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:139) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:81) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.action.bulk.TransportBulkAction$BulkOperation.doRun(TransportBulkAction.java:384) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.action.bulk.TransportBulkAction.executeBulk(TransportBulkAction.java:496) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.action.bulk.TransportBulkAction.executeIngestAndBulk(TransportBulkAction.java:243) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.action.bulk.TransportBulkAction.doExecute(TransportBulkAction.java:169) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.action.bulk.TransportBulkAction.doExecute(TransportBulkAction.java:89) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:167) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$apply$0(SecurityActionFilter.java:90) ~[?:?]
09:27:36 	at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:60) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$authorizeRequest$4(SecurityActionFilter.java:179) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authz.AuthorizationUtils$AsyncAuthorizer.maybeRun(AuthorizationUtils.java:173) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authz.AuthorizationUtils$AsyncAuthorizer.setRunAsRoles(AuthorizationUtils.java:167) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authz.AuthorizationUtils$AsyncAuthorizer.authorize(AuthorizationUtils.java:155) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.authorizeRequest(SecurityActionFilter.java:181) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$3(SecurityActionFilter.java:159) ~[?:?]
09:27:36 	at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:60) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$authenticateAsync$2(AuthenticationService.java:176) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$lookForExistingAuthentication$4(AuthenticationService.java:209) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lookForExistingAuthentication(AuthenticationService.java:220) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.authenticateAsync(AuthenticationService.java:174) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.access$000(AuthenticationService.java:134) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:104) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.applyInternal(SecurityActionFilter.java:158) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.apply(SecurityActionFilter.java:111) ~[?:?]
09:27:36 	at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:165) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:139) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.action.bulk.TransportSingleItemBulkWriteAction.doExecute(TransportSingleItemBulkWriteAction.java:69) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.action.bulk.TransportSingleItemBulkWriteAction.doExecute(TransportSingleItemBulkWriteAction.java:44) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:167) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$apply$0(SecurityActionFilter.java:90) ~[?:?]
09:27:36 	at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:60) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$authorizeRequest$4(SecurityActionFilter.java:179) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authz.AuthorizationUtils$AsyncAuthorizer.maybeRun(AuthorizationUtils.java:173) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authz.AuthorizationUtils$AsyncAuthorizer.setRunAsRoles(AuthorizationUtils.java:167) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authz.AuthorizationUtils$AsyncAuthorizer.authorize(AuthorizationUtils.java:155) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.authorizeRequest(SecurityActionFilter.java:181) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$3(SecurityActionFilter.java:159) ~[?:?]
09:27:36 	at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:60) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$authenticateAsync$2(AuthenticationService.java:176) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$lookForExistingAuthentication$4(AuthenticationService.java:209) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lookForExistingAuthentication(AuthenticationService.java:220) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.authenticateAsync(AuthenticationService.java:174) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.access$000(AuthenticationService.java:134) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:104) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.applyInternal(SecurityActionFilter.java:158) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.apply(SecurityActionFilter.java:111) ~[?:?]
09:27:36 	at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:165) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:139) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:81) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:87) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:76) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:395) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:384) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.client.support.AbstractClient.index(AbstractClient.java:402) ~[elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.xpack.watcher.actions.index.ExecutableIndexAction.lambda$execute$0(ExecutableIndexAction.java:100) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.core.ClientHelper.executeWithHeaders(ClientHelper.java:121) ~[x-pack-core-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.xpack.watcher.actions.index.ExecutableIndexAction.execute(ExecutableIndexAction.java:99) ~[?:?]
09:27:36 	at org.elasticsearch.xpack.core.watcher.actions.ActionWrapper.execute(ActionWrapper.java:144) [x-pack-core-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.xpack.watcher.execution.ExecutionService.executeInner(ExecutionService.java:455) [x-pack-watcher-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.xpack.watcher.execution.ExecutionService.execute(ExecutionService.java:295) [x-pack-watcher-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.xpack.watcher.execution.ExecutionService.lambda$executeAsync$5(ExecutionService.java:395) [x-pack-watcher-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.xpack.watcher.execution.ExecutionService$WatchExecutionTask.run(ExecutionService.java:544) [x-pack-watcher-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:624) [elasticsearch-6.5.2-SNAPSHOT.jar:6.5.2-SNAPSHOT]
09:27:36 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
09:27:36 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
09:27:36 	at java.lang.Thread.run(Thread.java:834) [?:?]

jakelandis · 2018-11-28T19:40:26Z

Ping @elastic/es-core-features

relates to elastic#30777, elastic#35361, elastic#29877.

relates to #30777, #35361, #29877.

jakelandis · 2019-10-07T20:02:54Z

Closing this issue out due to lack of actionable items.

This test was re-enabled via 70b6134 and any new failures should be on their own issue.

elasticmachine added :Data Management/Watcher >test Issues or PRs that are addressing/adding tests >test-failure Triaged test failures from CI labels Apr 25, 2018

elasticmachine assigned spinscale Apr 25, 2018

$@polyfractal$ polyfractal removed the >test Issues or PRs that are addressing/adding tests label May 9, 2018

jakelandis mentioned this issue Nov 5, 2018

watcher: Fix integration tests to ensure correct start/stop of Watcher #35271

Merged

jakelandis closed this as completed in #35271 Nov 7, 2018

jakelandis reopened this Nov 28, 2018

jakelandis self-assigned this Nov 28, 2018

talevy added a commit to talevy/elasticsearch that referenced this issue Feb 12, 2019

re-enable SmokeTestWatcherWithSecurityIT

bd6942a

relates to elastic#30777, elastic#35361, elastic#29877.

talevy mentioned this issue Feb 12, 2019

re-enable SmokeTestWatcherWithSecurityIT #38814

Merged

talevy added a commit that referenced this issue Feb 14, 2019

re-enable SmokeTestWatcherWithSecurityIT (#38814)

70b6134

relates to #30777, #35361, #29877.

jakelandis mentioned this issue May 22, 2019

Meta: Fix Watcher Test Failures #42409

Closed

jakelandis closed this as completed Oct 7, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[CI] SmokeTestWatcherWithSecurityIT.* failing #29877

[CI] SmokeTestWatcherWithSecurityIT.* failing #29877

elasticmachine commented Mar 14, 2018

elasticmachine commented Apr 11, 2018

elasticmachine commented Apr 13, 2018

elasticmachine commented Apr 13, 2018

jakelandis commented Nov 28, 2018

jakelandis commented Nov 28, 2018

jakelandis commented Oct 7, 2019

[CI] SmokeTestWatcherWithSecurityIT.* failing #29877

[CI] SmokeTestWatcherWithSecurityIT.* failing #29877

Comments

elasticmachine commented Mar 14, 2018

elasticmachine commented Apr 11, 2018

elasticmachine commented Apr 13, 2018

elasticmachine commented Apr 13, 2018

jakelandis commented Nov 28, 2018

jakelandis commented Nov 28, 2018

jakelandis commented Oct 7, 2019