EnrichIT.testBasicFlow failure on CI #48258
Assignees
Labels
:Data Management/Ingest Node
Execution or management of Ingest Pipelines including GeoIP
>test-failure
Triaged test failures from CI
Comments
gwbrown
added
>test-failure
|
Pinging @elastic/es-core-features (:Core/Features/Ingest) |
|
Another one in a PR build https://gradle-enterprise.elastic.co/s/aulxbxfqrnjxc |
martijnvg
added a commit
to martijnvg/elasticsearch
that referenced
this issue
Oct 21, 2019
…tarted.
Otherwise integration tests may fail if the monitoring interval is low:
```
[2019-10-21T09:57:25,527][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [integTest-0] fatal error in thread [elasticsearch[integTest-0][generic][T#4]], exiting
java.lang.AssertionError: initial cluster state not set yet
at org.elasticsearch.cluster.service.ClusterApplierService.state(ClusterApplierService.java:208) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT]
at org.elasticsearch.cluster.service.ClusterService.state(ClusterService.java:125) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT]
at org.elasticsearch.xpack.monitoring.MonitoringService$MonitoringExecution$1.doRun(MonitoringService.java:231) ~[?:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:703) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
at java.lang.Thread.run(Thread.java:835) [?:?]
```
I ran into this when lowering the monitoring interval when investigating
enrich monitoring test: elastic#48258
|
All the build failures contain these warnings: I suspect that this is the reason why no enrich stats monitoring docs are found. I want to lower the monitoring interval, but I ran into a test issue that I've opened this pr for: #48277 |
|
This failed in 2 PRs this morning I'm not muting this for now as it hasn't been affecting intajke |
martijnvg
added a commit
that referenced
this issue
Oct 24, 2019
martijnvg
added a commit
that referenced
this issue
Oct 24, 2019
martijnvg
added a commit
that referenced
this issue
Oct 24, 2019
martijnvg
added a commit
that referenced
this issue
Oct 31, 2019
martijnvg
added a commit
that referenced
this issue
Oct 31, 2019
This was added because it was suspected to cause the monitoring enrich verification to fail, but that is not the case. See #48258
martijnvg
added a commit
that referenced
this issue
Oct 31, 2019
This was added because it was suspected to cause the monitoring enrich verification to fail, but that is not the case. See #48258
|
I've re-enabled this test on master branch: 9536b79 I think due to circumstance it takes a while for the monitor index to be created (according to logs it does get created but it takes a long time), so I've increased the max wait time. |
martijnvg
added a commit
that referenced
this issue
Oct 31, 2019
This was added because it was suspected to cause the monitoring enrich verification to fail, but that is not the case. See #48258
|
This test hasn't failed since 9536b79 was pushed to master. I will re-enable this test on master and 7.5 branches. |
martijnvg
added a commit
that referenced
this issue
Nov 11, 2019
|
Closing this issue, please re-open if that test starts to fail again. |
martijnvg
added a commit
that referenced
this issue
Nov 11, 2019
martijnvg
added a commit
that referenced
this issue
Nov 21, 2019
…tarted. (#48277) Otherwise integration tests may fail if the monitoring interval is low: ``` [2019-10-21T09:57:25,527][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [integTest-0] fatal error in thread [elasticsearch[integTest-0][generic][T#4]], exiting java.lang.AssertionError: initial cluster state not set yet at org.elasticsearch.cluster.service.ClusterApplierService.state(ClusterApplierService.java:208) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] at org.elasticsearch.cluster.service.ClusterService.state(ClusterService.java:125) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] at org.elasticsearch.xpack.monitoring.MonitoringService$MonitoringExecution$1.doRun(MonitoringService.java:231) ~[?:?] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:703) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:835) [?:?] ``` I ran into this when lowering the monitoring interval when investigating enrich monitoring test: #48258
martijnvg
added a commit
to martijnvg/elasticsearch
that referenced
this issue
Nov 21, 2019
…tarted. (elastic#48277) Otherwise integration tests may fail if the monitoring interval is low: ``` [2019-10-21T09:57:25,527][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [integTest-0] fatal error in thread [elasticsearch[integTest-0][generic][T#4]], exiting java.lang.AssertionError: initial cluster state not set yet at org.elasticsearch.cluster.service.ClusterApplierService.state(ClusterApplierService.java:208) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] at org.elasticsearch.cluster.service.ClusterService.state(ClusterService.java:125) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] at org.elasticsearch.xpack.monitoring.MonitoringService$MonitoringExecution$1.doRun(MonitoringService.java:231) ~[?:?] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:703) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:835) [?:?] ``` I ran into this when lowering the monitoring interval when investigating enrich monitoring test: elastic#48258
martijnvg
added a commit
that referenced
this issue
Nov 21, 2019
…tarted. (#49426) Backport of #48277 Otherwise integration tests may fail if the monitoring interval is low: ``` [2019-10-21T09:57:25,527][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [integTest-0] fatal error in thread [elasticsearch[integTest-0][generic][T#4]], exiting java.lang.AssertionError: initial cluster state not set yet at org.elasticsearch.cluster.service.ClusterApplierService.state(ClusterApplierService.java:208) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] at org.elasticsearch.cluster.service.ClusterService.state(ClusterService.java:125) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] at org.elasticsearch.xpack.monitoring.MonitoringService$MonitoringExecution$1.doRun(MonitoringService.java:231) ~[?:?] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:703) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:835) [?:?] ``` I ran into this when lowering the monitoring interval when investigating enrich monitoring test: #48258
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This test has been failing off and on for a while, per build-stats. Appears to fail mostly (only?) on 7.x/7.5.
Build scan
Error & stack trace:
Reproduce line (does not reproduce locally):
The text was updated successfully, but these errors were encountered: