Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increate test timeouts for ServersCheckerTest #15032

Merged
merged 1 commit into from
Oct 30, 2019

Conversation

sleshchenko
Copy link
Member

What does this PR do?

From time to time these tests fail on our CI job.
I'm able to reproduce this issue locally if my laptop is quite loaded with another process like maven build. And as we can see the first test takes the most time.
Screenshot_20191030_155103
I did not find any information about why it may happen but I assume that it's related to memory allocation.
I wrote a simple test that does nothing except N Objects

  private static final int N = 10_000_000;

  @Test(invocationCount = 40)
  public void doNothing() throws Exception {
    List<Object> list = new ArrayList<>();
    for(int i = 0; i< N; i++) {
      list.add(new Object());
    }
  }

and it's what I get:

  1. N = 10_000
    Screenshot_20191030_154016
    I assume that it's a situation similar to what we have in ServersCheckerTest.java, when not much memory is needed and the first test always takes the most time. I assume that the process during the first test allocates some amount of memory and further tests just reuse it.
  2. N = 10_000_000
    Screenshot_20191030_153926
    I don't have a clear explanation of why 2nd and 3rd take the most time but it's not really needed.
    My testing is not accurate at all and its purpose was just to show that using timeout needs additional care since invocation time may take different time depending on which it will be run(1,2,3,...), which time(a system is loaded, a garbage collector is active).

I believe 5 seconds should be even more than enough for tests to be stable.
In the pessimistic scenario (when there is an error in code or tests) we may pay 7*5=35 seconds of build time, but such a situation must happen only feature branch, where code is broken but not in master.

What issues does this PR fix or reference?

N/A

Release Notes

N/A

Docs PR

N/A

@che-bot che-bot added the status/code-review This issue has a pull request posted for it and is awaiting code review completion by the community. label Oct 30, 2019
@che-bot
Copy link
Contributor

che-bot commented Oct 30, 2019

E2E Happy path tests of Eclipse Che Single User on K8S (minikube v1.1.1) has been successful:

@sleshchenko
Copy link
Member Author

ci-build

Signed-off-by: Sergii Leshchenko <sleshche@redhat.com>
@che-bot
Copy link
Contributor

che-bot commented Oct 30, 2019

E2E Happy path tests of Eclipse Che Single User on K8S (minikube v1.1.1) has been successful:

@che-bot
Copy link
Contributor

che-bot commented Oct 30, 2019

E2E tests of Eclipse Che Multiuser on OCP has been successful:

@che-bot
Copy link
Contributor

che-bot commented Oct 30, 2019

E2E tests of Eclipse Che Multiuser on OCP has been successful:

@sleshchenko sleshchenko merged commit c7b2c63 into eclipse-che:master Oct 30, 2019
@sleshchenko sleshchenko deleted the serversChecker branch October 30, 2019 15:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status/code-review This issue has a pull request posted for it and is awaiting code review completion by the community.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants