Skip to content

Commit

Permalink
[ftr] abort retry on invalid webdriver session (elastic#174092)
Browse files Browse the repository at this point in the history
## Summary

Since many our e2e tests use Retry service, we might have a situation
with retry running for some time (**20,30, 120 seconds** ) before
reaching timeout while browser already crashed and Webdriver is not
functioning properly.

This PR updates Retry service with checking error name before retry
attempt: if it is the WebDriver critical error, retry is aborted and we
fail test fast.

It should help with long useless logging messages as well:

Before:

```
         │ debg --- retry.try error: no such window: target window already closed
         │      from unknown error: web view not found
         │        (Session info: chrome=120.0.6099.129)
         │ debg Find.findByCssSelector('[data-test-subj="canvasExpressionInput"]') with timeout=10000
         │ debg --- retry.try failed again with the same message...
         │ debg Find.findByCssSelector('[data-test-subj="canvasExpressionInput"]') with timeout=10000
         │ debg --- retry.try failed again with the same message...
         ...
         │ERROR Browser is closed, no artifacts were captured for the failure
         └- ✖ fail: Canvas Canvas app expression editor shows autocomplete when typing
         │      retry.try timeout: NoSuchWindowError: no such window: target window already closed
         │ from unknown error: web view not found
         │   (Session info: chrome=120.0.6099.129)
         │     at Object.throwDecodedError (/Users/dmle/github/kibana/node_modules/selenium-webdriver/lib/error.js:524:15)
         │     at parseHttpResponse (/Users/dmle/github/kibana/node_modules/selenium-webdriver/lib/http.js:601:13)
         │     at Executor.execute (/Users/dmle/github/kibana/node_modules/selenium-webdriver/lib/http.js:529:28)
         │     at processTicksAndRejections (node:internal/process/task_queues:95:5)
         │     at Task.exec (prevent_parallel_calls.ts:28:20)
         │   Error: retry.try timeout: NoSuchWindowError: no such window: target window already closed
         │   from unknown error: web view not found
         │     (Session info: chrome=120.0.6099.129)
         │       at Object.throwDecodedError (node_modules/selenium-webdriver/lib/error.js:524:15)
         │       at parseHttpResponse (node_modules/selenium-webdriver/lib/http.js:601:13)
         │       at Executor.execute (node_modules/selenium-webdriver/lib/http.js:529:28)
         │       at processTicksAndRejections (node:internal/process/task_queues:95:5)
         │       at Task.exec (prevent_parallel_calls.ts:28:20)
         │       at onFailure (retry_for_success.ts:17:9)
         │       at retryForSuccess (retry_for_success.ts:59:13)
         │       at RetryService.try (retry.ts:31:12)
         │       at Proxy.clickByCssSelector (find.ts:417:5)
         │       at TestSubjects.click (test_subjects.ts:164:5)
         │       at Context.<anonymous> (expression.ts:92:7)
         │       at Object.apply (wrap_function.js:73:16)
         │
         │
       └-> "after all" hook for "shows autocomplete when typing"
         │ debg unloading docs from archive at /Users/dmle/github/kibana/x-pack/test/functional/fixtures/kbn_archiver/canvas/default.json
         │ info deleting 1 objects { space: undefined }
         │ succ 1 saved objects deleted
       └-> "after all" hook: afterTestSuite.trigger for "shows autocomplete when typing"
         │ERROR Browser window is already closed
```

After:

```
         │ debg --- retry.try error: no such window: target window already closed
         │      from unknown error: web view not found
         │        (Session info: chrome=120.0.6099.129)
         │ERROR Browser is closed, no artifacts were captured for the failure
         └- ✖ fail: Canvas Canvas app expression editor shows autocomplete when typing
         │      Error: WebDriver session is invalid, retry was aborted
         │       at retryForSuccess (retry_for_success.ts:64:13)
         │       at RetryService.try (retry.ts:31:12)
         │       at MonacoEditorService.getCodeEditorValue (monaco_editor.ts:25:5)
         │       at Context.<anonymous> (expression.ts:83:34)
         │       at Object.apply (wrap_function.js:73:16)
         │
         │
       └-> "after all" hook for "shows autocomplete when typing"
         │ debg unloading docs from archive at /Users/dmle/github/kibana/x-pack/test/functional/fixtures/kbn_archiver/canvas/default.json
         │ info deleting 1 objects { space: undefined }
         │ succ 1 saved objects deleted
       └-> "after all" hook: afterTestSuite.trigger for "shows autocomplete when typing"
         │ERROR Browser window is already closed
     └-> "after all" hook in "Canvas app"
       │ debg set roles = superuser
       │ debg creating user test_user
       │ debg created user test_user
     └-> "after all" hook: afterTestSuite.trigger in "Canvas app"
       │ERROR Browser window is already closed
   └-> "after all" hook: afterTestSuite.trigger in "Canvas"
     │ERROR Browser window is already closed

5 passing (17.0s)
1 failing

1)    Canvas
       Canvas app
         expression editor
           shows autocomplete when typing:

      Error: WebDriver session is invalid, retry was aborted
       at retryForSuccess (retry_for_success.ts:64:13)
       at RetryService.try (retry.ts:31:12)
       at MonacoEditorService.getCodeEditorValue (monaco_editor.ts:25:5)
       at Context.<anonymous> (expression.ts:83:34)
       at Object.apply (wrap_function.js:73:16)

```

(cherry picked from commit f9a4962)
  • Loading branch information
dmlemeshko committed Jan 2, 2024
1 parent a38fc92 commit a801563
Show file tree
Hide file tree
Showing 3 changed files with 13 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -52,12 +52,16 @@ export async function retryForSuccess<T>(log: ToolingLog, options: Options<T>) {

const start = Date.now();
const retryDelay = 502;
const criticalWebDriverErrors = ['NoSuchSessionError', 'NoSuchWindowError'];
let lastError;

while (true) {
if (Date.now() - start > timeout) {
await onFailure(lastError);
throw new Error('expected onFailure() option to throw an error');
} else if (lastError && criticalWebDriverErrors.includes(lastError.name)) {
// Aborting retry since WebDriver session is invalid or browser window is closed
throw new Error('WebDriver session is invalid, retry was aborted');
} else if (lastError && onFailureBlock) {
const before = await runAttempt(onFailureBlock);
if ('error' in before) {
Expand Down
5 changes: 4 additions & 1 deletion test/common/services/security/test_user.ts
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,10 @@ export class TestUser extends FtrService {
});

if (this.browser && this.testSubjects && !options?.skipBrowserRefresh) {
if (await this.testSubjects.exists('kibanaChrome', { allowHidden: true })) {
if (
(await this.browser.hasOpenWindow()) &&
(await this.testSubjects.exists('kibanaChrome', { allowHidden: true }))
) {
await this.browser.refresh();
// accept alert if it pops up
const alert = await this.browser.getAlert();
Expand Down
6 changes: 5 additions & 1 deletion test/functional/services/remote/remote.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
* Side Public License, v 1.
*/

import { NoSuchSessionError } from 'selenium-webdriver/lib/error';
import { NoSuchSessionError, NoSuchWindowError } from 'selenium-webdriver/lib/error';
import { FtrProviderContext } from '../../ftr_provider_context';
import { initWebDriver, BrowserConfig } from './webdriver';
import { Browsers } from './browsers';
Expand Down Expand Up @@ -37,6 +37,10 @@ export async function RemoteProvider({ getService }: FtrProviderContext) {
// Avoid duplicating NoSuchSessionError error output on each hook
// https://developer.mozilla.org/en-US/docs/Web/WebDriver/Errors/InvalidSessionID
log.error('WebDriver session is no longer valid');
} else if (error instanceof NoSuchWindowError) {
// Avoid duplicating NoSuchWindowError error output on each hook
// https://developer.mozilla.org/en-US/docs/Web/WebDriver/Errors
log.error('Browser window is already closed');
} else {
throw error;
}
Expand Down

0 comments on commit a801563

Please sign in to comment.