You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm writing a testsuite for aria behavior (in vanilla js). These tests require checking against DOM, such as document.activeElement to see if the expected element received focus.
While writing tests, I realized one of my tests started to fail. I was even more suspicious when I commented out the tests that happen before and my failing test become successful, huh?
I figured I could shuffle the tests with describe.shuffle() and yes, running them through shuffle makes them fail and succeed.
I also have some ideas, why that might be:
Some DOM operations are put on micro/macro task queue and need to be awaited to finish (closing and opening popovers as I do in the tests, wait for document.activeElement)
between suites, the browser isn't "resetted" and state leaks between them
Here is a video of me running tests (shuffle mode):
I don't think that this flakiness is something that we can address on a runner level - if you need to wait for an event to resolve, then you should await until it happens - Vitest provides vi.waitFor utility for example. You also need to write tests isolated from one another - so one test doesn't affect the other.
What we will do in the future though is provide API to deal with events because dispatchEvent is not something that actually happens when user interacts with the web page - the most correct implementation would be to call Devtools protocol, because even a simple click can have a lot of sub-events and different behaviors depending on the browser and the element. For example, in the next version you should be able to use the new commands API: #5097. In the future, it will be expanded to support more APIs.
Thanks for the explanation. I was writing tests for QUnit which does that cleanup and I incorrectly transported that mental model over. I massively restructured my tests and now it's clean and working.
Describe the bug
I'm writing a testsuite for aria behavior (in vanilla js). These tests require checking against DOM, such as
document.activeElement
to see if the expected element received focus.While writing tests, I realized one of my tests started to fail. I was even more suspicious when I commented out the tests that happen before and my failing test become successful, huh?
I figured I could shuffle the tests with
describe.shuffle()
and yes, running them through shuffle makes them fail and succeed.I also have some ideas, why that might be:
document.activeElement
)Here is a video of me running tests (shuffle mode):
Bildschirmaufnahme.2024-05-11.um.12.14.55.mov
Reproduction
My repro is the monorepo of my design system. Though the affected package in there is a leaf package. I committed the flaky state, here is the repro sha: https://github.com/hokulea/hokulea/tree/655f2ac11697dd5dc54bf2faafa56aceb142e40e
Repro steps:
pnpm install
in the rootcd incubator/aria-navigator
pnpm vitest:ui
is what I use)The affected tests are
tests/menu/navigation.test.ts
, thesubmenu navigation
suite.The test code is what matters, details the the things being tested shouldn't matter that much, but here is a summary for better understanding:
HTMLElement
methods:focus()
,showPopover()
,hidePopover()
Popover API
, more preciseauto
modeSystem Info
Used Package Manager
pnpm
Validations
The text was updated successfully, but these errors were encountered: