Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

raw reporter shouldn't output DqElements #1195

Closed
WilcoFiers opened this issue Oct 18, 2018 · 4 comments · Fixed by #1513
Closed

raw reporter shouldn't output DqElements #1195

WilcoFiers opened this issue Oct 18, 2018 · 4 comments · Fixed by #1513
Assignees
Labels
core Issues in the core code (lib/core) fix Bug fixes support

Comments

@WilcoFiers
Copy link
Contributor

While working on a new reporter I tried to run JSON.stringify() on the results of the raw reporter. Turns out that this is outputting DqElements, which can't be stringified after completing a run. Because the selectors are lazily constructed, and in order to construct them axe._selectorData is required, they can't be constructed after cleanup of axe._selectorData. To fix this we need to change the raw reporter to not output DqElement objects.

@WilcoFiers WilcoFiers added fix Bug fixes core Issues in the core code (lib/core) labels Oct 18, 2018
@jeeyyy
Copy link
Contributor

jeeyyy commented Nov 21, 2018

Note:
Go thru results and any Dq element, toJson() or equivalent for selector

@jeeyyy
Copy link
Contributor

jeeyyy commented Nov 22, 2018

@WilcoFiers - I was unable to find any DqElement in the raw report, so I could run a toJson on the same. Need more input.

@dbjorge
Copy link
Contributor

dbjorge commented Apr 10, 2019

We just hit this issue during development of https://github.com/Microsoft/axe-sarif-converter - the problematic elements are at JSON paths along the lines of root_raw_results[0].violations[0].node and root_raw_results[0].violations[0].any[0].relatedNodes[0].

One option for a low-churn fix that we tried out successfully would be updating the raw reporter to output JSON.parse(JSON.stringify(results)), rather than outputting results directly. I don't know that I'd recommend this, though, since DqElement.toJSON omits some properties from the original DqElement; I think it would be preferable for the output of the raw reporter to be as close as possible to the input that a new reporter library would be receiving (using the output of the raw reporter as test cases for the new reporter we're developing has been our primary use case for the raw reporter so far), so I think it might be better to do a more explicit step to replace the lazy properties in question with non-lazy ones in the existing finalization phase before the results get passed to reporters (https://github.com/dequelabs/axe-core/blob/develop/lib/core/utils/finalize-result.js).

@AutoSponge
Copy link
Contributor

@WilcoFiers so, the test at https://github.com/dequelabs/axe-core/blame/320e63380189f507a32c99fd19619f3c6a161764/test/core/utils/dq-element.js#L35 is not a valid test for your case?

That may defeat the purpose of the raw reporter. Could I suggest a string reporter that uses a JSON.stringify({}, replacer) function to remove or otherwise normalize DQElements?

@aellsey aellsey added this to the HTMLTools Sprint 2 milestone Apr 21, 2019
@WilcoFiers WilcoFiers modified the milestones: HTMLTools Sprint 2, Axe-core 3.3 Apr 23, 2019
@stephenmathieson stephenmathieson self-assigned this Apr 24, 2019
stephenmathieson added a commit that referenced this issue Apr 24, 2019
This patch updates our "raw" reporter, ensuring it does not pass `DqElement`s to its callback. Instead, it serializes the `DqElement` by using the `DqElement#toJSON()` method.

Unfortunately many of our tests pass non-results to the reporters, so several "guards" were added to the reporter ensuring we don't error while attempting to transform the provided results. In an ideal world, these tests would be rewritten to pass "proper" result objects to our reporters, but this felt out-of-scope for the task at hand.

Closes #1195
stephenmathieson added a commit that referenced this issue Apr 26, 2019
This patch updates our "raw" reporter, ensuring it does not pass `DqElement`s to its callback. Instead, it serializes the `DqElement` by using the `DqElement#toJSON()` method.

Unfortunately many of our tests pass non-results to the reporters, so several "guards" were added to the reporter ensuring we don't error while attempting to transform the provided results. In an ideal world, these tests would be rewritten to pass "proper" result objects to our reporters, but this felt out-of-scope for the task at hand.

Closes #1195
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Issues in the core code (lib/core) fix Bug fixes support
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants