-
Notifications
You must be signed in to change notification settings - Fork 53
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Filtering out all scenarios in the story file causes NPE #136
Comments
@valfirst I created test for the issue in the project and tried to fix it. |
@jsedlace , thank you for your contribution, I'll take a deep look today, but at first glance you're fixing symptom, anyway I'll use your test story to check behavior |
One more finding - according to JBehave grammar, there cannot be story without any scenario, if story contains just one scenario, the 'Scenario:' keyword is optional, but at least one step is mandatory. So if I tried story file without any scenario or step, it failed as well somewhere in JUnitScenarioReporter, but because it was invalid story file, it is acceptable (it created rootDescription with 3 stories
If all scenarios are filtered out, rootDescription contains just
Generation of descriptions happens in com.github.valfirst.jbehave.junit.monitoring.JUnitDescriptionGenerator#createDescriptionFrom(org.jbehave.core.embedder.PerformableTree) - storyDescription is added to list of all description only if it contains at least one allowed scenario - but in my case, all are filtered out, so no story is added. It looks that there is a conflict in behaviour of jbehave filtering - story is allowed, even if all scenarios are not allowed, and behavior of jbehave-junit-runner - story is not put to descriptions if at least one scenario is not allowed. |
@jsedlace thank you for investigation, I've looked deeper, and from my point of view the root cause is in JBehave core, I think story having only filtered out scenarios shouldn't be executed. I see 2 options:
What's impact of this issue? Does it have a high priority for you? |
@valfirst I already looked into jbehave-core little bit, because it seemed to me like the right place - filtering stories/scenarios (is/is not allowed to be run) is done on one place, so fix should be quite simple, I cannot think out all consequences now, but unit tests can lead me. Regarding impact/priority - I'll ask the team, but we have this issue for few weeks already, it actually started when we filtered out more tags because of some flakyness of tests/incompatible tests. I'll let you now, we can use our snapshot build for a while probably |
|
@valfirst Thanks, good job. jbehave-core 4.3.4 was released, do you plan to release also jbehave-junit-runner with this fix? |
@jsedlace , yes, I plan to release jbehave-junit-runner tomorrow |
@jsedlace, release 2.2.0 is published |
We use Jbehave with serenity, we also quite often use filtering of stories/scenarios with metafilters.
What is not working now:
one story file, 2 scenarios inside, each scenario is tagged with some tag (doesn't matter if same for all or different)
If metafilter is written so that all scenarios are filtered out, but the story itself is not filtered out, it fails with NPE:
It's quite unpleasant for us, because even if all scenarios pass (real tests), this fails the maven build.
If you can point me to some specific class where to focus with fixing, I'll be happy to provide PR, I have some ideas where to go, but the code is quite complex and even if I fix it somewhere, it does not need to be the right place (I may solve symptoms, not the root cause).
Reproducer is attached.
jbehave-reproducer.zip
The text was updated successfully, but these errors were encountered: