Karate engine re-write - exit from Cucumber but retain 100% syntax backwards compatibility

[vijayqeguy]

Any plans upgrade Cucumber version to 2 or 3?

We use Cucumber version 2 for our UI automation. Now if I import karate using maven, it conflicts with the version used for UI since karate using Cucumber version 1

[ptrthomas]

not right now unless we see more demand for this. advise stick to cucumber v1.2.5 as far as possible.

marking this ticket as "help wanted".

[ptrthomas]

plan to start work on this soon.

• use a custom parser and life-cycle for execution + reports
• support the existing Gherkin and keywords with 100% backwards compatibility
• possibly need a thin wrapper for IDE support to work - have to "trick" IDE-s into thinking that Karate is exposing standard step definitions

considering writing a grammar in ANTLR

[ptrthomas]

this work will involve taking a re-look at reports. would like to de-couple these from cucumber as well - but retain the cucumber JSON format

additional capability proposed:

• way for users to inject custom snippets and even images into the report.

[ptrthomas]

been doing analysis, this is going to be a lot of work ! but convinced this is top priority. not because we want to support that small percentage of users who insist on using Cucumber + Selenium. but because we want to be ready for Java 9/10 and we really don't need all the complexity and breaking changes that Cucumber v2 and v3 have introduced. I also personally do not agree with the decision the Cucumber team has made to use an out-of-JVM process (Go native binary) for Gherkin parsing in v3 v4 (?) onwards. https COLON SLASH SLASH github.com/cucumber/common/pull/424

for Karate - we are clear. we love the JVM. it is what allows us to do mocks (netty) and performance testing (gatling)

meanwhile, I can see that IntelliJ has yet to catch up with the v3 Cucumber changes: https://youtrack.jetbrains.com/issue/IDEA-193111

Not 100% sure about Eclipse, but we certainly don't care about the Lambda support: https COLON SLASH SLASH github.com/cucumber/cucumber-eclipse/issues/267

the worst case scenario is that Cucumber Java IDE support will show warnings because it can't find the "glue" etc. pefectly reasonable IMO.

would be interested in hearing what others think.

[ayoolaogunsola]

Would this affect the use of BDD-style tests (i.e feature files with scenarios/scenario outlines)? Also, would we lose the cucumber formatting and test run support within IDEs? The fact that Karate is based on cucumber was one of the attractions to this framework.

[ptrthomas]

@ayoolaogunsola thanks !

• as the title of this issue says we will have 100% backwards compatibility, yes Scenario / Outline, Given When etc.
• so yes, formatting support will be retained
• IDE integration by which I assume you mean "right-click-and-run" - this is where it gets hacky. I'm pretty confident though but cannot guarantee this 100%
• "based on Cucumber" - you may want to read this thread very carefully: Allow interoperability with cucumber or allow extensibility #398

[ayoolaogunsola]

@ptrthomas Thanks for clarifying. I don't have a need to integrate with Cucumber so as long as Karate keeps its simplicity, no concerns here. It's working well for our needs so far. Thanks for a great tool.

[ptrthomas]

missed tagging this ticket but this commit has the initial implementation of the antlr grammar and even the execution life-cycle is working :) the next big task is reporting.

99a2843

[thirdy]

Hi. Is there now a way to extend the DSL? I understand there was a discussion of it in #398. In my case, I want to make it simpler to build requests and provide reusable queries to responses. Something like:

* firstName Karl
* lastName Marx
* GetUser
* response.someComplexQuery == '2013-01-01'

GetUser will implicitly use the the given parameters to build the Request and also run the Http call. Then response will have some common fields to be tested. GetUser will also check config.channel if it's REST or SOAP, and will adopt accordingly.

So what I have now is a DSL specialized to our API.

Would appreciate any advice on how I should approach this. Maybe even write my own ANTLR grammar?

[ptrthomas]

@thirdy I think I have been very clear in this comment:
#398 (comment) - and strongly feel you can use Karate the way it is.

there's also some very early thoughts here, but this is not a priority until we move everything out of Cucumber:
#489 (comment)

[ptrthomas]

again forgot to tag this issue but this commit is a big milestone: d6e5f4b

we have successfully removed all dependencies on cucumber except for the critical few needed for backwards compatibility

the 2 reports (JSON, HTML) need to be worked on to bring them to the original state - but most of the pain is over

[ptrthomas]

I'm going to go ahead and mark this as fixed. this is a big deal and took far more work than I anticipated.

we don't depend on the fate of the Cucumber-JVM project anymore.
we control the entire life-cycle and reporting, which feels very liberating
even IDE support continues to work for cucumber-jvm 1.2.5 (karate default)

Please do try this out from the cukexit branch. We really need this help and feedback to ensure your existing karate-suites continue to work.

an interesting design decision is to continue to bundle the info.cukes:cucumber-java:1.2.5 dependency. this is purely for Eclipse and IntelliJ syntax coloring to work !

regarding the original issue by @vijayqaguy - you should be able to mix karate with newer cucumber versions. but IDE support may then break, which may require some community contribution to resolve. the new design of the step-definitions will make this easier (thin layer, using interfaces etc)

again, if you can clone and test this, please do help !

[ptrthomas]

found this twitter rant about Cucumber API breaking changes, saving this here for future reference

https://twitter.com/GarthGilmour/status/1031924983403044864

[ptrthomas]

saving this link for future reference, some discoveries the cucumber team made with the "go binary" approach: cucumber/common#425 (comment)

looks like we did the absolute right thing.

[ptrthomas]

HELL YEAH ! 0.9.0 released

[mpkorstanje]

I also personally do not agree with the decision the Cucumber team has made to use an out-of-JVM process (Go native binary) for Gherkin parsing in v3 v4 (?) onwards.

For the visitors from Stackoverflow and Google it is worth noting that Cucumber JVM never used the go binaries and that since v8 Gherkin has a native Java implementation again.

[ptrthomas]

@mpkorstanje thanks for the update !