Landed
Happy new year, everyone! Before I go into long discussion about what’s new, here’s the link to compiled phar version for your enjoyment:
link.
So here it is, Behat v3. This is actually a RC1 version if @stof merges it. In two words - it’s stable, it’s ready, but has some formatters missing (only pretty
and progress
are implemented at the moment).
So lets talk about what’s new. First, lets pretend that previous betas never happened and that it was a bad software designer dream. I was unhappy with the out coming architecture - it was hard to read, follow and extend. That’s the primary reason why v3 wasn’t been released months ago. I took time to take the Behat codebase and apply all that knowledge I learned in last year thanks to people like @mathiasverraes.
This new codebase represents all what good software design means to me and as a matter of fact, first behat version ever I’m 100% proud of (except maybe formatters code themselves :D ). It is truly layered, independent architecture which will solve our testing problems years ahead. No more huge rewrites, guys. Not because we wouldn’t want it, but because this new architecture allows us to gradually grown parts of Behat independently - either through core or third party extensions.
So let’s get started with shiny new features?
Testwork
First stop is the Behat\Testwork
. Testwork is an interesting, but totally unexpected side effect of this domain-driven redesign for Behat. You see, the way this last refactoring was going is separation of concerns and sharpening of Behat domain.
If somebody didn’t know, Behat domain is a “testing tool” :) So somewhere at later stages of redesign I noticed that there are some parts of the codebase that has limited to inexistent knowledge about Gherkin or Behat itself - those were the pure domain services on their own. So this was more than natural to move all this common functionality out of Behat\Behat
namespace. I ended up moving ~40% of Behat code out into Behat\Testwork
.
So what is this Testwork now? Testwork basically is a first ever meta-testing framework for php. It is a framework for building testing frameworks. Think about it - every time somebody wants to start his own testing framework he needs to solve all the same common problems again and again - services management, configuration, extension system, suite management, environment management and isolation, subject loading, testing services, user input and testers output management with formatters. Recreating all those things again and again could be a really boring task (trust creator of two popular testing frameworks) because it’s routine.
You don’t want to spend time on boiler plating those same services again and again - it’s a waste of time. You want to focus on things that matter most and things that differentiate your framework from others. That’s where Behat\Testwork
gives you a hand. It provides you with a foundation and set of services that cover all aspects of a testing tool development:
ServiceContainer
providing configuration loading and extension mechanisms for your framework
Call
management with exception catching and output buffering
Cli
management using CLI controllers
Environment
management using environment handlers and isolators
Exception
handling services
Hook
management allowing you to provide hooking mechanisms in your framework
Output
management using different formatters. Testwork even provides you with base formatter classes that have everything you need.
Subject
loading tools
Suite
management utils
Tester
services themselves
All those things are implemented as extensions on top of the really-really slim core (ServiceContainer
+ Cli
packages). It basically means two things - almost everything in Testwork is extension and all extensions could provide their own extension points. Those extension points of core extensions are used by Behat set of extensions on top of Testwork.
Testwork extension system have some really awesome, but completely unplanned implications:
- Behat itself from now is is just a core set of Testwork extensions
- From now on you can not only extend the Behat, but the foundation it was built on (Testwork)
- Every testing framework that’ll use Testwork as its foundation will have full support for all Testwork extensions. It’s like Rack or Stack, but for testing frameworks
Lets stop on point #3 for a second. What it means is that when parallel runner extension will land to Behat v3.1 it will be a Testwork extension. So all other testing frameworks (phpspec3?, phpunit3?, codeception2?) built on top of Testwork will have parallel running functionality absolutely for free.
As of its future plans, Testwork currently lives inside Behat repository. There are couple of good reasons for that:
- There could be some early design flaws (small ones if some) that I want to iron out before splitting both projects
- As Testwork was pretty much unplanned child, it does not have its own tests for now and solely depends on Behat test suite
Plan is for Behat v3.1 to extract Testwork into separate repository with its own suite of tests and contributors. For now, you can use it as a dependency by requesting behat/behat: ~3.0
.
Behat\Testwork
will hopefully open new era for testing frameworks in php. Same way as Behat and PhpSpec did. And I’m pretty much looking forward to what you guys can do with it. Documentation is coming, but for now, you can simply read the src/Behat/Testwork
codebase. I hope I did a good job to make it readable and understandable.
New Behat user land features
Now we can stop talking about architecture with design and shift our focus to something you’ll see in your day-to-day Behat-user life - shiny new Behat features. As a matter of fact, there are plenty of them in v3. Let me show you the bigger ones.
Multi-suite architecture
This is a big one. In v3 nor your contexts nor your features are first-class citizens of your test suite, suites are. Each behat exercise (single execution) could have one or more suites associated with it. And each suite could have one or more contexts and features associated with it. It means not only you could have multiple contexts in one suite - you could test same features against different context classes in one run. Here's how it looks like in reality: http://youtu.be/8RqFzqj79iY
Standard output buffering
Remember those nasty times when you needed to debug some step, you tried to echo something from it and it messed your entire output, confusing you even more? Well, happy to report that these times are gone. From now on, Behat captures all output your steps produce and delegates its printing to output formatters. And be sure that formatters do damn good job of printing them cleanly in the context of a step under question. Just try to echo anything in one of your test steps and run behat.phar I provided above ;)
Hooks as first class citizens
In prior versions of Behat, hooks were merely a way to execute something before of after something happens in the framework. The problem was that they never were a first class citizen. There was no clear exception handling and failing BeforeScenario
hook still didn’t prevent this scenario from running. Welp, no more. From now on, every @Before…
hook have an impact on the testers and if your @BeforeStep
fails, it means you can’t reliably test your step, thus it will be automatically marked as failed.
But what about debugging? Same output buffering and exception handling rules apply to hooks too. And pretty formatter does damn good job of presenting your hooks output or exceptions in a most readable and clean way. Just give it a shot - define @BeforeScenario
in your test context and enjoy the awesomeness :)
Tell - don’t ask snippets
We all know how awesome and impressive —append-snippets
option is (if you don’t - go read behat -h
). But more often than not we have more than one context class and want to add snippets to specific one instead of the main FeatureContext
. That’s the primary reason —append-to
was added, but it is clunky and unnatural to enter your context class every time. From now on, Behat will generate and append snippets only to contexts implementing one of those interfaces:
Behat\Behat\Context\TurnipAcceptingContext
Behat\Behat\Context\RegexAcceptingContext
Every other context class will be ignored. It basically means that from now on, you need to tell once which context accepts snippets instead of telling to which context put them every time. You can even create convention in your team that you don’t commit contexts implementing those interfaces, meaning that every team member could have his own context implementing them.
Turnip as a default step pattern language
We all know that nobody can beat flexibility of regexps. That's why they are still here in behat and will stay for versions ahead. But flexibility always comes at a cost - especially with regexps. Readability suffers a lot even when you define a simplest patterns. So that's why starting from 3.0, default behavior for your initial context would be to use Turnip-style snippets. Read more.
New Pretty and Progress formatters
Pretty and Progress formatters were reimplemented from ground up. And they are much cleaner and readable than ever before. For example, Pretty formatter now magically highlights outline row columns with different colours according to failure state. Keywords are highlighted and paths don’t have a tendency to take all available space. Just give them a try and I’m sure they’ll surprise you not once.
Speed and memory footprint improvements
New architecture doesn’t only mean better extensions or more features in the future. In some cases it actually means huge improvement on users day-to-day life. For example, new architecture of Behat v3 helped to eliminate all of the performance and memory issues of Behat v2. And oh my, there were some. I spent quite some time debugging Gherkin and finding crazy new ways to limit amount of undying references in the core. And the result is - incredible speed boost and memory footprint never been that low. As a matter of fact, I think if you ever had “not enough memory” exception - chances are really high you’ll not see them again.
I’m so impressed with Behat speed and memory consumption now that both those stats are now included by default in core formatters (so you can be impressed too).
Next steps
@stof will review this PR and hopefully would be able to merge it without finding problems that I’ll need to fix before :) In the mean time, I’ll switch my focus back to Behat core extensions (Mink, Symfony2) and new behat.org website. It will give community couple of weeks to one month for test-driving this new version and maybe even contributing those missing formatters (html, junit, json) :)
No more rewrites - because we really don’t need them anymore.
With love,
Happy new year, guys!