Event driven BDD test framework for PHP

Related tags

Testing testing php bdd
Overview

Peridot logo

Packagist Version Build Status HHVM Build Status Windows Build Status Scrutinizer Code Quality Codecov Coverage Gitter Chat

The highly extensible, highly enjoyable, PHP testing framework.

Read more at peridot-php.github.io or head over to the wiki.

Building PHAR

Peridot's phar is built using Box. Once box is installed, the phar can be built using the following command from the project directory:

box build

Generating Peridot's documentation

Peridot API documentation is generated using apigen. Once apigen is installed, run the following command from the project directory:

apigen generate

This will output documentation to the docs/ directory.

Running Peridot's tests

Peridot's test suite can be run using Peridot:

$ bin/peridot

And a sample of output:

Peridot output sample

Release

We use Robo for releases.

robo release [version] [site-path]

using assert for expectations

Peridot sets ASSERT_CALLBACK via assert_options in order to throw exceptions when using the native assert function. Peridot specs are considered passing if they don't throw an exception; if you are using assert for expectations and you find your specs are triggering false positives, you may need to update zend.assertions to 1 in your php.ini. This is set to -1 by default in PHP 7+

If you aren't too keen on using assert for testing, there are a ton of assertion/expectation libraries out there. Anything that throws exceptions in response to a failed assertion will do. The peridot-php org has authored Leo, and this library offers a richer assertion vocabulary for testing.

Comments
  • Failing quietly

    Failing quietly

    After a composer update moved us up to the new version today, our test that were working in 1.14 just did nothing. Command line would report a fail on the command.

    The peridot.php:

    <?php //peridot.php
    //For using prophecy in peridot in testing.  Allows it to be used globally
    require(dirname(__FILE__).'/../../vendor/autoload.php');
    ini_set('display_errors',0);
    error_reporting(0);
    use Evenement\EventEmitterInterface;
    use Peridot\Plugin\Prophecy\ProphecyPlugin;
    use Peridot\Plugin\Watcher\WatcherPlugin;
    
    return function(EventEmitterInterface $emitter) {
        $plugin = new ProphecyPlugin($emitter);
        $watcher = new WatcherPlugin($emitter);
        $watcher->track(__DIR__ . '/../../OS');
    };
    ?>
    
    opened by lawrenceoboylel 21
  • Updated dependencies, PHP>=7.1

    Updated dependencies, PHP>=7.1

    Hey there! Since I'm a big fan of testing with peridot, I can't help but notice there weren't that many updates as of late. I need Symfony Console >v4 in some of my projects, but can't, since Peridot still requires the v2.
    Hence I'd like to propose updating all dependencies to more recent versions, as well as raising the PHP version requirement to >= 7.1.

    I know, that shuns backwards compatibility. However, the definite EOL of PHP 5.6 is just ahead (Dec. 31st I think), so I'd say it's definitely okay to cut loose ends, too. Maybe in the shape of a new major version?

    Let me know what you think.

    Forgot to mention it - I ran the full test suite, which is all green with some minor modifications.

    opened by Radiergummi 14
  • Tests now fail when an error occurs.

    Tests now fail when an error occurs.

    Continuing from #175, this PR further improves the error handling in Peridot.

    The biggest change is that when an error occurs in a test, that test will now fail. This new behavior will honor the error_reporting setting, meaning that systems that typically ignore errors of a given severity can still be tested under Peridot. This is the same underlying mechanism used by the error control operator (@), so that will also work as intended.

    Regardless of the error_reporting level, a Peridot error event will still be emitted. It will be up to individual listener implementations to decide whether to honor the error_reporting level.

    Another change is to the behavior of tests in which multiple failures occur. Previously, if a test failed AND its teardown also failed, the teardown failure would "overwrite" the test failure. This behavior has changed so that the first failure is always the failure that is reported. This has particular significance given that a test can now fail because of an error, and then continue on to fail because of an assertion failure or other exception.

    I also added support for PHP 7 engine exceptions. Note that I have polyfilled the existence of Throwable and Error, so that the tests may also be run under PHP 5.

    This PR should also solve #158 and #132.

    opened by ezzatron 11
  • Tighten up CLI parsing

    Tighten up CLI parsing

    I have a multi-language project, so I want to organize my tests inside a main test folder.

    test/
      - php/
        - peridot.php
        - spec/
      - nodejs/
        - mocha.js
        - spec/
    

    I know that I can use --configuration=test/php/peridot.php to tell it where to look for the config. But how do I configure the folder containing tests?

    bug 
    opened by IngwiePhoenix 11
  • Memory limit error

    Memory limit error

    I suspect this issue may be related to issue 174, but I didn't want to hijack their issue, since it may not be related at all.

    Error: PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 3072 bytes).

    Recently, I created new test file and was running fine locally, but when I created the pull request, the tests failed on circleci. I believe circleci's memory limit is 128 (locally is 512), because when I lower local's memory limit to 128, the tests run out of memory as well.

    That was when I discovered how scopes were leaking as described in issue 174. We suspected that may be the issue and added an suite.end event that removed the properties from the scope. However, the issue still persists. I var_dumped the scope after unsetting the properties and noticed there is a ProphecyScope and it still has methodProphecies created by our tests.

    I'm wondering if perhaps the ProphecyScope is just getting bigger over the course of the tests and finally runs out of memory. Is there a way to reset/clear a ProphecyScope? Perhaps another suggestion to what the issue is?

    Below is the code for our peridot.php file.

    return function(EventEmitterInterface $emitter) {
        $plugin = new ProphecyPlugin($emitter);
        $watcher = new WatcherPlugin($emitter);
        $watcher->track(__DIR__ . '/../../OS');
    
        $testingScope = new TestingScope();
    
        $emitter->on('suite.start', function($test)use ($testingScope){
            $test->getScope()->peridotAddChildScope($testingScope);
        });
    
        $emitter->on('suite.end', function($test){
            $scope = $test->getScope();
    
            foreach($scope as $key => $value){
                unset($scope->$key);
            }
        });
    };
    

    Thanks for any suggestions.

    opened by djones8520 9
  • Add `test` alias for `it`

    Add `test` alias for `it`

    This adds the test alias for it and the other variants. This is similar to facebook's Jest. I think there are some cases where test can be more suiting fo the test description.

    Specifically, I've created a package which generates markdown documentation from your specs, and I've found that using it can make the wording weird, but test is more natural.

    Btw, thank you so much for making this package, it's such a great tool!

    Signed-off-by: RJ Garcia [email protected]

    opened by ragboyjr 7
  • Tests always reported as passing?

    Tests always reported as passing?

    I feel like I have to be missing something painfully simple here, but can't for the life of me figure it out. Started out by pulling in Peridot via Composer (along with a few plugins):

    "require-dev": {
        "mockery/mockery": "dev-master",
        "squizlabs/php_codesniffer": "~1.5",
        "peridot-php/peridot": "~1.15",
        "peridot-php/peridot-list-reporter": "^1.0",
        "peridot-php/peridot-code-coverage-reporters": "^2.0"
    }
    

    I then created a specs/ folder and within it a Sample.spec.php file containing the sample code from the GitHub page:

    describe('ArrayObject', function() {
        beforeEach(function() {
            $this->arrayObject = new ArrayObject(['one', 'two', 'three']);
        });
    
        describe('->count()', function() {
            it("should return the number of items", function() {
                $count = $this->arrayObject->count();
                assert($count === 3, "expected 3");
            });
        });
    });
    

    All looks peachy:

    $ vendor/bin/peridot specs/Sample.spec.php
    
      ArrayObject
          ->count()
            ✓ should return the number of items
    
    
      1 passing (0 ms)
    

    However, if I change the assertion in attempt to make the test fail...

    describe('ArrayObject', function() {
        beforeEach(function() {
            $this->arrayObject = new ArrayObject(['one', 'two', 'three']);
        });
    
        describe('->count()', function() {
            it("should return the number of items", function() {
                $count = $this->arrayObject->count();
                assert($count === 1, "expected 3");
            });
        });
    });
    

    ...the test still passes:

    $ vendor/bin/peridot specs/Sample.spec.php
    
      ArrayObject
          ->count()
            ✓ should return the number of items
    
    
      1 passing (0 ms)
    

    Using the List reporter also shows the test as passing:

    $ vendor/bin/peridot specs/Sample.spec.php -r list
      ArrayObject ->count() should return the number of items
    
    
      1 passing (0 ms)
    

    System info (Linux Mint 18):

    $ vendor/bin/peridot --version
    Peridot version 1.18.1
    $ php --version
    PHP 7.0.4-7ubuntu2.1 (cli) ( NTS )
    Copyright (c) 1997-2016 The PHP Group
    Zend Engine v3.0.0, Copyright (c) 1998-2016 Zend Technologies
        with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2016, by Zend Technologies
    $ composer show
    evenement/evenement                         v2.0.0             Événement is a very simp...
    hamcrest/hamcrest-php                       v2.0.0             This is the PHP port of Ha...
    mockery/mockery                             dev-master ee06e7b Mockery is a simple yet fl...
    peridot-php/peridot                         1.18.1             Event driven BDD test fram...
    peridot-php/peridot-code-coverage-reporters 2.0.1              Code coverage reporters fo...
    peridot-php/peridot-list-reporter           1.0.0              A list reporter for the Pe...
    peridot-php/peridot-scope                   1.3.0              Scopes for function bindin...
    phpunit/php-code-coverage                   2.2.4              Library that provides coll...
    phpunit/php-file-iterator                   1.4.1              FilterIterator implementat...
    phpunit/php-text-template                   1.2.1              Simple template engine.
    phpunit/php-timer                           1.0.8              Utility class for timing
    phpunit/php-token-stream                    1.4.8              Wrapper around PHP's token...
    sebastian/environment                       1.3.7              Provides functionality to ...
    sebastian/version                           1.0.6              Library that helps with ma...
    squizlabs/php_codesniffer                   1.5.6              PHP_CodeSniffer tokenises ...
    symfony/console                             v3.1.2             Symfony Console Component
    symfony/polyfill-mbstring                   v1.2.0             Symfony polyfill for the M...
    
    opened by ghost 7
  • Do not suppress errors

    Do not suppress errors

    I use Asplode in my peridot.php to set up an error handler that converts errors into error exceptions. I only recently realised that this was not working, and that my tests were passing silently despite errors.

    I see two problems here:

    1. Peridot is currently overwriting and ignoring any existing error handler.
    2. Peridot is currently silencing errors, because the error handler doesn't actually do anything except to raise a Peridot "error" event. That seems like a dangerous choice. Shouldn't a test fail if an error occurs during its execution, and shouldn't that happen without having to write a custom "error" event handler?

    This PR addresses these two problems in the following ways:

    1. If there is an existing error handler, Peridot will now "forward on" the error to the existing handler. This includes using the existing error handler's return value, which is important (see below).
    2. If there is no existing error handler, Peridot's error handler will now return false to indicate that the error should propagate to PHP's inbuilt error handler. This means that the error will at least be displayed to the user, instead of being silenced completely.

    This does not address the issue of whether tests should fail due to errors, but that's probably another discussion / PR.

    opened by ezzatron 7
  • [WIP] Support for multiple reporters

    [WIP] Support for multiple reporters

    This PR adds support for multiple reporters.

    Summary/justification of changes:

    • This is a not a BC break, unless you count setting the default value of the reporter command-line option in peridot.php.
    • Multiple reporters can now be specified on the command line with multiple --reporter options.
      • The docs will need a minor update to reflect this.
    • When multiple reporters are specified, they are composed into a 'composite' reporter.
      • The first reporter gets the "real" output interface, and subsequent reporters get a buffered output interface. Once the runner.end event fires, any buffered output is displayed after the primary reporter's output, separated by a single newline. This prevents interlaced output, while retaining streaming output for the primary reporter.
    • Retains support for getReporter() and setReporter().
      • We can deprecate them with the next major release, and remove them in the one after that. I'm not sure how we want to handle deprecations though.
    • I'm not sure what the PERIDOT_REPORTER environment variable is used for, but I also retained support for it, and added a similar PERIDOT_REPORTERS environment variable, which contains all the reporter names, separated by comma.

    Closes #186.

    opened by ezzatron 6
  • Adds codecov support and updates build matrix

    Adds codecov support and updates build matrix

    Switches us over to using codecov

    @ezzatron - I took a cue from you and introduced a Makefile. codecov says it aggregates reports across builds, so I thought it would be good to let code coverage run for all build targets in the matrix (with the exception of allowed failures)

    The Makefile currently only defines a coverage target, but we can define more for other build tasks as we need them.

    The coverage target is conditionally defined to skip nightly builds and to use phpdbg for PHP 7.0 and 7.1 (as it is the only driver supported for those versions with our version of phpunit/php-code-coverage)

    maintenance 
    opened by brianium 6
  • Allow to filter tests by pattern

    Allow to filter tests by pattern

    Currently peridot provide only ability to filter by file. What I propose is additional --filter options for CLI which will allows to filter by test case description. For example:

    describe('Some Component', function () {
        it('It works in most common cases (critical)', function () {
             // test case 1
        });
    
        it('It works in other cases', function () {
            // test case 2
        });
    });
    

    So by running

    peridot --filter "\(critical\)" specs/ 
    

    We will get only test case 1 executed. I could prepare a PR but I'm not sure about implementation and maybe this should be just plugin and not built in feature.

    feature request 
    opened by fesor 6
  • Peridot 2.0 Ideas

    Peridot 2.0 Ideas

    After a while of thinking about what a peridot 2.0 should look like, I don't think i want to try and compete with Kahlan and PHPUnit. Peridot 1.0 is basically a lite version of kahlan with a smaller community and resources behind it. I don't really see any area that peridot can try to tap into that Kahlan userbase in a way that Kahlan isn't already doing.

    The describe-it syntax is very popular in other languages and personally, I do like how expressive the system can be. That being said however, PHPUnit by far is the defacto testing framework and majority of the PHP community resources and integrations revolve around PHPUnit. If kahlan hasn't taken off by now, then it's very likely that the php community won't be ready to rally around a new framework.

    Personally, I actually like that the PHP community largely pools our resources into one testing framework. It's very attractive for new comers and also means that the community can focus our efforts on growing the PHPUnit ecosystem instead of dilluting our efforts among competing frameworks.

    All that said, I do think there might a space for a new frontend for PHPUnit. A library that piggy backs off the entire phpunit framework but provides a different interface for writing tests with the describe-it syntax that the peridot/kahlan community enjoys.

    I think peridot 2.0 would be a good place to explore this integration with phpunit. It'd allow us to focus on the parts of testing framework that are interesting and leave the heavy lifting to phpunit all the while piggy backing off of the wealth of documentation and third party plugins that phpunit provides.

    In a similar manner, we'd probably make peridot/leo be just a thin wrapper around the PHPUnit assertions library and include in the peridot 2.0 core.

    In the long run, I'd love to see peridot's new phpunit frontend be brought under the fold of PHPUnit with first class support and apart of the phpunit ecosystem instead of a separate library.

    Let me know what you all think about this idea, welcome to suggestions :)

    opened by ragboyjr 7
  • beforeAll on suite/file level and global beforeAll

    beforeAll on suite/file level and global beforeAll

    Is it possible to implement these with current API and add it to current DSL?

    beforeAll Runs a function before any of the tests in this file run. Example in Jest https://jestjs.io/docs/en/setup-teardown#one-time-setup

    Repeating Setup For Many Tests Run setup for each test, for ex. to cleanup database and cache storage. So it's a global beforeEach. Example in Jest https://jestjs.io/docs/en/setup-teardown#repeating-setup-for-many-tests

    opened by denblackstache 0
  • Data providers / parameterized tests

    Data providers / parameterized tests

    I think the concept of data providers is something that lots of people will look for when switching from PHPUnit to Peridot. I think I brought it up with @brianium during our first get-together, and there's #183 and peridot-php/peridot-core#16 related to this issue already.

    I had a conversation with my friend @koden-km about how to migrate some PHPUnit tests that use Data Providers over to Peridot.

    We came up with a solution that utilized an associative array and a foreach loop. Here's some sample code:

    describe('should produce a double double when there are double digit statistics for', function () {
        /**
         * Builds data for every combination of stats that can be a double double.
         */
        $doubleDoubleDataProvider = function () {
            $statistics = ['points', 'rebounds_defensive', 'assists', 'steals', 'blocks'];
            $statisticsCount = count($statistics);
            $data = [];
    
            for ($i = 0; $i < $statisticsCount; ++$i) {
                for ($j = $i + 1; $j < $statisticsCount; ++$j) {
                    $subStatistics = [$statistics[$i], $statistics[$j]];
                    $data[implode(', ', $subStatistics)] = $subStatistics;
                }
            }
    
            return $data;
        };
    
        foreach ($doubleDoubleDataProvider() as $label => $statistics) {
            it($label, function () use ($statistics) {
                $statistics = array_fill_keys($statistics, 10);
                $statistics['games_played'] = 1;
                $actual = $this->subject->calculateStatistics($statistics);
    
                expect($actual->doubleDoubles)->to->equal(1);
                expect($actual->tripleDoubles)->to->equal(0);
            });
        }
    });
    

    This produces output like this:

    should produce a double double when there are double digit statistics for
      ✓ points, rebounds_defensive
      ✓ points, assists
      ✓ points, steals
      ✓ points, blocks
      ✓ rebounds_defensive, assists
      ✓ rebounds_defensive, steals
      ✓ rebounds_defensive, blocks
      ✓ assists, steals
      ✓ assists, blocks
      ✓ steals, blocks
    

    This approach isn't too bad, but I think that's partly because this scenario only has one input to each test ($statistics). If each test required multiple inputs, either the inner closure would have to expand its list of use'd variables, or it would have to accept all inputs in a single use'd array/object, then access each by index/property.

    I looked for some prior art to guide us in implementing this feature. It seems as though the typical JavaScript approach is similar to this:

    [1,2,3].forEach(function (itemNumber) {
        describe("Test # " + itemNumber, function () {
            it("should be a number", function (done) {
                expect(itemNumber).to.be.a('number')
                expect(itemNumber).to.be(itemNumber) 
            });
        });
    });
    

    Unfortunately, this doesn't map to PHP very well because of the fact that PHP's closures don't automatically inherit variables from the parent scope. That is, at least until the Arrow Functions RFC is implemented.

    Maybe this calls for a new Peridot feature that could somehow exploit setDefinitionArguments(). There's a definite challenge though, as plugins like peridot-phony also use this method for injecting other arguments.

    Even if we make no code changes, I think having some solid documentation for how to achieve this kind of setup would help newcomers to justify the switch to Peridot.

    question 
    opened by ezzatron 4
  • ReporterInterface does not include the methods added by HasEventEmitterTrait

    ReporterInterface does not include the methods added by HasEventEmitterTrait

    Currently in some parts of the code, such as the new composite reporter, we assume that a reporter will have the setEventEmitter() method, which comes from HasEventEmitterTrait that we add in AbstractTest:

            array_map(function (ReporterInterface $reporter) use ($eventEmitter) {
                $reporter->setEventEmitter($eventEmitter);
            }, $this->reporters);
    

    There's been some other related stuff where we typehint on abstract classes when we should really use an interface instead. I think we need to audit our typehints, and interfaces.

    bug 
    opened by ezzatron 0
Releases(1.19.0)
Essence is a very flexible BDD style assertion framework for PHP that fits into existing PHPUnit projects nicely

Essence 1.5.1 Essence is a very flexible BDD style assertion framework for PHP that fits into existing PHPUnit projects nicely. Installation composer

bound1ess 2 Apr 7, 2015
:heavy_check_mark: PHP Test Framework for Freedom, Truth, and Justice

Kahlan is a full-featured Unit & BDD test framework a la RSpec/JSpec which uses a describe-it syntax and moves testing in PHP one step forward. Kahlan

Kahlan 1.1k Jan 2, 2023
vfsStream is a stream wrapper for a virtual file system that may be helpful in unit tests to mock the real file system. It can be used with any unit test framework, like PHPUnit or SimpleTest.

vfsStream vfsStream is a stream wrapper for a virtual file system that may be helpful in unit tests to mock the real file system. It can be used with

null 1.4k Dec 23, 2022
A Composer script to run a 'test' or 'spec' Composer script against multiple PHP versions.

composer-multitest composer-multitest is a Composer script that runs a test or spec Composer script against multiple PHP versions managed by PHPBrew o

Raphael Stolt 5 Aug 27, 2019
PHP Test Generator - A CLI tool which generates unit tests

This project make usages of PHPStan and PHPParser to generate test cases for a given PHP File.

Alexander Schranz 7 Dec 3, 2022
Infrastructure and testing helpers for creating CQRS and event sourced applications.

Broadway is a project providing infrastructure and testing helpers for creating CQRS and event sourced applications. Broadway tries hard to not get in your way.

null 1.5k Dec 30, 2022
⛽ Pest plugin to test Laravel applications powered by Octane.

⛽ Laravel Octane (Pest Plugin) Pest plugin to test Laravel applications powered by Octane. Install Via Composer composer require --dev cerbero/pest-pl

Andrea Marco Sartori 21 Apr 5, 2022
Magic Test allows you to write browser tests by simply clicking around on the application being tested, all without the slowness of constantly restarting the testing environment.

Magic Test for Laravel Magic Test allows you to write browser tests by simply clicking around on the application being tested, all without the slownes

null 400 Jan 5, 2023
A sample RESTful API in Laravel with PHPunit test.

Laravel PHP Framework URL | URI | Action |

Fasil 9 Jul 11, 2020
PHPUnit Application Architecture Test

PHPUnit Application Architecture Test Idea: write architecture tests as well as feature and unit tests Installation Install via composer composer requ

null 19 Dec 11, 2022
Patchstack Test task Laravel&Vue CRUD

Patchstack Test Task - Laravel & Vue CRUD SPA Written with Laravel and Vue2 using mix. Installation Clone this repository Run "composer update" comman

Crypto Rookie 3 Aug 25, 2022
The Phoronix Test Suite is the most comprehensive testing and benchmarking platform

The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available for Linux, Solaris, macOS, Windows, and BSD operating systems.

Phoronix Test Suite 1.9k Jan 7, 2023
Prevent none-test output in your Pest tests.

Pest Plugin Silence Often, when writing tests, we echo and dump test code to debug and check everything is working correctly. It can be easy to forget

Worksome 5 Feb 23, 2022
Test requests in Laravel without all the boilerplate.

Request Factories Test requests in Laravel without all the boilerplate. ?? Psst. Although our examples use Pest PHP, this works just as well in PHPUni

Worksome 391 Jan 1, 2023
The modern, simple and intuitive PHP unit testing framework.

atoum PHP version atoum version 5.3 -> 5.6 1.x -> 3.x 7.2 -> 8.x 4.x (current) A simple, modern and intuitive unit testing framework for PHP! Just lik

atoum 1.4k Nov 29, 2022
Full-stack testing PHP framework

Codeception Modern PHP Testing for everyone Codeception is a modern full-stack testing framework for PHP. Inspired by BDD, it provides an absolutely n

Codeception Testing Framework 4.6k Jan 7, 2023
AST based PHP Mutation Testing Framework

Infection - Mutation Testing framework Please read documentation here: infection.github.io Twitter: @infection_php Discord: https://discord.gg/ZUmyHTJ

Infection - Mutation Testing Framework for PHP 1.8k Jan 2, 2023
PHP Mocking Framework

Phake Phake is a framework for PHP that aims to provide mock objects, test doubles and method stubs. Phake was inspired by a lack of flexibility and e

Phake 469 Dec 2, 2022