PHP Benchmarking framework

Last update: May 22, 2022

CI Documentation Latest Stable Version Total Downloads License

PHPBench is a benchmark runner for PHP analogous to PHPUnit but for performance rather than correctness.

Features include:

  • Revolutions: Repeat your code many times to determine average execution time.
  • Iterations: Sample your revolutions many times and review aggregated statistical data.
  • Process Isolation: Each iteration is executed in a separate process.
  • Reporting: Customizable reports and various output formats (e.g. console, CSV, Markdown, HTML).
  • Report storage and comparison: Store benchmarks locally to be used as a baseline reference, or to reference them later.
  • Memory Usage: Keep an eye on the amount of memory used by benchmarking subjects.
  • Assertions: Assert that code is performing within acceptable limits, or that it has not regressed from a previously recorded baseline.

See the documentation to find out more.

Installation

$ composer require phpbench/phpbench --dev

See the installation instructions for more options.

Documentation

Documentation is hosted on readthedocs.

Community

Screenshots

Default output:

phpbench-standard

Blinken logger:

phpbench-blinken

HTML report:

phpbench2

Contributing

PHPBench is an open source project. If you find a problem or want to discuss new features or improvements please create an issue, and/or if possible create a pull request.

GitHub

https://github.com/phpbench/phpbench
Comments
  • 1. General discussion for usability

    I'd like to suggest a couple of usability issues that came up using PHPBench for the first time:

    1. There should be a setUp() and tearDown() etc. by default - I don't like to add @BeforeMethods({"setUp"}) to every docblock
    2. The syntax of the data-provider was confusing to me - you should stick to PHPUnit and provide $param1, $param2 and $param3 instead of the single $params array
    3. Is there a way to use PHPBench with a continuous integration service like Travis CI? If not, I suggest you add something like a threshold that returns an error code if the benchmarks failed
    4. Tabular is great but you should truncate values that are longer than 20 characters: http://i.snag.gy/UNsRd.jpg
    5. Please use semantic versioning... 0.6.0 instead of 0.6

    Keep in mind, this is just a discussion - please create new issues for each one of the issues

    Thank you

    Reviewed by henryruhs at 2015-11-01 11:42
  • 2. Resources are not available with @BeforeMethods

    I am trying to initialize some hash contexts (resources) as a #BeforeMethods, but it won't let me use it, because something is changes them.

    I attached a file and both bench functions should actually work, because they are using exactly the same code, except that one of them is using @BeforeMethods to initialize the hash, the other one is not.

    PhpHashBench.php.txt

    Reviewed by ogmueller at 2016-08-31 14:59
  • 3. composer deps for php8 ( php > 7.2)

    Hey. Die festen bindungen an eine php version neven total weil 0.16 grundsätzlich auch unter php8 laufen kann. "php": "^7.2", (composer.json) ist in diversen tags hinterlegt und composer will nur mit gewallt installieren :-(

    also: php >= Version mit der es laufen muss oder "php": "^7.2 || ^7.3 || ^8.0" funktioniert wunderbar wenn man composer mit phpbench/phpbench: "*" configuriert (auf user seite) und composer --prefer-source nutzt um dessen unterstützung der dependeny checks erfolgreich zu erhalten. ich brauch eine version die laufen kann, nicht die stricte reduktion nichts mehr nutzen zu können :-) LG aus HH

    Reviewed by flobee at 2021-03-10 21:44
  • 4. Sanitize/calibrate functionality

    I try to figure out if I can replace my own benchmark micro-framework with phpbench. I did a cursory search on documentation and the benchmarks-examples repository but failed to find what would be the best way to reimplement sanitize/calibrate functionality within phpbench (for example, check this class to see what I mean). Any suggestions?

    Reviewed by rybakit at 2018-05-25 09:53
  • 5. Comparing two dump-files does not work

    To reproduce:

    phpbench --report=aggregate --dump-file=fileA.xml --context="A"
    phpbench --report=aggregate --dump-file=fileB.xml --context="B"
    
    phpbench report --file=fileA.xml --file=fileB.xml --report=compare
    

    right from the docs.

    [InvalidArgumentException]
      Column "vcs_branch" does not exist, valid columns: "suite", "date", "stime", "benchmark_full", "groups", "params", "revs", "its", "mem_real", "mem_final", "mem_peak", "min", "max", "sum", "stdev", "mean", "mode", "variance", "rstdev", "best", "worst", "uname_os", "uname_host", "uname_release", "uname_version", "un
      ame_machine", "php_version", "php_xdebug", "baseline_nothing", "baseline_md5", "baseline_file_rw"  	
    Exception trace:
     PhpBench\Report\Generator\Table\Row->offsetGet() at C:\project\vendor\phpbench\phpbench\lib\Report\Generator\TableGenerator.php:261
     PhpBench\Report\Generator\TableGenerator->PhpBench\Report\Generator\{closure}() at C:\project\vendor\lstrojny\functional-php\src\Functional\Map.php:43
     Functional\map() at C:\project\vendor\phpbench\phpbench\lib\Report\Generator\TableGenerator.php:265
     PhpBench\Report\Generator\TableGenerator->PhpBench\Report\Generator\{closure}() at C:\project\vendor\lstrojny\functional-php\src\Functional\Map.php:43
     Functional\map() at C:\project\vendor\phpbench\phpbench\lib\Report\Generator\TableGenerator.php:266
     PhpBench\Report\Generator\TableGenerator->processCols() at C:\project\vendor\phpbench\phpbench\lib\Report\Generator\TableGenerator.php:106
     PhpBench\Report\Generator\TableGenerator->generate() at C:\project\vendor\phpbench\phpbench\lib\Report\ReportManager.php:56
     PhpBench\Report\ReportManager->generateReports() at C:\project\vendor\phpbench\phpbench\lib\Report\ReportManager.php:84
     PhpBench\Report\ReportManager->renderReports() at C:\project\vendor\phpbench\phpbench\lib\Console\Command\Handler\ReportHandler.php:41
     PhpBench\Console\Command\Handler\ReportHandler->reportsFromInput() at C:\project\vendor\phpbench\phpbench\lib\Console\Command\ReportCommand.php:87
     PhpBench\Console\Command\ReportCommand->execute() at C:\project\vendor\symfony\console\Command\Command.php:252
     Symfony\Component\Console\Command\Command->run() at C:\project\vendor\symfony\console\Application.php:936
     Symfony\Component\Console\Application->doRunCommand() at C:\project\vendor\symfony\console\Application.php:240
     Symfony\Component\Console\Application->doRun() at C:\project\vendor\symfony\console\Application.php:148
     Symfony\Component\Console\Application->run() at C:\project\vendor\phpbench\phpbench\lib\PhpBench.php:58
     PhpBench\PhpBench::run() at C:\project\vendor\phpbench\phpbench\bin\phpbench.php:24
     require() at C:\project\vendor\phpbench\phpbench\bin\phpbench:4
    
    Reviewed by fhein at 2018-01-05 00:10
  • 6. [RuntimeException] Could not launch script:

    I get this Exception then i try to run phpbench on windows.

    [RuntimeException]
      Could not launch script:
    

    the command I am executing

    vendor\bin\phpbench.bat run benchmarks/StringifyBench.php
    

    Windows 10 1511 php 5.5.8

    please help me, I would love to use phpbench :)

    Reviewed by Kanti at 2015-11-19 14:02
  • 7. Parameter sets broken in master?

    When using c35e70b3a6ee7cdaf1a7048809b8e3dcb9f0f10f I'm seeing the following error:

    \Relay\Benchmarks\MgetBench::benchMgetUsingPredis
    
        PhpBench\Executor\Exception\ExecutionError
        Fatal error: Uncaught TypeError: {closure}(): Argument #1 ($value) must be of type string, array given in /private/var/folders/cw/70xgjbbx7ps2mlhyqx24snbc0000gn/T/PhpBenchPz3t08:27
        Stack trace:
        #0 [internal function]: {closure}(Array)
        #1 /private/var/folders/cw/70xgjbbx7ps2mlhyqx24snbc0000gn/T/PhpBenchPz3t08(34): array_map(Object(Closure), Array)
        #2 {main}
          thrown in /private/var/folders/cw/70xgjbbx7ps2mlhyqx24snbc0000gn/T/PhpBenchPz3t08 on line 27
    
    
        #0 /Users/Till/Development/Repositories/relay/src/PhpBench/RedisExecutor.php(50): PhpBench\Executor\Benchmark\TemplateExecutor->execute(Object(PhpBench\Executor\ExecutionContext), Object(PhpBench\Registry\Config))
        #1 /Users/Till/Development/Repositories/relay/vendor/phpbench/phpbench/lib/Benchmark/Runner.php(366): Relay\PhpBench\RedisExecutor->execute(Object(PhpBench\Executor\ExecutionContext), Object(PhpBench\Registry\Config))
        #2 /Users/Till/Development/Repositories/relay/vendor/phpbench/phpbench/lib/Benchmark/Runner.php(308): PhpBench\Benchmark\Runner->runIteration(Object(Relay\PhpBench\RedisExecutor), Object(PhpBench\Registry\Config), Object(PhpBench\Model\Iteration), Object(PhpBench\Benchmark\Metadata\SubjectMetadata))
        #3 /Users/Till/Development/Repositories/relay/vendor/phpbench/phpbench/lib/Benchmark/Runner.php(279): PhpBench\Benchmark\Runner->runVariant(Object(Relay\PhpBench\RedisExecutor), Object(PhpBench\Registry\Config), Object(PhpBench\Benchmark\RunnerConfig), Object(PhpBench\Benchmark\Metadata\SubjectMetadata), Object(PhpBench\Model\Variant))
        #4 /Users/Till/Development/Repositories/relay/vendor/phpbench/phpbench/lib/Benchmark/Runner.php(197): PhpBench\Benchmark\Runner->runSubject(Object(Relay\PhpBench\RedisExecutor), Object(PhpBench\Benchmark\RunnerConfig), Object(PhpBench\Model\Subject), Object(PhpBench\Benchmark\Metadata\SubjectMetadata))
        #5 /Users/Till/Development/Repositories/relay/vendor/phpbench/phpbench/lib/Benchmark/Runner.php(123): PhpBench\Benchmark\Runner->runBenchmark(Object(PhpBench\Benchmark\RunnerConfig), Object(PhpBench\Model\Benchmark), Object(PhpBench\Benchmark\Metadata\BenchmarkMetadata))
        #6 /Users/Till/Development/Repositories/relay/vendor/phpbench/phpbench/lib/Console/Command/Handler/RunnerHandler.php(141): PhpBench\Benchmark\Runner->run(Object(Generator), Object(PhpBench\Benchmark\RunnerConfig))
        #7 /Users/Till/Development/Repositories/relay/vendor/phpbench/phpbench/lib/Console/Command/RunCommand.php(134): PhpBench\Console\Command\Handler\RunnerHandler->runFromInput(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\StreamOutput), Object(PhpBench\Benchmark\RunnerConfig))
        #8 /Users/Till/Development/Repositories/relay/vendor/symfony/console/Command/Command.php(299): PhpBench\Console\Command\RunCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\StreamOutput))
        #9 /Users/Till/Development/Repositories/relay/vendor/symfony/console/Application.php(978): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\StreamOutput))
        #10 /Users/Till/Development/Repositories/relay/vendor/symfony/console/Application.php(295): Symfony\Component\Console\Application->doRunCommand(Object(PhpBench\Console\Command\RunCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\StreamOutput))
        #11 /Users/Till/Development/Repositories/relay/vendor/symfony/console/Application.php(167): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\StreamOutput))
        #12 /Users/Till/Development/Repositories/relay/vendor/phpbench/phpbench/lib/PhpBench.php(51): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\StreamOutput))
        #13 /Users/Till/Development/Repositories/relay/vendor/phpbench/phpbench/bin/phpbench.php(25): PhpBench\PhpBench::run()
        #14 /Users/Till/Development/Repositories/relay/vendor/phpbench/phpbench/bin/phpbench(4): require('/Users/Till/Dev...')
        #15 {main}
    

    This is how the @ParamProviders used to work:

    class GetThroughputBench extends BenchCase
    {
        /**
         * @ParamProviders("provideKeys")
         */
        public function benchGetThroughputOfRelayWarmed($params)
        {
            foreach ($params['keys'] as $key) {
                $this->relay->get($key);
            }
        }
    
        /**
         * Provides the keys for each benchmark.
         *
         * @return Generator
         */
        public function provideKeys()
        {
            $keys = [];
    
            foreach (static::loadJson('dataset-meteorites.json') as $landing) {
                $keys[] = "meteorite:{$landing->id}";
            }
    
            yield 'dataset' => [
                'keys' => $keys,
            ];
        }
    }
    
    Reviewed by tillkruss at 2021-06-29 23:13
  • 8. `Could not find parselet for "none" token` error

    I have put a simple example benchark into benchmarks/CustomerSoapBench.php:

    <?php
    
    namespace POM\benchmarks;
    
    use WebConnector\Customer\CustomerClientFactory;
    use WebConnector\Customer\Type\Read;
    
    class CustomerSoapBench {
        /**
         * @return void
         */
        public function benchRead() {
            $client = CustomerClientFactory::factory('https://'. WEBCONNECTOR_HOST .'/test_1094/CustomerIISWebService.svc?wsdl');
            $read = new Read('D83756');
            $client->read($read);
        }
    }
    

    placed my phpbench.json config beside it in benchmarks/ and invoked it from the project root like

    $ php vendor/bin/phpbench run benchmarks/CustomerSoapBench.php --report=default --config=benchmarks/phpbench.json  -vvv
    [DEBUG] Spawning process: C:\tools\php74\php.exe  "C:\Users\mstaab\AppData\Local\Temp\Php5FFA.tmp"
    
    [DEBUG] Spawning process: C:\tools\php74\php.exe  "C:\Users\mstaab\AppData\Local\Temp\Php60D6.tmp"
    
    PHPBench (dev-master) running benchmarks...
    with configuration file: C:/dvl/GitHub/philipp/benchmarks/phpbench.json
    with PHP version 7.4.13, xdebug ❌, opcache ❌
    
    [DEBUG] Spawning process: C:\tools\php74\php.exe  "C:\Users\mstaab\AppData\Local\Temp\Php629D.tmp"
    
    \POM\benchmarks\CustomerSoapBench
    
        benchRead...............................I0 [DEBUG] Spawning process: C:\tools\php74\php.exe -dmax_execution_time=0 "C:\Users\mstaab\AppData\Local\Temp\Php
    6389.tmp"
    
    
    In SyntaxError.php line 30:
    
      [PhpBench\Expression\Exception\SyntaxError]
      Could not find parselet for "none" token:
    
     " (" ~ rstdev(variant.time.avg) ~ ")"  mode) ~
            ----------------------------------------^
    
    
    Exception trace:
      at C:\dvl\GitHub\philipp\vendor\phpbench\phpbench\lib\Expression\Exception\SyntaxError.php:30
     PhpBench\Expression\Exception\SyntaxError::forToken() at C:\dvl\GitHub\philipp\vendor\phpbench\phpbench\lib\Expression\Parser.php:89
     PhpBench\Expression\Parser->parseExpression() at C:\dvl\GitHub\philipp\vendor\phpbench\phpbench\lib\Expression\Parselet\ConcatParselet.php:23
     PhpBench\Expression\Parselet\ConcatParselet->parse() at C:\dvl\GitHub\philipp\vendor\phpbench\phpbench\lib\Expression\Parser.php:105
     PhpBench\Expression\Parser->parseExpression() at C:\dvl\GitHub\philipp\vendor\phpbench\phpbench\lib\Expression\Parselet\ConcatParselet.php:23
     PhpBench\Expression\Parselet\ConcatParselet->parse() at C:\dvl\GitHub\philipp\vendor\phpbench\phpbench\lib\Expression\Parser.php:105
     PhpBench\Expression\Parser->parseExpression() at C:\dvl\GitHub\philipp\vendor\phpbench\phpbench\lib\Expression\Parser.php:64
     PhpBench\Expression\Parser->parseList() at C:\dvl\GitHub\philipp\vendor\phpbench\phpbench\lib\Expression\Parser.php:49
     PhpBench\Expression\Parser->parse() at C:\dvl\GitHub\philipp\vendor\phpbench\phpbench\lib\Expression\ExpressionLanguage\RealExpressionLanguage.php:29
     PhpBench\Expression\ExpressionLanguage\RealExpressionLanguage->parse() at C:\dvl\GitHub\philipp\vendor\phpbench\phpbench\lib\Expression\ExpressionLanguage\Me
    moisedExpressionLanguage.php:31
     PhpBench\Expression\ExpressionLanguage\MemoisedExpressionLanguage->parse() at C:\dvl\GitHub\philipp\vendor\phpbench\phpbench\lib\Progress\VariantSummaryForma
    tter.php:73
     PhpBench\Progress\VariantSummaryFormatter->formatVariant() at C:\dvl\GitHub\philipp\vendor\phpbench\phpbench\lib\Progress\Logger\PhpBenchLogger.php:157
     PhpBench\Progress\Logger\PhpBenchLogger->formatIterationsFullSummary() at C:\dvl\GitHub\philipp\vendor\phpbench\phpbench\lib\Progress\Logger\VerboseLogger.ph
    p:87
     PhpBench\Progress\Logger\VerboseLogger->variantEnd() at C:\dvl\GitHub\philipp\vendor\phpbench\phpbench\lib\Benchmark\Runner.php:348
     PhpBench\Benchmark\Runner->endVariant() at C:\dvl\GitHub\philipp\vendor\phpbench\phpbench\lib\Benchmark\Runner.php:310
     PhpBench\Benchmark\Runner->runVariant() at C:\dvl\GitHub\philipp\vendor\phpbench\phpbench\lib\Benchmark\Runner.php:268
     PhpBench\Benchmark\Runner->runSubject() at C:\dvl\GitHub\philipp\vendor\phpbench\phpbench\lib\Benchmark\Runner.php:196
     PhpBench\Benchmark\Runner->runBenchmark() at C:\dvl\GitHub\philipp\vendor\phpbench\phpbench\lib\Benchmark\Runner.php:122
     PhpBench\Benchmark\Runner->run() at C:\dvl\GitHub\philipp\vendor\phpbench\phpbench\lib\Console\Command\Handler\RunnerHandler.php:136
     PhpBench\Console\Command\Handler\RunnerHandler->runFromInput() at C:\dvl\GitHub\philipp\vendor\phpbench\phpbench\lib\Console\Command\RunCommand.php:134
     PhpBench\Console\Command\RunCommand->execute() at C:\dvl\GitHub\philipp\vendor\symfony\console\Command\Command.php:256
     Symfony\Component\Console\Command\Command->run() at C:\dvl\GitHub\philipp\vendor\symfony\console\Application.php:971
     Symfony\Component\Console\Application->doRunCommand() at C:\dvl\GitHub\philipp\vendor\symfony\console\Application.php:290
     Symfony\Component\Console\Application->doRun() at C:\dvl\GitHub\philipp\vendor\symfony\console\Application.php:166
     Symfony\Component\Console\Application->run() at C:\dvl\GitHub\philipp\vendor\phpbench\phpbench\lib\PhpBench.php:51
     PhpBench\PhpBench::run() at C:\dvl\GitHub\philipp\vendor\phpbench\phpbench\bin\phpbench.php:25
     require() at C:\dvl\GitHub\philipp\vendor\bin\phpbench(21) : eval()'d code:3
     eval() at C:\dvl\GitHub\philipp\vendor\bin\phpbench:21
    
    run [--filter FILTER] [--group GROUP] [--parameters PARAMETERS] [--assert ASSERT] [--format FORMAT] [--revs REVS] [-l|--progress PROGRESS] [-b|--bootstrap BOO
    TSTRAP] [--executor EXECUTOR] [--stop-on-error] [--php-binary PHP-BINARY] [--php-config PHP-CONFIG] [--php-wrapper PHP-WRAPPER] [--php-disable-ini] [--report
    REPORT] [-o|--output OUTPUT] [--ref REF] [--file FILE] [--time-unit TIME-UNIT] [--precision PRECISION] [--mode MODE] [-d|--dump-file [DUMP-FILE]] [--dump] [--
    iterations ITERATIONS] [--warmup WARMUP] [-r|--retry-threshold RETRY-THRESHOLD] [--sleep SLEEP] [--tag TAG] [--store] [--tolerate-failure] [--] [<path>]
    
    

    my benchmarks/phpbench.json file looks like

    {
      "runner.bootstrap": "../app/portal/phpstan-bootstrap.php"
    }
    

    the phpstan-bootstrap.php is the same one I use successfully for phpstan and similar tools:

    <?php
    
    define('APP_ROOT', dirname(__DIR__, 2));
    define('APP_NAME', basename(__DIR__));
    define('APP_PATH', APP_ROOT.'/app/'.APP_NAME);
    define('PUBLIC_PATH', APP_ROOT.'/public/'.APP_NAME);
    
    $_SERVER['APP_MODE'] = 'development';
    define('APP_MODE', $_SERVER['APP_MODE']);
    
    // \cms\ViewTypeProvider requires a rocket path via vhost
    $_SERVER['ROCKET_PATH'] = APP_ROOT.'/vendor/plugins/rocket/';
    
    require_once APP_ROOT.'/vendor/autoload.php';
    include_once APP_ROOT.'/config/app.php';
    
    

    running on [email protected] from today at 8a5da09d52f44d223b467f98974b0bc5113f4c46

    $ php -v
    PHP 7.4.13 (cli) (built: Nov 24 2020 12:43:30) ( NTS Visual C++ 2017 x64 )
    Copyright (c) The PHP Group
    Zend Engine v3.4.0, Copyright (c) Zend Technologies
    

    on windows bash

    Reviewed by staabm at 2021-04-21 13:33
  • 9. Ensure opcache optimizations are enabled when running benchmarks?

    Related: https://wiki.php.net/rfc/opcache.no_cache

    In practice, most benchmarks run with the CLI SAPI, which has no opcache by default.

    I was wondering if phpbench should (at all) enforce opcache to be enabled, or warn when it isn't.

    Raised from https://github.com/lyrixx/php-bench/pull/5#issuecomment-637418989

    Reviewed by Ocramius at 2020-06-02 09:35
  • 10. Environment variables through phpbench.json

    {
        "bootstrap": "vendor/autoload.php",
        "path": "src/Benchmark",
        "env": [
            {
                "name": "APP_ENV",
                "value": "test",
                "force": true
            },
            {
                "name": "SOME_VAR",
                "value": "foo bar"
            }
        ]
    }
    

    Are there any plans to implement something like this? This would help a lot when writing benchmarks when working with a database in a test environment...

    What do you think about this?

    Reviewed by fenric at 2020-02-11 00:09
  • 11. Configure @dependabot for automatic dependency upgrades

    Today I came across the usual "which dependency is holding my upgrades back?" scenario :D

    In practice, I would suggest:

    • adding @dependabot core to this repository, to automate upgrades (perhaps weekly upgrades?)
    • adding auto-merge for dependabot upgrades
    • adding release automation, to allow for quick releases of improvements (I know you have your own tooling, so unsure if this is too much scope creep)
    Reviewed by Ocramius at 2021-12-21 16:25
  • 12. Feature request: partitioning (break) by groups

    Request:

    Please, create partitioning (break) by class annotation groups.

    Description:

    Every benchmark class can have annotation with the list of groups. e.g.

    /**
     * @Groups({"group1", "group2"})
     */
    class myNewBenchmark
    {
    ..
    }
    

    Several classes can have the same group. Lets say we have 2 classes: myNewBenchmark and myOldBenchmark with the same group group1. Both classes are in the same directory all_benchmarks So we can execute the cli: $ phpbench run --group=group1 ./all_benchmarks

    An we get similar report results:

    group1
    +----------------+----------------+-------+
    | benchmark      | subject        | mode  |
    +----------------+----------------+-------+
    | myNewBenchmark | benchEvaluate1 | 457μs |
    | myNewBenchmark | benchEvaluate2 | 337μs |
    | myOldBenchmark | benchEvaluate1 | 349μs |
    | myOldBenchmark | benchEvaluate2 | 341μs |
    +----------------+----------------+-------+
    

    At the same time there is a possibility to create reporting with partitioning by benchmarks. e.g.

    "report.generators": {
          "custom": {
              "extends": "expression",
              "break": ["benchmark"],
              "cols": {
                  "benchmark": null,
                  "subject": null,
                  "mode": null,
              }
          },
    }
    

    So we can execute cli: $ phpbench run --report=custom ./all_benchmarks

    And reporting results would give the results divided/partitioned by benchmarks:

    myNewBenchmark
    +----------------+----------------+-------+
    | benchmark      | subject        | mode  |
    +----------------+----------------+-------+
    | myNewBenchmark | benchEvaluate1 | 457μs |
    | myNewBenchmark | benchEvaluate2 | 337μs |
    +----------------+----------------+-------+
    
    myOldBenchmark
    +----------------+----------------+-------+
    | benchmark      | subject        | mode  |
    +----------------+----------------+-------+
    | myOldBenchmark | benchEvaluate1 | 457μs |
    | myOldBenchmark | benchEvaluate2 | 337μs |
    +----------------+----------------+-------+
    

    Feature:

    It would be nice to have partitioning/breaking by class annotation groups, similar to benchmarks breaking.

    e.g. Lets say there are several classes in all_benchmarks directory : 3 of them have different groups and 1 of them has no group.

    /**
     * @Groups({"group1"})
     */
    class myNewBenchmark
    {
    ..
    }
    
    /**
     * @Groups({"group1", "group-standard"})
     */
    class myOldBenchmark
    {
    ..
    }
    
    /**
     * @Groups({"group-standard"})
     */
    class myStandardBenchmark
    {
    ..
    }
    
    /**
     * there is no group here
     */
    class myGeneralBenchmark
    {
    ..
    }
    

    Then we create the following reporting config, where groups contains the list of all existing groups (here: ["group1", "group-standard"]) and iterates through them.

    "report.generators": {
          "custom_groups": {
              "extends": "expression",
              "break": ["groups"],
              "cols": {
                  "benchmark": null,
                  "subject": null,
                  "mode": null,
              }
          },
    }
    

    So if we execute cli: $ phpbench run --report=custom_groups ./all_benchmarks

    we get the reporting result like:

    group1
    +----------------+----------------+-------+
    | benchmark      | subject        | mode  |
    +----------------+----------------+-------+
    | myNewBenchmark | benchEvaluate1 | 457μs |
    | myNewBenchmark | benchEvaluate2 | 337μs |
    | myOldBenchmark | benchEvaluate1 | 349μs |
    | myOldBenchmark | benchEvaluate2 | 341μs |
    +----------------+----------------+-------+
    
    group-standard
    +---------------------+----------------+-------+
    | benchmark           | subject        | mode  |
    +---------------------+----------------+-------+
    | myOldBenchmark      | benchEvaluate1 | 349μs |
    | myOldBenchmark      | benchEvaluate2 | 341μs |
    | myStandardBenchmark | benchEvaluate1 | 361μs |
    | myStandardBenchmark | benchEvaluate2 | 350μs |
    +---------------------+----------------+-------+
    
    (no group)
    +--------------------+----------------+----------+
    | benchmark          | subject        | mode     |
    +--------------------+----------------+----------+
    | myGeneralBenchmark | benchAssert    | 1.204ms  |
    +--------------------+----------------+----------+
    

    Thank you in advance!

    Reviewed by aGiqs at 2022-04-07 10:48
  • 13. Testing all benchmark groups at once?

    Hi,

    Thank you a lot for benchmark tool. I find it very useful :-)!

    I created all my benchmark tests with group annotations and used the following reporting settings in benchmark config:

    "reports": {
            "custom": {
                "extends": "aggregate",
                "sort": {"benchmark": "asc", "subject": "asc"}
            },
            "unittests": {
                "extends": "aggregate",
                "sort": {"benchmark": "asc", "subject": "asc"}
            }
        },
    

    So if I want to generate a becnhmark report for one group, i run:

    $ phpbench run --group=my-group-name ./my-bechmark-tests

    Benchmark tool runs perfectly and I get my results.

    Let's say now I want to create reports for all my benchmark tests but per group.

    Is it possible to configure benchmark reporting so, that it detects all existing groups and tests one group after another?

    Thank you in advance for all kind of tips!

    Reviewed by aGiqs at 2022-02-25 10:49
  • 14. Remove webmozart/path-util

    Currently, composer complains when installing phpbench, with the following message :

    Package webmozart/path-util is abandoned, you should avoid using it. Use symfony/filesystem instead.
    

    This problem was brought up in issue #978

    Indeed, the symfony/filesystem package provides the same functionalities with the class Symfony\Component\Filesystem\Path

    As a caveat, it is only available since v5.4 so it could be considered a breaking change for some end users.

    Reviewed by Arzaroth at 2022-02-17 13:39
  • 15. Feature request: ability to import baseline/tagged benchmarks for runs

    Hi!

    I want to be able to export a specific tag to store it in git so I can compare future revisions of the code against it.

    One possibility I considered was storing the storage.xml_storage_path in git, but then my local runs would also be stored there and I would need to check if I accidentally added one to git every time. I find this too tedious and think there is a better way.

    Another example of what I have in mind is the way psalm handles baselines: you can provide a baseline file via command line or config and all errors it finds will be checked against it. I can also store the baseline file in git and it will be used by the CI to check if new errors where added by the changed code.

    Any idea if this is possible and desirable for enough people?

    Reviewed by ricardoboss at 2022-02-08 12:32
  • 16. Avoid using webmozart/path-util

    Apparently, webmozart/path-util is no longer maintained and phpbench has it as a dependency.

    You should consider changing it to a maintained package or the proposed one: symfony/filesystem.

    Reviewed by ricardoboss at 2022-01-07 00:07
Slim Framework Tracy Debugger Bar
 Slim Framework Tracy Debugger Bar

Slim Framework Tracy Debugger Bar configure it by mouse now in package: Panel Description Slim Framework - Slim Environment RAW data Slim Container RA

Feb 26, 2022
🔴 First class integration of Sentry to Nette Framework (@nette)
🔴 First class integration of Sentry to Nette Framework (@nette)

Website ?? contributte.org | Contact ????‍?? f3l1x.io | Twitter ?? @contributte Usage To install latest version of contributte/sentry use Composer. co

Jan 27, 2022
Handle PHP errors, dump variables, execute PHP code remotely in Google Chrome

PHP Console server library PHP Console allows you to handle PHP errors & exceptions, dump variables, execute PHP code remotely and many other things u

May 19, 2022
PHP APM (Alternative PHP Monitor)

APM (Alternative PHP Monitor) APM (Alternative PHP Monitor) is a monitoring extension enabling native Application Performance Management (APM) for PHP

May 19, 2022
Zipkin PHP is the official PHP Tracer implementation for Zipkin

Zipkin PHP is the official PHP Tracer implementation for Zipkin, supported by the OpenZipkin community. Installation composer require openz

May 14, 2022
Debug bar for PHP
Debug bar for PHP

PHP Debug Bar Displays a debug bar in the browser with information from php. No more var_dump() in your code! Features: Generic debug bar Easy to inte

May 11, 2022
Xdebug — Step Debugger and Debugging Aid for PHP

Xdebug Xdebug is a debugging tool for PHP. It provides step-debugging and a whole range of development aids, such as stack traces, a code profiler, fe

May 13, 2022
Kint - a powerful and modern PHP debugging tool.
Kint - a powerful and modern PHP debugging tool.

Kint - debugging helper for PHP developers What am I looking at? At first glance Kint is just a pretty replacement for var_dump(), print_r() and debug

May 14, 2022
😎 Tracy: the addictive tool to ease debugging PHP code for cool developers. Friendly design, logging, profiler, advanced features like debugging AJAX calls or CLI support. You will love it.
😎 Tracy: the addictive tool to ease debugging PHP code for cool developers. Friendly design, logging, profiler, advanced features like debugging AJAX calls or CLI support. You will love it.

Tracy - PHP debugger Introduction Tracy library is a useful helper for everyday PHP programmers. It helps you to: quickly detect and correct errors lo

May 20, 2022
The Interactive PHP Debugger

The interactive PHP debugger Implemented as a SAPI module, phpdbg can exert complete control over the environment without impacting the functionality

Apr 26, 2022
Dontbug is a reverse debugger for PHP
Dontbug is a reverse debugger for PHP

Dontbug Debugger Dontbug is a reverse debugger (aka time travel debugger) for PHP. It allows you to record the execution of PHP scripts (in command li

May 8, 2022
PHP Debug Console
PHP Debug Console

PHP Console A web console to try your PHP code into Creating a test file or using php's interactive mode can be a bit cumbersome to try random php sni

May 14, 2022
Php Debugger to run in terminal to debug your code easily.
Php Debugger to run in terminal to debug your code easily.

What is Dephpugger? Dephpugger (read depugger) is an open source lib to make a debug in php direct in terminal, without necessary configure an IDE. Th

Jan 12, 2022
PCOV - CodeCoverage compatible driver for PHP

PCOV A self contained CodeCoverage compatible driver for PHP Requirements and Installation See INSTALL.md API /** * Shall start recording coverage in

May 6, 2022
Low-overhead sampling profiler for PHP 7+
Low-overhead sampling profiler for PHP 7+

phpspy phpspy is a low-overhead sampling profiler for PHP. For now, it works with Linux 3.2+ x86_64 non-ZTS PHP 7.0+ with CLI, Apache, and FPM SAPIs.

May 14, 2022
The VarDumper component provides mechanisms for walking through any arbitrary PHP variable. It provides a better dump() function that you can use instead of var_dump().

VarDumper Component The VarDumper component provides mechanisms for walking through any arbitrary PHP variable. It provides a better dump() function t

May 18, 2022
PHP errors for cool kids
PHP errors for cool kids

whoops PHP errors for cool kids whoops is an error handler framework for PHP. Out-of-the-box, it provides a pretty error interface that helps you debu

May 18, 2022
Clockwork - php dev tools in your browser - server-side component
Clockwork - php dev tools in your browser - server-side component

Clockwork is a development tool for PHP available right in your browser. Clockwork gives you an insight into your application runtime - including requ

May 18, 2022
Laravel Debugbar (Integrates PHP Debug Bar)
Laravel Debugbar (Integrates PHP Debug Bar)

Laravel Debugbar This is a package to integrate PHP Debug Bar with Laravel. It includes a ServiceProvider to register the debugbar and attach it to th

May 24, 2022