Hi!

It would be nice to have an ability to install phpunit the way composer is installed.

So running tests like php phpunit.phar ./ and downloading phpunit like curl -s http://phpunit.de/installation | php or just wget http://phpunit.de/pnpunit.phar is very easy and does not require to have pear installed

There already is an attempt to create a phar from PHPUnit https://github.com/igorw/phpunit-phar

Also having composer.json inside phpunit for all it's dependencies is also a good thing. And a package registered at packagist.org as well.

Links:

• http://getcomposer.org/
• http://packagist.org/

PHPUnit 9 is being tested against nightly builds of PHP 8 for quite a while and it is safe to say that -- as of right now -- PHPUnit 9 works on PHP 8. While some things might still change until PHP 8.0.0 is released, it is time to make the support for PHP 8 official.

However, this is not as simple as applying

diff --git a/composer.json b/composer.json
index 0f8416d1d..2bf0fa21e 100644
--- a/composer.json
+++ b/composer.json
@@ -22,7 +22,7 @@
     "prefer-stable": true,
     "minimum-stability": "dev",
     "require": {
-        "php": "^7.3",
+        "php": "^7.3 || ^8.0",

to this repository for phpunit/phpunit.

PHP 8.0 also needs to be allowed in the composer.json files of PHPUnit's dependencies:

• [x] phpunit/php-code-coverage
• [x] phpunit/php-file-iterator
• [x] phpunit/php-invoker
• [x] phpunit/php-text-template
• [x] phpunit/php-timer
• [x] phpunit/php-token-stream
• [x] sebastian/code-unit
• [x] sebastian/code-unit-reverse-lookup
• [x] sebastian/comparator
• [x] sebastian/diff
• [x] sebastian/environment
• [x] sebastian/exporter
• [x] sebastian/global-state
• [x] sebastian/object-enumerator
• [x] sebastian/object-reflector
• [x] sebastian/recursion-context
• [x] sebastian/resource-operations
• [x] sebastian/type
• [x] sebastian/version
• [X] doctrine/instantiator (has "php": "^7.1 || ^8.0" in 1.3.1)
• [X] symfony/polyfill-ctype (has "php": ">=5.3.3" in 1.17.1)
• [x] myclabs/deep-copy CC @mnapoli and @theofidry
• [x] phar-io/manifest CC @theseer
• [x] phar-io/version CC @theseer
• [x] phpdocumentor/reflection-common CC @jaapio and @mvriel
• [x] phpdocumentor/reflection-docblock CC @jaapio and @mvriel
• [x] phpdocumentor/type-resolver CC @jaapio and @mvriel
• [x] phpspec/prophecy-phpunit CC @ciaranmcnulty and @stof
• [x] phpspec/prophecy CC @ciaranmcnulty and @stof
• [x] theseer/tokenizer CC @theseer
• [x] webmozart/assert CC @webmozart

$ composer info | sort -d
doctrine/instantiator              1.3.1             A small, lightweight utility to instantiate objects in PHP without invoking their constructors
myclabs/deep-copy                  1.9.5             Create deep copies (clones) of your objects
phar-io/manifest                   1.0.3             Component for reading phar.io manifest information from a PHP Archive (PHAR)
phar-io/version                    2.0.1             Library for handling version information and constraints
phpdocumentor/reflection-common    2.1.0             Common reflection classes used by phpdocumentor to reflect the code structure
phpdocumentor/reflection-docblock  5.1.0             With this component, a library can provide support for annotations via DocBlocks or otherwise retrieve information that is embedded in a DocBlock.
phpdocumentor/type-resolver        1.2.0             A PSR-5 based resolver of Class names, Types and Structural Element Names
phpspec/prophecy-phpunit           v2.0.0            PhpUnit test case integrating the Prophecy mocking library
phpspec/prophecy                   v1.10.3           Highly opinionated mocking framework for PHP 5.3+
phpunit/php-code-coverage          9.0.x-dev 4b3293f Library that provides collection, processing, and rendering functionality for PHP code coverage information.
phpunit/php-file-iterator          3.0.2             FilterIterator implementation that filters files based on a list of suffixes.
phpunit/php-invoker                3.0.1             Invoke callables with a timeout
phpunit/php-text-template          2.0.1             Simple template engine.
phpunit/php-timer                  5.0.0             Utility class for timing
phpunit/php-token-stream           4.0.2             Wrapper around PHP's tokenizer extension.
sebastian/code-unit                1.0.3             Collection of value objects that represent the PHP code units
sebastian/code-unit-reverse-lookup 2.0.1             Looks up which function or method a line of code belongs to
sebastian/comparator               4.0.2             Provides the functionality to compare PHP values for equality
sebastian/diff                     4.0.1             Diff implementation
sebastian/environment              5.1.1             Provides functionality to handle HHVM/PHP environments
sebastian/exporter                 4.0.1             Provides the functionality to export PHP variables for visualization
sebastian/global-state             5.0.x-dev 836ce3a Snapshotting of global state
sebastian/object-enumerator        4.0.1             Traverses array structures and object graphs to enumerate all referenced objects
sebastian/object-reflector         2.0.1             Allows reflection of object attributes, including inherited and non-public ones
sebastian/recursion-context        4.0.1             Provides functionality to recursively process PHP variables
sebastian/resource-operations      3.0.1             Provides a list of PHP built-in functions that operate on resources
sebastian/type                     2.1.0             Collection of value objects that represent the types of the PHP type system
sebastian/version                  3.0.0             Library that helps with managing the version number of Git-hosted PHP projects
symfony/polyfill-ctype             v1.17.1           Symfony polyfill for ctype functions
theseer/tokenizer                  1.1.3             A small library for converting tokenized PHP source code into XML and potentially other formats
webmozart/assert                   1.9.0             Assertions to validate method input/output with nice error messages.

Of course, I will take care of all dependencies that live in the sebastian/ and phpunit/ namespaces myself.

PHPUnit is currently supported on PHP 5.3.3 and newer. PHP 5.3.3 was released in July 2010 and stopped being supported by the PHP project in August 2014. PHP 5.6 was released in August 2014 and is supported by the PHP project until August 2016.

The release process of PHPUnit requires that a PHPUnit release series that is actively supported by the PHPUnit project must be compatible with all versions of PHP that are actively supported by the PHP project. It also states that the minimum required version of PHP may only be bumped in a new major release of PHPUnit.

I would like to bump the required version of PHP to PHP 5.6 for PHPUnit 5 in an effort to lessen the burden to support outdated PHP versions for the PHPUnit development team and to motivate the PHP community to adopt newer versions of PHP. PHPUnit 5 should be the last major version of PHPUnit to support PHP 5. PHPUnit 6 should only support PHP 7.

I do not want to leave users of PHPUnit behind. However, there are quite a few language features in newer versions of PHP, especially in PHP 7, that I would like to leverage for the development of PHPUnit. According to PHPUnit's release process, PHPUnit 4 would be supported through bugfix releases for at least one year after PHPUnit 5 comes out.

With regards to a timeline, I am currently thinking of releasing PHPUnit 5 in December 2015 and PHPUnit 6 in December 2016.

Hi :smile:

I've got into some situations involving partial assertion of arrays that could be easily solved if we had the following implementation:

$data = [
    'a' => 'item a',
    'b' => 'item b',
    'c' => 'item c',
    'd' => 'item d'
];

// passes
$this->assertArrayPart(['c' => 'item c', 'a' => 'item a'], $data); 

// fails
$this->assertArrayPart(['c' => 'item c', 'e' => 'item e'], $data); 
$this->assertArrayPart(['c' => 'wrong value'], $data); 

It could also assert recursions:

$data = [
    'a' => 'item a',
    'b' => 'item b',
    'c' => ['a2' => 'item a2', 'b2' => 'item b2']
];

$needle = [
    'a' => 'item a',
    'c' => ['a2' => 'item a2']
];

$this->assertArrayPart($needle, $data);  // passes

I searched through all the docs and found nothing equivalent.

As discussed in #522, this set of changes adds support for composer installation.

The package-composer.json file, coupled with package.xml and the package2composer script that is part of the Conductor package at http://pear.claylo.com/ is all that is needed to update the composer.json file.

Simply run package2composer in the top level of the checkout to regenerate composer.json to reflect changes made to package.xml

Suggestion: Link your repository at http://packagist.com. It's a one-time GitHub Service hookup, which will keep those using composer up to date as you produce releases using your normal process.

Followed the installation instructions (I have had this working before in another project but try a new install). Seems I have PHPUnit install

laravel-rest/vendor/phpunit/phpunit/phpunit

which is pointed to by an symlink in vendor/bin

phpunit -> ../phpunit/phpunit/composer/bin/phpunit

regardless, running vendor/bin/phpunit, nothing happens (No errors etc) just like the command runs but does nothing.

"require-dev": {

"phpunit/phpunit": "3.7.27"

},

As stated, I have had this working in other projects, but this one has me a bit stumped?

| Q | A | --------------------| --------------- | PHPUnit version | 8.4.1 | PHP version | PHP 7.3.9-1~deb10u1 (cli) (built: Sep 18 2019 10:33:23) ( NTS ) | PHPStorm version | PhpStorm 2019.2.3, Build #PS-192.6817.20, built on September 25, 2019 | Installation Method | Composer

Summary

I'm using the mentioned PHP / PHPUnit setup in a Vagrant VirtualBox VM. Since PHPUnit 8.4.1 I cannot run a single test case from the context menu of PHPStorm without a fatal error.

Previous Behavior

Until 8.4.0 it was possible to run single test cases.

Current Behavior

Testing started at 9:27 PM ...
[vagrant:///path/to/project]:/usr/bin/php /vagrant/vendor/phpunit/phpunit/phpunit --configuration /vagrant/build/phpunit.xml Vendor\\ProjectNamespace\\Tests\\UnitTests\\SomeClassTest /vagrant/tests/UnitTests/SomeClassTest.php --teamcity

Fatal error: Uncaught PHPUnit\Runner\Exception: Class 'Vendor\\ProjectNamespace\\Tests\\UnitTests\\SomeClassTest' could not be found in '/vagrant/tests/UnitTests/SomeClassTest.php'. in /vagrant/vendor/phpunit/phpunit/src/Runner/StandardTestSuiteLoader.php on line 69

PHPUnit\Runner\Exception: Class 'Vendor\\ProjectNamespace\\Tests\\UnitTests\\SomeClassTest' could not be found in '/vagrant/tests/UnitTests/SomeClassTest.php'. in /vagrant/vendor/phpunit/phpunit/src/Runner/StandardTestSuiteLoader.php on line 69
PHP Fatal error:  Uncaught PHPUnit\Runner\Exception: Class 'Vendor\\ProjectNamespace\\Tests\\UnitTests\\SomeClassTest' could not be found in '/vagrant/tests/UnitTests/SomeClassTest.php'. in /vagrant/vendor/phpunit/phpunit/src/Runner/StandardTestSuiteLoader.php:69

Stack trace:
Call Stack:
    0.0001     398208   1. {main}() /vagrant/vendor/phpunit/phpunit/phpunit:0
    0.0065     851376   2. PHPUnit\TextUI\Command::main() /vagrant/vendor/phpunit/phpunit/phpunit:61
#0 /vagrant/vendor/phpunit/phpunit/src/Runner/BaseTestRunner.php(141): PHPUnit\Runner\StandardTestSuiteLoader->load('Vendor\\\\Pro...', '/vagrant/tests/...')
    0.0065     851488   3. PHPUnit\TextUI\Command->run() /vagrant/vendor/phpunit/phpunit/src/TextUI/Command.php:159
#1 /vagrant/vendor/phpunit/phpunit/src/Runner/BaseTestRunner.php(101): PHPUnit\Runner\BaseTestRunner->loadSuiteClass('Vendor\\\\Pro...', '/vagrant/tests/...')
    0.0152    1546696   4. PHPUnit\TextUI\TestRunner->getTest() /vagrant/vendor/phpunit/phpunit/src/TextUI/Command.php:177
#2 /vagrant/vendor/phpunit/phpunit/src/TextUI/Command.php(177): PHPUnit\Runner\BaseTestRunner->getTest('Vendor\\\\Pro...', '/vagrant/tests/...', Array)
    0.0152    1546696   5. PHPUnit\TextUI\TestRunner->loadSuiteClass() /vagrant/vendor/phpunit/phpunit/src/Runner/BaseTestRunner.php:101
#3 /vagrant/vendor/phpunit/phpunit/src/TextUI/Command.php(159): PHPUnit\TextUI\Command->run(Array, true)
    0.0165    1558304   6. PHPUnit\Runner\StandardTestSuiteLoader->load() /vagrant/vendor/phpunit/phpunit/src/Runner/BaseTestRunner.php:141
#4 /vagrant/vendor/phpunit/phpunit/phpunit(61): PHPUnit\TextUI\Command::main()
#5 {main}
  thrown in /vagrant/vendor/phpunit/phpunit/src/Runner/StandardTestSuiteLoader.php on line 69

Process finished with exit code 255

Running all test cases from a folder context menu or running a single test from inside a test case are not affected. They still run as expected.

How to reproduce

Install PHPUnit 8.4.1 and select Run 'SomeClassTest (PHPUnit)'.

| Q | A | --------------------| --------------- | PHPUnit version | 8.0.6 | PHP version | 7.3.2 | Installation Method | PHAR scoped

When running the 8.0.6 scoped phar on a test suite with code coverage I get this error.

phpunit
PHPUnit 8.0.6 by Sebastian Bergmann and contributors.

PHP Fatal error:  Uncaught Error: Call to undefined function PHPUnit\xdebug_start_code_coverage() in phar:///usr/local/bin/phpunit/php-code-coverage/Driver/Xdebug.php:53
Stack trace:
#0 phar:///usr/local/bin/phpunit/php-code-coverage/CodeCoverage.php(210): SebastianBergmann\CodeCoverage\Driver\Xdebug->start(true)
#1 phar:///usr/local/bin/phpunit/phpunit/Framework/TestResult.php(532): SebastianBergmann\CodeCoverage\CodeCoverage->start(Object(CaptainHook\App\Composer\CmdTest))
#2 phar:///usr/local/bin/phpunit/phpunit/Framework/TestCase.php(635): PHPUnit\Framework\TestResult->run(Object(CaptainHook\App\Composer\CmdTest))
#3 phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php(556): PHPUnit\Framework\TestCase->run(Object(PHPUnit\Framework\TestResult))
#4 phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php(556): PHPUnit\Framework\TestSuite->run(Object(PHPUnit\Framework\TestResult))
#5 phar:///usr/local/bin/phpunit/phpunit/TextUI/TestRunner.php(414): PHPUnit\Framework\TestSuite->run(Object(PHPUnit\Framework\TestRe in phar:///usr/local/bin/phpunit/php-code-coverage/Driver/Xdebug.php on line 53

Fatal error: Uncaught Error: Call to undefined function PHPUnit\xdebug_start_code_coverage() in phar:///usr/local/bin/phpunit/php-code-coverage/Driver/Xdebug.php on line 53

Error: Call to undefined function PHPUnit\xdebug_start_code_coverage() in phar:///usr/local/bin/phpunit/php-code-coverage/Driver/Xdebug.php on line 53

Call Stack:
    0.0013     550656   1. {main}() /usr/local/bin/phpunit:0
    0.0838    9760864   2. PHPUnit\TextUI\Command::main() /usr/local/bin/phpunit:619
    0.0838    9760976   3. PHPUnit\TextUI\Command->run() phar:///usr/local/bin/phpunit/phpunit/TextUI/Command.php:71
    0.2137   17031528   4. PHPUnit\TextUI\TestRunner->doRun() phar:///usr/local/bin/phpunit/phpunit/TextUI/Command.php:102
    0.2340   17125680   5. PHPUnit\Framework\TestSuite->run() phar:///usr/local/bin/phpunit/phpunit/TextUI/TestRunner.php:414
    0.2380   17128592   6. PHPUnit\Framework\TestSuite->run() phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php:556
    0.2398   17129248   7. CaptainHook\App\Composer\CmdTest->run() phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php:556
    0.2399   17129248   8. PHPUnit\Framework\TestResult->run() phar:///usr/local/bin/phpunit/phpunit/Framework/TestCase.php:635
    0.2400   17129336   9. SebastianBergmann\CodeCoverage\CodeCoverage->start() phar:///usr/local/bin/phpunit/phpunit/Framework/TestResult.php:532
    0.2400   17129336  10. SebastianBergmann\CodeCoverage\Driver\Xdebug->start() phar:///usr/local/bin/phpunit/php-code-coverage/CodeCoverage.php:210

When I checkout the latest 8.0 branch and create the PHAR myself it works just fine. Is it just me or does someone get the same error with code coverage

Follow up to #4743 and per the discussion in that issue.

This commit splits the GH action job to "build & test the phar" into two separate jobs:

1. Build the Phar. Test building the phar against the various supported PHP versions to ensure that part of the release process is compatible with the supported PHP versions.
2. Test the Phar. Only one Phar is actually released, the scoped Phar as build on PHP 8.0. This job will now test that this particular Phar file functions as expected, which makes this job more closely match the reality of running tests with the Phar as would be released.

Notes:

• In the "Build Phar" job, the PHP 8.1 task is marked as "experimental" and is - for now - allowed to fail. That building of the Phar failing on PHP 8.1 will not inhibit the phar as build on PHP 8.0 from running on PHP 8.1 and will not block a PHP 8.1 compatible release. The build is failing due to PHP 8.1. incompatibilities in PHP_Scoper and Symfony Finder. Issues have already been opened in both repos about this. Additionally, the build shows numerous deprecation notices for PHPAB, ConsoleTools and Box Project. Issues/PRs have been opened in each.
• The scoped Phar as build on PHP 8.0 is uploaded as an artifact during the "Build Phar" job and available for download after each CI run.
• This Phar artifact is subsequently downloaded in the "Test Phar" job to be used to run the jobs.
• The "Test Phar" job need a Composer install to run the PHPUnit native test suite. To make absolutely sure that the Phar will also function correctly without the Composer install making all the PHPUnit dependencies available, a single stand-alone test is run purely with the Phar, before the Composer install is run.

|Note: updated documentation| |:---| |There is a wiki page with more documentation with examples while the official docs are updated.|

Inspiration

Are your unit tests truly independent units?

In the past I have often been bitten by test suites that do not test what they claim to test. A common reason is hidden dependencies in the runtime context. Worst case this has caused applications to fail when deployed in production environments.

Changing the order of unit tests turns out to be a simple and effective way of bringing these assumptions out in the open.

Thanks to @crazybus and @geogdog for providing real-world scenarios and feedback during development.

Summary of functionality

The test ordering functionality can be configured using command line parameters:

• --reverse-order run tests in reverse order
• --random-order run tests in random order
• --random-order-seed=<number> fix the random order, allowing random runs to be reproduced
• --ignore-dependencies disable the bundled dependency resolver

Design decisions

Low impact, high performance

The feature will only wake up when there is work to do. The iterative dependency resolver is designed to do as little work as possible, using few iterations and minimal memory. It uses @depends annotations to keep as many dependent tests as possible in the required order.

No unnecessary extra output is generated. Only the randomizer outputs the random seed using the default results printer.

Clean abstraction

The reordering is built into the main test runner and only changes the order of the tests of each test suite. Other parts of PHPUnit are not affected, with the exception of a bit of configuration handling.

Changes to the built-in testsuite

• added a fourth MultiDependencyTest and updated existing tests to match
• added new TextUI tests for --reverse-order, --random-order and --ignore-dependencies
• added @depends for testGlobalsBackupPre() to PHPUnit's own testsuite

Future work

• update the main PHPUnit documentation to reflect the changes in (optional) order and dependency handling
• add configuration options for phpunit.xml in addition to the command line flags
• add options and/or annotations for finer grained reordering config (e.g. "never reorder this suite")

| Q | A | --------------------| --------------- | PHPUnit version | 8.4.2 | PHP version | 7.3.11 | Installation Method | Composer

Composer info:

| package | version | description |-------------|------------|---------------- |doctrine/instantiator | 1.2.0 | A small, lightweight utility to instantiate objects in PHP without invoking their constructors |myclabs/deep-copy | 1.9.3 |Create deep copies (clones) of your objects |phar-io/manifest | 1.0.3 |Component for reading phar.io manifest information from a PHP Archive (PHAR) |phar-io/version | 2.0.1 | Library for handling version information and constraints |phpdocumentor/reflection-common | 2.0.0 |Common reflection classes used by phpdocumentor to reflect the code structure |phpdocumentor/reflection-docblock| 4.3.2 | With this component, a library can provide support for annotations via DocBlocks or otherwise retrieve information that is embedded in a DocBlock. |phpdocumentor/type-resolver | 1.0.1 | A PSR-5 based resolver of Class names, Types and Structural Element Names |phpspec/prophecy | 1.9.0 | Highly opinionated mocking framework for PHP 5.3+ |phpunit/php-code-coverage | 7.0.8 | Library that provides collection, processing, and rendering functionality for PHP code coverage information. |phpunit/php-file-iterator | 2.0.2 | FilterIterator implementation that filters files based on a list of suffixes. |phpunit/php-text-template | 1.2.1 | Simple template engine. |phpunit/php-timer | 2.1.2 | Utility class for timing |phpunit/php-token-stream | 3.1.1 | Wrapper around PHP's tokenizer extension. |phpunit/phpunit | 8.4.2 | The PHP Unit Testing framework. |sebastian/code-unit-reverse-lookup | 1.0.1| Looks up which function or method a line of code belongs to |sebastian/comparator | 3.0.2 | Provides the functionality to compare PHP values for equality |sebastian/diff | 3.0.2 | Diff implementation |sebastian/environment | 4.2.2 | Provides functionality to handle HHVM/PHP environments |sebastian/exporter | 3.1.2 | Provides the functionality to export PHP variables for visualization |sebastian/global-state | 3.0.0 | Snapshotting of global state |sebastian/object-enumerator | 3.0.3 | Traverses array structures and object graphs to enumerate all referenced objects |sebastian/object-reflector | 1.1.1 | Allows reflection of object attributes, including inherited and non-public ones |sebastian/recursion-context | 3.0.0 | Provides functionality to recursively process PHP variables |sebastian/resource-operations | 2.0.1 | Provides a list of PHP built-in functions that operate on resources |sebastian/type | 1.1.3 | Collection of value objects that represent the types of the PHP type system |sebastian/version | 2.0.1 | Library that helps with managing the version number of Git-hosted PHP projects |symfony/polyfill-ctype | v1.12.0 | Symfony polyfill for ctype functions |theseer/tokenizer | 1.1.3 | A small library for converting tokenized PHP source code into XML and potentially other formats |webmozart/assert | 1.5.0 | Assertions to validate method input/output with nice error messages.

Summary

PhpUnit 8 consumes much more memory for test run comparing with PhpUnit 7.

Current behavior

We run the same set of tests with the same amount of assertions and see that PhpUnit 8 consumes ~30MB more than PhpUnit 7.

How to reproduce

Write code

<?php

use PHPUnit\Framework\TestCase;

class SomeTest extends TestCase
{
    /**
     * @dataProvider provider
     */
    public function testOne()
    {
        $this->assertTrue(true);
    }

    public function provider()
    {
        for ($a = 0; $a < 20000; $a++){
            yield "constant a${a}" => [$a];
        }
    }
}

And run test using PhpUnit 8.4.2 and PhpUnit 7.5.17. Results will be similar to

PHPUnit 7.5.17 by Sebastian Bergmann and contributors.

Time: 831 ms, Memory: 40.00 MB

OK (20000 tests, 20000 assertions)

and

PHPUnit 8.4.2 by Sebastian Bergmann and contributors.

Time: 941 ms, Memory: 56.76 MB

OK (20000 tests, 20000 assertions)

Expected behavior

It would be nice if PhpUnit 8 consumes not so much memory comparing with PhpUnit 7.

| Q | A | --------------------| --------------- | PHPUnit version |9.5.27 | PHP version | 8.1.10 | Installation Method | Composer

Summary

When compare strings (teorically) identically but contain space, result is failed

Current behavior

1) QueryTest::testGetInfo
Failed asserting that 'SANTAGATI SALVATORE' contains "SANTAGATI SALVATORE".

QueryTest.php:30
phpunit:98   

How to reproduce

<?php

declare(strict_types=1);

use PHPUnit\Framework\TestCase;
use Salsan\Members;

final class QueryTest extends TestCase{
    private $paramters = array(
        'id' => '165714',
        'membershipYear' => '2022'
    );

    public function testInit(): object
    {
        $members = new Members\Query($this->paramters);
        $this->assertIsObject($members);

        return $members;
    }

     /**
     * @depends testInit
     */
    public function testGetInfo($members): void
    {
        $info = $members->getList();

        foreach ($info as $member) {
            $this->assertStringContainsString("SANTAGATI SALVATORE", $member['name']);
            $this->assertIsBool($member['isRookie']);     
        }
    }


}

Expected behavior

The result to be "true" echo gettype($member["name"]); // result is string echo ($member["name") // result is SANTAGATI SALVATORE

I tried with others string without whitespace and work correctly

| Q | A | --------------------| --------------- | PHPUnit version | 9.5.27 | PHP version | 7.3, 7.4, 8.0, 8.1, 8.2 | Installation Method | Composer

Summary

Testdox does not output the data provide index if an argument is "empty".

How to reproduce

I have created a project to reproduce this bug and all details such as output in all PHP versions and composer install logs can be seen in the pipeline.

// tests/ReproduceTest.php
<?php

use PHPUnit\Framework\TestCase;

class ReproduceTest extends TestCase
{
    public function iterableDataProvider(): iterable
    {
        yield 'no arg' => []; // will not have "no arg" printed out when using `--testdoz`
        yield 'one arg' => [1];
        yield 'two args' => [1, 2];
    }

    /**
     * @test
     * @dataProvider iterableDataProvider
     */
    public function reproduceWithIterableDataProvider(int ...$args): void
    {
        self::assertTrue(true);
    }

    public function arrayDataProvider(): array
    {
        return [
            'no arg' => [],
            'one arg' => [1],
            'two args' => [1, 2],
        ];
    }

    /**
     * @test
     * @dataProvider arrayDataProvider
     */
    public function reproduceWithArrayDataProvider(int ...$args): void
    {
        self::assertTrue(true);
    }
}

Current behavior

$ composer exec phpunit -- tests/ReproduceTest.php --testdox
PHPUnit 9.5.27 by Sebastian Bergmann and contributors.

Reproduce
 ✔ Reproduce with iterable data provider
 ✔ Reproduce with iterable data provider with one·arg
 ✔ Reproduce with iterable data provider with two·args
 ✔ Reproduce with array data provider
 ✔ Reproduce with array data provider with one·arg
 ✔ Reproduce with array data provider with two·args

Expected behavior

$ composer exec phpunit -- tests/ReproduceTest.php --testdox
PHPUnit 9.5.27 by Sebastian Bergmann and contributors.

Reproduce
 ✔ Reproduce with iterable data provider with no·arg
 ✔ Reproduce with iterable data provider with one·arg
 ✔ Reproduce with iterable data provider with two·args
 ✔ Reproduce with array data provider with no·arg
 ✔ Reproduce with array data provider with one·arg
 ✔ Reproduce with array data provider with two·args

| Q | A | --------------------| --------------- | PHPUnit version | 9.5 | PHP version | 8.1 | Installation Method | Composer

Summary

--migrate-configuration

removes the 4 space indentation and creates 2 space one That needs to be fixed for all our repos. Would be nice if the commend would more aware of the indentation (4 is default anyway) and only does changes needed. This would also make diff report easier to review and validate, and then commit the changes done

Right now the diff is literally the whole file.

--- a/phpunit.xml.dist	(revision 745b83726c67a5b55f75322f2f7fad7288f2a084)
+++ b/phpunit.xml.dist	(date 1672264287952)
@@ -1,37 +1,30 @@
 <?xml version="1.0" encoding="UTF-8"?>
-<phpunit
-    colors="true"
-    bootstrap="tests/bootstrap.php"
-    >
-    <php>
-        <ini name="memory_limit" value="-1"/>
-        <ini name="apc.enable_cli" value="1"/>
-		<!-- E_ALL => 32767 -->
-		<!-- E_ALL & ~E_USER_DEPRECATED => 16383 -->
-		<ini name="error_reporting" value="16383"/>
-    </php>
-
-    <!-- Add any additional test suites you want to run here -->
-    <testsuites>
-        <testsuite name="ide-helper">
-            <directory>tests/TestCase/</directory>
-        </testsuite>
-    </testsuites>
-
-    <!-- Setup a listener for fixtures -->
-    <listeners>
-        <listener class="Cake\TestSuite\Fixture\FixtureInjector">
-            <arguments>
-                <object class="Cake\TestSuite\Fixture\FixtureManager"/>
-            </arguments>
-        </listener>
-    </listeners>
-
-    <!-- Prevent coverage reports from looking in tests and vendors -->
-    <filter>
-        <whitelist>
-            <directory suffix=".php">src/</directory>
-        </whitelist>
-    </filter>
-
+<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" colors="true" bootstrap="tests/bootstrap.php" xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/9.3/phpunit.xsd">
+  <coverage>
+    <include>
+      <directory suffix=".php">src/</directory>
+    </include>
+  </coverage>
+  <php>
+    <ini name="memory_limit" value="-1"/>
+    <ini name="apc.enable_cli" value="1"/>
+    <!-- E_ALL => 32767 -->
+    <!-- E_ALL & ~E_USER_DEPRECATED => 16383 -->
+    <ini name="error_reporting" value="16383"/>
+  </php>
+  <!-- Add any additional test suites you want to run here -->
+  <testsuites>
+    <testsuite name="ide-helper">
+      <directory>tests/TestCase/</directory>
+    </testsuite>
+  </testsuites>
+  <!-- Setup a listener for fixtures -->
+  <listeners>
+    <listener class="Cake\TestSuite\Fixture\FixtureInjector">
+      <arguments>
+        <object class="Cake\TestSuite\Fixture\FixtureManager"/>
+      </arguments>
+    </listener>
+  </listeners>
+  <!-- Prevent coverage reports from looking in tests and vendors -->
 </phpunit>

Side note: Even if the indentation is fixed btw, the re-ordering of all the high level keys also further makes the diffing harder. Filter vs Coverage

The methods createStubForIntersectionOfInterfaces() and createMockForIntersectionOfInterfaces() accept a list of interfaces (list<class-string>) and return an intersection of these interface plus Stub or MockObject, respectively.

This fact needs to be document through annotations that can be understood by static analysis tools.

Some object may have properties with unpredictable values, like random UUID, and you want to compare the predictable ones.

In Java, with AssertJ it’s possible with: assertThat(actual).usingRecursiveComparison().ignoringFields("id").isEqualTo(expected).

Maybe a signature like assertEqualsIgnoringFields($expected, $actual, string ...$fields)

<?php

class Foo {

    private readonly DateTimeImmutable $date;
    public function __construct(private readonly string $name)
    {
        $this->date = new DateTimeImmutable();
    }
}

/**
* @test
*/
public function passingTest(): void
{
    $foo1 = new Foo("fooName");
    $foo2 = new Foo("fooName");

    self::assertEqualsIgnoringFields($foo1, $foo2, "date");
}

Sorry if this is a long post, it was a bit of a journey for me!

When trying to implement my own Constraint, I was a bit confused by the API. My understanding is that the core of execution is this:

// Constraint.php
public function evaluate($other, string $description = '', bool $returnResult = false): ?bool
{
    $success = false;

    if ($this->matches($other)) {
        $success = true;
    }

    if ($returnResult) {
        return $success;
    }

    if (!$success) {
        $this->fail($other, $description);
    }

    return null;
}

The documentation however suggest that the right extension point is Constraint::matches() and by extension, Constraint::failureDescription().

Looking at PHPUnit codebase for more examples I see a mix of the two which makes me confused.

Indeed, overriding matches looks very simple and the best solution. However, when wanting to provide a more detailed and helpful failure description, I feel you are just "duplicating" matches but returning the description string instead, because depending of the step or the why matches return false, the failure description changes.

You can see how this is a problem even within PHPUnit with the constraint IsJson:

/**
 * Evaluates the constraint for parameter $other. Returns true if the
 * constraint is met, false otherwise.
 *
 * @param mixed $other value or object to evaluate
 */
protected function matches($other): bool
{
    if ($other === '') {
        return false;
    }

    json_decode($other);

    if (json_last_error()) {
        return false;
    }

    return true;
}

/**
 * Returns the description of the failure.
 *
 * The beginning of failure messages is "Failed asserting that" in most
 * cases. This method should return the second part of that sentence.
 *
 * @param mixed $other evaluated value or object
 *
 * @throws \SebastianBergmann\RecursionContext\InvalidArgumentException
 */
protected function failureDescription($other): string
{
    if ($other === '') {
        return 'an empty string is valid JSON';
    }

    json_decode($other);
    $error = (string) JsonMatchesErrorMessageProvider::determineJsonError(
        (string) json_last_error()
    );

    return sprintf(
        '%s is valid JSON (%s)',
        $this->exporter()->shortenedExport($other),
        $error
    );
}

Looking at both methods, we can notice that:

• $other is effectively being evaluated twice
• how with a slightly more complex scenario, you repeat the matching process to provide a more tailored error

I think this is not really desirable for both performance (a failure does not necessarily stop the process, so having it's evaluation expansive for no reason is not desired) and robustness (duplicating the evaluation feels brittle).

My first impression when seeing this was to either create the failure description directly in matches as a constraint state via a property or to have a memoized method that does "evaluate and provide error message upon non match". I am not too found of neither.†

Fishing around for more examples, I actually saw a few constraints such as:

• ObjectEquals that instead directly fail in matches()
• IsEqual that overrides evaluate().

The second case actually looks more right to me compared to †, but completely forgo the nice utils provided Constraint::fail(). At least with the current API. I find strange however that evaluate() return type is bool when clearly it is true|throw ExpectationFailedException.

This opinion is completely uninformed about the historical reasons, future plans and BC regards, so maybe it's just silly, but WDYT of changing the API of matches() to return true|string|null instead, string replacing failureDescription() in usage, which is only need if the return is null (to keep the default).

And regardless if feels that evaluate() return type could be changed to void?