💥 Collision is a beautiful error reporting tool for command-line applications

OS: Windows 10 Build 18363.657 Windows Terminal (beta): 0.9.433.0 Laravel Version: 7.0-dev

How to reproduce:

• Run php artisan test
• The output doesn't include the highlighting from this package's printer class.
• Note: when I tried using PHPUnit\TextUI\ResultPrinter in the printer parameter for the artisan command, the colors aren't respected either.
• With the printer class from this package specified in phpunit.xml, the colors are respected when running via vendor\bin\phpunit

replace abandoned package jakub-onderka/php-console-highlighter with "php-parallel-lint/php-console-highlighter 0.5

closes https://github.com/nunomaduro/collision/issues/172

After composer update I get a warning about deprecated packages

composer update
Loading composer repositories with package information                                                                                                                                                                                                    Updating dependencies (including require-dev)         
Package operations: 1 install, 6 updates, 0 removals
  - Updating league/commonmark (1.3.2 => 1.3.3): Downloading (100%)         
  - Updating psy/psysh (v0.10.2 => v0.10.3): Downloading (100%)         
  - Updating laravel/framework (v6.18.3 => v6.18.6): Downloading (100%)         
  - Updating laravel/tinker (v2.3.0 => v2.4.0): Downloading (100%)         
  - Installing sebastian/code-unit (1.0.0): Downloading (100%)         
  - Updating phpunit/phpunit (9.0.2 => 9.1.1): Downloading (100%)         
  - Updating composer/ca-bundle (1.2.6 => 1.2.7): Downloading (100%)         
Package jakub-onderka/php-console-color is abandoned, you should avoid using it. Use php-parallel-lint/php-console-color instead.
Package jakub-onderka/php-console-highlighter is abandoned, you should avoid using it. Use php-parallel-lint/php-console-highlighter instead.
Writing lock file
Generating optimized autoload files

I think there was a closed issue related to this. I use this feature pretty often when running the complete test suite so I can walk away, get some coffee, come back and see what a mess I made during my refactor. Just wondering if it is possible to modify the PHPUnit listener to respect this flag.

Coming from pest https://github.com/pestphp/pest/issues/73

While PHPUnit's DefaultResultPrinter emits output to stdout anyways if beStrictAboutOutputDuringTests sets false, collision might want to follow the same behavior.

When running php artisan test environment specific .env files aren't loaded.

Creating a fresh Laravel installation comes with a phpunit.xml file where APP_ENV=testing is defined. When running phpunit it would load a .env.testing file if it exists in the project root and include those variables when testing.

This should fix and mimic the same way as Laravel loads environment specific .env files when running php artisan test

Hi I have problem to upgrade to laravel 6.0.

When I run composer update I get:

If I put this package to dont-discover like this:

Every is OK.

On Laravel 5.8 I don't have this problem.

Hey, love the package so far :) Very helpful!

I have found one gripe which is annoying, and maybe a very edge case - but if you deem it fixable then I think it's definitely an improvement.

So if you type something like

php artisan migrates instead of php artisan migrate it throws a CommandNotFoundException

However artisan already handles this and gives you suggestions - the suggestions get a bit battered due to the exception also rendering.

Would it be possible to ignore that exception?

Example test:

<?php

namespace Tests\Feature;

use Illuminate\Foundation\Testing\RefreshDatabase;
use Tests\TestCase;

class ExampleTest extends TestCase
{
    /**
     * A basic test example.
     *
     * @return void
     */
    public function testBasicTest()
    {
        $response = $this->get('/')->dump();

        $response->assertStatus(200);
    }
}

Only a certain number of lines from the dump output are shown. While not a huge world shaking bug it makes switching from using PHPUnit directly a bit of a pain as something dumping the output is required for debugging a test.

Hey @nunomaduro,

Would it be possible to for the collision package to integrate with Flare, so one can share the public link with anybody they'll like to show their error with?

Every status icon is now using a semantic symbol to allow for quick scanning and visual cohesion:

• RISKY & WARN both use ! (reasoning: PHPUnit provides an explanation of the error to understand what's going on.)
• INCOMPLETE uses … (reasoning: an ellipsis indicates an intentional omission of something. Here, a test.)
• SKIPPED uses - (reasoning : run the suite, "minus" this test)

Php is dockerized in it's container:

PHP 8.1.1 (cli) (built: Dec 18 2021 01:38:53) (NTS)
Copyright (c) The PHP Group
Zend Engine v4.1.1, Copyright (c) Zend Technologies
    with Xdebug v3.1.1, Copyright (c) 2002-2021, by Derick Rethans

Laravel Framework 8.83.26

Phpunit:

composer info phpunit/phpunit
name     : phpunit/phpunit
descrip. : The PHP Unit Testing framework.
keywords : phpunit, testing, xunit
versions : * 9.5.26
type     : library
license  : BSD 3-Clause "New" or "Revised" License (BSD-3-Clause) (OSI approved) https://spdx.org/licenses/BSD-3-Clause.html#licenseText
homepage : https://phpunit.de/
source   : [git] https://github.com/sebastianbergmann/phpunit.git 851867efcbb6a1b992ec515c71cdcf20d895e9d2
dist     : [zip] https://api.github.com/repos/sebastianbergmann/phpunit/zipball/851867efcbb6a1b992ec515c71cdcf20d895e9d2 851867efcbb6a1b992ec515c71cdcf20d895e9d2
path     : /app/vendor/phpunit/phpunit
names    : phpunit/phpunit

When running:

$ docker-compose exec php_fpm bash
$ php artisan test

NunoMaduro\Collision\Adapters\Phpunit\Printer::__construct(): Argument #1 ($output) must be of type ?Symfony\Component\Console\Output\ConsoleOutputInterface, string given, called in /app/vendor/phpunit/phpunit/src/TextUI/TestRunner.php on line 1230

I traced it a bit a fallen back to:

vendor/phpunit/phpunit/src/TextUI/TestRunner.php:1223

 private function createPrinter(string $class, array $arguments): ResultPrinter
    {
        $object = new $class(
            (isset($arguments['stderr']) && $arguments['stderr'] === true) ? 'php://stderr' : null,
            $arguments['verbose'],
            $arguments['colors'],
            $arguments['debug'],
            $arguments['columns'],
            $arguments['reverseList']
        );

        assert($object instanceof ResultPrinter);

        return $object;
    }

Let me know if you need more details.

FYI if I set that part (isset($arguments['stderr']) && $arguments['stderr'] === true) ? 'php://stderr' : null, to null only it works further.

Before this

./artisan test --no-ansi
PHPUnit 9.5.26 by Sebastian Bergmann and contributors.

Unknown option "--no-ansi"

But with this change I get uncolored output when wanted

I have only tested by using this in my local Laravel project - but it works OK there

NB./artisan test --help

suggests

--ansi|--no-ansi Force (or disable --no-ansi) ANSI output

Let me start by saying that this is kind of a weird corner case, and I am 100% to blame for ending up in this situation. And I don't know if Collision could (or should) display this type of error in a different way to avoid confusion, but I'll post this in case you want to.

Here's a simple test that demonstrates the issue

class ExampleTest extends TestCase
{
    public function setUp(): void
    {
        parent::setUp();
        // Simulating a custom trait on the base TestCase printing a message to STDERR
        fwrite(STDERR, "\033[32mEnvironment validated.\033[0m\n");
    }

    /**
     * @runInSeparateProcess
     * @preserveGlobalState disabled
     */
    public function test_mocking_a_hard_dependency()
    {
        // Simulating a test that has to be run in a separate process in
        // order to allow mocking a static method call on a class that might already be loaded.
        $this->assertTrue(true);
    }
}

Normally PhpUnit doesn't care if you write to STDERR during tests, but if a test runs in a separate process, it will result in an exception getting thrown.

Here's the output of from php artisan test:

Possibly because I'm an idiot, but looking at this output several times, I didn't realize that "Environment validated" was the exception. Especially because it's written in green, and also I'm used to seeing it every time I run my test suite. So in stead I focused on the highlighted code, which doesn't really provide any value in this case.

Here's the output from vanilla PhpUnit:

Displayed like this, I instantly realized what the problem was.

• Collision version: 6.3.1
• PHPUnit version: 9.5.25
• Laravel Version: 9.36.2
• PHP Version: 8.0.22
• Database Driver & Version: MySQL 5.7

I'm noticing some large differences between the printed stack traces when running tests with the phpunit command and php artisan test, using the verbose flag.

Most of the time i'm running php artisan test but the exception stack trace is not helpful at all. I'm not seeing application code in the stack lines, so I cannot figure out the origin of the exception in my code.

When running php artisan test -v:

  • Tests\Feature\ExampleTest > example
   PHPUnit\Framework\ExceptionWrapper

  Attempted to lazy load [posts] on model [App\Models\User] but lazy loading is disabled.

  at vendor/laravel/framework/src/Illuminate/Database/Eloquent/Concerns/HasAttributes.php:559
    555▕         if (! $this->exists || $this->wasRecentlyCreated) {
    556▕             return;
    557▕         }
    558▕
  ➜ 559▕         throw new LazyLoadingViolationException($this, $key);
    560▕     }
    561▕
    562▕     /**
    563▕      * Get a relationship value from a method.

  1   vendor/phpunit/phpunit/src/Framework/TestCase.php:904
      PHPUnit\Framework\TestResult::run()

  2   vendor/phpunit/phpunit/src/Framework/TestSuite.php:673
      PHPUnit\Framework\TestCase::run()

  3   vendor/phpunit/phpunit/src/Framework/TestSuite.php:673
      PHPUnit\Framework\TestSuite::run()

  4   vendor/phpunit/phpunit/src/Framework/TestSuite.php:673
      PHPUnit\Framework\TestSuite::run()

  5   vendor/phpunit/phpunit/src/TextUI/TestRunner.php:673
      PHPUnit\Framework\TestSuite::run()

  6   vendor/phpunit/phpunit/src/TextUI/Command.php:144
      PHPUnit\TextUI\TestRunner::run()

  7   vendor/phpunit/phpunit/src/TextUI/Command.php:97
      PHPUnit\TextUI\Command::run()

  8   vendor/phpunit/phpunit/phpunit:98
      PHPUnit\TextUI\Command::main()

When running phpunit -v:

Tests\Feature\ExampleTest::example
Illuminate\Database\LazyLoadingViolationException: Attempted to lazy load [posts] on model [App\Models\User] but lazy loading is disabled.

vendor/laravel/framework/src/Illuminate/Database/Eloquent/Concerns/HasAttributes.php:559
vendor/laravel/framework/src/Illuminate/Database/Eloquent/Concerns/HasAttributes.php:518
vendor/laravel/framework/src/Illuminate/Database/Eloquent/Concerns/HasAttributes.php:453
vendor/laravel/framework/src/Illuminate/Database/Eloquent/Model.php:2172
app/Actions/ExampleAction.php:122
app/Actions/ExampleAction.php:61
app/Jobs/ExampleJob.php:201
app/Jobs/ExampleJob.php:117
app/Jobs/ExampleJob.php:85
vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php:36
vendor/laravel/framework/src/Illuminate/Container/Util.php:41
vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php:93
vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php:37
vendor/laravel/framework/src/Illuminate/Container/Container.php:651
vendor/laravel/framework/src/Illuminate/Bus/Dispatcher.php:128
vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php:141
vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php:116
vendor/laravel/framework/src/Illuminate/Bus/Dispatcher.php:132
vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php:124
vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php:141
vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php:116
vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php:126
vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php:70
vendor/laravel/framework/src/Illuminate/Queue/Jobs/Job.php:98
vendor/laravel/framework/src/Illuminate/Queue/SyncQueue.php:43
vendor/laravel/framework/src/Illuminate/Bus/Dispatcher.php:253
vendor/laravel/framework/src/Illuminate/Bus/Dispatcher.php:229
vendor/laravel/framework/src/Illuminate/Bus/Dispatcher.php:77
vendor/laravel/framework/src/Illuminate/Support/Facades/Facade.php:338
tests/Feature/ExampleTest.php:646

Here the stack trace is much more evident, I can see which parts of my code caused the exception, so I can pinpoint my mistakes.

Unlike PHPUnit, and most PHPUnit printers, Pest does not emit output after each test in a test case, but rather, it buffers the entire collection of tests in a test case until the whole test file has completed. This is presumably because it wants to emit PASS/FAIL on the header line for the test case, which it can't know until all sub-tests have finished, but the upshot of this is there is no output for some time, especially for long-running tests, which is not a good user experience.

We should like to know, as soon as the test runner knows, what the result of running each test is. If that means reformatting the output so it is less pretty, and the PASS/FAIL summary appears at the foot of each test case rather than in the head, then so be it: I strongly support functionality over form when the two are otherwise in conflict.

The following is just a suggestion of an alternative layout that would permit immediate feedback for each test case.