The tool converts different error reporting standards for deep compatibility with popular CI systems (TeamCity, IntelliJ IDEA, GitHub Actions, etc).

Overview

JBZoo / CI-Report-Converter

CI Docker Cloud Build Status codecov Psalm Coverage Scrutinizer Code Quality
PHP Version PHP Strict Types Stable Version Total Downloads Docker Pulls GitHub Issues GitHub License

Why?

I believe you are familiar with the huge zoo of various utilities for testing, checking code standards, linters etc. It's really often the output of utilities is not supported in popular CI systems (TeamCity, GitHub, etc...). I guess you are lucky if the utility saves the error report in the junit.xml format, because it works pretty fine with almost all modern dev software.

But... My experience tells me it's the exception rather than the rule. For example, I really like the good old phpmd utility (perhaps you have another right opinion about the benefits. At least, it's just an example). It doesn't integrate well with TeamCity/PhpStorm/GitHub. Everytime I spend a lot of time looking for results in the logs. Though I really want to see instant and well-printed error report.

Therefore, I developed a converter that changes the report format for deep integration with CI systems and JetBrain IDEs.

Well... It may seem to you it's a useless thing, and your favorite super tool works fine in TeamCity/PhpStorm. Just take a look at the examples below.

Installing

composer require jbzoo/ci-report-converter
php ./vendor/bin/ci-report-converter --help

# OR use phar file
wget https://github.com/JBZoo/CI-Report-Converter/releases/latest/download/ci-report-converter.phar
chmod +x ./ci-report-converter.phar
./ci-report-converter.phar --help

# OR just pull the Docker Image
docker run --rm jbzoo/ci-report-converter --help 

Using as GitHub Action

Action allows you to convert error reports to the GitHub Annotations format

- uses: jbzoo/ci-report-converter@master # or see the specific version on releases page
  with:
    # File path with the original report format. If not set or empty, then the STDIN is used.
    # Required: true
    input-file: ./build/checkstyle.xml

    # Source format. Available options: checkstyle, junit, phpmd-json, phpmnd, pmd-cpd, psalm-json
    # Default value: checkstyle
    # Required: true
    input-format: checkstyle

    # Will exit with the code=1, if any violations are found.
    # Default value: no
    non-zero-code: yes

    # File path with the result report format. If not set or empty, then the STDOUT is used.
    output-file: ./build/junit.xml

    # Target format. Available options: gitlab-json, github-cli, junit, tc-inspections, tc-tests
    # Default value: github-cli
    # Required: true
    output-format: junit

    # Set custom name of root group/suite (if it's possible).
    # Required: true
    suite-name: My Tests

Example GitHub Action workflow

name: Linters

on:
  pull_request:
    branches:
      - "*"

jobs:
  linters:
    name: PHPcs
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Code
        uses: actions/checkout@v2
        
      - name: PHP Code Sniffer
        run: ./vendor/bin/phpcs --report=checkstyle --standard=PSR12 -q ./src > ./build/phpcs-checkstyle.xml
        
      - name: Converting checkstyle.xml to Github Annotations
        uses: jbzoo/ci-report-converter@master
        with:
          input-file: build/phpcs-checkstyle.xml

Available Directions

At the moment, converting of reports works with

Also, you can follow metrics around your code in TeamCity via teamcity:stats

Help description in terminal

Converting

$ php ./vendor/bin/ci-report-converter convert --help
Description:
  Convert one report format to another one

Usage:
  convert [options]

Options:
  -S, --input-format=INPUT-FORMAT      Source format. Available options: checkstyle, junit, phpmd-json, phpmnd, pmd-cpd, psalm-json [default: "checkstyle"]
  -I, --input-file[=INPUT-FILE]        File path with the original report format. If not set or empty, then the STDIN is used.
  -T, --output-format=OUTPUT-FORMAT    Target format. Available options: gitlab-json, github-cli, junit, tc-inspections, tc-tests [default: "tc-tests"]
  -O, --output-file[=OUTPUT-FILE]      File path with the result report format. If not set or empty, then the STDOUT is used.
  -R, --root-path[=ROOT-PATH]          If option is set, all absolute file paths will be converted to relative once. [default: "."]
  -N, --suite-name=SUITE-NAME          Set custom name of root group/suite (if it's possible).
  -F, --tc-flow-id[=TC-FLOW-ID]        Custom flowId in TeamCity output. Default value is PID of the tool.
  -Q, --non-zero-code[=NON-ZERO-CODE]  Will exit with the code=1, if any violations are found. [default: "no"]
  -h, --help                           Display help for the given command. When no command is given display help for the list command
  -q, --quiet                          Do not output any message
  -V, --version                        Display this application version
      --ansi|--no-ansi                 Force (or disable --no-ansi) ANSI output
  -n, --no-interaction                 Do not ask any interactive question
  -v|vv|vvv, --verbose                 Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug

Custom Metrics in TeamCity

To clarify the use of the method, take a look at the examples and screenshots below, please.

$ php ./vendor/bin/ci-report-converter teamcity:stats --help
Description:
  Push code metrics to TeamCity Stats

Usage:
  teamcity:stats [options]

Options:
  -S, --input-format=INPUT-FORMAT  Source format. Available options: junit-xml, pdepend-xml, phploc-json, phpmetrics-xml, phpunit-clover-xml
  -I, --input-file[=INPUT-FILE]    File path with the original report format. If not set or empty, then the STDIN is used.
  -O, --output-file[=OUTPUT-FILE]  File path with the result report format. If not set or empty, then the STDOUT is used.
  -R, --root-path[=ROOT-PATH]      If option is set, all absolute file paths will be converted to relative once. [default: "."]
  -F, --tc-flow-id[=TC-FLOW-ID]    Custom flowId in TeamCity output. Default value is PID of the tool.
  -h, --help                       Display help for the given command. When no command is given display help for the list command
  -q, --quiet                      Do not output any message
  -V, --version                    Display this application version
      --ansi|--no-ansi             Force (or disable --no-ansi) ANSI output
  -n, --no-interaction             Do not ask any interactive question
  -v|vv|vvv, --verbose             Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug

Examples

JetBrains IDE (IntelliJ IDEA, PhpStorm, WebStorm, etc)

One of the unique features of the tool is converting reports to a unit test format compatible with JetBrains IDE and TeamCity. The following examples show how to use JetBrains IDE UI to display any kind of style issues. Yeah, I know that the integration is not the cleanest, and it's not super beautiful. However, this code/screenshots demonstrate the usability of the approach.

PHPcs in JetBrains PhpStorm

NOTE: I believe that coding style issues have the same level of severity as any other sort of errors. Therefore, I prefer to use the same workflow to check the quality of the code as I do with regular PHPUnit tests. It's like "just one click!" Also, you will have the awesome bonus - navigating the project and gives the most detailed information about errors.

The general idea is pretty simple:

  • We take almost any utility for testing.
  • It saves report in the file or outputs error to StdOut as xml/json.
  • CI-Report-Converter changes the report format. It saves result somewhere or just outputs it in StdOut.
  • ???
  • Profit.

In the next example we will see how to integrate JetBrains IDE UI with Code Sniffer deeply. I use PHPcs just as example. This is the most popular linter in PHP. However, the approach is independent of the programming language or unit testing framework.

<?php declare(strict_types=1);

use PHPUnit\Framework\Assert;
use PHPUnit\Framework\TestCase;

/**
 * Here's an example based on PHPUnit and PhpStorm, but you are not limited to PHP language.
 * This is just an example to show the idea.
 * So you can use any sort language to integrate style tests with JetBrains IDE.
 */
class CheckStyleExamplesTest extends TestCase
{
    /**
     * The short example which uses pipe as way to pass error report.
     */
    public function testPipelineWay(): void
    {
        echo shell_exec(                       # Execute command via shell and return the complete output as a string.
            'php ./vendor/bin/phpcs' .         # Path to bin of third-party tool (PHP Code Sniffer is just example).
            ' --report=checkstyle' .           # Output format of PHPcs. ci-report-converter expects it by default as `--input-format` option.
            ' --standard=PSR12 -q ./src' .     # The custom tool options. For phpcs `-q` is important!
            ' | ' .                            # The pipe operator, it passes the output of one command as input to another. See https://en.wikipedia.org/wiki/Pipeline_(Unix)
            ' ./ci-report-converter.phar'      # The converter does all the magic. Look at help description ( --help) to lean more about options and default values.
        );

        # Usually PHPUnit expects at least one assert in a test.
        # Otherwise, it may show useless warning messages. It depends on PHPUnit version and your configurations.
        # So, just in case, we make a fake assertion.
        Assert::assertTrue(true);
    }

    /**
     * The super detailed example which uses files as way to pass error report.
     */
    public function testXmlFileWay(): void
    {
        shell_exec(                                     # Execute command via shell and return the complete output as a string.
            'php ./vendor/bin/phpcs' .                  # Path to bin of third-party tool (PHP Code Sniffer is just example).
            ' --report=checkstyle' .                    # Output format of PHPcs. CI-Report-Converter expects it by default as `--input-format` option.
            ' --report-file=./build/phpcs-report.xml' . # Save result of phpcs work in XML file in "checkstyle" format.
            ' --standard=PSR12 -q ./src' .              # The custom tool options. For phpcs `-q` is important!
            ' || true > /dev/null'                      # We don't expect any output of phpcs and ignore error exit codes.
                                                        # Lol, we are very self-confident. Actually, we need only XML file, that's it.
        );

        // I've shown all the options explicitly just to add transparency.
        // In fact, this example does the same thing as the code above in `testPipelineWay()`.
        echo shell_exec(
            './ci-report-converter.phar' .              # The path to bin file of CI-Report-Converter. It depends of your installation way. See above.
            ' --input-format=checkstyle' .              # Source reporting format. Default value is "checkstyle".
            ' --input-file=./build/phpcs-report.xml' .  # Using prepared file on previous step as source.
            ' --output-format=tc-tests' .               # Target reporting format. Default value is "tc-tests".
            ' --suite-name=PHPcs' .                     # Define the name of group. See screenshot below.
            ' --root-path=`pwd`'                        # Specify the project path for pretty printing paths in UI. Default value is `.` (current dir). 
        );

        # The same reason like in `testPipelineWay()`.
        Assert::assertTrue(true);
    }
}

What happens under the hood. Also, see source file as ready-to-use examples.

cd  ~/your/project/root/directory
php ./vendor/bin/phpunit ./tests/examples/CheckStyleExamples.php --teamcity

# or
php ./vendor/bin/phpcs --report=checkstyle --standard=PSR12 -q ./src | ./ci-report-converter.phar

Mess Detector (phpmd-json)

Screenshot

PHPmd in JetBrains PhpStorm

php ./vendor/bin/phpmd ./src json cleancode | ./ci-report-converter.phar -Sphpmd-json

Magic Number Detector (phpmnd)

Screenshot

PHPmd in JetBrains PhpStorm

php ./vendor/bin/phpmnd ./src --hint --quiet --xml-output=./build/phpmnd-report.xml
./ci-report-converter.phar --root-path=./src --input-file=./build/phpmnd-report.xml --input-format=phpmnd 

Copy/Paste Detector (pmd-cpd)

Screenshot

PHPcpd in JetBrains PhpStorm

php ./vendor/bin/phpcpd ./src --log-pmd=./build/phpcpd-report.xml
./ci-report-converter.phar --input-file=./build/phpcpd-report.xml --input-format=pmd-cpd

PHPStan (checkstyle)

Screenshot

PHPstan in JetBrains PhpStorm

php ./vendor/bin/phpstan analyse --error-format=checkstyle --no-progress ./src | ./ci-report-converter.phar

Psalm (psalm-json)

Screenshot

PHP Psalm in JetBrains PhpStorm

php ./vendor/bin/psalm --output-format=json | ./ci-report-converter.phar --input-format="psalm-json"

Phan (checkstyle)

Screenshot

PHP Psalm in JetBrains PhpStorm

php ./vendor/bin/phan.phar --directory=./src --output-mode=checkstyle |  ./ci-report-converter.phar

TeamCity - Style Issue As Failed Unit test

Use the option --output-format=tc-tests to convert any style report to unit test format in TeamCity

Style issues as failed tests in TeamCity

TeamCity - Style Issue As Code Inspections

Use the option --output-format=tc-inspections to convert any style report to unit test format in TeamCity

Style Issue As Code Inspections in TeamCity

TeamCity - Reported Statistic Values

PHPDepend.xml => Example in TeamCity

./vendor/bin/pdepend.phar --summary-xml="./build/pdepend-summary.xml" --quiet ./src
./ci-report-converter.phar teamcity:stats --input-format="pdepend-xml" --input-file="./build/pdepend-summary.xml"

PHPloc.json => Example in TeamCity

./vendor/bin/phploc ./src --log-json="./build/phploc.json" --quiet
./ci-report-converter.phar teamcity:stats --input-format="phploc-json" --input-file="./build/phploc.json"

PHPUnitClover.xml => Example in TeamCity

./vendor/bin/phpunit --coverage-clover ./build/phpunit-report.xml
./ci-report-converter.phar teamcity:stats --input-format="phpunit-clover-xml" --input-file="./build/phpunit-report.xml"

JUnit.xml => Example in TeamCity

./vendor/bin/phpunit --coverage-xml ./build/phpunit-junit.xml
./ci-report-converter.phar teamcity:stats --input-format="junit-xml" --input-file="./build/phpunit-junit.xml"

PHPMetrics.xml => Example in TeamCity

./vendor/bin/phpmetrics  ./src --report-violations="./build/phpmetrics-report.xml"
./ci-report-converter.phar teamcity:stats --input-format="phpmetrics-xml" --input-file="./build/phpmetrics-report.xml"

Style issues as failed tests in TeamCity

GitHub Actions

You can find a lot of life examples here and see real examples in YML file

GitHub Actions 1

GitHub Actions 1

GitLab CI

Use the option --output-format=gitlab-json to convert the report to Gitlab JSON format.

Als, see how to implemente GitLab Custom Report

GitLab Code Quality Report

Use tool as SDK to generate reports

Also, you can use source of tool as SDK for reports in your tools/project.

PS: More examples are coming soon.

JUnit.xml (API)

use JBZoo\CiReportConverter\Converters\JUnitConverter;
use JBZoo\CiReportConverter\Formats\Source\SourceCaseOutput;
use JBZoo\CiReportConverter\Formats\Source\SourceSuite;

$class = \JBZoo\PHPUnit\ExampleTest::class;
$className = str_replace('\\', '.', $class);
$filename = './tests/ExampleTest.php';
$line = 28;

$suite = new SourceSuite('Suite');
$case = $suite->addTestCase('Test Name');
$case->time = 0.001824;
$case->file = $filename;
$case->line = $line;
$case->class = $class;
$case->classname = $className;
$case->assertions = 5;
$case->stdOut = 'Some std output';
$case->errOut = 'Some err output';
$case->failure = new SourceCaseOutput('Failure', 'Failure Message', 'Failure Details');
$case->error = new SourceCaseOutput('Error', 'Error Message', 'Error Details');
$case->warning = new SourceCaseOutput('Warning', 'Warning Message', 'Warning Details');
$case->skipped = new SourceCaseOutput('Skipped', 'Skipped Message', 'Skipped Details');

echo (new JUnitConverter())->fromInternal($suite);
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="Suite" tests="1" assertions="5" errors="1" warnings="1" failures="1" skipped="1" time="0.001824">
    <testcase name="Test Name" class="JBZoo\PHPUnit\ExampleTest" classname="JBZoo.PHPUnit.ExampleTest" file="./tests/ExampleTest.php" line="28" assertions="5" time="0.001824">
      <failure type="Failure" message="Failure Message">Failure Details</failure>
      <warning type="Warning" message="Warning Message">Warning Details</warning>
      <error type="Error" message="Error Message">Error Details</error>
      <system-out>Some std output
Some err output</system-out>
      <skipped/>
    </testcase>
  </testsuite>
</testsuites>

GitHub Actions (API)

use JBZoo\CiReportConverter\Formats\GithubActions\GithubActions;

$ghActions = new GithubActions();
$case0 = $ghActions->addCase('src/Root.php');
$case0->line = 789;
$case0->column = null;
$case0->message = 'Something went wrong #0';

$suite1 = $ghActions->addSuite('src/File.php');
$case1 = $suite1->addCase('src/Class.php');
$case1->line = 123;
$case1->column = 4;
$case1->message = 'Something went wrong #1';

echo $ghActions->__toString();
::error file=src/Root.php,line=789::Something went wrong #0

::group::src/File.php
::error file=src/Class.php,line=123,col=4::Something went wrong #1
::endgroup::

Contributing

# Fork the repo and build project
make build

# Make your local changes

# Run all tests and check code style
make test
make codestyle

# Create your pull request and check all tests in CI

License

MIT

See Also

  • Composer-Diff - See what packages have changed after composer update.
  • Composer-Graph - Dependency graph visualization of composer.json based on mermaid-js.
  • Mermaid-PHP - Generate diagrams and flowcharts with the help of the mermaid script language.
  • Utils - Collection of useful PHP functions, mini-classes, and snippets for every day.
  • Image - Package provides object-oriented way to manipulate with images as simple as possible.
  • Data - Extended implementation of ArrayObject. Use files as config/array.
  • Retry - Tiny PHP library providing retry/backoff functionality with multiple backoff strategies and jitter support.
  • SimpleTypes - Converting any values and measures - money, weight, exchange rates, length, ...
You might also like...
DeepCopy - Create deep copies (clones) of your objects
DeepCopy - Create deep copies (clones) of your objects

DeepCopy DeepCopy helps you create deep copies (clones) of your objects. It is designed to handle cycles in the association graph. Table of Contents H

Xplico is a Network Forensic Analisys Tool NFAT, for Unix and Unix-like operating systems

Xplico is a Network Forensic Analisys Tool NFAT, for Unix and Unix-like operating systems. It uses libpcap, a packet capture and filtering library.

Standards Alignment Tool

Standards Alignment Tool (SALT) [] Overview This is a prototype for testing the IMS Global Learning Consortium® CASE™ Specification and proving its us

A tool to automatically fix Twig Coding Standards issues

Twig CS Fixer Installation This standard can be installed with the Composer dependency manager. Add the coding standard as a dependency of your projec

A tool to automatically fix PHP Coding Standards issues
A tool to automatically fix PHP Coding Standards issues

PHP Coding Standards Fixer The PHP Coding Standards Fixer (PHP CS Fixer) tool fixes your code to follow standards; whether you want to follow PHP codi

GPT-3 powered business idea generator

The idea (lol) is to give you inspiration to make something cool, if you lack inspiration right now. Many ideas might not be perfect but they might give you the spark to start thinking to get to a really good idea further on.

Converts any PocketMine-MP 3.0 extended blocks into PM4 native blocks!

ExtendedBlocksConverter Converts any PocketMine-MP 3.0 extended blocks into PM4 native blocks! Yes, you heard right, this plugin can convert any lefto

This is a plugin based on Vokuro ACL idea

IMPORATNT! You should switch to branch v3.0.0 We have switched to facebook/graph-sdk 5.4 ! $ composer require crada/phalcon-user-plugin:^3.0 Phalcon U

JSONFinder - a library that can find json values in a mixed text or html documents, can filter and search the json tree, and converts php objects to json without 'ext-json' extension.

JSONFinder - a library that can find json values in a mixed text or html documents, can filter and search the json tree, and converts php objects to json without 'ext-json' extension.

Comments
  • Output format

    Output format "plain" doesn't work

    Using on Github actions as:

          - uses: jbzoo/[email protected]
            with:
              input-file: ./build/mypy-report.xml
              input-format: junit
              output-format: plain
    

    it fails:

     In Convert.php line 105:
      [JBZoo\CiReportConverter\Commands\Exception]                                 
      Format "plain" not found. See the option "--output-format".                  
      Available options: checkstyle,gitlab-json,github-cli,junit,phpmd-json,phpmn  
      d,pmd-cpd,psalm-json,tc-inspections,tc-tests                                 
    Exception trace:
      at /app/src/Commands/Convert.php:105
     JBZoo\CiReportConverter\Commands\Convert->getFormat() at /app/src/Commands/Convert.php:82
     JBZoo\CiReportConverter\Commands\Convert->executeAction() at /app/src/Commands/AbstractCommand.php:50
     JBZoo\CiReportConverter\Commands\AbstractCommand->execute() at /app/vendor/symfony/console/Command/Command.php:255
     Symfony\Component\Console\Command\Command->run() at /app/vendor/symfony/console/Application.php:1009
     Symfony\Component\Console\Application->doRunCommand() at /app/vendor/symfony/console/Application.php:273
     Symfony\Component\Console\Application->doRun() at /app/vendor/symfony/console/Application.php:149
     Symfony\Component\Console\Application->run() at /app/ci-report-converter.php:45
     require_once() at /app/ci-report-converter:19
    convert [-S|--input-format INPUT-FORMAT] [-I|--input-file [INPUT-FILE]] [-T|--output-format OUTPUT-FORMAT] [-O|--output-file [OUTPUT-FILE]] [-R|--root-path [ROOT-PATH]] [-N|--suite-name SUITE-NAME] [-F|--tc-flow-id [TC-FLOW-ID]] [-Q|--non-zero-code [NON-ZERO-CODE]]
    
    opened by danielkza 3
Releases(2.10.0)
Owner
JBZoo Toolbox
Awesome developer tools and libs
JBZoo Toolbox
Check modules in app/code and vendor for PHP 8 compatibility status - PHP_CodeSniffer & php-compatibility standard

M2 PHP version compatibility check How To use Requires PHP 7.3+ | PHP 8 This app will run PHP_CodeSniffer with phpcompatibility/php-compatibility on t

William Tran 24 Oct 13, 2022
Better error reporting for PHP, and prettier too!

PHP Error | Improve Error Reporting for PHP THIS IS NO LOBGER MAINTAINED, Please leave me alone. PHP errors are not good enough for development, it's

Joseph Lenton 1.4k Dec 27, 2022
A super simple, clean and pretty error handler that replace the default error handler of PHP. You need only include this file!

php-custom-error-handler A super simple, clean and pretty error handler that replace the default error handler of PHP. You need just include only this

null 6 Nov 7, 2022
A Twitter bot powered by Github Actions tweeing Wikidata milestones

Twitter WikidataMeter Bot I'm the code, and deployment of Twitter bot WikidataMeter. I tweet a few fun things about Wikidata as it grows. Tweets Feel

null 9 Dec 9, 2022
This package is aimed to be a simplistic PHP workflow package that works in a similar fashion to GitHub Actions.

Workflow This package is aimed to be a simplistic PHP workflow package that works in a similar fashion to GitHub Actions. Installation To install this

Steve McDougall 4 Sep 26, 2022
GitHub Actions for Magento 2 Extensions

GitHub Actions for Magento 2 Extensions This repository's aim is to provide a set of open sourced GitHub actions to write better tested Magento 2 exte

ExtDN 104 Jan 1, 2023
This repository aims to build a fairly complete CI/CD example using GitHub workflows and actions.

CI/CD example This repository aims to build a fairly complete CI/CD example using GitHub workflows and actions. Keep in mind that the toolset used in

Robin Ingelbrecht 4 Nov 1, 2022
A high-level machine learning and deep learning library for the PHP language.

Rubix ML A high-level machine learning and deep learning library for the PHP language. Developer-friendly API is delightful to use 40+ supervised and

Rubix 1.7k Jan 1, 2023
Rubix ML - A high-level machine learning and deep learning library for the PHP language.

A high-level machine learning and deep learning library for the PHP language. Developer-friendly API is delightful to use 40+ supervised and

Rubix 1.7k Jan 6, 2023
A list of ICs and IPs for AI, Machine Learning and Deep Learning.

AI Chip (ICs and IPs) Editor S.T.(Linkedin) Welcome to My Wechat Blog StarryHeavensAbove for more AI chip related articles Latest updates Add news of

Shan Tang 1.4k Jan 3, 2023