Static Analysis Results Baseliner

Related tags

Code Analysis sarb
Overview

Static Analysis Results Baseliner (SARB)

PHP versions: 7.3 to 8.1 Latest Stable Version License Total Downloads

Continuous Integration Scrutinizer Code Quality Type coverage PHPStan level 8 PHPStan level 8 Code Coverage Infection MSI

Why SARB?

If you've tried to introduce advanced static analysis tools (e.g. Psalm, PHPStan) to legacy projects the tools have probably reported thousands of problems. It's unrealistic to fix all but the most critical ones before continuing development.

SARB is used to create a baseline of these results. As work on the project progresses SARB can takes the latest static analysis results, removes those issues in the baseline and report the issues raised since the baseline. SARB does this, in conjunction with git, by tracking lines of code between commits. Currently SARB only supports git but it is possible to add support for other SCMs.

SARB is written in PHP, however it can be used to baseline results for any language and any static analysis tool.

Why not SARB?

SARB should not be used on greenfield projects. If you're lucky enough to work on a greenfield project make sure you fix all problems raised by static analysis as you go along.

Requirements

Currently SARB only supports projects that use git.

SARB requires PHP >= 7.3 to run. The project being analysed does not need to run PHP 7.3 or even be a PHP project at all.

Installing

You can either add directly to the project you wish to run analysis on:

composer require --dev dave-liddament/sarb

Or you can install SARB globally (e.g. if you want to use it on a non PHP project):

composer global require dave-liddament/sarb

If you install globally make sure the composer bin directory is in your path.

Using SARB

If you're using version 0.x see the old documentation and how to upgrade.

1. Make sure the current git commit is the one to be used in the baseline

When creating the baseline, SARB needs to know the git commit SHA of the baseline. Make sure your code is in the state you want it to be in for the baseline and that the current commit represents that state.

2. Create the baseline

Run the static analyser of choice and pipe the results into SARB:

E.g. using Psalm's JSON output:

vendor/bin/psalm --output-format=json | vendor/bin/sarb create --input-format="psalm-json" psalm.baseline

This creates a baseline file called psalm.baseline. You'll want to check this in to your repository.

3. Update code and then use SARB to remove baseline results

Continue coding. Then rerun static analyser and pipe results into SARB:

vendor/bin/psalm --output-format=json | vendor/bin/sarb remove psalm.baseline

Running SARB from a global installation

If you are running SARB from a global installation you will need to specify the root of the project (where the .git directory lives). The above would become:

psalm --output-format=json | sarb create --project-root=/path/to/project/root --input-format="psalm-json" psalm.baseline

Supported tools

To see a list of supported tools and formats use:

vendor/bin/sarb list-static-analysis-tools

How to create and remove baseline for each supported tool:

PHP CodeSniffer

vendor/bin/phpcs src --report=json | vendor/bin/sarb create --input-format="phpcodesniffer-json" phpcs.baseline
vendor/bin/phpcs src --report=json | vendor/bin/sarb remove phpcs.baseline

Phan

vendor/bin/phan -m json | vendor/bin/sarb create --input-format="phan-json" phan.baseline
vendor/bin/phan -m json | vendor/bin/sarb remove phan.baseline

See notes on relative paths.

Exakat

-format sarb | vendor/bin/sarb remove exakat.baseline">
php exakat.phar report -p <project> -format sarb | vendor/bin/sarb create --input-format="exakat-sarb" exakat.baseline
php exakat.phar report -p <project> -format sarb | vendor/bin/sarb remove exakat.baseline

PHPMD

| vendor/bin/sarb remove phpmd.baseline">
vendor/bin/phpmd src json <ruleset> | vendor/bin/sarb create --input-format="phpmd-json" phpmd.baseline
vendor/bin/phpmd src json <ruleset> | vendor/bin/sarb remove phpmd.baseline

Psalm

vendor/bin/psalm --output-format=json | vendor/bin/sarb create --input-format="psalm-json" psalm.baseline
vendor/bin/psalm --output-format=json | vendor/bin/sarb remove psalm.baseline

NOTE: Checkout Psalm's built in baseline feature. Learn how it differs from SARB.

PHPStan

vendor/bin/phpstan analyse --error-format=json | vendor/bin/sarb create --input-format="phpstan-json" phpstan.baseline
vendor/bin/phpstan analyse --error-format=json | vendor/bin/sarb remove phpstan.baseline

NOTE: Checkout PHPStan's built in baseline feature. Learn how it differs from SARB.

PHP Magic Number Detector

vendor/bin/phpmnd . | vendor/bin/sarb create --input-format="phpmnd" phpmnd.baseline
vendor/bin/phpmnd . | vendor/bin/sarb remove phpmnd.baseline

See notes on relative paths.

My tool isn't supported...

That's no problem there are 3 methods to integrate a static analysis tool with SARB.

Output formats

The format for showing issues after the baseline is removed can be specified using --output-format option. Possible values are: table, text, json or github (for Github actions).

SARB with Github Actions

If you're using actions/checkout@v2 to checkout your code you'll need to add set fetch-depth to 0. By default checkout only gets that latest state of the code and none of the history. SARB uses git, which needs the full git history, to track file changes since the baseline. To get the full history checked out use this:

- uses: actions/checkout@v2
  with:
    fetch-depth: 0

Also don't forget to use the SARB option --output-format=github. It will annotate your PR with any issues that have been added since the baseline.

Further Reading

Authors

Comments
  • Introduction of the concept of Severity in order to distinguish errors from warnings.

    Introduction of the concept of Severity in order to distinguish errors from warnings.

    It seems that SARB is only aware of the concept of Error; it doesn't know anything about warnings or what it should do with them.

    In its most simplistic form, I could say that a warning (or a list of warnings) should be shown, but the tool should return success instead of failure, in case the baseline contains only warnings.

    Since this is a fairly complex feature to implement and extra caution must be taken in order not to introduce any BC-breaking changes, I thought it would be better to ask, is submitting a Pull Request that introduces the concept of severity to SARB, has any meaning.

    Thanks in advance, Alex.

    opened by MisterIcy 6
  • False positive: error introduced since baseline

    False positive: error introduced since baseline

    Hi, i am using PHPStan (tried either with text and json format) on file with error, and just copied it as latest errors, created sarb baseline and got false positive.

    Steps to reproduce:

    Output of vendor/bin/phpstan analyse --error-format=raw > reports/phpstan_errors_baseline.txt (phpstan_errors_baseline.txt):

    /app/presenters/ExitPresenter.php:165:Cannot access property $brand on bool|Klarka\Item\Item.
    

    cp reports/phpstan_errors_baseline.txt reports/phpstan_errors_latest.txt to simulate error was not fixed.

    Create baseline: vendor/bin/sarb create-baseline reports/phpstan_errors_baseline.txt reports/sarb_baseline.json phpstan-text-tmp

    And then compare: vendor/bin/sarb remove-baseline-results reports/phpstan_errors_latest.txt reports/sarb_baseline.json reports/errors_since_baseline.txt -f

    (false positive) Output is:

    Baseline uses ResultsParser [phpstan-text-tmp] and HistoryAnalyser [git]
    Errors before baseline 1
    Errors in baseline 1
    Errors introduced since baseline 1
    
    FILE: app/presenters/ExitPresenter.php
    +------+---------------------------------------------------------+
    | Line | Description                                             |
    +------+---------------------------------------------------------+
    | 165  | Cannot access property $brand on bool|Klarka\Item\Item. |
    +------+---------------------------------------------------------+
    

    Here is content of sarb_baseline.json if that helps:

    {"historyAnalyser":"git","historyMarker":"8f7184af1d682f5ba2acff3e06c2b2196d6f5afc","resultsParser":"phpstan-text-tmp","analysisResults":[{"lineNumber":165,"fileName":"app\/presenters\/ExitPresenter.php","type":"Cannot access property $brand on","message":"Cannot access property $brand on bool|Klarka\\Item\\Item.","fullDetails":"\/app\/presenters\/ExitPresenter.php:165:Cannot access property $brand on bool|Klarka\\Item\\Item."}]}
    

    Do you have any idea what am i doing wrong?

    opened by JanMikes 6
  • Add support php code sniffer json format

    Add support php code sniffer json format

    Add support for php code sniffer json format.

    @DaveLiddament Got any better idea how to deal with the closures in PhpCodeSnifferJsonResultsParser::convertToString()? I need information from the full details, and didn't find anything else...

    opened by temp 5
  • Add Code_Sniffer

    Add Code_Sniffer

    It will be nice to add Code_Sniffer here. Why? The same thing as phpstan. If applying for old project, it can show you tons of issues that you have in your code.

    enhancement 
    opened by vladyslavstartsev 4
  • webmozart/path-util usages

    webmozart/path-util usages

    In v1.4.0 webmozart/path-util was removed - https://github.com/DaveLiddament/sarb/pull/110

    Usages however still exist: https://github.com/DaveLiddament/sarb/search?q=PathUtil

    Resulting in:

    $ cat /tmp/phpcpd-sarb.json | php vendor/bin/sarb remove phpcpd.baseline
    Unexpected critical error: Class 'Webmozart\PathUtil\Path' not found
    
    opened by bytestream 3
  • False positive: error when using baseline with history on windows

    False positive: error when using baseline with history on windows

    Hi, I'm using phpcs with sarb on windows and encountered an issue today.

    Steps to reproduce:

    1. Create a file with an error
    2. Create baseline
    3. Add some lines (error now on a different line)
    4. Run phpcs and remove errors with sarb

    Expected behavior

    Still no errors (the history feature works in my pipeline and my docker containers, which run on linux)

    Actual behavior

    Errors showing, despite being in the baseline

    I'm working on a windows machine. Despite that git commands return \n as line breaks. This means that DaveLiddament\StaticAnalysisResultsBaseliner\Domain\HistoryAnalyser\UnifiedDiffParser\Parser on line 62 doesn't work for me. As far as I know git returns \n on all OS and not just linux. I attached the simplest change in a PR. I'm not quite sure if git is outputting somewhere something else than \n, but if we would want to make sure that would work too, the other proposal would be something like this: preg_split("/\r\n|\n|\r/", $diffAsString)

    opened by XonneX 3
  • [phpcs] Not in project root

    [phpcs] Not in project root

    Long directory paths in phpcs text reports are truncated

    FILE: ...eran/test/foo/bar/quz/test.php
    --------------------------------------------------------------------------------
    FOUND 1 ERROR AFFECTING 1 LINE
    

    Links:

    • https://github.com/squizlabs/PHP_CodeSniffer/blob/9c90cbfafe32be55a0b00e2bda4035d48f76a7d9/src/Reports/Full.php#L85
    • https://github.com/squizlabs/PHP_CodeSniffer/wiki/Configuration-Options#setting-the-default-report-width

    I'd be grateful if the PHPCS documentation was updated to account for this:

    vendor/bin/phpcs src/ > /tmp/phpcs.txt
    

    becomes

    vendor/bin/phpcs --report-width=auto src/ > /tmp/phpcs.txt
    

    Perhaps with a note explaining what --report-width is for in case the user has to run a different command, for example I used a modified command:

    ./vendor/bin/phpcs --report-width=auto --standard=phpcs.xml . > /tmp/phpcs.txt
    

    Edit

    Alternative workaround - use the undocumented JSON parser https://github.com/DaveLiddament/sarb/pull/60:

    $ ./vendor/bin/phpcs --report=json --standard=phpcs.xml . > /tmp/phpcs.txt
    $ ./vendor/bin/sarb create-baseline /tmp/phpcs.txt phpcs.baseline phpcodesniffer-json
    Baseline created
    Errors in baseline 12617
    
    opened by bytestream 3
  • PHPCS baseline empty

    PHPCS baseline empty

    Creating baseline for PHPCS results in empty one for me, vendor/bin/sarb create-baseline /tmp/phpcs.txt phpcs.baseline phpcodesniffer-full:

    Baseline created
    Errors in baseline 0
    

    I was going by https://github.com/DaveLiddament/sarb/blob/master/docs/PhpCodeSniffer.md

    Tried also json output for PHPCS but baseline just stays empty. My git repository is clean, everything committed.

    {"historyAnalyser":"git","historyMarker":"97fe7e1fb7c5be68bccb3f86ff97cfb4fe633f1e","resultsParser":"phpcodesniffer-full","analysisResults":[]}
    

    I have suspicion maybe I'm missing something.

    opened by Ludo444 3
  • Add support for deptrac

    Add support for deptrac

    Hi @DaveLiddament, you were in contact with some colleagues of mine at the Symfony Live in London. They told me you are interested in supporting deptrac with SARB. Deptrac supports the jUnit format we might could adapt here. There is still an issue https://github.com/sensiolabs-de/deptrac/issues/205 but should be fixable soon. Otherwise we could introduce another formatter. WDYT?

    opened by smoench 3
  • Unexpected critical error: Line number must be positive integer. Got: 0

    Unexpected critical error: Line number must be positive integer. Got: 0

    Hello!

    We had following error reported by Phpstan:

    ProfilePresenter.php:0:ParseError (syntax error, unexpected end of file, expecting function (T_FUNCTION) or const (T_CONST)) thrown while autoloading class Klarka\TagHide\TagHide.
    

    I understand what the error says and it is completely legitimate. Though when running SARB: vendor/bin/sarb remove-baseline-results reports/phpstseline.json reports/errors_since_baseline.txt

    There is following error: Unexpected critical error: Line number must be positive integer. Got: 0

    I think SARB should allow number 0 in line, because it is legit error and should be reported.

    What do you think?

    enhancement 
    opened by JanMikes 3
  • Remove webmozart/path-util usages

    Remove webmozart/path-util usages

    $ composer why webmozart/path-util
    vimeo/psalm  4.13.1  requires  webmozart/path-util (^2.3)  
    
    $ composer why vimeo/psalm
    dave-liddament/sarb   dev-master  requires (for development)  vimeo/psalm (^4.13)
    
    opened by bytestream 1
  • PHAR distribution by PHIVE

    PHAR distribution by PHIVE

    Please consider phar file distribution https://phar.io/

    1. create a phar file with https://github.com/humbug/box
    2. add GPG signature
    3. upload them as release attachments
    4. upload your public key to keyservers
    5. add a "sarb" alias to https://github.com/phar-io/phar.io/blob/master/data/repositories.xml
    opened by szepeviktor 0
  • Add support for infection.

    Add support for infection.

    Example output is below. The failures are listed in the 'Escaped mutants' section of the infection-log.txt file. For the example below the failure would be file /projects/danack/github/params/lib/Params/Params.php on line 36 and the error type would be OneZeroInteger.

    Escaped mutants:
    ================
    
    
    1) /projects/danack/github/params/lib/Params/Params.php:36    [M] OneZeroInteger
    
    --- Original
    +++ New
    @@ @@
         {
             foreach ($namedRules as $parameterName => $rules) {
                 // TODO - test for packed array?
    -            if (count($rules) === 0) {
    +            if (count($rules) === 1) {
                     throw new RulesEmptyException();
                 }
                 $firstRule = $rules[0];
    
    PHPUnit 8.1.2 by Sebastian Bergmann and contributors.
    
    ...                                                                 3 / 3 (100%)
    
    Time: 29 ms, Memory: 4.00 MB
    
    OK (3 tests, 13 assertions)
    
    Timed Out mutants:
    ==================
    
    Killed mutants:
    ===============
    
    
    1) /projects/danack/github/params/lib/Params/Exception/ValidationException.php:29    [M] OneZeroInteger
    
    opened by Danack 1
Releases(1.6.0)
  • 1.6.0(Dec 1, 2022)

    Adds concept of severity. Issues can be either ERROR or WARNING. By default both are reported. However it is possible to exclude warnings from the results by using the --ignore-warnings flag.

    Thanks to @MisterIcy who suggested the idea and came up with an initial implementation, from which this PR is inspired by.

    Source code(tar.gz)
    Source code(zip)
  • 1.5.0(Aug 7, 2022)

  • 1.4.1(Dec 24, 2021)

  • 1.4.0(Dec 23, 2021)

    New features

    • Adds SARB format to support tools that only provide relative and not absolute paths (#96)
    • Removed dependency on deprecated library webmozart/path (#110) (Thanks @a4blue and @jdressen)

    Other updates

    • Updated contributing notes for information about using libraries (#111)
    • Bumped PHPStan analysis level to max (#112)
    Source code(tar.gz)
    Source code(zip)
  • 1.3.0(Nov 29, 2021)

  • 1.2.0(Nov 25, 2021)

  • 1.1.0(Oct 1, 2021)

  • 1.0.1(Jul 13, 2021)

  • 1.0.0(Mar 4, 2021)

    Major updates including:

    See instructions and script to help upgrading from version 0.x.

    Improved UX

    Pipe output from static analysis tool straight to SARB (much easier than before). E.g.

    Create a baseline from PHP CodeSniffer output:

    phpcs  src --report=json | vendor/bin/sarb create  --input-format="phpcodesniffer-json" phpcs.baseline
    

    Show issues introduced since baseline made:

    phpcs  src --report=json | vendor/bin/sarb remove phpcs.baseline
    

    Updated documentation

    Including how to integrate SARB with almost any static analysis tool, even non PHP tools.

    Support for all maintained versions of PHP

    PHP 7.3, 7.4 and 8.0 supported.

    Tidy up and improved code coverage.

    Test coverage increased to 100%. Infection MSI ~90%.

    Source code(tar.gz)
    Source code(zip)
  • 0.8.0(Jul 28, 2020)

  • 0.5.1(Jul 17, 2020)

  • 0.7.0(Apr 16, 2020)

    Add support for PHP code sniffer's json format.

    Thanks to https://github.com/temp for PR #60 to add support for PHP Code Sniffer JSON format.

    Source code(tar.gz)
    Source code(zip)
  • 0.6.0(Jan 27, 2020)

  • 0.5.0(Oct 9, 2019)

    Support for tool agnostic sarb.json format

    Added support for a generic sarb.json format. Any static analysis tool that produces output in this format will be compatible with SARB. (#49)

    Minor updates

    • Fix typos (thanks @tonivdv)
    • Bump versions of dependancies
    • Add deptrac (#53) to enforce architectural constraints
    Source code(tar.gz)
    Source code(zip)
  • 0.4.1(Sep 15, 2019)

  • 0.4.0(Aug 9, 2019)

  • 0.3.3(Apr 28, 2019)

  • 0.3.2(Jan 8, 2019)

  • 0.3.1(Dec 31, 2018)

  • 0.3.0(Dec 30, 2018)

    Enhancements to SARB to improve developer experience (DX).

    NOTE: This is NOT compatible with baselines produced by earlier versions of SARB. You'll need to re-create baselines using this version of SARB.

    • FEATURE: Improve DX by providing summary of issues raised since baseline in CLI output.
    • FEATURE: Use new format for Psalm .txt. output which now includes violation type for each issues. Requires Psalm >= v3.0.6
    • FEATURE: Widen range of Symfony components. Now ^3.4 || ^4.0 for DI, config, console, process and yaml.
    • FIX: Compare baseline against current code NOT most recent commit
    Source code(tar.gz)
    Source code(zip)
  • 0.2.0(Dec 20, 2018)

    Adds ResultsParsers for:

    • Psalm text format psalm-text-tmp
    • PHPStan JSON format phpstan-json-tmp
    • PHPStan text formatphpstan-text-tmp

    NOTE: The above ResultsParsers are only interim solutions. The current implementations can not accurately determine the type of the static analysis violations, they use a violation type guesser which has limitations.

    An updated Psalm text formatter ResultsParser will be available once PR 1139 released.

    An update to the PHPStan ResultsParsers will be available once enhancement 1686 has been released.

    Source code(tar.gz)
    Source code(zip)
  • 0.1.0(Nov 28, 2018)

    Initial release of SARB.

    Check out the documentation for the problem SARB solves and how to use it.

    NOTE: This is still very much in beta.

    Source code(tar.gz)
    Source code(zip)
Owner
Dave Liddament
Dave Liddament
Beautiful and understandable static analysis tool for PHP

PhpMetrics PhpMetrics provides metrics about PHP project and classes, with beautiful and readable HTML report. Documentation | Twitter | Contributing

PhpMetrics 2.3k Dec 22, 2022
Static code analysis to find violations in a dependency graph

PhpDependencyAnalysis PhpDependencyAnalysis is an extendable static code analysis for object-oriented PHP-Projects to generate dependency graphs from

Marco Muths 546 Dec 7, 2022
Performs advanced static analysis on PHP code

PHP Analyzer Please report bugs or feature requests via our website support system ? in bottom right or by emailing [email protected]. Contri

Continuous Inspection 443 Sep 23, 2022
The Exakat Engine : smart static analysis for PHP

Exakat The Exakat Engine is an automated code reviewing engine for PHP. Installation Installation with the phar Phar is the recommended installation p

Exakat 370 Dec 28, 2022
A static php code analysis tool using the Graph Theory

Mondrian Ok guyz, you have a master degree in Graph Theory, you follow Law of Demeter and you live on S.O.L.I.D principles ? Let's have some Fun ! (^Ο‰

Florent Genette 391 Nov 30, 2022
A static analysis tool for finding errors in PHP applications

Psalm Psalm is a static analysis tool for finding errors in PHP applications. Installation To get started, check out the installation guide. Live Demo

Vimeo 5k Jan 2, 2023
Deptrac is a static code analysis tool for PHP that helps you communicate, visualize and enforce architectural decisions in your projects

Deptrac is a static code analysis tool for PHP that helps you communicate, visualize and enforce architectural decisions in your projects. You can freely define your architectural layers over classes and which rules should apply to them.

QOSSMIC GmbH 2.2k Dec 30, 2022
A static analysis tool for security

progpilot A static analyzer for security purposes Only PHP language is currently supported Installation Option 1: use standalone phar Download the lat

null 271 Dec 27, 2022
Infection Static Analysis Plugin

Static analysis on top of mutation testing - prevents escaped mutants from being invalid according to static analysis

Roave, LLC 108 Jan 2, 2023
A set of tools for lexical and syntactical analysis written in pure PHP.

Welcome to Dissect! master - this branch always contains the last stable version. develop - the unstable development branch. Dissect is a set of tools

Jakub LΓ©dl 221 Nov 29, 2022
A project to add Psalm support for Drupal for security testing, focused only on taint analysis.

psalm-plugin-drupal A Drupal integration for Psalm focused on security scanning (SAST) taint analysis. Features Stubs for sinks, sources, and sanitize

Samuel Mortenson 38 Aug 29, 2022
Phan is a static analyzer for PHP. Phan prefers to avoid false-positives and attempts to prove incorrectness rather than correctness.

Phan is a static analyzer for PHP that prefers to minimize false-positives. Phan attempts to prove incorrectness rather than correctness. Phan looks f

null 5.4k Jan 7, 2023
A static analyzer for PHP version migration

PHP Migration Readme in Chinese δΈ­ζ–‡ This is a static analyzer for PHP version migration and compatibility checking. It can suppose your current code ru

Yuchen Wang 194 Sep 27, 2022
Parse: A Static Security Scanner

Parse: A PHP Security Scanner PLEASE NOTE: This tool is still in a very early stage. The work continues... The Parse scanner is a static scanning tool

psec.io 342 Jan 2, 2023
SonarPHP: PHP static analyzer for SonarQube & SonarLint

Code Quality and Security for PHP This SonarSource project is a static code analyser for PHP language used as an extension for the SonarQube platform.

SonarSource 343 Dec 25, 2022
Sri Lanka Grade Exam Results Bot πŸ“š can collect Grade 5 O/L A/L Exam Results πŸ“š in Second Powerd By Sri lanka Department Of Examination πŸš€

Sri-Lanka-Exam-Results-Telegram-Bot How to use the bot /start : Start Sri Lanka Grade 5 O/L A/L Exam Results Bot. /help : More information about Sri L

GD Hiruna 8 Oct 2, 2022
Class used for health analysis results. It returns BMI, BMR, TDEE, Pollock 7 sinfolds stats (body fat, body density, lean mass, fat mass)

php-health Class used for health analysis results. It returns BMI, BMR, TDEE, Polock 7 sinfolds stats (body fat, body density, lean mass, fat mass) Th

Vinicius Marques de Souza 2 Oct 5, 2022
PHP Static Analysis Tool - discover bugs in your code without running it!

PHPStan - PHP Static Analysis Tool PHPStan focuses on finding errors in your code without actually running it. It catches whole classes of bugs even b

PHPStan 11.6k Dec 30, 2022
Beautiful and understandable static analysis tool for PHP

PhpMetrics PhpMetrics provides metrics about PHP project and classes, with beautiful and readable HTML report. Documentation | Twitter | Contributing

PhpMetrics 2.3k Dec 22, 2022
Static code analysis to find violations in a dependency graph

PhpDependencyAnalysis PhpDependencyAnalysis is an extendable static code analysis for object-oriented PHP-Projects to generate dependency graphs from

Marco Muths 546 Dec 7, 2022