This is a first PR in a series to improve the tests and test coverage for this package.

The PR will probably be easiest to review by looking at the commits individually.

Notes:

• All previously existing test cases, still exist.
• The additional new tests document the existing functionality. No changes were made to the functionality of the package.

Composer: improve PHPUnit version constraints

... as PHPUnit < 5.7.21 doesn't contain the forward compatibility layer with namespaced classes.

PHPUnit: improve configuration

• Add more detailed basic test configuration
• Unless junit is used somewhere, generating this report for code coverage is unnecessary.
• Generating the text report, however, is useful.

Tests: rename the test class/file to TokenizeTest

Realistically, all tests in the HighligherTest file are testing the handling of various token types and only incidentally testing the rest of the logic of the Highlighter class.

With that in mind, I'm proposing to rename the test file to make it more obvious what is being tested. This will then also allow for adding additional new test classes to test the rest of the functionality of the Highlighter class.

Includes adding minimal documentation and a minor method order tweak.

TokenizeTest: use nowdocs instead of heredocs

For most tests, the text within the heredoc should not be interpreted (interpolated). For those tests, it makes more sense to use nowdocs instead of heredocs.

Similar to single quoted versus double quoted strings, text within nowdocs is not interpolated (variables are not expanded), while within heredocs, they are.

Nowdocs are available since PHP 5.3, so can be safely used within this test suite.

Refs:

• https://www.php.net/manual/en/language.types.string.php#language.types.string.syntax.heredoc
• https://www.php.net/manual/en/language.types.string.php#language.types.string.syntax.nowdoc

TokenizeTest: move "empty" tests to new test setup with data providers

This sets up a new structure for this test class using test methods with data providers. All other existing tests will be converted to data sets in data providers in follow on commits.

TokenizeTest: add tests for PHP tags

Note that the new test data uses NOWDOC format instead of HEREDOC format.

TokenizeTest: move magic constant test cases to data provider

Note: this removes the looping from the test (which was a bad thing).

Advantages of using data providers compared to using a loop within the test function, are:

1. When one of the "test cases" fails, the rest of the tests cases will still be run (not so when using a loop). This prevents one test failure "hiding" behind another failure.
2. When there are multiple assertions is a test, it is often hard to determine which of the assertions (test cases) failed. Using a data provider - especially with named cases like used in this commit - will make the error coming from PHPUnit more descriptive and will make debugging which test case failed easier.
3. The test cases will now show (and be counted) as individual tests in progress reports and in testdox output.

Ref: https://phpunit.readthedocs.io/en/stable/writing-tests-for-phpunit.html#data-providers

TokenizeTest: move miscellaneous token test cases to data provider

TokenizeTest: add dedicated test for a T_STRING type token

TokenizeTest: move comment token test cases to data provider

TokenizeTest: add test cases for multi-line comments

TokenizeTest: move text string test cases to data provider

TokenizeTest: add additional test cases for text string handling

This adds extra tests for:

• Double quoted strings with interpolated variables.
• Nowdoc
• Heredocs with and without interpolated variables.

Note: for the nowdoc and heredoc tests, the test input is put in single quotes to prevent "nested here/nowdoc", which could skew the tests. These test cases also need to have a new line after the ; as otherwise PHP does not tokenize these correctly (and the highlighter is not concerned with fixing incorrect tokenization by PHP itself).

TokenizeTest: add test for inline HTML

TokenizeTest: move function call test cases to data provider

TokenizeTest: move keyword test case to data provider

TokenizeTest: add additional tests for keywords and operators

🆕 GH Actions: set short_open_tag ini setting for PHP 5.3 test run

The tests for this PR were failing after support for PHP 5.3 had been added back.

Reason being that the short PHP open echo tags <?= are only recognized as such with the ini setting short_open_tag turned on in PHP 5.3.

I would normally add this to the matrix to do a complete test run with and without short_open_tag, but as this only affects PHP 5.3, I think that's a little over the top.

Instead this commit adds a tweak to the test workflow to turn that ini setting on for the PHP 5.3 test run.

Ref: https://www.php.net/manual/en/ini.core.php#ini.short-open-tag

Currently installing this pulls in Jakub's now abandoned version, rather than php-parallel-lint/php-console-color. This throws the following warning on running composer install:

Package jakub-onderka/php-console-color is abandoned, you should avoid using it. Use php-parallel-lint/php-console-color instead.

This should probably be switched over.

Composer/Ant: move the build scripts to the Composer configuration

As this is a PHP project and the build steps are relatively simple, it will make it more intuitive for contributors to use Composer scripts to run CI checks, than to have to install Ant for this.

This commit creates the same build/CI scripts as were previously available via Ant in the Composer configuration file.

Notes:

• The scripts are set up to respect the PHP version used by Composer (instead of a system default).
• The scripts are set up to work equally well on Linux/Windows/Mac.
• The only two differences are:
  • The mkdir and file deletions won't automatically be done anymore. Generally speaking, the mkdir should only need to be run once anyway and the file deletes should not be necessary as the log files will just be overwritten by the tools, so I deemed this an acceptable trade-off.
  • The colors argument won't be set anymore. For most tools, this means, the tool will default to automatically determining whether the CLI used supports colourized output. As the determination of this in most tools has become pretty good, I, again, deemed this an acceptable trade-off.

CI: switch to GitHub Actions

This commit:

• Adds a GH Actions cs workflow for the code style and vardump checks which were previously run on Travis. These checks only need to run against one PHP version as the results will be the same independently of the PHP version these checks are run on.
• Adds a GH Actions test workflow for the linting and unit test checks which were previously run on Travis.
• Uses the cs2pr tool to display the results of the PHP linting and the results of the PHP code style check in-line in the file diff in the PR.
• Removes the, now redundant, .travis.yml configuration.
• Updates the .gitattributes file.
• Updates the "Build Status" badge in the Readme to use the results from the GH Actions runs.

Fixes #13 Fixes #22

Commit Details

PHPUnit: fix file name

The test file did not reflect the name of the class within it.

PHPUnit setup: rename and update the config file

It is recommended to use the phpunit.xml.dist file name for the PHPUnit configuration file to allow individual developers to overload it with a phpunit.xml file.

To this end, I've:

• Renamed the PHPUnit configuration file.
• Adjusted the corresponding export-ignore in .gitattributes.
• Added phpunit.xml to the .gitignore file so overload files from individual developers do not get committed.
• Adjusted the references to the file in the Ant build.xml file.

As for the config file itself:

• Added a name for the testsuite as is required in later PHPUnit versions.
• Changed the coverage filter directive to be more precise.
• Minor other tweaks.

PHPUnit: use a type-safe assertion

assertEquals() is to assertSame() as == is to ===, so always use assertSame() unless you need a loose comparison.

PHPUnit: make the tests platform independent

The tests would fail on non-*nix systems as the "expected output" contains hard-coded \n line endings, while the code in the Highlighter class uses PHP_EOL.

This allows the tests to run (and pass) on non-*nix platforms.

PHPUnit: widen the version requirements

Tested & found the unit tests working on PHPUnit 4.x - 7.x, so allow for those versions.

PHPUnit: use annotations for fixtures / cross-version compat up to PHPUnit 9.x

As of PHPUnit 8.x, the method signature for the setUpBeforeClass(), setUp(), tearDown() and tearDownAfterClass() fixture methods has changed to require the void return type.

As the void return type isn't available until PHP 7.1, this cannot be implemented.

Anntations to the rescue. By renaming the setUp() method to another, descriptive name and using the @before annotation, the tests can be made cross-version compatible up to PHPUnit 9.x.

With this change, the unit tests can now be run on PHPUnit 4 - 9.

As PHPUnit has a caching feature build in as of PHPUnit 8, we need to add the .phpunit.result.cache file to .gitignore.

Ant: fail the build on unit test failures

~~Travis: allow for installing PHPUnit on PHP 8~~

Unless there is a specific reason to use --prefer-source, always use --prefer-dist for the composer install command to allow Travis to optimally use the caching feature.

A number of predefined actions have had major release, which warrant an update to the workflow(s).

These updates don't actually contain any changed functionality, they are mostly just a change of the Node version used by the action itself (from Node 14 to Node 16).

Refs:

• https://github.com/actions/checkout/releases

• Updated the changelog to include all changes currently merged and in the 1.0.0 milestone.
• Simplified the changelog for the non-user facing changes.

Note: PR #40 is not currently included in the changelog and is not a user-facing change, so can be moved to the next milestone/release.

Tests: add tests covering namespaced names

PHP 8.0: handle changed tokenization of namespaced names

Namespaced names are now represented using the T_NAME_QUALIFIED (Foo\Bar), T_NAME_FULLY_QUALIFIED (\Foo\Bar) and T_NAME_RELATIVE (namespace\Foo\Bar) tokens. T_NS_SEPARATOR is only used for standalone namespace separators, and only syntactially valid in conjunction with group use declarations.

Ref: https://www.php.net/manual/en/migration80.incompatible.php#migration80.incompatible.tokenizer

This commit:

• Adds recognition for the new PHP 8.0 tokens for namespaced names and highlights these as TOKEN_DEFAULT, same as a non-namespaced T_STRING name.
• Adjusts the new unit tests to document the different highlighting between PHP < 8.0 and PHP 8.0+.

I've chosen not to make force the same highlighting across versions, but to let the highlighting respect the different tokenization across PHP versions.

The php-parallel-lint/php-console-color package has tagged a new release.

The most notable change is the namespace update.

This updates the PHP Console Highlighter package to use the new version of PHP Console Color.

Refs:

• https://github.com/php-parallel-lint/PHP-Console-Color/releases/tag/v1.0

Non-dev:

• PHP Console Color has released version 0.3 in May. I've set it to allow both 0.2 as well as 0.3 to prevent conflicts. Packages which use this package which may have the Console Color package explicitly set as a requirement.

Dev:

• PHP Parallel Lint has released version 1.2.0 in April.
• PHP Var dump check has released version 0.4 in April. I've set this one to be a flexible requirement to prevent having to update it on each new version.
• PHP_CodeSniffer has moved on a lot since version 1.5, so let's use a more recent one.

As it was, installing with PHP Console Color 0.3 was not allowed as Composer treats minors < 1.0 as majors. So updating to that version requires an update to the composer.json requirements.

For pre-1.0 versions it also acts with safety in mind and treats ^0.3 as >=0.3.0 <0.4.0.

Refs:

• https://getcomposer.org/doc/articles/versions.md#caret-version-range-
• https://github.com/php-parallel-lint/PHP-Console-Color/releases/tag/v0.3
• https://github.com/php-parallel-lint/PHP-Parallel-Lint/releases/tag/v1.2.0
• https://github.com/php-parallel-lint/PHP-Var-Dump-Check/releases/tag/v0.4

GitHub has deprecated the use of set-output (and set-state) in favour of new environment files.

This commit updates workflows to use the new methodology.

Refs:

• https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
• https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#environment-files