I met a leading \0 string when running tests, is this normal?

There was 1 failure:

1) Tuupola\Base62\Base62Test::testShouldEncodeAndDecodeRandomBytes
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-Binary String: 0x00486eea2de87439fc081b892616a3b0f1f098df86e2cdd23e7d21f5f046a30a1a6662fff6c3c017b1d4853a1fdd7dc00975016d9c2801b9df659fadc6abe1109b1e1f3960367603e75bb9ddf9d8097af5948f74df585d05bbee61aff992f3d35577e31aafce7d4342d3a68da0d5ca8d46bde2f7e7f555cf6a1938c4f52bdd43
+Binary String: 0x486eea2de87439fc081b892616a3b0f1f098df86e2cdd23e7d21f5f046a30a1a6662fff6c3c017b1d4853a1fdd7dc00975016d9c2801b9df659fadc6abe1109b1e1f3960367603e75bb9ddf9d8097af5948f74df585d05bbee61aff992f3d35577e31aafce7d4342d3a68da0d5ca8d46bde2f7e7f555cf6a1938c4f52bdd43

/private/tmp/base62/tests/Base62Test.php:41

FAILURES!

The Problem

When I tried to use this with my phpexpertsinc/ConciseUUID project, the tests all failed on low value UUIDs.

InvalidArgumentException: $bytes string should contain 16 characters.

Here is a test case:

use Ramsey\Uuid\Uuid;
use Tuupola\Base62Proxy as Base62;

$uuid = Uuid::fromBytes(Base62::decode($conciseUuid));

Here's the test data:

    $badUuids = [
            '0023a441-a3a3-4d9e-bd65-de3381c3a226' => '00GHs6XflJ51yCvZ4TwH4g',
            '1ee9a026-48ef-4592-9d87-88ceea7bc35e' => '0wKXIE87UgfjIvSPLkAHao',
            '0e0aa2a8-1a10-45e4-a67a-c97b9c5a7d19' => '0QUkgNC86JAY1A8JhVZ7iT',
            '1ad0d525-97c9-4c08-ad56-59acd47e3f7c' => '0obEi3noEliUnbTQbhMrLo',
   ];

The Solution

I solved this using ext-gmp via:

    // 3. We pad zeros to the beginning, as the result returned by gmp_strval after base conversion
    // is not always 22 characters long.
    $uuid = str_pad($uuid, 22, '0', STR_PAD_LEFT);

$hex = '123456abcdef';
Base62::encode($hex);  // "JngPBzse6O99qumM"
Base62::encode(hex2bin($hex)); // "5gOMRIbf"

I think it is better to add encodeHex and decodeHex methods, then we can encode hex directly like Base62::encodeHex($hex). And for internal, we may call encodeHex for string encoding like:

public function encodeHex($data)
{
    return ctype_xdigit($data) ? gmp_strval(gmp_init($data, 16), 62) : '';
}

public function encode($data, $integer = false)
{
    if (is_integer($data) || true === $integer) {
        return gmp_strval(gmp_init($data, 10), 62);
    }

    return $this->encodeHex(bin2hex($data));
}

Changed log

• add the suggested extension: ext-gmp and ext-bcmath.
• using the class-based PHPUnit namespace.
• consider the php-7.2 test will output some deprecated and warning message when using the PHPUnit version 4.8. See more details about the Travis build log.

When using this library in a project, I ran into an issue when I wrote a test that passed intentionally "bad data" to the decode method.

Specifically, when passing the invalid data to the decode method of the GmpEncoder, which uses gmp_init(), the GMP extension was throwing an unchecked warning about the data not being a valid number within the specified base (62).

I've recreated this error in the provided tests, along with a fix:

... Unfortunately, there's not a great way to check for this error at a higher level, due to GMP's usage of warnings and a false return value. Interestingly enough, I've actually run into this problem before when building my own library years ago, so I've actually reported this as a PHP bug, but it was closed with a "Won't Fix" status: https://bugs.php.net/bug.php?id=68002

In any case, I've created this fix to handle the issue for both the GMP encoder's case and for the base encoder, as otherwise not handling this case could lead to an invalid decode result or "data-loss".

Since the encoding algorithm is based on arithmetic division, the current algorithm is discarding 0x00 bytes at the beginning of the data stream.

Examples: 0x0001 -(encode)-> 0x31 -(decode)-> 0x01 0x00000000 -(encode)-> 0x00 -(decode)-> 0x00 0x303030 -(decode)-> 0x00

The fix I came up with involves chopping these leading zero-bytes from the data stream and appending their encoded (or decoded) form after performing the division. There's also a new test case for this corner case.

PS Fixes #4. Had not seen the issue.