Hello. I'd like to propose adding an alternative to the standard matrix multiplication, the multiplication using Strassen algorithm, which asymptotic complexity is lower, O(n^2.81) vs ϴ(n^3)

I can't install tensor in a PHP 7.2 Alpine image with pecl.

Steps to reproduce inside the docker image (docker run --rm -it php:7.2-cli-alpine sh)

apk update
apk add $PHPIZE_DEPS lapack-dev libexecinfo-dev openblas-dev
pecl install tensor

Here's the last few lines of the output:

 cc -O3 -ffast-math -I. -I/tmp/pear/temp/tensor/ext -DPHP_ATOM_INC -I/tmp/pear/temp/pear-build-defaultuserDNKfcM/tensor-2.1.4/include -I/tmp/pear/temp/pear-build-defaultuserDNKfcM/tensor-2.1.4/main -I/tmp/pear/temp/tensor/ext -I/usr/local/include/php -I/usr/local/include/php/main -I/usr/local/include/php/TSRM -I/usr/local/include/php/Zend -I/usr/local/include/php/ext -I/usr/local/include/php/ext/date/lib -DHAVE_CONFIG_H -g -O2 -c /tmp/pear/temp/tensor/ext/tensor/tensor.zep.c  -fPIC -DPIC -o tensor/.libs/tensor.o
/bin/sh /tmp/pear/temp/pear-build-defaultuserDNKfcM/tensor-2.1.4/libtool --mode=compile cc -O3 -ffast-math -I. -I/tmp/pear/temp/tensor/ext -DPHP_ATOM_INC -I/tmp/pear/temp/pear-build-defaultuserDNKfcM/tensor-2.1.4/include -I/tmp/pear/temp/pear-build-defaultuserDNKfcM/tensor-2.1.4/main -I/tmp/pear/temp/tensor/ext -I/usr/local/include/php -I/usr/local/include/php/main -I/usr/local/include/php/TSRM -I/usr/local/include/php/Zend -I/usr/local/include/php/ext -I/usr/local/include/php/ext/date/lib  -DHAVE_CONFIG_H  -g -O2   -c /tmp/pear/temp/tensor/ext/tensor/vector.zep.c -o tensor/vector.lo
 cc -O3 -ffast-math -I. -I/tmp/pear/temp/tensor/ext -DPHP_ATOM_INC -I/tmp/pear/temp/pear-build-defaultuserDNKfcM/tensor-2.1.4/include -I/tmp/pear/temp/pear-build-defaultuserDNKfcM/tensor-2.1.4/main -I/tmp/pear/temp/tensor/ext -I/usr/local/include/php -I/usr/local/include/php/main -I/usr/local/include/php/TSRM -I/usr/local/include/php/Zend -I/usr/local/include/php/ext -I/usr/local/include/php/ext/date/lib -DHAVE_CONFIG_H -g -O2 -c /tmp/pear/temp/tensor/ext/tensor/vector.zep.c  -fPIC -DPIC -o tensor/.libs/vector.o
/bin/sh /tmp/pear/temp/pear-build-defaultuserDNKfcM/tensor-2.1.4/libtool --mode=compile cc -O3 -ffast-math -I. -I/tmp/pear/temp/tensor/ext -DPHP_ATOM_INC -I/tmp/pear/temp/pear-build-defaultuserDNKfcM/tensor-2.1.4/include -I/tmp/pear/temp/pear-build-defaultuserDNKfcM/tensor-2.1.4/main -I/tmp/pear/temp/tensor/ext -I/usr/local/include/php -I/usr/local/include/php/main -I/usr/local/include/php/TSRM -I/usr/local/include/php/Zend -I/usr/local/include/php/ext -I/usr/local/include/php/ext/date/lib  -DHAVE_CONFIG_H  -g -O2   -c /tmp/pear/temp/tensor/ext/tensor/columnvector.zep.c -o tensor/columnvector.lo
 cc -O3 -ffast-math -I. -I/tmp/pear/temp/tensor/ext -DPHP_ATOM_INC -I/tmp/pear/temp/pear-build-defaultuserDNKfcM/tensor-2.1.4/include -I/tmp/pear/temp/pear-build-defaultuserDNKfcM/tensor-2.1.4/main -I/tmp/pear/temp/tensor/ext -I/usr/local/include/php -I/usr/local/include/php/main -I/usr/local/include/php/TSRM -I/usr/local/include/php/Zend -I/usr/local/include/php/ext -I/usr/local/include/php/ext/date/lib -DHAVE_CONFIG_H -g -O2 -c /tmp/pear/temp/tensor/ext/tensor/columnvector.zep.c  -fPIC -DPIC -o tensor/.libs/columnvector.o
/bin/sh /tmp/pear/temp/pear-build-defaultuserDNKfcM/tensor-2.1.4/libtool --mode=compile cc -O3 -ffast-math -I. -I/tmp/pear/temp/tensor/ext -DPHP_ATOM_INC -I/tmp/pear/temp/pear-build-defaultuserDNKfcM/tensor-2.1.4/include -I/tmp/pear/temp/pear-build-defaultuserDNKfcM/tensor-2.1.4/main -I/tmp/pear/temp/tensor/ext -I/usr/local/include/php -I/usr/local/include/php/main -I/usr/local/include/php/TSRM -I/usr/local/include/php/Zend -I/usr/local/include/php/ext -I/usr/local/include/php/ext/date/lib  -DHAVE_CONFIG_H  -g -O2   -c /tmp/pear/temp/tensor/ext/tensor/decompositions/cholesky.zep.c -o tensor/decompositions/cholesky.lo
/tmp/pear/temp/pear-build-defaultuserDNKfcM/tensor-2.1.4/libtool: line 1283: can't create tensor/decompositions/cholesky.loT: nonexistent directory
mkdir tensor/decompositions/.libs
mkdir: can't create directory 'tensor/decompositions/.libs': No such file or directory
make: *** [Makefile:252: tensor/decompositions/cholesky.lo] Error 1
ERROR: `make' failed

We have decided to direct some efforts toward researching a new Tensor 3.0 extension with the API code written in Zephir and optimized code written in C with the goal of drastically accelerating tensor operations. The Tensor 2.0 extension gave us a 2.5X speedup over the PHP implementation, this research is to develop a set of minimum features necessary to achieve at least a 10X speedup. Possible ways to achieve this are BLAS library integration, cache-efficient blocking algorithms, a fixed or strided array implementation, and vectorized operations. Optional but desirable enhancements to the extension include n-d array operations and CPU and GPU parallelization. In addition, incremental backwards compatible optimizations can be applied to the 2.0.0 branch as part of this research.

With a 10X+ speedup over the PHPland code we should be able to implement more advanced deep learning models including LSTMs and potentially Convnets (if we figure out a sensible n-d array implementation).

Other things to consider ...

• Will it be easier to reverse-engineer the Zephir-generated code or write it from scratch?
• Will it be easier/better to interface to TensorFlow under the hood?

Resources

• https://github.com/RubixML/Tensor/tree/master/ext
• https://www.zend.com/resources/writing-php-extensions
• https://github.com/numpy/numpy

modColumnVector does not return a correct result and PHPUnit is also expecting the wrong values. The true mod (checked using NumPy) for the implemented test (Tests/MatrixTest::modColumnVector) is:

[
     [ 2. ,  0.5,  2. ],
     [-0. , -0. , -0. ],
     [ 0.8,  3.6,  0.6]
 ]

PHP modulus operand "%" is made for integer only operations, witch can cause unexpected results when used for floating points.

fmod for some reason also failed to return the correct value and it is also famous for having problems with floating points, so it had to be implemented manually.

Without a respective lib, linking would inevitably fail. We also add a check for cblas.h, and if either is not found, disable ext/tensor with a respective warning.

See also https://github.com/RubixML/Tensor/issues/22.

When using the tensor PHP extension with PHP 8.1, at startup PHP displays the warnings reported at #28

For example. by simply running an empty PHP code, here's what we have:

Deprecated: Return type of Tensor\Vector::offsetGet($index) should either be compatible with ArrayAccess::offsetGet(mixed $offset): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in Unknown on line 0

Deprecated: Return type of Tensor\Vector::getIterator() should either be compatible with IteratorAggregate::getIterator(): Traversable, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in Unknown on line 0

Deprecated: Return type of Tensor\Matrix::getIterator() should either be compatible with IteratorAggregate::getIterator(): Traversable, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in Unknown on line 0

What about making it so tests fails in this case?

After some failed tests with CRubix interface I realized the expected eigenvalues and vectors from MatrixTest are actually wrong compared to other libraries. The following matrix was used:

[
      [22, -17, 12],
      [4, 11, -2],
      [20, -6, -9],
]

Tensor with JAMA

$values = [25.108706520450326, -15.096331148319537, 13.9876246278692];
$vectors = [
            [-0.5029346679560592, -0.1309992382037118, -0.33107976181279675],
            [0.15580805853732102, -0.08643645234319261, -0.6918777439682378],
            [0.8501650243704214, 0.987607178637524, 0.641631809310763],
        ];

Tensor with CArray

$values = [-15.09633115,  25.10870652,  13.98762463];
$vectors = [[  0.25848695, -0.86227193, -0.66844722 ]
            [ -0.11314538, -0.17721180, -0.61268791 ]
            [ -0.95936574,  -0.47442924, -0.42165370 ]];

Computed using LAPACK_dgeev

NumPy

values = [-15.09633115,  25.10870652,  13.98762463]
vectors = [[ 0.25848695, -0.86227193, -0.66844722],
           [-0.11314538, -0.1772118 , -0.61268791],
           [-0.95936574, -0.47442924, -0.4216537 ]]

Computed using LAPACK_dgeev

When using the tensor PHP extension with PHP 8.1 we have these PHP warnings:

Deprecated: Return type of Tensor\Vector::offsetGet($index) should either be compatible with ArrayAccess::offsetGet(mixed $offset): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in Unknown on line 0

Deprecated: Return type of Tensor\Vector::getIterator() should either be compatible with IteratorAggregate::getIterator(): Traversable, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in Unknown on line 0

Deprecated: Return type of Tensor\Matrix::getIterator() should either be compatible with IteratorAggregate::getIterator(): Traversable, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in Unknown on line 0

a simple fix is to simply add this line right before these methods:

#[\ReturnTypeWillChange]

See https://www.php.net/manual/en/migration81.incompatible.php#migration81.incompatible.core.type-compatibility-internal

I got three errors when trying to install the tensor package using pecl. Error 1: /tmp/pear/temp/tensor/ext/tensor.c:25:10: fatal error: cblas.h: No such file or directory Error 2: /tmp/pear/temp/tensor/ext/include/linear_algebra.c:7:10: fatal error: lapacke.h: No such file or directory Error 3:

/usr/bin/ld: cannot find -lgfortran
collect2: error: ld returned 1 exit status

The solution is to install libopenblas, libraries and gfortran.

apt-get install libopenblas-dev
apt-get install liblapacke-dev
apt-get install gfortran

Singular Value Decomposition (SVD) is used in a few machine learning methods such as Latent Semantic Analysis (LSA), collaborative filtering recommender systems, and more. It generalizes the eigendecomposition of a square normal matrix to any m x n matrix via an extension of the polar decomposition. This ticket is to research the best SVD algorithm (or set of algorithms) to implement that covers the two aforementioned use cases. Note that we'll need both PHP and Zephir/C implementations of SVD, however, for the prototype it would be perfectly fine to focus on one (PHP) or the other (Zephir/C).

https://en.wikipedia.org/wiki/Singular_value_decomposition

https://en.wikipedia.org/wiki/Matrix_factorization_(recommender_systems)

https://en.wikipedia.org/wiki/Latent_semantic_analysis

https://numpy.org/devdocs/reference/generated/numpy.linalg.svd.html

The code we use for eigenvalue/eigenvector computation is ancient and I have no idea how it works. I feel we can do better or at least if someone knew what they were doing could refactor the monster algorithm. I believe the code was written in the PHP 4 days.

Here's what we're working with currently

https://github.com/RubixML/Tensor/blob/master/lib/JAMA/EigenvalueDecomposition.php

In case you need to move to PHP 8.2 before @andrewdalpino find's time to update Tensor:

https://github.com/myfluxi/Tensor/commit/a6c229b762af5227a85db295e1d7451ab7edc8e9

Hello, How is the research on https://github.com/RubixML/DL/issues/1 going? Whole repository seems to be archived few years ago.. Will there ever be a GPU support on PHP with Tensor extension? Or is there already an alternative to it?

Guys ar Tensorflow, according to https://discuss.tensorflow.org/t/tensorflow-for-php/1373 are not planning to implement it as independant wrapper. Instead they are advising to run TensorFlow Serving server for computing and interact with it through REST API

Would be glad to hear your thoughts on what is current globally best approach to utilize GPU while using PHP. Can't find much info on internet aswell, most posts are 4years old..

Multi-threading using RubixML/Tensor is a great oportunity, but CPU performance still can't compare to GPU, especially with specialised matrix multiplication cores from nvidia RTX.

Versions according to https://pkgs.alpinelinux.org/packages

.Dockerfile for pecl install tensor

FROM php:7.4-fpm-alpine

RUN apk update && apk --no-cache add --update \
    # Optional To Compile Extension
    # A C compiler v12.1.1_git20220630-r5
    autoconf gcc musl-dev g++ \
    # A Fortran compiler v12.1.1_git20220630-r5
    bash bash-doc bash-completion musl-dev gfortran gdb make \
    # OpenBLAS development package v0.3.21-r0
    openblas-dev \
    # LAPACKE C interface to LAPACK v3.10.1-r0
    lapack \
    # re2c v3.0-r0
    re2c \
    # make v4.3-r0
    make \
    # autoconf v2.71-r0
    autoconf \
    # automake v1.16.5-r1
    automake \
    # build-essentials for Alpine 
    alpine-sdk build-base
    # execinfo.h missing error: 
    # libexecinfo-dev

RUN pecl install tensor

Error from pecl install tensor:

#0 10.20 /tmp/pear/temp/tensor/ext/kernel/backtrace.c:15:10: fatal error: execinfo.h: No such file or directory
#0 10.20    15 | #include <execinfo.h>
#0 10.20       |          ^~~~~~~~~~~~
#0 10.20 compilation terminated.
#0 10.20 make: *** [Makefile:202: kernel/backtrace.lo] Error 1
#0 10.20 ERROR: `make' failed

Fixed by apk add libexecinfo-dev https://github.com/awslabs/aws-lambda-cpp/issues/124#issuecomment-968302646

But then other error pops up:

#0 85.60 /usr/lib/gcc/x86_64-alpine-linux-musl/11.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find -llapacke: No such file or directory
#0 85.61 collect2: error: ld returned 1 exit status
#0 85.61 make: *** [Makefile:289: tensor.la] Error 1
#0 85.62 ERROR: `make' failed

musl/bin/ld file is there.

bash-5.1# ls /usr/lib/gcc/x86_64-alpine-linux-musl/11.2.1/../../../../x86_64-alpine-linux-musl/bin/ -l
total 4936
-rwxr-xr-x    2 root     root         59536 Jul  7 13:57 ar
-rwxr-xr-x    2 root     root        659448 Jul  7 13:57 as
-rwxr-xr-x    4 root     root       1469904 Jul  7 13:57 ld
-rwxr-xr-x    4 root     root       1469904 Jul  7 13:57 ld.bfd
-rwxr-xr-x    2 root     root         52392 Jul  7 13:57 nm
-rwxr-xr-x    2 root     root        157808 Jul  7 13:57 objcopy
-rwxr-xr-x    2 root     root        332720 Jul  7 13:57 objdump
-rwxr-xr-x    2 root     root         59568 Jul  7 13:57 ranlib
-rwxr-xr-x    2 root     root        620928 Jul  7 13:57 readelf
-rwxr-xr-x    2 root     root        157808 Jul  7 13:57 strip

Also tried building from source - same error on make step pops up.

.Dockerfile for build tensor from source

RUN apk update && apk --no-cache add --update \
    # A C compiler v12.1.1_git20220630-r5
    autoconf gcc musl-dev g++ cmake \
    # A Fortran compiler v12.1.1_git20220630-r5
    bash bash-doc bash-completion musl-dev gfortran gdb make \
    # OpenBLAS development package v0.3.21-r0
    openblas-dev \
    # LAPACKE C interface to LAPACK v3.10.1-r0
    lapack \
    # re2c v3.0-r0
    re2c \
    # make v4.3-r0
    make \
    # autoconf v2.71-r0
    autoconf \
    # automake v1.16.5-r1
    automake \
    # build-essentials for Alpine 
    alpine-sdk build-base \
    # execinfo.h missing: https://github.com/awslabs/aws-lambda-cpp/issues/124#issuecomment-968302646
    libexecinfo-dev

# RUN pecl install tensor

RUN git clone https://github.com/RubixML/Tensor Tensor

WORKDIR /var/www/html/Tensor/ext

RUN phpize \
    && ./configure \
    && make \
    && make install

Error from build tensor from source:

#0 86.23 /usr/lib/gcc/x86_64-alpine-linux-musl/11.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find -llapacke: No such file or directory
#0 86.27 collect2: error: ld returned 1 exit status
#0 86.28 make: *** [Makefile:289: tensor.la] Error 1

What am I missing here?

here is the error i'm getting: /bin/sh /home/trendalix1/Tensor/ext/libtool --mode=compile cc -O3 -ffast-math -I. -I/home/trendalix1/Tensor/ext -DPHP_ATOM_INC -I/home/trendalix1/Tensor/ext/include -I/home/trendalix1/Tensor/ext/main -I/home/trendalix1/Tensor/ext -I/opt/cpanel/ea-php74/root/usr/include/php -I/opt/cpanel/ea-php74/root/usr/include/php/main -I/opt/cpanel/ea-php74/root/usr/include/php/TSRM -I/opt/cpanel/ea-php74/root/usr/include/php/Zend -I/opt/cpanel/ea-php74/root/usr/include/php/ext -I/opt/cpanel/ea-php74/root/usr/include/php/ext/date/lib -DHAVE_CONFIG_H -g -O2 -c /home/trendalix1/Tensor/ext/include/linear_algebra.c -o include/linear_algebra.lo libtool: compile: cc -O3 -ffast-math -I. -I/home/trendalix1/Tensor/ext -DPHP_ATOM_INC -I/home/trendalix1/Tensor/ext/include -I/home/trendalix1/Tensor/ext/main -I/home/trendalix1/Tensor/ext -I/opt/cpanel/ea-php74/root/usr/include/php -I/opt/cpanel/ea-php74/root/usr/include/php/main -I/opt/cpanel/ea-php74/root/usr/include/php/TSRM -I/opt/cpanel/ea-php74/root/usr/include/php/Zend -I/opt/cpanel/ea-php74/root/usr/include/php/ext -I/opt/cpanel/ea-php74/root/usr/include/php/ext/date/lib -DHAVE_CONFIG_H -g -O2 -c /home/trendalix1/Tensor/ext/include/linear_algebra.c -fPIC -DPIC -o include/.libs/linear_algebra.o /home/trendalix1/Tensor/ext/include/linear_algebra.c:7:21: fatal error: lapacke.h: No such file or directory #include <lapacke.h> ^ compilation terminated. make: *** [include/linear_algebra.lo] Error 1

I can see that i have the lapacke.h files on the server with : find / -name lapacke.h which returns:

/usr/include/openblas/lapacke.h
/usr/include/lapacke/lapacke.h

I also tried adding the include paths so it would find it with:

export PATH="$HOME/usr/include:$PATH"
export PATH="$HOME/usr/include/openblas:$PATH"
export PATH="$HOME/usr/include/lapacke:$PATH"
export PATH="$HOME/include:$PATH"

but it doesn't seem to find the file. Sorry if this is something obvious, i don't spend much time in the command line.

Thanks, Mark

All versions fail to compile

fatal error C1083: Cannot open include file: 'cblas.h': No such file or directory

See https://windows.php.net/downloads/pecl/releases/tensor/