[Numpy-discussion] Tensor Contraction (HPTT) and Tensor Transposition (TCL)

Paul Springer pavdev at gmx.de
Wed Aug 16 05:39:26 EDT 2017


> What version of Numpy are you comparing to? Note that in 1.13 you can
> enable some optimization in einsum, and the coming 1.14 makes that the
> default and uses CBLAS when possible.
I was using 1.10.4; however, I am currently running the benchmark with
1.13.1 and 'optimize=True'; this, however, seems to yield even worse
performance (see attached).
If you are interested, you can check the performance difference yourself
via: ./benchmark/python/bechmark.sh
> If you want to get it into Numpy, it would be worth checking if the
> existing functions can be improved before adding new ones.
>
> Note that Numpy transposition method just rearranges the indices, so
> the advantage of actual transposition is to have better cache
> performance or allow direct use of CBLAS. I assume TCL uses some
> tricks to do transposition in a way that is more cache friendly?
HPTT is a sophisticated library for tensor transpositions, as such it
blocks the tensors such that (1) spatial locality can be exploited.
Moreover, (2) it uses explicit vectorization to take advantage of the
CPU's vector units.

TCL uses the Transpose-Transpose-GEMM-Transpose approach where all
tensors are flattened into matrices (via HPTT) and then contracted via
GEMM; the final result is eventually folded (via HPTT) into the desired
output tensor.

Would it be possible to expose HPTT and TCL as optional packages within
NumPY? This way I don't have to redo the work that I've already put into
those libraries.
> Might check the license if your work uses code from a publication.
As far as licenses are concerned that should not be a problem since I
wrote to code myself and it doesn't use code from publications other
than mine.

Best regards,
Paul
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20170816/b16d3461/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tcl_vs_numpy_vs_eigen.pdf
Type: application/pdf
Size: 111882 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20170816/b16d3461/attachment-0001.pdf>


More information about the NumPy-Discussion mailing list