Hi all,
We are pleased to announce the release of the xtensor
library and its python bindings xtensor-python
, by Johan Mabille (@JohanMabille) and Sylvain Corlay (@SylvainCorlay).
xtensor
xtensor
is a C++ template library for manipulating multi-dimensional array expressions.
It provides
pybind
.More details on lazy computing with xtensor
are available below.
xtensor-cookiecutter
Besides, xtensor
and xtensor-python
, we provide a cookiecutter template project to help extension authors get started. The xtensor-cookiecutter
generates a simple project for a Python C++ extension with
setup.py
compiling the extension modulextensor
exposed to python, and the example of a vectorized function exposed to python.You can try the xtensor
live on the project website. The Try it Now button is powered by
Cling
C++ interpreter.Jupyter
notebook.Binder
project.xtensor
requires a modern C++ compiler supporting C++14. The following C+ compilers are supported:
xtensor
and xtensor-python
are header-only libraries. We provide packages for the conda package manager.
conda install -c conda-forge xtensor # installs xtensor
conda install -c conda-forge xtensor-python # installs xtensor and xtensor-python
Initialize a 2-D array and compute the sum of one of its rows and a 1-D array.
#include <iostream>
#include "xtensor/xarray.hpp"
#include "xtensor/xio.hpp"
xt::xarray<double> arr1
{{1.0, 2.0, 3.0},
{2.0, 5.0, 7.0},
{2.0, 5.0, 7.0}};
xt::xarray<double> arr2
{5.0, 6.0, 7.0};
xt::xarray<double> res = xt::make_xview(arr1, 1) + arr2;
std::cout << res;
Outputs:
{7, 11, 14}
Initialize a 1-D array and reshape it inplace.
#include <iostream>
#include "xtensor/xarray.hpp"
#include "xtensor/xio.hpp"
xt::xarray<int> arr
{1, 2, 3, 4, 5, 6, 7, 8, 9};
arr.reshape({3, 3});
std::cout << arr;
Outputs:
{{1, 2, 3},
{4, 5, 6},
{7, 8, 9}}
xtensor
We can operate on arrays of different shapes of dimensions in an elementwise fashion. Broadcasting rules of xtensor are similar to those of numpy and libdynd.
In an operation involving two arrays of different dimensions, the array with the lesser dimensions is broadcast across the leading dimensions of the other.
For example, if A
has shape (2, 3)
, and B
has shape (4, 2, 3)
, the result of a broadcasted operation with A
and B
has shape (4, 2, 3)
.
(2, 3) # A
(4, 2, 3) # B
---------
(4, 2, 3) # Result
The same rule holds for scalars, which are handled as 0-D expressions. If matched up dimensions of two input arrays are different, and one of them has size 1
, it is broadcast to match the size of the other. Let's say B has the shape (4, 2, 1)
in the previous example, so the broadcasting happens as follows:
(2, 3) # A
(4, 2, 1) # B
---------
(4, 2, 3) # Result
With xtensor
, if x
, y
and z
are arrays of broadcastable shapes, the return type of an expression such as x + y * sin(z)
is not an array. It is an xexpression
object offering the same interface as an N-dimensional array, which does not hold the result. Values are only computed upon access or when the expression is assigned to an xarray object. This allows to operate symbolically on very large arrays and only compute the result for the indices of interest.
We provide utilities to vectorize any scalar function (taking multiple scalar arguments) into a function that will perform on xexpression
s, applying the lazy broadcasting rules which we just described. These functions are called xfunctions. They are xtensor
's counterpart to numpy's universal functions.
In xtensor
, arithmetic operations (+
, -
, *
, /
) and all special functions are xfunctions.
xexpression
s and Broadcasting IteratorsAll xexpression
s offer two sets of functions to retrieve iterator pairs (and their const
counterpart).
begin()
and end()
provide instances of xiterator
s which can be used to iterate over all the elements of the expression. The order in which elements are listed is row-major
in that the index of last dimension is incremented first.xbegin(shape)
and xend(shape)
are similar but take a broadcasting shape as an argument. Elements are iterated upon in a row-major way, but certain dimensions are repeated to match the provided shape as per the rules described above. For an expression e
, e.xbegin(e.shape())
and e.begin()
are equivalent.Two container classes implementing multi-dimensional arrays are provided: xarray
and xtensor
.
xarray
can be reshaped dynamically to any number of dimensions. It is the container that is the most similar to numpy arrays.xtensor
has a dimension set at compilation time, which enables many optimizations. For example, shapes and strides of xtensor
instances are allocated on the stack instead of the heap.xarray
and xtensor
container are both xexpression
s and can be involved and mixed in universal functions, assigned to each other etc...
Besides, two access operators are provided:
operator()
which can take multiple integral arguments or none.operator[]
which takes a single multi-index argument, which can be of size determined at runtime. operator[]
also supports access with braced initializers.The python bindings are built upon the pybind11 library, a lightweight header-only for creating bindings between the Python and C++ programming languages.
C++ code
#include <numeric> // std::accumulate
#include "pybind11/pybind11.h" // pybind11
#include "xtensor/xmath.hpp" // C++ universal functions
#include "xtensor-python/pyarray.hpp" // numpy bindings
double sum_of_sines(xt::pyarray<double> &m)
{
auto sines = xt::sin(m); // sines does not hold any value
return std::accumulate(sines.begin(), sines.end(), 0.0);
}
PYBIND11_PLUGIN(xtensor_python_test)
{
pybind11::module m("xtensor_python_test",
"Test module for xtensor python bindings");
m.def("sum_of_sines",
sum_of_sines,
"Return the sum of the sines");
return m.ptr();
}
Python Code
import numpy as np
import xtensor_python_test as xt
a = np.arange(15).reshape(3, 5)
s = xt.sum_of_sines(v)
s
Outputs
1.2853996391883833
C++ code
#include "pybind11/pybind11.h"
#include "xtensor-python/pyvectorize.hpp"
#include <numeric>
#include <cmath>
namespace py = pybind11;
double scalar_func(double i, double j)
{
return std::sin(i) - std::cos(j);
}
PYBIND11_PLUGIN(xtensor_python_test)
{
py::module m("xtensor_python_test",
"Test module for xtensor python bindings");
m.def("vectorized_func", xt::pyvectorize(scalar_func), "");
return m.ptr();
}
Python Code
import numpy as np
import xtensor_python_test as xt
x = np.arange(15).reshape(3, 5)
y = [1, 2, 3]
z = xt.vectorized_func(x, y)
z
Outputs
[[-1. , 0.30116, 1.32544, 1.13111, -0.10315],
[-1.95892, -0.81971, 1.07313, 1.97935, 1.06576],
[-1.54402, -1.54029, -0.12042, 1.41016, 1.64425]]