ANN: ndtypes/xnd 0.2.0b1 released
stefan at bytereef.org
Sun Jan 21 11:50:49 EST 2018
this is the first release (beta) of two Python modules for describing and
manipulating unboxed in-memory data.
ndtypes is based on the libndtypes C-library. libndtypes has four main
1) Typing a memory block (abstract type).
2) Storing the exact layout information of a memory block (concrete type).
3) Constructing the type from a close variant of the datashape type language.
3) Dynamic type checking of memory blocks by pattern-matching an abstract
function signature against the types of concrete input arguments.
ndtypes implements all the heavy-lifting for typing and handling memory blocks
and is meant to be used as a foundation for other modules.
xnd is based on the libxnd C-library and the ndtypes module. It defines and
manages typed memory blocks. Because libndtypes has full layout information,
the xnd memory block is a small C-struct that contains bitmaps for optional
data, a linear index for addressing bitmaps, the type and the data pointer.
This makes it easy to implement lightweight memory views that support the
whole range of C types relevant for scientific computing.
Since libndtypes uses a standard algebraic data type to store recursive
type and layout information, mapping and unpacking nested Python values
directly to memory is straightforward.
Unless the type contains explicit pointer types like references, bytes or
strings, a single, pointer-free memory block is allocated.
List of xnd's features
1) Missing values (NA) are supported for all data. The bitmaps are
2) ndarrays with arbitrarily nested element types and optional values.
3) Ragged arrays using offset addressing with nested element types and
optional values. The ragged array type is compatible with the Arrow
4) Slicing and multi-dimensional indexing. Slicing and indexing returns
fully typed views on the data.
5) First class tuples and records with detailed explicit alignment
6) Constructor types (Joule(float32) != Ampere(float32)).
7) Reference types for compatibility with other memory block libraries.
8) Buffer protocol support for importing other memory blocks while
translating PEP-3118 format strings to datashape.
More information about the Python-announce-list