[Numpy-discussion] ANN: numexpr 2.4 RC1

Christoph Gohlke cgohlke at uci.edu
Sun Apr 6 16:13:55 EDT 2014


On 4/6/2014 3:51 AM, Francesc Alted wrote:
> ===========================
>    Announcing Numexpr 2.4 RC1
> ===========================
>
> Numexpr is a fast numerical expression evaluator for NumPy.  With it,
> expressions that operate on arrays (like "3*a+4*b") are accelerated
> and use less memory than doing the same calculation in Python.
>
> It wears multi-threaded capabilities, as well as support for Intel's
> MKL (Math Kernel Library), which allows an extremely fast evaluation
> of transcendental functions (sin, cos, tan, exp, log...)  while
> squeezing the last drop of performance out of your multi-core
> processors.  Look here for a some benchmarks of numexpr using MKL:
>
> https://github.com/pydata/numexpr/wiki/NumexprMKL
>
> Its only dependency is NumPy (MKL is optional), so it works well as an
> easy-to-deploy, easy-to-use, computational engine for projects that
> don't want to adopt other solutions requiring more heavy dependencies.
>
> What's new
> ==========
>
> A new `contains()` function has been added for detecting substrings in
> strings.  Thanks to Marcin Krol.
>
> Also, there is a new version of setup.py that allows better management
> of the NumPy dependency during pip installs.  Thanks to Aleks Bunin.
>
> This is the first release candidate before 2.4 final would be out,
> so please give it a go and report back any problems you may have.
>
> In case you want to know more in detail what has changed in this
> version, see:
>
> https://github.com/pydata/numexpr/wiki/Release-Notes
>
> or have a look at RELEASE_NOTES.txt in the tarball.
>
> Where I can find Numexpr?
> =========================
>
> The project is hosted at GitHub in:
>
> https://github.com/pydata/numexpr
>
> You can get the packages from PyPI as well (but not for RC releases):
>
> http://pypi.python.org/pypi/numexpr
>
> Share your experience
> =====================
>
> Let us know of any bugs, suggestions, gripes, kudos, etc. you may
> have.
>
>
> Enjoy data!
>

Hi Francesc,

some minor issues building with msvc/MKL on Windows:

* the `numexpr\win32` folder is missing in the PyPI package 
numexpr-2.4-rc1.tar.gz
* `numexpr\interpreter.cpp` uses POSIX syslog.h, which is not available 
with msvc9/10
* `numexpr\interpreter.cpp` uses the C99 fminl function, which is not 
available with msvc9/10

I'll submit a PR later for the latter two.

Some tests fail on Python 3 (attached).

Christoph

-------------- next part --------------
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Numexpr version:   2.4-rc1
NumPy version:     1.8.1
Python version:    3.3.5 (v3.3.5:62cf4e77f785, Mar  9 2014, 10:35:05) [MSC v.1600 64 bit (AMD64)]
AMD/Intel CPU?     True
VML available?     True
VML/MKL version:   Intel(R) Math Kernel Library Version 11.1.2 Product Build 20140122 for Intel(R) 64 architecture appli
cations
Number of threads used by default: 8 (out of 8 detected cores)
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
........EEEEEEEEE........EEEEEEEEE......................................................................................
<snip>
======================================================================
ERROR: test_str_contains_basic0 (numexpr.tests.test_numexpr.test_numexpr)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 735, in evaluate
    compiled_ex = _numexpr_cache[numexpr_key]
KeyError: ('contains("abc", "ab")', (('optimization', 'aggressive'), ('truediv', False)), ())

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numexpr\tests\test_numexpr.py", line 166, in test_str_contains_basic0
    res = evaluate('contains("abc", "ab")')
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 738, in evaluate
    NumExpr(ex, signature, **context)
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 554, in NumExpr
    precompile(ex, signature, context)
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 511, in precompile
    constants_order, constants = getConstants(ast)
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 294, in getConstants
    for a in constants_order]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 294, in <listcomp>
    for a in constants_order]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 284, in convertConstantToKind
    return kind_to_type[kind](x)
TypeError: string argument without an encoding

======================================================================
ERROR: test_str_contains_basic1 (numexpr.tests.test_numexpr.test_numexpr)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numexpr\tests\test_numexpr.py", line 171, in test_str_contains_basic1
    res = evaluate('contains(haystack, "ab")')
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in evaluate
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in <listcomp>
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 629, in getType
    raise ValueError("unkown type %s" % a.dtype.name)
ValueError: unkown type str96

======================================================================
ERROR: test_str_contains_basic2 (numexpr.tests.test_numexpr.test_numexpr)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numexpr\tests\test_numexpr.py", line 176, in test_str_contains_basic2
    res = evaluate('contains("abcd", haystack)')
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in evaluate
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in <listcomp>
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 629, in getType
    raise ValueError("unkown type %s" % a.dtype.name)
ValueError: unkown type str96

======================================================================
ERROR: test_str_contains_basic3 (numexpr.tests.test_numexpr.test_numexpr)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numexpr\tests\test_numexpr.py", line 182, in test_str_contains_basic3
    res = evaluate('contains(haystacks, needles)')
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in evaluate
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in <listcomp>
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 629, in getType
    raise ValueError("unkown type %s" % a.dtype.name)
ValueError: unkown type str224

======================================================================
ERROR: test_str_contains_basic4 (numexpr.tests.test_numexpr.test_numexpr)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numexpr\tests\test_numexpr.py", line 188, in test_str_contains_basic4
    res = evaluate('contains("test abc here", needles)')
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in evaluate
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in <listcomp>
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 629, in getType
    raise ValueError("unkown type %s" % a.dtype.name)
ValueError: unkown type str160

======================================================================
ERROR: test_str_contains_basic5 (numexpr.tests.test_numexpr.test_numexpr)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numexpr\tests\test_numexpr.py", line 193, in test_str_contains_basic5
    res = evaluate('contains("test abc here", needles)')
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in evaluate
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in <listcomp>
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 629, in getType
    raise ValueError("unkown type %s" % a.dtype.name)
ValueError: unkown type str160

======================================================================
ERROR: test_str_contains_listproduct (numexpr.tests.test_numexpr.test_numexpr)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numexpr\tests\test_numexpr.py", line 246, in test_str_contains_listproduct
    res = [bool(x) for x in evaluate('contains(b, a)')]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in evaluate
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in <listcomp>
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 629, in getType
    raise ValueError("unkown type %s" % a.dtype.name)
ValueError: unkown type str192

======================================================================
ERROR: test_str_contains_withemptystr1 (numexpr.tests.test_numexpr.test_numexpr)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numexpr\tests\test_numexpr.py", line 251, in test_str_contains_withemptystr1
    res = evaluate('contains("abcd", withemptystr)')
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in evaluate
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in <listcomp>
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 629, in getType
    raise ValueError("unkown type %s" % a.dtype.name)
ValueError: unkown type str96

======================================================================
ERROR: test_str_contains_withemptystr2 (numexpr.tests.test_numexpr.test_numexpr)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numexpr\tests\test_numexpr.py", line 256, in test_str_contains_withemptystr2
    res = evaluate('contains(withemptystr, "")')
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in evaluate
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in <listcomp>
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 629, in getType
    raise ValueError("unkown type %s" % a.dtype.name)
ValueError: unkown type str96

======================================================================
ERROR: test_str_contains_basic0 (numexpr.tests.test_numexpr.test_numexpr2)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 735, in evaluate
    compiled_ex = _numexpr_cache[numexpr_key]
KeyError: ('contains("abc", "ab")', (('optimization', 'aggressive'), ('truediv', False)), ())

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numexpr\tests\test_numexpr.py", line 166, in test_str_contains_basic0
    res = evaluate('contains("abc", "ab")')
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 738, in evaluate
    NumExpr(ex, signature, **context)
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 554, in NumExpr
    precompile(ex, signature, context)
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 511, in precompile
    constants_order, constants = getConstants(ast)
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 294, in getConstants
    for a in constants_order]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 294, in <listcomp>
    for a in constants_order]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 284, in convertConstantToKind
    return kind_to_type[kind](x)
TypeError: string argument without an encoding

======================================================================
ERROR: test_str_contains_basic1 (numexpr.tests.test_numexpr.test_numexpr2)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numexpr\tests\test_numexpr.py", line 171, in test_str_contains_basic1
    res = evaluate('contains(haystack, "ab")')
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in evaluate
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in <listcomp>
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 629, in getType
    raise ValueError("unkown type %s" % a.dtype.name)
ValueError: unkown type str96

======================================================================
ERROR: test_str_contains_basic2 (numexpr.tests.test_numexpr.test_numexpr2)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numexpr\tests\test_numexpr.py", line 176, in test_str_contains_basic2
    res = evaluate('contains("abcd", haystack)')
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in evaluate
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in <listcomp>
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 629, in getType
    raise ValueError("unkown type %s" % a.dtype.name)
ValueError: unkown type str96

======================================================================
ERROR: test_str_contains_basic3 (numexpr.tests.test_numexpr.test_numexpr2)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numexpr\tests\test_numexpr.py", line 182, in test_str_contains_basic3
    res = evaluate('contains(haystacks, needles)')
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in evaluate
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in <listcomp>
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 629, in getType
    raise ValueError("unkown type %s" % a.dtype.name)
ValueError: unkown type str224

======================================================================
ERROR: test_str_contains_basic4 (numexpr.tests.test_numexpr.test_numexpr2)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numexpr\tests\test_numexpr.py", line 188, in test_str_contains_basic4
    res = evaluate('contains("test abc here", needles)')
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in evaluate
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in <listcomp>
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 629, in getType
    raise ValueError("unkown type %s" % a.dtype.name)
ValueError: unkown type str160

======================================================================
ERROR: test_str_contains_basic5 (numexpr.tests.test_numexpr.test_numexpr2)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numexpr\tests\test_numexpr.py", line 193, in test_str_contains_basic5
    res = evaluate('contains("test abc here", needles)')
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in evaluate
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in <listcomp>
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 629, in getType
    raise ValueError("unkown type %s" % a.dtype.name)
ValueError: unkown type str160

======================================================================
ERROR: test_str_contains_listproduct (numexpr.tests.test_numexpr.test_numexpr2)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numexpr\tests\test_numexpr.py", line 246, in test_str_contains_listproduct
    res = [bool(x) for x in evaluate('contains(b, a)')]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in evaluate
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in <listcomp>
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 629, in getType
    raise ValueError("unkown type %s" % a.dtype.name)
ValueError: unkown type str192

======================================================================
ERROR: test_str_contains_withemptystr1 (numexpr.tests.test_numexpr.test_numexpr2)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numexpr\tests\test_numexpr.py", line 251, in test_str_contains_withemptystr1
    res = evaluate('contains("abcd", withemptystr)')
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in evaluate
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in <listcomp>
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 629, in getType
    raise ValueError("unkown type %s" % a.dtype.name)
ValueError: unkown type str96

======================================================================
ERROR: test_str_contains_withemptystr2 (numexpr.tests.test_numexpr.test_numexpr2)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numexpr\tests\test_numexpr.py", line 256, in test_str_contains_withemptystr2
    res = evaluate('contains(withemptystr, "")')
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in evaluate
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 730, in <listcomp>
    signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)]
  File "X:\Python33\lib\site-packages\numexpr\necompiler.py", line 629, in getType
    raise ValueError("unkown type %s" % a.dtype.name)
ValueError: unkown type str96

----------------------------------------------------------------------
Ran 5431 tests in 5.938s

FAILED (errors=18)


More information about the NumPy-Discussion mailing list