[Numpy-discussion] Decision tree-like algorithm on numpy arrays

[Francesc Alted]

A Friday 07 May 2010 08:18:44 Martin Raspaud escrigué:
> Francesc Alted skrev:
> > Hi Martin,
> 
> [...]
> 
> > and the output for my machine:
> >
> > result_array1: [4 2 4 ..., 1 3 4] 1.819
> > result_array2: [4 2 4 ..., 1 3 4] 0.308
> >
> > which is a 6x speed-up.  I suppose this should be pretty close of what
> > you can get with C.
> 
> Hi Francesc,
> Thanks a lot for the idea !
> This looks nice, I wasn't aware of numexpr.
> 
> I guess it's no problem to run "evaluate" on user defined functions ?

No problem.  You only have to express your functions in terms of numexpr 
expressions.  Look at this simple example:

#------------------------------------------------------------------
import numpy as np
import numexpr as ne

N = 1e7
array1 = np.random.random(N)
array2 = np.random.random(N)
array3 = np.random.random(N)

# An user-defined function
def some_test_on(arr, value):
    return arr > value

result_array1 = np.where(some_test_on(array1, 0.6),
                         np.where(some_test_on(array2, 0.5), 1, 2),
                         np.where(some_test_on(array3, 0.3), 3, 4))

print "result_array1:", result_array1

# The same user-defined function than above,
# but return a numexpr expression instead
def ne_some_test_on(arr, value):
    return "(" + arr + " > %s" % value + ")"

expr1 = "where("+ne_some_test_on("array2", 0.5) + ", 1, 2)"
expr2 = "where("+ne_some_test_on("array3", 0.3) + ", 3, 4)"
expr = "where("+ne_some_test_on("array1", 0.6) + ", %s, %s)" % (expr1, expr2)

result_array2 = ne.evaluate(expr)
print "result_array2:", result_array2

assert np.allclose(result_array1, result_array2)
#------------------------------------------------------------------

and the output:

result_array1: [2 3 4 ..., 3 1 3]
result_array2: [2 3 4 ..., 3 1 3]

-- 
Francesc Alted