[Numpy-discussion] ANN: numexpr 2.4.3 released
njs at pobox.com
Mon Apr 27 19:42:26 EDT 2015
On Mon, Apr 27, 2015 at 4:23 PM, Neil Girdhar <mistersheik at gmail.com> wrote:
> I was told that numba did similar ast parsing, but maybe that's not true.
> Regarding the ast, I don't know about reliability, but take a look at
> get_ast in pyautodiff:
> It looks up the __file__ attribute and passes that through compile to get
> the ast. Of course that won't work when you don't have source code (a .pyc
> only module, or when else?)
> Since I'm looking into this kind of solution for the future of my code, I'm
> curious if you think that's too unreliable for some reason?
I'd certainly hesitate to rely on it for anything I cared about or
would be used by a lot of people... it's just intrinsically pretty
hacky. No guarantee that the source code you find via __file__ will
match what was used to compile the function, doesn't work when working
interactively or from the ipython notebook, etc. Or else you have to
trust a decompiler, which is a pretty serious complex chunk of code
just to avoid typing quote marks.
> From a
> usability standpoint, I do think that's better than feeding in strings,
> * are not syntax highlighted, and
> * require porting code from regular numpy expressions to numexpr strings
> (applying a decorator is so much easier).
Yes, but then you have to write a program that knows how to port code
from numpy expressions to numexpr strings :-). numexpr only knows a
tiny restricted subset of Python...
The general approach I'd take to solve these kinds of problems would
be similar to that used by Theano or dask -- use regular python source
code that generates an expression graph in memory. E.g. this could
def do_stuff(arr1, arr2):
arr1 = deferred(arr1)
arr2 = deferred(arr2)
arr3 = np.sum(arr1 + (arr2 ** 2))
return force(arr3 / np.sum(arr3))
Nathaniel J. Smith -- http://vorpus.org
More information about the NumPy-Discussion