Mailman 3 April 2016 - NumPy-Discussion

Re: [Numpy-discussion] Starting work on ufunc rewrite
by Irwin Zaid 03 Apr '16

03 Apr '16

Hey guys, I figured I'd just chime in here. Over in DyND-town, we've spent a lot of time figuring out how to structure DyND callables, which are actually more general than NumPy gufuncs. We've just recently got them to a place where we are very happy, and are able to represent a wide range of computations. Our callables use a two-fold approach to evaluation. The first pass is a resolution pass, where a callable can specialize what it is doing based on the input types. It is able to deduce the return type, multidispatch, or even perform some sort of recursive analysis in the form of computations that call themselves. The second pass is construction of a kernel object that is exactly specialized to the metadata (e.g., strides, contiguity, ...) of the array. The callable itself can store arbitrary data, as can each pass of the evaluation. Either (or both) of these passes can be done ahead of time, allowing one to have a callable exactly specialized for your array. If NumPy is looking to change it's ufunc design, we'd be happy to share our experiences with this. Irwin On Thu, Mar 31, 2016 at 4:00 PM, Jaime Fernández del Río <jaime.frio at gmail.com <https://mail.scipy.org/mailman/listinfo/numpy-discussion>> wrote: >* I have started discussing with Nathaniel the implementation of the ufunc ABI *>* break that he proposed in a draft NEP a few months ago: *>>* http://thread.gmane.org/gmane.comp.python.numeric.general/61270 <http://thread.gmane.org/gmane.comp.python.numeric.general/61270> *>>* His original proposal was to make the public portion of PyUFuncObject be: *>>* typedef struct { *>* PyObject_HEAD *>* int nin, nout, nargs; *>* } PyUFuncObject; *>>* Of course the idea is that internally we would use a much larger struct that *>* we could change at will, as long as its first few entries matched those of *>* PyUFuncObject. My problem with this, and I may very well be missing *>* something, is that in PyUFunc_Type we need to set the tp_basicsize to the *>* size of the extended struct, so we would end up having to expose its *>* contents. This is somewhat similar to what now happens with PyArrayObject: *>* anyone can #include "ndarraytypes.h", cast PyArrayObject* to *>* PyArrayObjectFields*, and access the guts of the struct without using the *>* supplied API inline functions. Not the end of the world, but if you want to *>* make something private, you might as well make it truly private. *>>* I think it would be to have something similar to what NpyIter does:: *>>* typedef struct { *>* PyObject_HEAD *>* NpyUFunc *ufunc; *>* } PyUFuncObject; *>>* where NpyUFunc would, at this level, be an opaque type of which nothing *>* would be known. We could have some of the NpyUFunc attributes cached on the *>* PyUFuncObject struct for easier access, as is done in NewNpyArrayIterObject. *>* This would also give us more liberty in making NpyUFunc be whatever we want *>* it to be, including a variable-sized memory chunk that we could use and *>* access at will. NpyIter is again a good example, where rather than storing *>* pointers to strides and dimensions arrays, these are made part of the *>* NpyIter memory chunk, effectively being equivalent to having variable sized *>* arrays as part of the struct. And I think we will probably no longer trigger *>* the Cython warnings about size changes either. *>>* Any thoughts on this approach? Is there anything fundamentally wrong with *>* what I'm proposing here? *>>* Also, this is probably going to end up being a rewrite of a pretty large and *>* complex codebase. I am not sure that working on this on my own and *>* eventually sending a humongous PR is the best approach. Any thoughts on how *>* best to handle turning this into a collaborative, incremental effort? Anyone *>* who would like to join in the fun? *>>* Jaime *>>* -- *>* (\__/) *>* ( O.o) *>* ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de *>* dominación mundial. *>>* _______________________________________________ *>* NumPy-Discussion mailing list *>* NumPy-Discussion at scipy.org <https://mail.scipy.org/mailman/listinfo/numpy-discussion> *>* https://mail.scipy.org/mailman/listinfo/numpy-discussion <https://mail.scipy.org/mailman/listinfo/numpy-discussion> *>

2 1

Starting work on ufunc rewrite
by Jaime Fernández del Río 03 Apr '16

03 Apr '16

I have started discussing with Nathaniel the implementation of the ufunc ABI break that he proposed in a draft NEP a few months ago: http://thread.gmane.org/gmane.comp.python.numeric.general/61270 His original proposal was to make the public portion of PyUFuncObject be: typedef struct { PyObject_HEAD int nin, nout, nargs; } PyUFuncObject; Of course the idea is that internally we would use a much larger struct that we could change at will, as long as its first few entries matched those of PyUFuncObject. My problem with this, and I may very well be missing something, is that in PyUFunc_Type we need to set the tp_basicsize to the size of the extended struct, so we would end up having to expose its contents. This is somewhat similar to what now happens with PyArrayObject: anyone can #include "ndarraytypes.h", cast PyArrayObject* to PyArrayObjectFields*, and access the guts of the struct without using the supplied API inline functions. Not the end of the world, but if you want to make something private, you might as well make it truly private. I think it would be to have something similar to what NpyIter does:: typedef struct { PyObject_HEAD NpyUFunc *ufunc; } PyUFuncObject; where NpyUFunc would, at this level, be an opaque type of which nothing would be known. We could have some of the NpyUFunc attributes cached on the PyUFuncObject struct for easier access, as is done in NewNpyArrayIterObject. This would also give us more liberty in making NpyUFunc be whatever we want it to be, including a variable-sized memory chunk that we could use and access at will. NpyIter is again a good example, where rather than storing pointers to strides and dimensions arrays, these are made part of the NpyIter memory chunk, effectively being equivalent to having variable sized arrays as part of the struct. And I think we will probably no longer trigger the Cython warnings about size changes either. Any thoughts on this approach? Is there anything fundamentally wrong with what I'm proposing here? Also, this is probably going to end up being a rewrite of a pretty large and complex codebase. I am not sure that working on this on my own and eventually sending a humongous PR is the best approach. Any thoughts on how best to handle turning this into a collaborative, incremental effort? Anyone who would like to join in the fun? Jaime -- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.

3 3

Call for Proposals || PyCon India 2016
by Ayush Kesarwani 02 Apr '16

02 Apr '16

Hello Everyone The Call for Proposals (CFP) for PyCon India 2016, New Delhi are live now. We have started accepting proposals. Those interested to submit a proposal for a talk/proposal should submit the same at the given link [1]. More information about the event is present at the official website [2]. Kindly adhere to the guidelines mentioned for the submission of proposals. Please help us spread the word. Kindly use #inpycon in your social updates. Any queries regarding the CFP could be sent to contact(a)in.pycon.org . Regards Team InPycon [1] bit.ly/inpycon2016cfp [2] http://bit.ly/inpycon2016

1 0

openopt.org down for months?
by Pierre Haessig 01 Apr '16

01 Apr '16

Hello, I noticed some weeks (or months) ago that the openopt.org website is down. Today, discussing optimization packages in Python with a colleague , I noticed it is still down today. Has somebody reading the numpy list more information about the state of the OpenOpt project? Beyond the code which is still reachable on PyPI, I remember the website was a useful resource on the topic with its Wikipedia-style pages. I've investigated related websites and noticed that : * last PyPI update is August 2015 https://pypi.python.org/pypi/openopt * last tweet is July 2015 https://twitter.com/dmitrey15 (with tweets coming every other months or so until that). best, Pierre

1 0