Hello, First, I'd like to report a bug. It seems ndarray does not implement tp_traverse or tp_clear, so if you have a reference cycle in an ndarray with dtype object none of those objects will ever be collected. Secondly, please bear with me, I'm not a NumPy expert, but would it be possible to have NumPy export an API that can be called without the GIL? In the upcoming Cython release 0.16 (soon to be released) we will have what is called typed memoryviews [1], which allow you to obtain a typed view on any PEP 3118 buffer, and it allows you to do a lot more things without the GIL. e.g. it allows you to pass these views around, transpose them, slice them (in the same way as in NumPy but slightly more restricted, it doesn't support masks and such), index them etc without the GIL. However, there is a lot of good functionality in NumPy that is simply not accessible without going through a Python and GIL layer, only to have NumPy (possibly) release the GIL. So instead we could have an API that takes a C-level ndarray view descriptor (shape/strides/ndim (possibly suboffsets), base type description) that would do the actual work (and perhaps doesn't even need to know anything about Python) and won't need the GIL (this wouldn't work for the dtype object, of course). The Py_buffer struct comes to mind, but format strings are hard to deal with. It would be the caller's responsibility to do any necessary synchronization. This API would simply be wrapped by a Python API that gets this "view" from the PyArrayObject, and which may or may not decide to release the GIL, and call the nogil API. This wrapping API could even be written easily in Cython. As for exceptions and error handling, there are many ways we can think of doing this without requiring the GIL. One of the reasons that I think this is important is that when you're using cython.parallel [2] you don't want to hold the gil, but you do want your NumPy goodies. Cython re-implements a very small subset of NumPy to get you the core functionality, but to get back to NumPy world you have to acquire the GIL, convert your memoryview slice to an ndarray (using the buffer interface through numpy.asarray()) and then have NumPy operate on that. It's a pain to write and it's terrible for performance. Even if you forget the GIL part, there's still the (expensive and explicit) conversion. In general I think there might be many advantages to such functionality other than for Cython. There shouldn't really be a reason to tie NumPy only to the CPython platform. Anyway, what do you guys think, does this make any sense? Mark [1]: https://sage.math.washington.edu:8091/hudson/job/cython-docs/doclinks/1/src/... [2]: http://docs.cython.org/src/userguide/parallelism.html
30.10.2011 21:48, mark florisson kirjoitti:
First, I'd like to report a bug. It seems ndarray does not implement tp_traverse or tp_clear, so if you have a reference cycle in an ndarray with dtype object none of those objects will ever be collected.
Indeed, this is missing. http://projects.scipy.org/numpy/ticket/1003 If I recall correctly, there was something painful in implementing this for Numpy arrays, though...
Secondly, please bear with me, I'm not a NumPy expert, but would it be possible to have NumPy export an API that can be called without the GIL? In the upcoming Cython release 0.16 (soon to be released) we will have what is called typed memoryviews [1], which allow you to obtain a typed view on any PEP 3118 buffer, and it allows you to do a lot more things without the GIL. e.g. it allows you to pass these views around, transpose them, slice them (in the same way as in NumPy but slightly more restricted, it doesn't support masks and such), index them etc without the GIL.
The closest thing to making this to happen is the work made on porting Numpy to IronPython. Basically, a major part of that involved ripping the innards of Numpy out to a more reusable C library. It's been in a merge-limbo for some time now, however. -- Pauli Virtanen
On 30 October 2011 21:01, Pauli Virtanen <pav@iki.fi> wrote:
30.10.2011 21:48, mark florisson kirjoitti:
First, I'd like to report a bug. It seems ndarray does not implement tp_traverse or tp_clear, so if you have a reference cycle in an ndarray with dtype object none of those objects will ever be collected.
Indeed, this is missing. http://projects.scipy.org/numpy/ticket/1003
If I recall correctly, there was something painful in implementing this for Numpy arrays, though...
Secondly, please bear with me, I'm not a NumPy expert, but would it be possible to have NumPy export an API that can be called without the GIL? In the upcoming Cython release 0.16 (soon to be released) we will have what is called typed memoryviews [1], which allow you to obtain a typed view on any PEP 3118 buffer, and it allows you to do a lot more things without the GIL. e.g. it allows you to pass these views around, transpose them, slice them (in the same way as in NumPy but slightly more restricted, it doesn't support masks and such), index them etc without the GIL.
The closest thing to making this to happen is the work made on porting Numpy to IronPython. Basically, a major part of that involved ripping the innards of Numpy out to a more reusable C library. It's been in a merge-limbo for some time now, however.
Ah, that's too bad. Is it anywhere near ready, or was it abandoned for ironclad? Could you point me to the code?
-- Pauli Virtanen
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
31.10.2011 09:44, mark florisson kirjoitti: [clip]
Ah, that's too bad. Is it anywhere near ready, or was it abandoned for ironclad? Could you point me to the code?
It's quite ready and working, and as far as I understand, Enthought is shipping it. I haven't used it, though. The code is here: https://github.com/numpy/numpy-refactor Pauli
Mark: I'm just wondering what you wanted to do with NumPy from Cython -- a stopgap solution for SIMD, iterator support, or something else? SIMD using NumPy really isn't the best idea long-term because of all the temporaries needed in compound expressions, which is really bad on the memory bus for anything but tiny arrays. For that I'd rather look at finding a nogil core of numexpr or similar. Of course, there is a number of convenient NumPy utility functions which would be cool to have in nogil mode... But given that the GIL is a problem in so many cases, I wonder how far it is really possible to go even given the refactored numpy core. -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. Pauli Virtanen <pav@iki.fi> wrote: 31.10.2011 09:44, mark florisson kirjoitti: [clip] > Ah, that's too bad. Is it anywhere near ready, or was it abandoned for > ironclad? Could you point me to the code? It's quite ready and working, and as far as I understand, Enthought is shipping it. I haven't used it, though. The code is here: https://github.com/numpy/numpy-refactor Pauli_____________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On 31 October 2011 10:03, Dag Sverre Seljebotn <d.s.seljebotn@astro.uio.no> wrote:
Mark: I'm just wondering what you wanted to do with NumPy from Cython -- a stopgap solution for SIMD, iterator support, or something else?
SIMD using NumPy really isn't the best idea long-term because of all the temporaries needed in compound expressions, which is really bad on the memory bus for anything but tiny arrays. For that I'd rather look at finding a nogil core of numexpr or similar.
Yes I'm aware of numexpr and the general problem with array expressions in NumPy. It's not just about SIMD or iterators, it's as you say below, there's lots of stuff that wouldn't be available even if we get SIMD. And if NumPy would get such an API, Cython could figure out how many (or if) temporaries are actually needed and call into the NumPy API with inplace operations. The thing is, how much of NumPy (and numexpr or theano) does Cython want to reimplement? Will you stop at SIMD with elemental functions? And will it run on my GPU? I suppose from a purity perspective I'd just like this functionality to be available in a library and have my language use the library efficiently behind my back, instead of implementing everything itself.
Of course, there is a number of convenient NumPy utility functions which would be cool to have in nogil mode... But given that the GIL is a problem in so many cases, I wonder how far it is really possible to go even given the refactored numpy core. -- Sent from my Android phone with K-9 Mail. Please excuse my brevity.
Pauli Virtanen <pav@iki.fi> wrote:
31.10.2011 09:44, mark florisson kirjoitti: [clip] > Ah, that's too bad. Is it anywhere near ready, or was it abandoned for > ironclad? Could you point me to the code? It's quite ready and working, and as far as I understand, Enthought is shipping it. I haven't used it, though. The code is here: https://github.com/numpy/numpy-refactor Pauli ________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On 10/31/2011 11:48 AM, mark florisson wrote:
On 31 October 2011 10:03, Dag Sverre Seljebotn <d.s.seljebotn@astro.uio.no> wrote:
Mark: I'm just wondering what you wanted to do with NumPy from Cython -- a stopgap solution for SIMD, iterator support, or something else?
SIMD using NumPy really isn't the best idea long-term because of all the temporaries needed in compound expressions, which is really bad on the memory bus for anything but tiny arrays. For that I'd rather look at finding a nogil core of numexpr or similar.
Yes I'm aware of numexpr and the general problem with array expressions in NumPy. It's not just about SIMD or iterators, it's as you say below, there's lots of stuff that wouldn't be available even if we get SIMD. And if NumPy would get such an API, Cython could figure out how many (or if) temporaries are actually needed and call into the NumPy API with inplace operations.
The thing is, how much of NumPy (and numexpr or theano) does Cython want to reimplement? Will you stop at SIMD with elemental functions? And will it run on my GPU?
I suppose from a purity perspective I'd just like this functionality to be available in a library and have my language use the library efficiently behind my back, instead of implementing everything itself.
I do totally agree, but I'm also afraid that this is a neverending quest as long as the GIL is present in CPython. There will always be stuff I'd like to call without the GIL. Only NumPy is not sufficient; I'd also like to use all the scientific libraries which relies on and extends NumPy (all of SciPy for starters), and so on. I do feel that what we have + SIMD covers just enough situations that it is useful for writing "numerical cores", without needing the rest of NumPy. If one starts to pull in more conveniences then I feel I might equally likely need something in SciPy. I'm not really against what you try to do; any progress at all on how much one can do without the GIL is great, I'm just playing the devil's advocate for a bit. Dag Sverre
On 10/31/2011 12:01 PM, Dag Sverre Seljebotn wrote:
On 10/31/2011 11:48 AM, mark florisson wrote:
On 31 October 2011 10:03, Dag Sverre Seljebotn <d.s.seljebotn@astro.uio.no> wrote:
Mark: I'm just wondering what you wanted to do with NumPy from Cython -- a stopgap solution for SIMD, iterator support, or something else?
SIMD using NumPy really isn't the best idea long-term because of all the temporaries needed in compound expressions, which is really bad on the memory bus for anything but tiny arrays. For that I'd rather look at finding a nogil core of numexpr or similar.
Yes I'm aware of numexpr and the general problem with array expressions in NumPy. It's not just about SIMD or iterators, it's as you say below, there's lots of stuff that wouldn't be available even if we get SIMD. And if NumPy would get such an API, Cython could figure out how many (or if) temporaries are actually needed and call into the NumPy API with inplace operations.
The thing is, how much of NumPy (and numexpr or theano) does Cython want to reimplement? Will you stop at SIMD with elemental functions? And will it run on my GPU?
I suppose from a purity perspective I'd just like this functionality to be available in a library and have my language use the library efficiently behind my back, instead of implementing everything itself.
I do totally agree, but I'm also afraid that this is a neverending quest as long as the GIL is present in CPython. There will always be stuff I'd like to call without the GIL. Only NumPy is not sufficient; I'd also like to use all the scientific libraries which relies on and extends NumPy (all of SciPy for starters), and so on.
As an example, it'd be nice to have scipy.ndimage available without the GIL: http://docs.scipy.org/doc/scipy/reference/ndimage.html Now, this *can* easily be done as the core is written in C++. I'm just pointing out that some people may wish more for calling scipy.ndimage inside their prange than for some parts of NumPy. Although if the proposal is some way of writing Cython code that makes skipping the GIL possible to a larger degree than today then this is indeed highly relevant. People are already talking loudly about reimplementing NumPy in Cython (e.g., Travis Oliphant on his blog)., and SciPy is already largely reimplemented in Cython for the .NET port. Dag Sverre
As an example, it'd be nice to have scipy.ndimage available without the GIL: http://docs.scipy.org/doc/scipy/reference/ndimage.html
Now, this *can* easily be done as the core is written in C++. I'm just pointing out that some people may wish more for calling scipy.ndimage inside their prange than for some parts of NumPy.
Not exactly to your larger point wrt the GIL, but I *think* some skimage (née scikits.image) folks are trying to rewrite most of ndimage's functionality in cython. I don't know what the status of this effort is though... Zach
On Mon, Oct 31, 2011 at 11:28 AM, Zachary Pincus <zachary.pincus@yale.edu> wrote:
As an example, it'd be nice to have scipy.ndimage available without the GIL: http://docs.scipy.org/doc/scipy/reference/ndimage.html
Now, this *can* easily be done as the core is written in C++. I'm just pointing out that some people may wish more for calling scipy.ndimage inside their prange than for some parts of NumPy.
Not exactly to your larger point wrt the GIL, but I *think* some skimage (née scikits.image) folks are trying to rewrite most of ndimage's functionality in cython. I don't know what the status of this effort is though...
We still rely on scipy.ndimage in some places, but since we don't have to support N-dimensional arrays, we can often do things in a slightly simpler and faster way. Almost all the performance code in the scikit is written in Cython, which makes it trivial to release the GIL on internal loops. I am actively soliciting feedback from current or prospective users, so that we can drive the scikit in the right direction. Already, it's an entirely different project from what is was a couple of months ago. We stopped trying to duplicate the MATLAB toolbox functionality, and have been adding some exciting new algorithms. The number of pull requests have tripled since the 0.3 release, and we're aiming to have 0.4 done this week. Regards Stéfan
2011/10/31 Stéfan van der Walt <stefan@sun.ac.za>:
On Mon, Oct 31, 2011 at 11:28 AM, Zachary Pincus <zachary.pincus@yale.edu> wrote:
As an example, it'd be nice to have scipy.ndimage available without the GIL: http://docs.scipy.org/doc/scipy/reference/ndimage.html
Now, this *can* easily be done as the core is written in C++. I'm just pointing out that some people may wish more for calling scipy.ndimage inside their prange than for some parts of NumPy.
Not exactly to your larger point wrt the GIL, but I *think* some skimage (née scikits.image) folks are trying to rewrite most of ndimage's functionality in cython. I don't know what the status of this effort is though...
We still rely on scipy.ndimage in some places, but since we don't have to support N-dimensional arrays, we can often do things in a slightly simpler and faster way. Almost all the performance code in the scikit is written in Cython, which makes it trivial to release the GIL on internal loops.
That's good to hear. However, as was the point of my inquiry, it would be a lot nicer if it didn't release the GIL itself and require the GIL to be called, but if it wouldn't require the GIL to be called in the first place. Such an API (especially if written in Cython), can be easily exposed to both C and Cython. You then want a wrapper function that is callable from Python (i.e., requires the GIL), which decides whether or not it should release the GIL, and which calls the nogil equivalent. In the simplest of cases, one may use a cpdef nogil function, but I expect that often you'd have to use cdef nogil with a def wrapper. It does mean that the nogil API part would not take any kind of objects, but only C structures and such (which could be encapsulated by the respective objects). Of course, it may not always be possible, feasible or useful to have such an API, but when it comes to arrays and images it probably will be.
I am actively soliciting feedback from current or prospective users, so that we can drive the scikit in the right direction. Already, it's an entirely different project from what is was a couple of months ago. We stopped trying to duplicate the MATLAB toolbox functionality, and have been adding some exciting new algorithms. The number of pull requests have tripled since the 0.3 release, and we're aiming to have 0.4 done this week.
Regards Stéfan _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On 31 October 2011 09:50, Pauli Virtanen <pav@iki.fi> wrote:
31.10.2011 09:44, mark florisson kirjoitti: [clip]
Ah, that's too bad. Is it anywhere near ready, or was it abandoned for ironclad? Could you point me to the code?
It's quite ready and working, and as far as I understand, Enthought is shipping it. I haven't used it, though.
The code is here: https://github.com/numpy/numpy-refactor
Pauli
Cool, thanks.
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
participants (5)
-
Dag Sverre Seljebotn -
mark florisson -
Pauli Virtanen -
Stéfan van der Walt -
Zachary Pincus