__array__struct__: about using PyCapsule instead of PyCObject for Python 2.7
Do we really need this for NumPy 2? What about using the old PyCObject for all Py 2.x versions? If this is not done, perhaps NumPy 2 on top of Py 2.x should still accept the __array_struct__ being a PyCObject? As reference, Cython still uses PyCObject for Py<3.1 http://hg.cython.org/cython-devel/rev/8a58f1544bd8#l1.33 . -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169
On Tue, Jun 29, 2010 at 8:21 PM, Lisandro Dalcin <dalcinl@gmail.com> wrote:
Do we really need this for NumPy 2? What about using the old PyCObject for all Py 2.x versions? If this is not done, perhaps NumPy 2 on top of Py 2.x should still accept the __array_struct__ being a PyCObject?
As reference, Cython still uses PyCObject for Py<3.1 http://hg.cython.org/cython-devel/rev/8a58f1544bd8#l1.33 .
Blame the Python 2.7 developers, those suckers deprecated the PyCObject. Grrr... I didn't see the point, myself, I'm tempted to deprecate 2.7 just to get even. There are some routines in the numpy/core/src includes that you might want to copy, they will allow you to use a common interface for PyCObject and PyCapsule if you need to do that. Chuck
On 30 June 2010 02:48, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Tue, Jun 29, 2010 at 8:21 PM, Lisandro Dalcin <dalcinl@gmail.com> wrote:
Do we really need this for NumPy 2? What about using the old PyCObject for all Py 2.x versions? If this is not done, perhaps NumPy 2 on top of Py 2.x should still accept the __array_struct__ being a PyCObject?
As reference, Cython still uses PyCObject for Py<3.1 http://hg.cython.org/cython-devel/rev/8a58f1544bd8#l1.33 .
Blame the Python 2.7 developers, those suckers deprecated the PyCObject.
Oh! Sorry! Now I realize that!
Grrr... I didn't see the point, myself, I'm tempted to deprecate 2.7 just to get even. There are some routines in the numpy/core/src includes that you might want to copy, they will allow you to use a common interface for PyCObject and PyCapsule if you need to do that.
I've already fixed my code for PyCapsule. What's not clear to me is how to build (i.e, use the old cobject or the new capsules) __array_struct__ across NumPy and Python versions combinations. Will NumPy 1.x series ever support Python 2.7? In such case, should I use cobjects or capsules? Still, I think NumPy should still accept cobjects from extension types on a Python 2.x runtime, the fix for this support is trival: accept cobjects in NpyCapsule_Check() and NpyCapsule_AsVoidPtr() for PY_MAJOR_VERSION==2. People can still live with PendingDeprecationWarning (remember, they are shown if explicitly requested with -W), and such warning is triggered when cobjects are created, not when they are accessed. -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169
On Wed, Jun 30, 2010 at 10:57 AM, Lisandro Dalcin <dalcinl@gmail.com> wrote:
On 30 June 2010 02:48, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Tue, Jun 29, 2010 at 8:21 PM, Lisandro Dalcin <dalcinl@gmail.com>
wrote:
Do we really need this for NumPy 2? What about using the old PyCObject for all Py 2.x versions? If this is not done, perhaps NumPy 2 on top of Py 2.x should still accept the __array_struct__ being a PyCObject?
As reference, Cython still uses PyCObject for Py<3.1 http://hg.cython.org/cython-devel/rev/8a58f1544bd8#l1.33 .
Blame the Python 2.7 developers, those suckers deprecated the PyCObject.
Oh! Sorry! Now I realize that!
Do I detect a touch of sarcasm?
Grrr... I didn't see the point, myself, I'm tempted to deprecate 2.7 just to get even. There are some routines in the numpy/core/src includes that you might want to copy, they will allow you to use a common interface for PyCObject and PyCapsule if you need to do that.
I've already fixed my code for PyCapsule. What's not clear to me is how to build (i.e, use the old cobject or the new capsules) __array_struct__ across NumPy and Python versions combinations. Will NumPy 1.x series ever support Python 2.7? In such case, should I use cobjects or capsules?
Still, I think NumPy should still accept cobjects from extension types on a Python 2.x runtime, the fix for this support is trival: accept cobjects in NpyCapsule_Check() and NpyCapsule_AsVoidPtr() for PY_MAJOR_VERSION==2.
That might work. The destructors are different between the two, but if you just need access to the pointer things should be simpler. If you test it out and put together a patch I don't see why it shouldn't be applied. If it works. I'm a bit dubious about changing the check however.
People can still live with PendingDeprecationWarning (remember, they are shown if explicitly requested with -W), and such warning is triggered when cobjects are created, not when they are accessed.
Yeah, but it is kind of ugly to have to deal with the warnings. Chuck
On 30 June 2010 15:08, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Wed, Jun 30, 2010 at 10:57 AM, Lisandro Dalcin <dalcinl@gmail.com> wrote:
On 30 June 2010 02:48, Charles R Harris <charlesr.harris@gmail.com> wrote:
Oh! Sorry! Now I realize that!
Do I detect a touch of sarcasm?
No, no sarcasm at all! I just realized that PyCObject were (pending)deprecated in 2.7 ... Anyway. let me say I'm so annoyed and upset as you.
Grrr... I didn't see the point, myself, I'm tempted to deprecate 2.7 just to get even. There are some routines in the numpy/core/src includes that you might want to copy, they will allow you to use a common interface for PyCObject and PyCapsule if you need to do that.
I've already fixed my code for PyCapsule. What's not clear to me is how to build (i.e, use the old cobject or the new capsules) __array_struct__ across NumPy and Python versions combinations. Will NumPy 1.x series ever support Python 2.7? In such case, should I use cobjects or capsules?
Still, I think NumPy should still accept cobjects from extension types on a Python 2.x runtime, the fix for this support is trival: accept cobjects in NpyCapsule_Check() and NpyCapsule_AsVoidPtr() for PY_MAJOR_VERSION==2.
That might work. The destructors are different between the two, but if you just need access to the pointer things should be simpler.
That's my point. I'm fine with numpy creating PyCapsule's in 2.7, as long as PyCObject's are still accepted. Given a cobject, NumPy only needs to get the pointer.
If you test it out and put together a patch I don't see why it shouldn't be applied. If it works. I'm a bit dubious about changing the check however.
Well, if the check is not modified to accept PyCObject's, the things will simply not work. NumPy uses NpyCapsule_Check in order to typecheck __array_struct__ attributes. That check should succeed for cobjects, then you know can call NpyCapsule_AsVoidPtr.
People can still live with PendingDeprecationWarning (remember, they are shown if explicitly requested with -W), and such warning is triggered when cobjects are created, not when they are accessed.
Yeah, but it is kind of ugly to have to deal with the warnings.
Yes, I understand. However, I'll insist about accepting PyCObject's, this will not make warnings show up. So, in short, if you accept that both NpyCapsule_Check() and NpyCapsule_AsVoidPtr() should accept PyCObject's for Py < 3.1, I can provide the patch... I can stress-test this because i'm using __array_struct__ in order to share memory between numpy arrays and PETSc vectors. -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169
Lisandro Dalcin skrev:
No, no sarcasm at all! I just realized that PyCObject were (pending)deprecated in 2.7 ... Anyway. let me say I'm so annoyed and upset as you.
PyCapsule should be used instead. It has two main advantages over PyCObject: First, it associates a 'name' with the void pointer, to provide some sort of type safety. (The 'name' could have been named 'password' to make its intention clear.) Second, the PyCapsule API makes it easier to implement destructors. PyCObject is a severe security hole and stability problem. It can crash the interpreter or run exploit code, as no checks are made before destructors are executed. PyCObject will never be missed. And personally I am glad it was deprecated because it should be avoided. It is better to include a backport of PyCapsule than continue to use PyCObject for Python 2.6, 2.5 and 2.4. Sturla
On Wed, Jun 30, 2010 at 10:57 AM, Lisandro Dalcin <dalcinl@gmail.com> wrote:
On 30 June 2010 02:48, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Tue, Jun 29, 2010 at 8:21 PM, Lisandro Dalcin <dalcinl@gmail.com>
wrote:
Do we really need this for NumPy 2? What about using the old PyCObject for all Py 2.x versions? If this is not done, perhaps NumPy 2 on top of Py 2.x should still accept the __array_struct__ being a PyCObject?
As reference, Cython still uses PyCObject for Py<3.1 http://hg.cython.org/cython-devel/rev/8a58f1544bd8#l1.33 .
Blame the Python 2.7 developers, those suckers deprecated the PyCObject.
Oh! Sorry! Now I realize that!
Grrr... I didn't see the point, myself, I'm tempted to deprecate 2.7 just to get even. There are some routines in the numpy/core/src includes that you might want to copy, they will allow you to use a common interface for PyCObject and PyCapsule if you need to do that.
I've already fixed my code for PyCapsule. What's not clear to me is how to build (i.e, use the old cobject or the new capsules) __array_struct__ across NumPy and Python versions combinations. Will NumPy 1.x series ever support Python 2.7? In such case, should I use cobjects or capsules?
We do support 2.7, but with PyCapsule. You might want to take a look at f2py also, as it also uses PyCapsule for Python >= 2.7. <snip> Chuck
Wed, 30 Jun 2010 12:13:38 -0600, Charles R Harris wrote: [clip]
Grrr... I didn't see the point, myself, I'm tempted to deprecate 2.7 just to get even. There are some routines in the numpy/core/src includes that you might want to copy, they will allow you to use a common interface for PyCObject and PyCapsule if you need to do that.
I've already fixed my code for PyCapsule. What's not clear to me is how to build (i.e, use the old cobject or the new capsules) __array_struct__ across NumPy and Python versions combinations. Will NumPy 1.x series ever support Python 2.7? In such case, should I use cobjects or capsules?
We do support 2.7, but with PyCapsule. You might want to take a look at f2py also, as it also uses PyCapsule for Python >= 2.7.
I think we need to change this decision. PyCObject is still available on Python 2.7, and will only raise a PendingDeprecationWarning, which does not show up by default. I believe the Python devs reversed the full deprecation before the final 2.7 release. So I think we should just stick with PyCObject on 2.x, as we have done so far. I'll just bump the version checks so that PyCapsule is used only on 3.x. I'll commit this to Numpy SVN soon. -- Pauli Virtanen
On Sat, Jul 17, 2010 at 8:51 AM, Pauli Virtanen <pav@iki.fi> wrote:
Wed, 30 Jun 2010 12:13:38 -0600, Charles R Harris wrote: [clip]
Grrr... I didn't see the point, myself, I'm tempted to deprecate 2.7 just to get even. There are some routines in the numpy/core/src includes that you might want to copy, they will allow you to use a common interface for PyCObject and PyCapsule if you need to do that.
I've already fixed my code for PyCapsule. What's not clear to me is how to build (i.e, use the old cobject or the new capsules) __array_struct__ across NumPy and Python versions combinations. Will NumPy 1.x series ever support Python 2.7? In such case, should I use cobjects or capsules?
We do support 2.7, but with PyCapsule. You might want to take a look at f2py also, as it also uses PyCapsule for Python >= 2.7.
I think we need to change this decision. PyCObject is still available on Python 2.7, and will only raise a PendingDeprecationWarning, which does not show up by default. I believe the Python devs reversed the full deprecation before the final 2.7 release.
So I think we should just stick with PyCObject on 2.x, as we have done so far. I'll just bump the version checks so that PyCapsule is used only on 3.x.
I'll commit this to Numpy SVN soon.
Now that npy_3kcompat.h is public we can probably remove some of the version checks. Chuck
On Jun 30, 2010, at 12:48 AM, Charles R Harris wrote:
On Tue, Jun 29, 2010 at 8:21 PM, Lisandro Dalcin <dalcinl@gmail.com> wrote: Do we really need this for NumPy 2? What about using the old PyCObject for all Py 2.x versions? If this is not done, perhaps NumPy 2 on top of Py 2.x should still accept the __array_struct__ being a PyCObject?
As reference, Cython still uses PyCObject for Py<3.1 http://hg.cython.org/cython-devel/rev/8a58f1544bd8#l1.33 .
Blame the Python 2.7 developers, those suckers deprecated the PyCObject. Grrr... I didn't see the point, myself, I'm tempted to deprecate 2.7 just to get even. There are some routines in the numpy/core/src includes that you might want to copy, they will allow you to use a common interface for PyCObject and PyCapsule if you need to do that.
Did we complain loudly to these guys. I fear that our voice is not loud enough with the current crop of Python developers. -Travis
Chuck
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
--- Travis Oliphant Enthought, Inc. oliphant@enthought.com 1-512-536-1057 http://www.enthought.com
participants (5)
-
Charles R Harris -
Lisandro Dalcin -
Pauli Virtanen -
Sturla Molden -
Travis Oliphant