[Cython] cython-devel Digest, Vol 53, Issue 9

Stephen LARROQUE lrq3000 at gmail.com
Fri Jul 3 03:16:22 CEST 2015


Thank you very much Stefan.

I have just read the ticket, and it seems that Guido van Rossum consider
this to be a feature, not a bug (???), so I guess they won't fix it.

For me it would be a nice plus to have this problem fixed, but I can live
with my current workaround, so I'm not sure if Cython should spend dev time
to fix that if that's expensive.

At least, I think the workaround should be mentioned in the github wiki so
that others may implement finite fields extension types without MemoryError
issues. Should I add a wiki page?

2015-06-19 12:00 GMT+02:00 <cython-devel-request at python.org>:

> Send cython-devel mailing list submissions to
>         cython-devel at python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://mail.python.org/mailman/listinfo/cython-devel
> or, via email, send a message with subject or body 'help' to
>         cython-devel-request at python.org
>
> You can reach the person managing the list at
>         cython-devel-owner at python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of cython-devel digest..."
>
>
> Today's Topics:
>
>    1. Re: Bug: Extension Type inheriting from int cause a
>       MemoryError (Stefan Behnel)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Fri, 19 Jun 2015 08:38:32 +0200
> From: Stefan Behnel <stefan_ml at behnel.de>
> To: cython-devel at python.org
> Subject: Re: [Cython] Bug: Extension Type inheriting from int cause a
>         MemoryError
> Message-ID: <5583B8E8.9050902 at behnel.de>
> Content-Type: text/plain; charset=utf-8
>
> Stephen LARROQUE schrieb am 15.06.2015 um 12:34:
> > I am trying to make an extension type inheriting from int or cython.int
> (to
> > do arithmetic operations in Galois Fields). However, this causes a
> > MemoryError because it seems such extension type is not freed correctly.
> > Other than that, it works perfectly well.
> >
> > cdef class ExtendedInt(int): pass
> >
> > for j in xrange(10000000):
> >     ExtendedInt(j)
>
> It looks like a bug in the "int" type in Python 2.x. Python 2.7 has this
> code in intobject.c:
>
> """
> static void
> int_dealloc(PyIntObject *v)
> {
>     if (PyInt_CheckExact(v)) {
>         Py_TYPE(v) = (struct _typeobject *)free_list;
>         free_list = v;
>     }
>     else
>         Py_TYPE(v)->tp_free((PyObject *)v);
> }
>
> static void
> int_free(PyIntObject *v)
> {
>     Py_TYPE(v) = (struct _typeobject *)free_list;
>     free_list = v;
> }
> """
>
> Your extension type automatically inherits the "int_free" slot function
> from its base type, so the "else" case in "int_dealloc()" will in fact call
> "int_free()" and append the object to the free list despite *not* being
> exactly of type PyInt. Then, when creating a new ExtendedInt instance,
> Python's int-subtype instantiation code *ignores* the free list and instead
> creates a completely new object.
>
> Thus, the free list keeps growing until it fills all memory and the process
> dies. I created a CPython ticket.
>
> https://bugs.python.org/issue24469
>
> I guess we could hack up a work around for this in Cython somehow (Python
> 2.7 is an easy target being a dead end, after all), but let's wait what the
> CPython devs have to say about this. Note, however, that any fix in a
> future CPython 2.7.x release will not fix it in earlier Python 2.x
> versions, so my guess is that we'd end up having to add a work-around on
> our side anyway.
>
> Stefan
>
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
>
> ------------------------------
>
> End of cython-devel Digest, Vol 53, Issue 9
> *******************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20150703/26d47ca0/attachment-0001.html>


More information about the cython-devel mailing list