tp_finalize vs tp_del sematics
Hi everybody, I'm trying to get sense of PEP-0442 [1]. Most of the looks clear, however I wasn't able to answer myself one simple question: why it wasn't possible to implement proposed CI disposal scheme on top of tp_del? Common sense suggests that tp_del and tp_finalize have different semantics. For instance, with tp_finalize there is a guarantee that the object will be in a safe state, as documented at [2]. However, tp_del is not documented, and I have only vague idea of its guarantees. Are there any? Thanks for the clarification. 1. https://www.python.org/dev/peps/pep-0442/ 2. https://docs.python.org/3/c-api/typeobj.html -- Best regards, Valentine Sinitsyn
Hi Valentine, On 19 August 2015 at 09:53, Valentine Sinitsyn <valentine.sinitsyn@gmail.com> wrote:
why it wasn't possible to implement proposed CI disposal scheme on top of tp_del?
From the point of view of someone writing a C extension module, both tp_del and tp_finalize are called with the same guarantee that the object is still valid at that point. The difference is only that the
I'm replying here as best as I understand the situation, which might be incomplete or wrong. presence of tp_del prevents the object from being collected at all if it is part of a cycle. Maybe the same could have been done without duplicating the function pointer (tp_del + tp_finalize) with a Py_TPFLAGS_DEL_EVEN_IN_A_CYCLE. A bientôt, Armin.
Hi Valentine,
On 19 August 2015 at 09:53, Valentine Sinitsyn <valentine.sinitsyn@gmail.com> wrote:
why it wasn't possible to implement proposed CI disposal scheme on top of tp_del?
I'm replying here as best as I understand the situation, which might be incomplete or wrong.
From the point of view of someone writing a C extension module, both tp_del and tp_finalize are called with the same guarantee that the object is still valid at that point. The difference is only that the presence of tp_del prevents the object from being collected at all if it is part of a cycle. Maybe the same could have been done without duplicating the function pointer (tp_del + tp_finalize) with a Py_TPFLAGS_DEL_EVEN_IN_A_CYCLE. So you mean that this was to keep things backwards compatible for
Hi Armin, Thanks for replying. On 23.08.2015 17:14, Armin Rigo wrote: third-party extensions? I haven't thought about it this way, but this makes sense. However, the behavior of Python code using objects with __del__ has changed nevertheless: they are collectible now, and __del__ is always called exactly once, if I understand everything correctly. Thanks, Valentine
Hi Valentine, On 24 August 2015 at 20:43, Valentine Sinitsyn <valentine.sinitsyn@gmail.com> wrote:
So you mean that this was to keep things backwards compatible for third-party extensions? I haven't thought about it this way, but this makes sense. However, the behavior of Python code using objects with __del__ has changed nevertheless: they are collectible now, and __del__ is always called exactly once, if I understand everything correctly.
Yes, I think so. There is a *highly obscure* corner case: __del__ will still be called several times if you declare your class with "__slots__=()". A bientôt, Armin.
Hi Armin, On 25.08.2015 12:51, Armin Rigo wrote:
Hi Valentine,
On 24 August 2015 at 20:43, Valentine Sinitsyn <valentine.sinitsyn@gmail.com> wrote:
So you mean that this was to keep things backwards compatible for third-party extensions? I haven't thought about it this way, but this makes sense. However, the behavior of Python code using objects with __del__ has changed nevertheless: they are collectible now, and __del__ is always called exactly once, if I understand everything correctly.
Yes, I think so. There is a *highly obscure* corner case: __del__ will still be called several times if you declare your class with "__slots__=()". Even on "post-PEP-0442" Python 3.4+? Could you share a link please?
Thanks, Valentine
Hi Valentine, On 25 August 2015 at 09:56, Valentine Sinitsyn <valentine.sinitsyn@gmail.com> wrote:
Yes, I think so. There is a *highly obscure* corner case: __del__ will still be called several times if you declare your class with "__slots__=()".
Even on "post-PEP-0442" Python 3.4+? Could you share a link please?
class X(object): __slots__=() # <= try with and without this def __del__(self): global revive revive = self print("hi") X() revive = None revive = None revive = None --Armin
Hi Armin, On 25.08.2015 13:00, Armin Rigo wrote:
Hi Valentine,
On 25 August 2015 at 09:56, Valentine Sinitsyn <valentine.sinitsyn@gmail.com> wrote:
Yes, I think so. There is a *highly obscure* corner case: __del__ will still be called several times if you declare your class with "__slots__=()".
Even on "post-PEP-0442" Python 3.4+? Could you share a link please?
class X(object): __slots__=() # <= try with and without this def __del__(self): global revive revive = self print("hi")
X() revive = None revive = None revive = None Indeed, that's very strange. Looks like a bug IMHO. Thanks for pointing out.
Valentine
Hi Armin, On 25.08.2015 13:00, Armin Rigo wrote:
Hi Valentine,
On 25 August 2015 at 09:56, Valentine Sinitsyn <valentine.sinitsyn@gmail.com> wrote:
Yes, I think so. There is a *highly obscure* corner case: __del__ will still be called several times if you declare your class with "__slots__=()".
Even on "post-PEP-0442" Python 3.4+? Could you share a link please?
class X(object): __slots__=() # <= try with and without this def __del__(self): global revive revive = self print("hi")
X() revive = None revive = None revive = None By accident, I found a solution to this puzzle:
class X(object): __slots__ = () class Y(object): pass import gc gc.is_tracked(X()) # False gc.is_tracked(Y()) # True An object with _empty_ slots is naturally untracked, as it can't create back references. Valentine
On Thu, Sep 3, 2015 at 9:23 AM, Valentine Sinitsyn <valentine.sinitsyn@gmail.com> wrote:
Hi Armin,
On 25.08.2015 13:00, Armin Rigo wrote:
Hi Valentine,
On 25 August 2015 at 09:56, Valentine Sinitsyn <valentine.sinitsyn@gmail.com> wrote:
Yes, I think so. There is a *highly obscure* corner case: __del__ will still be called several times if you declare your class with "__slots__=()".
Even on "post-PEP-0442" Python 3.4+? Could you share a link please?
class X(object): __slots__=() # <= try with and without this def __del__(self): global revive revive = self print("hi")
X() revive = None revive = None revive = None
By accident, I found a solution to this puzzle:
class X(object): __slots__ = ()
class Y(object): pass
import gc gc.is_tracked(X()) # False gc.is_tracked(Y()) # True
An object with _empty_ slots is naturally untracked, as it can't create back references.
Valentine
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/fijall%40gmail.com
That does not make it ok to have del called several time, does it?
Hi Maciej, On 04.09.2015 00:08, Maciej Fijalkowski wrote:
On Thu, Sep 3, 2015 at 9:23 AM, Valentine Sinitsyn <valentine.sinitsyn@gmail.com> wrote:
Hi Armin,
On 25.08.2015 13:00, Armin Rigo wrote:
Hi Valentine,
On 25 August 2015 at 09:56, Valentine Sinitsyn <valentine.sinitsyn@gmail.com> wrote:
Yes, I think so. There is a *highly obscure* corner case: __del__ will still be called several times if you declare your class with "__slots__=()".
Even on "post-PEP-0442" Python 3.4+? Could you share a link please?
class X(object): __slots__=() # <= try with and without this def __del__(self): global revive revive = self print("hi")
X() revive = None revive = None revive = None
By accident, I found a solution to this puzzle:
class X(object): __slots__ = ()
class Y(object): pass
import gc gc.is_tracked(X()) # False gc.is_tracked(Y()) # True
An object with _empty_ slots is naturally untracked, as it can't create back references.
Valentine
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/fijall%40gmail.com
That does not make it ok to have del called several time, does it? That's a tricky question. Python's data model [1,2] doesn't say anything about how many times __del__ can be called. PEP-0442 guarantees it will be called only once, but it implicitly covers GC-objects only.
For me, it looks like destructor behaviour for non-GC object is undefined, but I agree it makes sense to call them exactly once as well. 1. https://docs.python.org/3/reference/datamodel.html 2. https://docs.python.org/2/reference/datamodel.html -- Valentine
Hi Valentine, On Thu, Sep 3, 2015 at 9:15 PM, Valentine Sinitsyn <valentine.sinitsyn@gmail.com> wrote:
That does not make it ok to have del called several time, does it?
That's a tricky question.
If the Python documentation now says something like ``the __del__ method is never called more than once on the same instance'' without acknowledging this corner case, then it could be regarded as documentation bug. I didn't check, though. But feel free to open an issue and mention everything I said above, if you want to. A bientôt, Armin.
Hi Armin, On 04.09.2015 02:29, Armin Rigo wrote:
Hi Valentine,
On Thu, Sep 3, 2015 at 9:15 PM, Valentine Sinitsyn <valentine.sinitsyn@gmail.com> wrote:
That does not make it ok to have del called several time, does it?
That's a tricky question.
If the Python documentation now says something like ``the __del__ method is never called more than once on the same instance'' without acknowledging this corner case, then it could be regarded as documentation bug. I didn't check, though. But feel free to open an issue and mention everything I said above, if you want to. I've checked the places I remembered, but didn't find any guarantees for __del__ to be called exactly once in general. Looks like it was only important to ensure safe finalization.
Looks a bit inconsistent though so it's probably appropriate to file the report. Valentine
participants (3)
-
Armin Rigo -
Maciej Fijalkowski -
Valentine Sinitsyn