[Python-bugs-list] [ python-Bugs-535905 ] Evil Trashcan and GC interaction

[noreply@sourceforge.net]

Bugs item #535905, was opened at 2002-03-27 15:47
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=535905&group_id=5470

Category: Python Interpreter Core
Group: Python 2.1.2
Status: Open
Resolution: None
Priority: 7
Submitted By: Guido van Rossum (gvanrossum)
Assigned to: Neil Schemenauer (nascheme)
Summary: Evil Trashcan and GC interaction

Initial Comment:
This bug is only a hypothetical bug -- from source 
code inspection it appears clear that it can happen, 
and we have a Zope user who is reporting crashes that 
are consistent with this, but it isn't confirmed that 
this is indeed the cause.  Tim is attempting to 
reproduce it while I'm typing this.

The trashcan code stores NULL or other values in the 
ob_type field of an object, while that object is still 
in the GC doubly-linked list of container objects.  
When the collector is invoked, it dereferences ob_type 
(specifically, in subtract_refs, it retrieves ob_type-
>tp_traverse and calls it).  Obviously, the value 
stored in ob_type by the trashcan code causes this to 
fail.

This bug occurs in all versions of Python that have 
the GC code enabled (i.e. 2.0 and above).  In 2.1 and 
before, the code looks different than in 2.2, but the 
effect is the same: the trashcan code abuses ob_type 
of an object that's in a GC generational chain.

If we fix this, a 2.1.3 release including the fix 
should be issued, and we should hold up the final 
release of 2.2.1 to include a fix as well.

----------------------------------------------------------------------

Comment By: Matthew T. Kromer (zerohero)
Date: 2002-03-28 09:10

Message:
Logged In: YES 
user_id=378059

Leo in Brazil (the Zope customer experiencing the problem)
has bumped the trashcan depth limit up to 5 million, and
been stable for the entire overnight and morning production
periods.

He confirms the description and I believe he will be trying
the patch today.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-03-27 21:15

Message:
Logged In: YES 
user_id=6380

Here's a patch for the 2.1 maintenance branch. AFAICT this
fixes the problem for lists, tuples and dicts, which is a
100% fix (the other trashcan objects, frames and tracebacks,
don't use GC).

Note: the dict test case is just like the tuple or list
testcase, but has
  t = {1: t, 2: Ouch()}
as the last line of f().

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-03-27 20:17

Message:
Logged In: YES 
user_id=6380

Alas, that fixes it for tuples, but not for lists. I've
uploaded the list example as lis.py.

We also need a fix for 2.1.2 (to help the Zope user who
originally ran into this) and one for 2.2.1 -- the latter is
needed quickly if we want to make the final release.

----------------------------------------------------------------------

Comment By: Neil Schemenauer (nascheme)
Date: 2002-03-27 17:18

Message:
Logged In: YES 
user_id=35752

Well, not as simple as I had hoped.  Attached is a rough
patch that fixes the bug (I think, I haven't had time to
really study the interaction between GC and the trashcan
macros).

----------------------------------------------------------------------

Comment By: Neil Schemenauer (nascheme)
Date: 2002-03-27 16:55

Message:
Logged In: YES 
user_id=35752

The fix should be simple.  Coming soon. :-)

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-03-27 16:04

Message:
Logged In: YES 
user_id=31435

Attaching a brief program (tup.py) that provokes a crash 
quickly, at least on Windows.  The crash goes away if the 
trashcan limit is boosted above the depth of tuple nesting 
this program creates.

----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=535905&group_id=5470