[Python-bugs-list] [ python-Bugs-793822 ] gc.get_referrers() is inherently dangerous

Sat Sep 6 16:51:03 EDT 2003

Bugs item #793822, was opened at 2003-08-23 10:17
Message generated for change (Comment added) made by bcannon
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=793822&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Armin Rigo (arigo)
Assigned to: Nobody/Anonymous (nobody)
Summary: gc.get_referrers() is inherently dangerous

Initial Comment:
gc.get_referrers() can be used to crash any Python

interpreter because it allows the user to obtain

objects which are still under construction.

Here is an example where an iterator can use it to

obtain a tuple while it is still being populated by the

'tuple' built-in function. The following example

triggers a SystemError, but as the tuple 't' is at the

moment still full of null values it can easily generate

segfaults instead.

from gc import get_referrers

def iter():

    tag = object()

    yield tag   # 'tag' gets stored in the result tuple

    lst = [x for x in get_referrers(tag)

           if isinstance(x, tuple)]

    t = lst[0]  # this *is* the result tuple

    print t     # full of nulls !

tuple(iter())

Unless someone has more ideas than me as to how to

prevent this problem, I'd suggest that

gc.get_referrers() should be deemed 'officially

dangerous' in the docs.

----------------------------------------------------------------------

>Comment By: Brett Cannon (bcannon)
Date: 2003-09-06 15:51

Message:
Logged In: YES 
user_id=357491

The wording Armin proposes in his patch makes sense to me.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2003-09-02 11:11

Message:
Logged In: YES 
user_id=4771

Here is the doc patch (attached).

A clean solution to this problem would involve delaying GC

registration, which I think would imply changes in the C

API. It is probably not worth it. I don't think it would be

a problem for the GC not to be able to browse through

objects still under constructions because such objects are

never part of a dead cycle anyway, and probably not part of

a cycle at all until later mutated.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-08-31 16:36

Message:
Logged In: YES 
user_id=31435

Martin, it's easy to transform the example into one that 

crashes.  For example, adding "print t[3]" as the last line of 

the iter() function causes it to die with a segfault on my box.  

Virtually anything that fetches a NULL from the tuple will try 

to Py_INCREF it (that's all "t[3]" does), and that's an instant 

NULL-pointer dereferencing death.

That said, it would be more helpful <wink> if Armin submitted 

a doc patch.  The introspective features of the gc module are 

intended to be used by consenting adults facing hard 

debugging problems, and I don't care if they can be tricked 

into blowing up.  I agree the docs should point out the 

possibility, though.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2003-08-31 10:18

Message:
Logged In: YES 
user_id=21627

I see no harm in your example. Even though the tuple has

NULLs in it, is in a stable state. The lesson learned is

that an object should become gc-tracked only if it is in a

stable state, as gc-tracking means to expose references to

the object. This is true independent of get_referrers, as

gc-tracking means that tp_traverse might be called for the

object, so it *has* to be in a stable state.

I fail to see how the example "crashes" the Python

interpreter. I causes a SystemError when the tuple is

resized, that's all. There are many ways to cause a

SystemError, including

raise SystemError

I recommend not to declare a function as "dangerous" in the

docs. Instead, the actual problem should be explained, both

in the GC part of the C API, and in gc module docs (for both

get_referrers, and get_objects).

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2003-08-28 06:23

Message:
Logged In: YES 
user_id=4771

But it would be very difficult to fix the code base to avoid

the problem. The 'tuple' constructor was only an example; it

is actually a quite common pattern everywhere in the C code

base of both the core and extension modules. Expecting an

object not to be seen before you first hand it out is

extremely common, and get_referrers() breaks that

assumption. Hence the claim that the problem really lies in

get_referrers().

----------------------------------------------------------------------

Comment By: Denis S. Otkidach (ods)
Date: 2003-08-28 00:35

Message:
Logged In: YES 
user_id=63454

I guess it's dangerous to make object that is not properly 

initialized reachable from other code. Even if it's reachable 

with get_referrers() only. There is no danger in 

get_referrers() itself.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=793822&group_id=5470