[ python-Bugs-984722 ] Py_BuildValue loses reference counts on error

[SourceForge.net]

Bugs item #984722, was opened at 2004-07-03 21:19
Message generated for change (Comment added) made by mwh
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=984722&group_id=5470

Category: Python Interpreter Core
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Submitted By: Pete Shinners (shredwheat)
Assigned to: Michael Hudson (mwh)
Summary: Py_BuildValue loses reference counts on error

Initial Comment:
Py_BuildValue has the convenient "N" type argument to
take a PyObject* without a reference count. To the
programmer this means your are transferring the object
ownership to the Py_BuildValue function (and the tuple
it creates).

If Py_BuildValue encounters an error processing an
argument it aborts and returns NULL. But the remaining
arguments are ignored. Therefore objects are leaked
from a leftover reference count.

>From looking at the code in Python/modsupport.c it
looks like a reasonable solution would be to insert a
Py_None into the tuple/list/dict being created and
internally set some sort of error flag. When the
function is returning it would check the error flag and
if set, Py_DECREF the created object and return NULL.

At first this may seem like a lot of work to do when we
already know the function will fail. But it is no more
work than we would be doing when the function succeeeds.


----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2004-07-14 12:11

Message:
Logged In: YES 
user_id=6656

Looks OK, I'll probably check it in (with slightly rearranged 
comments) after make test finishes.

Is this testable?  Maybe in testcapimodule. Maybe it's not worth it.

----------------------------------------------------------------------

Comment By: Pete Shinners (shredwheat)
Date: 2004-07-13 21:59

Message:
Logged In: YES 
user_id=1076442

Here is patch, please review and let me know if further
revisions are necessary. Not sure if the comment line in
each error case is helpful or a distraction.

Performance difference is negligable for the common
'success' case. If multiple arguments raise exceptions it is
undefined which one is current when the function returns. A
NULL will still be returned if there are any exceptions.

(my first Python patch, exciting)

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2004-07-12 18:59

Message:
Logged In: YES 
user_id=80475

Thanks, that will be nice.

Do make a special effort to make sure this has zero
performance impact on the common case.  No sense in mucking
up the works to satisfy a corner case that rarely impacts
real apps.

----------------------------------------------------------------------

Comment By: Pete Shinners (shredwheat)
Date: 2004-07-12 15:29

Message:
Logged In: YES 
user_id=1076442

I will attempt a patch, this does not look difficult. I am
currently trying to get straight CVS Python to run the test
suite, but having difficulty. I will have a patch ready or
more comments otherwise later this week.


----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2004-07-12 14:36

Message:
Logged In: YES 
user_id=80475

I thought you would have an interest and be in a good
position to know whether and how this should be fixed.  Feel
free to unassign.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2004-07-12 13:30

Message:
Logged In: YES 
user_id=6656

Um.  Any particular reason for assigning to me?  I could probably 
review and apply any patch that appears, but I'm unlikely to get to 
creating one myself...

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=984722&group_id=5470