[ python-Bugs-1534630 ] Python 2.5 svn crash in _elementtree.c

[SourceForge.net]

Bugs item #1534630, was opened at 2006-08-04 17:29
Message generated for change (Comment added) made by effbot
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1534630&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: Python 2.5
Status: Open
Resolution: None
>Priority: 7
Submitted By: Barry A. Warsaw (bwarsaw)
Assigned to: Fredrik Lundh (effbot)
Summary: Python 2.5 svn crash in _elementtree.c

Initial Comment:
/F's sourceforge screen scraper tool triggered a crash
in _elementtree.c in the latest Python 2.5 svn.  It
seems that a Py_DECREF should probably be a Py_XDECREF
unless there's some other logic bug I'm missing. 
Here's the patch if it's the former.

----------------------------------------------------------------------

>Comment By: Fredrik Lundh (effbot)
Date: 2006-08-16 17:30

Message:
Logged In: YES 
user_id=38376

Alright, I've finally identified the problem: you get this
crash if you call the "data" method on a builder *before*
you call any other method.  E.g.

    bob = ET.TreeBuilder()
    bob.data("data") # <-- crash!
    bob.start("tag", {})
    bob.end("tag")
    bob.close()

In this case, self->last will point to Py_None, instead of
an Element object, and self->last->text will thus be a
mostly random pointer...

Note that his cannot happen if you're using the Expat
parser; the parser will raise a syntax error before calling
any TreeBuilder method.

This should be easy to fix (make treebuilder_handle_data
return immediately if self->last == Py_None); I'll try
getting this in before the code freeze.

----------------------------------------------------------------------

Comment By: Fredrik Lundh (effbot)
Date: 2006-08-11 19:23

Message:
Logged In: YES 
user_id=38376

I'm pretty sure this only happens if you're explicitly using
the TreeBuilder class, which is a rather specialized use of
ET.  I'm also pretty sure that this is a cElementTree
regression; I have a vague memory that I did some
"optimizations" around 1.0.4 or 1.0.5 that might have caused
this.

I hope to find some time to look at this this weekend.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2006-08-11 17:30

Message:
Logged In: YES 
user_id=33168

Yes, for all the reasons you mention.  Also, I'm not sure
that anyone other than /f can address this problem and I
don't know when he will be able to fix it.  I would very
much prefer this be addressed prior to 2.5, but if it has to
wait for 2.5.1 so be it.  Also, I would guess this same
problem is in the external version, so it's probably not a
'regression' either, even though this is a new module.

Realize that there's still a month before release.  So this
is a non-issue if you can convince /f to fix it soon-ish. :-)

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2006-08-11 14:42

Message:
Logged In: YES 
user_id=12800

You might be right about the basic invariant problem, but
why shouldn't it block release?  Is your thinking that
because it's a problem in an extension module (and a new and
probably rarely used one at that)?  It's definitely severe
enough to hit people who use the module, not that it can't
be fixed after the 2.5 release of course.


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2006-08-11 06:21

Message:
Logged In: YES 
user_id=33168

I don't think this patch will address the underlying
problem.  There are many DECREFs (and no XDECREFs) of
last->text.  It seems that the code believes there cannot be
a NULL.  Perhaps all the DECREFs of last->text should be
XDECREF?

I would really like this to get done for 2.5, but it
shouldn't block release.  Dropping priority.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1534630&group_id=5470