[Expat-bugs] [ expat-Bugs-615606 ] Buffer overrun with XML_ParserCreateNS

noreply@sourceforge.net noreply@sourceforge.net
Sat, 28 Sep 2002 18:15:59 -0700


Bugs item #615606, was opened at 2002-09-27 11:06
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=615606&group_id=10127

Category: None
Group: Test Required
Status: Open
Resolution: Fixed
Priority: 5
Submitted By: Daniel Bowen (daniel_bowen)
Assigned to: Nobody/Anonymous (nobody)
Summary: Buffer overrun with XML_ParserCreateNS

Initial Comment:
Expat 1.95.5 on Win32 using MSVC 6, 7

A buffer overrun occurs under the following set of 
circumstances:
* In a test program, create an XML_Parser using 
XML_ParserCreateNS (instead of XML_ParserCreate)
* Parse a file (or stdin) where the number of bytes is 
greater than the size of your buffer (so that you have to 
do multiple passes).  It seems to happen with both 
XML_GetBuffer / XML_ParseBuffer as well as allocating 
your own buffer and calling XML_Parse.

To see that an error occurs:
* Compile a debug version of expat (DLL or static library)
* Compile a debug version of your test program that 
uses the debug version of expat
* You get a dialog similar to the following:

---------------------------
Microsoft Visual C++ Debug Library
---------------------------
Debug Error!

Program: ...\Expat-1.95.5
\Source\examples\Debug\elements.exe

DAMAGE: after Normal block (#88) at 0x002F7798.


(Press Retry to debug the application)
---------------------------
Abort   Retry   Ignore   
---------------------------

If you click "Retry", it takes you to the _free_dbg 
function in dbgheap.c, line 1066 in MSVC 6, as a result 
of a "CheckBytes" call.  The call stack indicates that 
this is on the "XML_ParserFree" call.  In the output 
window, it lists a handful of addresses where the bytes 
should be "0xFD", such as:

memory check error at 0x002F77BF = 0x69, should be 
0xFD.

If you view memory at this address, you see parts of the 
input XML file have been written there.  If you set a 
breakpoint to break when data at this location in 
memory changes, you break at line 2110 in xmlparse.c 
in the function doContent, at the line:

            /* don't need to check for space - already done 
in storeAtts() */
            while (*localPart) *uri++ = *localPart++;

If you watch memory changing as you do multiple 
passes through XML_Parse/XML_ParseBuffer, it seems 
that instead of reusing the internal buffer starting at the 
beginning, the internal buffer keeps getting appended to 
(beyond the size of its allocation).


This should be easily reproducable.  For example, in 
the "elements" sample, change the "XML_ParserCreate
(NULL)" line on line 32 in elements.c 
to "XML_ParserCreateNS(NULL, '|')".

I haven't tested this scenario in builds other than 1.95.5, 
so I'm not sure if this is a new bug or a bug that hasn't 
yet been tripped across.

Thanks!
-Daniel

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-09-28 18:15

Message:
Logged In: NO 

Ouch!  I've been working for the past two days tracking down 
this bug.  Thanks for the patch!

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2002-09-28 07:56

Message:
Logged In: YES 
user_id=290026

I could trace this bug to the function storeRawNames.
What happens is this: when namespace processing
is turned on, tag->name.str does not point to tag->buf,
but to the binding->uri buffer of the associated
PREFIX structure. Therefore, when tag->buf is reallocated,
one should update tag->name.str only when namespace
processing is off. This condition was not tested and
tag->name.str was wrongly updated in those cases
when namespace processing was turned on and the
storeRawNames function reallocated tag->buf.

Fix is already committed to CVS. Patch attached.
I tested with your example and it works fine now.

----------------------------------------------------------------------

Comment By: Daniel Bowen (daniel_bowen)
Date: 2002-09-27 12:09

Message:
Logged In: YES 
user_id=619351

I had to re-upload with the compiled elements.exe taken out 
(it made the file too big to upload).  You can just rebuild all in 
debug mode.

----------------------------------------------------------------------

Comment By: Daniel Bowen (daniel_bowen)
Date: 2002-09-27 12:02

Message:
Logged In: YES 
user_id=619351

I've attached a copy of the "elements" project along with the 
expat (static) that it uses.  I compiled both in debug, and 
included in the upload elements.exe.  Also in the 
examples\Debug directory is an XML file that will trigger the 
bug.

I had tried it with only a couple of input XML files.  I've tried it 
with a few more, and I see that to reproduce the problem is 
not always as simple as "Parse a file (or stdin) where the 
number of bytes is greater than the size of your buffer".  I'm 
not sure what characteristics trigger the bug, but from what I 
could tell, this was at least one symptom.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2002-09-27 11:30

Message:
Logged In: YES 
user_id=290026

I am using XML_ParserCreateNS with multiple buffers
all the time and have never had that problem.

Could you please attach a complete but simple
test case, so that we can reproduce the error.

Thanks,

Karl

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=615606&group_id=10127