[ python-Bugs-1122301 ] marshal may crash on truncated input

[SourceForge.net]

Bugs item #1122301, was opened at 2005-02-14 03:14
Message generated for change (Comment added) made by nnorwitz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1122301&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Submitted By: Fredrik Lundh (effbot)
Assigned to: Fredrik Lundh (effbot)
Summary: marshal may crash on truncated input

Initial Comment:
marshal doesn't behave well on truncated or otherwise 
malformed input.  here's a short demo script, from a 
recent comp.lang.python thread:

:::

the problem is that the following may or may not reach 
the "done!" statement, somewhat depending on python 
version, memory allocator, and what data you pass to 
dumps.

import marshal

data = marshal.dumps((1, 2, 3, "hello", 4, 5, 6))

for i in range(len(data), -1, -1):
    try:
        print marshal.loads(data[:i])
    except EOFError:
        print "EOFError"
    except ValueError:
        print "ValueError"

print "done!"

(try different data combinations, to see how far you get 
on your platform...)

fixing this should be relatively easy, and should result in 
a safe unmarshaller (your application will still have to 
limit the amount of data fed into load/loads, of course).

:::

(also note that marshal may raise either EOFError or 
ValueError exceptions, again somewhat depending on 
how the file is damaged.  a little consistency wouldn't 
hurt, but I'm not sure if/how this can be fixed...)


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2005-09-29 22:19

Message:
Logged In: YES 
user_id=33168

This works in CVS, but still not in 2.4.  I guess the patch
didn't get backported.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2005-05-27 02:36

Message:
Logged In: YES 
user_id=6656

Ping!

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2005-04-19 07:58

Message:
Logged In: YES 
user_id=6656

I think the attached fixes this example, and another involving marshalled 
sets.

I spent a while feeding random data to marshal a few days ago and found 
that the commonest problem was attempting to allocate really huge 
sequences.  Also, the TYPE_STRINGREF is horribly fragile, but I'm 
hoping Martin's going to fix that (he has a bug filed against him, anyway).

Can you test/check it in?  My marshal.c has rather a lot of local changes.

Also, a test suite entry would be nice...

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1122301&group_id=5470