[ python-Bugs-1185883 ] PyObject_Realloc bug in obmalloc.c

[SourceForge.net]

Bugs item #1185883, was opened at 2005-04-19 08:07
Message generated for change (Comment added) made by tim_one
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185883&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Kristján Valur (krisvale)
>Assigned to: Nobody/Anonymous (nobody)
Summary: PyObject_Realloc bug in obmalloc.c

Initial Comment:
obmalloc.c:835
If the previous block was not handled by obmalloc, and 
the realloc is for growing the block, this memcpy may 
cross a page boundary and cause a segmentation 
fault.  This scenario can happen if a previous allocation 
failed to successfully allocate from the obmalloc pools, 
due to memory starvation or other reasons, but was 
successfully allocated by the c runtime.

The solution is to query the actual size of the allocated 
block, and copy only so much memory.  Most modern 
platforms provide size query functions complementing 
the malloc()/free() calls.  on Windows, this is the _msize
() function.

----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2005-04-19 11:00

Message:
Logged In: YES 
user_id=31435

mwh:  Umm ... I don't understand what the claim is.  For 
example, what HW does Python run on where memcpy 
segfaults just because the address range crosses a page 
boundary?  If that's what's happening, sounds more like a 
bug in the platform memcpy.  I can memcpy blocks spanning 
thousands of pages on my box -- and so can you <wink>.

krisvale:  which OS and which C are you using?

It is true that this code may try to access a bit of memory 
that wasn't allocated.  If that's at the end of the address 
space, then I could see a segfault happening.  If it is, I doubt 
there's any portable way to fix it short of PyObject_Realloc 
never trying to take over small blocks it didn't control to begin 
with.  Then the platform realloc() will segfault instead <wink>.


----------------------------------------------------------------------

Comment By: Kristján Valur (krisvale)
Date: 2005-04-19 10:39

Message:
Logged In: YES 
user_id=1262199

I can only say that I´ve been seeing this happeing with our 
software.  Admittedly it's because we are eating up all 
memory due to other reasons, but we would like to deal with 
that with a MemoryError rather than a crash.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2005-04-19 10:30

Message:
Logged In: YES 
user_id=6656

Tim, what do you think?

This is a pretty unlikely scenario, it seems to me.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185883&group_id=5470