[Python-bugs-list] [ python-Bugs-585792 ] Invalid mmap crashes Python interpreter

noreply@sourceforge.net noreply@sourceforge.net
Mon, 12 Aug 2002 08:47:01 -0700


Bugs item #585792, was opened at 2002-07-24 04:52
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=585792&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: paul rubin (phr)
>Assigned to: Martin v. Löwis (loewis)
Summary: Invalid mmap crashes Python interpreter

Initial Comment:
This is under Red Hat 7.2, Linux kernel 2.4.7.   "x" is
an empty file, i.e. zero bytes long.

import mmap
f = open(x,"r+")m = mmap.mmap(f.fileno(), 10)
print m[1]

Crashes Python with a bus error.  Python should catch
the bus error signal
and raise an appropriate exception that the script can
catch.


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2002-08-12 11:47

Message:
Logged In: YES 
user_id=33168

I thought the st_size was updated by the code in
new_buffersize.  It was only the position, so the comment
should go.  ValueError is fine with me.  Should I add an
entry to NEWS?  New patch attached w/NEWS and test.

There is another issue--Windows.  I how Windows behaves and
I can't test it.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-08-12 10:45

Message:
Logged In: YES 
user_id=21627

I'd say it is a ValueError if the map size is larger than
the file size, instead of silently decreasing the size:
Errors should never pass silently, Unless explicitly silenced.

On the trick of new_buffersize: I'm not sure I understand
what you are referring to: we don't use stdio in mmap at all.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2002-08-12 09:30

Message:
Logged In: YES 
user_id=33168

I wasn't sure if I could change the size.  For Solaris at
least, it seems this is the right thing to do.  Is it likely
that other systems allow the file size to change?  I've
forced the size to be the max(size_arg, size_of_file).  This
has the added benefit of working for writing too.  Prior to
this patch, writing past the end crashed on Linux.

There are 2 comments in the patch:  one worrying about the
file changing, the other producing a warning message if the
size argument is larger than the file.  Do we have to worry
about the size changing?  Should we tell the user what they did?

Sorry about the old style diff.  I remembered to do the cvs
-f diff part, but forgot the -C5.  My defaults are unified.
New context patch attached.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-08-12 03:45

Message:
Logged In: NO 

Please don't even think of adding a stat call to every
access.  The whole idea of mmap is to speed up access by
eliminating system calls when you touch the file contents. 
Checking the file size at map creation is reasonable.  Some
note should also be added to the mmap doc about possible
crashes if the file size changes.

>From a Python script writer's viewpoint, the most convenient
fix is for Python to catch the bus error signal and inspect
the stack to figure out that the trap occurred in the mmap
routine, then generate and return an appropriate Python
exception.  However, doing that is pretty
platform-dependent, and attempting it in all the Python
ports that support mmap doesn't sound practical.  

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-08-12 03:15

Message:
Logged In: YES 
user_id=21627

Ah, an old-style diff. I assume that goes into mmap_item?

I'd rather avoid the stat call on every access, and check
the size on map creation.

The Solaris man page says this about changing files:

     If the size of the mapped file changes  after  the 
call  to
     mmap()  as  a  result  of some other operation on the
mapped
     file, the effect of references to  portions  of  the 
mapped
     region  that  correspond to added or removed portions
of the
     file is unspecified.

So it appears that this is a case of "don"t do this, then",
and that it atleast won't crash - but we might want to
perform a few experiments.



----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2002-08-11 11:05

Message:
Logged In: YES 
user_id=33168

Martin, could you please review the patch?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2002-07-28 11:00

Message:
Logged In: YES 
user_id=33168

Attached is a patch which fixes this problem.  I'm not sure
this patch it correct nor appropriate.  But it stops the
crash.  I'm hoping some mmap expert can shed some light on this.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=585792&group_id=5470