[ python-Bugs-1341031 ] mmap does not accept length as 0

[SourceForge.net]

Bugs item #1341031, was opened at 2005-10-28 22:26
Message generated for change (Comment added) made by birkenfeld
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1341031&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.4
>Status: Closed
>Resolution: Out of Date
Priority: 5
Submitted By: liturgist (liturgist)
Assigned to: Nobody/Anonymous (nobody)
Summary: mmap does not accept length as 0

Initial Comment:
Creating an mmap object does not accept a length
parameter of zero on Linux FC4 and Cygwin.  However, it
does on Windows XP.

$ ls -al t.dat
-rw-r--r--  1 pwatson mkgroup-l-d 64 Oct 28 10:13 t.dat

$ python
Python 2.4.1 (#1, May 27 2005, 18:02:40)
[GCC 3.3.3 (cygwin special)] on cygwin
Type "help", "copyright", "credits" or "license" for
more information.
>>> import mmap
>>> fp = open('t.dat', 'rb')
>>> b = mmap.mmap(fp.fileno(), 0, access=mmap.ACCESS_READ)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
EnvironmentError: [Errno 22] Invalid argument
>>> b = mmap.mmap(fp.fileno(), 64, access=mmap.ACCESS_READ)
>>> b.size()
64

=====
$ ls -al .profile
-rwxrwxr-x  1 pwatson pwatson 1920 Jul 22 06:57 .profile

$ python
Python 2.4.1 (#1, Jul 19 2005, 14:16:43)
[GCC 4.0.0 20050519 (Red Hat 4.0.0-8)] on linux2
Type "help", "copyright", "credits" or "license" for
more information.
>>> import mmap
>>> fp = open('.profile', 'rb')
>>> b = mmap.mmap(fp.fileno(), 0, access=mmap.ACCESS_READ)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
EnvironmentError: [Errno 22] Invalid argument
>>> b = mmap.mmap(fp.fileno(), 1920,
access=mmap.ACCESS_READ)
>>> b.size()
1920

----------------------------------------------------------------------

>Comment By: Reinhold Birkenfeld (birkenfeld)
Date: 2005-12-18 11:57

Message:
Logged In: YES 
user_id=1188172

This was addressed with patch #1144555, which has been
applied to the 2.5 branch only (as it is a new feature).

Closing as "Out of Date".

----------------------------------------------------------------------

Comment By: Josiah Carlson (josiahcarlson)
Date: 2005-11-08 20:37

Message:
Logged In: YES 
user_id=341410

You could just pass fp.fstat(fp.fileno()).st_size to all of
your instances, not just the unix ones.

Right now, Python exposes the underlying mmap
implementation.  Windows automatically handles the
statting/etc., for using the full size of the file in an
mmap.  Other platforms don't.

While it would seem reasonable to have the unix specific
implementation automatically perform an os.fstat(), actually
calling the requisite code in os from mmapmodule.c is a pain
in the ass (in my experience), and duplicating the code in
mmapmodule.c is a poor idea.

I would suggest you offer a documentation patch in the patch
tracker to state the behavior of Unix mmap and suggested
cross-platform workarounds.

----------------------------------------------------------------------

Comment By: liturgist (liturgist)
Date: 2005-11-08 20:15

Message:
Logged In: YES 
user_id=197677

The bug report is still right here in this listing.

What are the possible solutions?

1) Not have the Windows version map the entire file length.
 Could break existing code, so not likely.

2) Have the UNIX version map the entire file when a length
of 0 is passed.  Since the fp is passed, the code can get
length from os.fstat(fp.fileno()).st_size.  There could be
some code handling this exception whose operation would
change if this modification were made.

3) Modify the documentation to be more explicit about valid
and invalid length values on the UNIX version.  Providing
and example using fp.fstat(fp.fileno()).st_size would be
very helpful.

I see the third option as the only viable path at this
point.  Letting the Windows path use zero as a magic
indicator is an impediment to writing cross-platform code. 
An example in the documentation of how it could be done is
sorely needed.

----------------------------------------------------------------------

Comment By: Josiah Carlson (josiahcarlson)
Date: 2005-11-08 19:11

Message:
Logged In: YES 
user_id=341410

According to the documentation, in Windows, passing a
'length' of 0 will map the entire file.  On 'Unix', it does
not discuss what happens when you use a length of 0.

The behavior on undefined arguments in the case of linux
could almost be expected (though perhaps should be clarified
in documentation), and being that the 'cygwin' platform
isn't Windows (it's a unix-like environment in Windows),
expecting it to perform like Windows, is probably 

Wasn't your bug report that you could not map 0 bytes from
the file, or that passing '0' did not do what you expected
on Linux and/or cygwin?  If so, then maybe the documentation
should be updated.

To respond to "wouldn't os.stat(filename).st_size be good to
pass during mmap creation time?"  Of course, I do that
myself, and I would expect most sane people to do the same.
 os.fstat(fp.fileno()).st_size also works.

So, what is your bug report again, and why is this bug
report open?

----------------------------------------------------------------------

Comment By: liturgist (liturgist)
Date: 2005-11-08 16:16

Message:
Logged In: YES 
user_id=197677

If the file was created by another process or specified as a
parameter, the code might not be free to write into it. 
Asking os.stat() for the file size works on both platforms.
 I have not tested on any other platform, but I do not see
any reason it would not work.

fp = open(fn, 'rb')
b = mmap.mmap(fp.fileno(), os.stat(fp.name).st_size,
access=mmap.ACCESS_READ)

Even if the file were just created, surely asking os.stat()
for the file size would not have more overhead than writing
to the file.  Does it?

----------------------------------------------------------------------

Comment By: Josiah Carlson (josiahcarlson)
Date: 2005-11-08 07:17

Message:
Logged In: YES 
user_id=341410

On Windows and Linux, I have been able to map positive,
nonzero portions of files for quite some time.  The only
use-case I can see in using an mmap of size 0 is if the file
was just created, and a user wanted to immediately resize
the mmap to be larger.

This can be worked-around with a simple fp.write('\0'),
followed by a mmap call with size 1.  Resize as
desired/necessary.

----------------------------------------------------------------------

Comment By: liturgist (liturgist)
Date: 2005-10-28 23:09

Message:
Logged In: YES 
user_id=197677

It would be helpful to creating cross-platform code for all
platforms to have the same requirements.  If this is marked
as Not-A-Bug, then how about changing it to a documentation
bug so that and example could be provided?

fp = open(fn, 'rb')
b = mmap.mmap(fp.fileno(), os.stat(fp.name).st_size,
access=mmap.ACCESS_READ)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1341031&group_id=5470