[Python-Dev] large file support

[Andreas Jung]

I remember that we had several times trouble with compiling
Python 2.1 under Redhat with LF support. Also the way described in the docs
did not work in all cases and we had to tweak the sources at bit
(I think it was posixmodule.c).

-aj

--On Monday, June 17, 2002 15:22 -0400 Jeremy Hylton <jeremy@zope.com> 
wrote:

> I've run into a problem with large files using Python 2.1.2 and a
> Linux 2.4.9 box.  We've got a large file -- almost 6GB -- that Python
> chokes on even though regular shell tools seem to be fine.
>
> In particular, os.stat() of the file fails with EOVERFLOW and open()
> of the file fails with EFBIG.  The stat() failure is really bad
> because it means os.path.exists() returns false.
>
> strace tells me that other tools open the file passing O_LARGEFILE,
> but Python does not.  (They pass it even for small files.) I can't
> find any succient explanation of O_LARGEFILE, but Google turns up all
> sorts of pages that mention it.  It looks like the right way to open
> large files, but it only seems to be defined in <asm/fcntl.h> on the
> Linux box in question.
>
> I haven't had any luck searching for a decent way to invoke stat() and
> have it be prepared for a very large file.
>
> I think Python is definitely broken here.  Can anyone offer any clues
> or pointers to documentation?  Better yet, a fix.  I'm happy to help
> integrate and test it.
>
> Jeremy
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev





    ---------------------------------------------------------------------
   -    Andreas Jung                     http://www.andreas-jung.com   -
  -   EMail: andreas at andreas-jung.com                              -
   -            "Life is too short to (re)write parsers"               -
    ---------------------------------------------------------------------