Too many open files
duncan.booth at invalid.invalid
Mon Feb 4 16:56:35 CET 2008
Steven D'Aprano <steve at REMOVE-THIS-cybersource.com.au> wrote:
> On Mon, 04 Feb 2008 13:57:39 +0100, AMD wrote:
>> The problem I have under windows is that as soon as I get to 500 files I
>> get the Too many open files message. I tried the same thing in Delphi
>> and I can get to 3000 files. How can I increase the number of open files
>> in Python?
> Windows XP has a limit of 512 files opened by any process, including
> stdin, stdout and stderr, so your code is probably failing after file
> number 509.
No, the C runtime has a limit of 512 files, the OS limit is actually 2048.
> I don't know how Delphi works around that issue. Perhaps one of the
> Windows gurus can advise if there's a way to increase that limit from
Call the C runtime function _setmaxstdio(n) to set the maxmimum the number
of open files to n up to 2048. Alternatively os.open() and os.write()
should bypass the C runtime limit.
It would probably be better though to implement some sort of caching scheme
in memory and avoid having to mess with the limits at all. Or do it in two
passes: creating 100 files on the first pass and splitting each of those in
a second pass.
More information about the Python-list