tempfile(s) problem (Re: cgi.FieldStorage() problem)

[Richard van de Stadt]

Richard van de Stadt wrote:
> 
> Erno Kuusela wrote:
> >
> > In article <3A7F20D9.4BE57828 at cs.utwente.nl>, Richard van de Stadt
> > <stadt at cs.utwente.nl> writes:
> >
> > | OSError: [Errno 24] Too many open files: '/var/tmp/@24739.61'
> >
> > you probably just need to increase the resource limits for the account
> > that your http daemon runs under. i don't know what kind of startup
> > procedure you're using for the daemon so it's hard to say exactly how
> > to fix it, but the /bin/sh command for manipulating resource limits is
> > "ulimit" ("limit" with csh descendants).
> >
> > if it still blows up after you've upped the limit, it might
> > be a problem with the cgi module going haywire and opening
> > new files boundlessly, but that seems pretty unlikely.
> 
> But still, it looks like that, although only 1 at the time.
> The webform it gets to work with has 140 <input ...> lines,
> while the number of temporary files that were created is 61.
> The limit is probably set at 64, If you count the opening of
> the script, and some modules, this makes sense.
> 
> It seems like each <input ...> line causes one temporary file to be
> opened, although only 1 exists at the same time. Does cgi.FieldStorage()
> work like that???
> 
> calling ulimit from within the script, which should, I think, do what you
> suggest:
> 
> ulimitCommand = "/usr/bin/ulimit -n 512"
> os.system (ulimitCommand)
> 
> doesn't help.
> 
> Richard.

I copied the data to another system where ulimit -n shows the number 260 for
the maximum number of open files, and there it works!

So, indeed it seems that cgi.FieldStorage() creates a temporary file for each
<input> line of a webform, only one at the time, however, each time with a
different file descriptor. I think this looks bad: "You can use this program, but
in order to run it, Python requires you to change your system settings"...

(I'm using Python for CyberChair, an online paper submission and reviewing system.
The webform under discussion has one input field for each reviewer, stating the
papers he or she is supposed to be reviewing. ICML'01 has almost 140 reviewers...).

Richard.
To boldly go where no webform has gone before :-)