[Python-bugs-list] [Bug #119806] cgi.py uses too much memory on large file uploads

[noreply@sourceforge.net]

Bug #119806, was updated on 2000-Oct-30 13:35
Here is a current snapshot of the bug.

Project: Python
Category: Modules
Status: Open
Resolution: None
Bug Group: Feature Request
Priority: 5
Summary: cgi.py uses too much memory on large file uploads

Details: When uploading files through a web form and parsing the form using cgi.py - the entire fil(s are read into memory, which is intolerable with very large (I'm thinking multiple 10's of megabyte) files.

The culprit seems to be in the FieldStorage class, basically the three calls to:

   self.lines.append(line)

in the read_lines_to_eof, read_lines_to_outerboundary, and skip_lines methods.  

Commenting those calls out seems to fix the problem, as long as you don't care to access the 'lines' member of the FieldStorage instance.  A real fix would possibly be to add a "keep_lines=1" keyword parameter to FieldStorage.__init__(), save that value in the instance, and check it before accumulating form lines.

For detailed info, follow this link:
http://sourceforge.net/bugs/?func=detailbug&bug_id=119806&group_id=5470