[Web-SIG] WSGI, cgi.FieldStorage incompatibility

[Robert Brewer]

James Y Knight wrote:
> On Sep 29, 2006, at 3:31 PM, Guido van Rossum wrote:
> 
> > On 9/29/06, Michael Kerrin <michael.kerrin at openapp.biz> wrote:
> >>   But the current implementation of cgi.FieldStorage in the 2.4.4  
> >> branch
> >> and on Python 2.5 does call readline with the size argument. It has
> >> started to do this in response to the Python bug #1112549 -
> >> cgi.FieldStorage memory usage can spike in line-oriented ops. See
> >> http://sourceforge.net/tracker/index.php? 
> >> func=detail&aid=1112549&group_id=5470&atid=105470
> >>
> >>   Since it is reasonable for a WSGI application to use  
> >> cgi.FieldStorage
> >> I am wondering whether cgi.FieldStorage or the WSGI specification  
> >> needs
> >> to changed in order to solve this incompatibility.
> >>
> >>   Originally I thought it was cgi.FieldStorage that needs to be  
> >> changed,
> >> and hence tried to fix it by wrapping the input stream so that the
> >> readline method always uses the read method on the input stream.  
> >> While
> >> this seems to work for me it introduces a level of 
> complexity in the
> >> cgi.py file, and possible some other bugs, that makes me think that
> >> adding the size argument for readline into the WSGI specification  
> >> isn't
> >> such bad idea after all.
> >
> > Since that change to cgi.py was a security fix I would strongly
> > recommend not to remove it and to change the WSGI spec instead.
> 
> Given that this change is now part of python 2.4.4 and python 
> 2.5, it  
> seems to me it is now a defacto requirement that all WSGI server  
> implementations must support readline with a size argument in order  
> to run any interesting software, despite the spec explicitly saying  
> that you shouldn't. I suspect simply modifying the spec to 
> follow the  
> current reality would be the least bad option.
> 
> But this kind of destabilizing breakage really shouldn't be allowed  
> to happen again. Once the error was discovered, the cgi.py change  
> should have been immediately reverted until either a decision was  
> made to change the WSGI spec, or else the change fixed to not break  
> WSGI compliant servers. This limbo situation is pretty bad.

...and it's still pretty bad. What can I do to speed up this process?
Write a change proposal for the WSGI spec?


Robert Brewer
System Architect
Amor Ministries
fumanchu at amor.org