CGIHTTPServer and POST with Netscape versus IE

[Kevin Altis]

Well I was going to say this is fixed in the Python 2.3a1 release because I
thought Steve Holden had committed a fix, but I just checked the
CGIHTTPServer.py file and it doesn't appear to have changed, so maybe he'll
chime in here.

The PythonCard webserver sample contains a fix for the run_cgi() method of
CGIHTTPRequestHandler; it may be different than what Steve was going to
patch with but it seems to work fine. Here are the two code fragments, one
for the Unix section and the second one for the Windows section. The basic
idea is to just throw away bytes paste the content-length but not block
while doing so.
            if pid != 0:
                # Parent
                pid, sts = os.waitpid(pid, 0)
                # throw away additional data [see bug #427345]
                while select.select([self.rfile], [], [], 0)[0]:
                    waste = self.rfile.read(1)
                if sts:
                    self.log_error("CGI script exit status %#x", sts)
                return
            # Child


            if self.command.lower() == "post" and nbytes > 0:
                data = self.rfile.read(nbytes)
                fi.write(data)
                # KEA now throw away data past Content-length
                while select.select([self.rfile._sock], [], [], 0)[0] != []:
                    waste = self.rfile._sock.recv(1)

The one problem area I remember is that apparently someone added support for
Keep-Alive in BaseHTTPRequestHandler which might cause problems with the
code above. If you need a robust server, you probably shouldn't be using the
CGIHTTPServer module, but it is perfectly fine for light personal use and
testing. Since the HTTP/1.1 support in the request handler code is shallow
at best, adding Keep-Alive support was probably not a good idea.

It would probably be beneficial to look at these changes in context, in
which case you can get the 0.6.9 source

http://sourceforge.net/project/showfiles.php?group_id=19015

or just check out the webserver.py file in cvs

http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/pythoncard/PythonCardPrototyp
e/samples/webserver/

ka
---
Kevin Altis
altis at semi-retired.com
http://radio.weblogs.com/0102677/

"Peter Hansen" <peter at engcorp.com> wrote in message
news:3E1A4A4D.50C5DB52 at engcorp.com...
> (This may be my first question in this newsgroup.  Please
> be gentle! :-)
>
> We've encountered a little problem in a test program
> involving the standard module CGIHTTPServer.py and a POST
> operation from Internet Explorer.
>
> In summary, Netscape always works, but when using
> Internet Explorer 6.0 (SP1, probably all versions fail)
> a POST operation will result in a message (not a 404 or
> any other recognizable HTTP error) that reads:
>
>    "The page cannot be displayed"
>
> along with a page full of friendly advice about what I
> might have done wrong, and a title bar reading "Cannot
> find server".
>
> After extensive investigation, we discovered that IE is
> appending a CR LF pair to the request body, but is not
> reporting it in the Content-length field.  (The actual
> data is 23 bytes long, or 25 with these two.)  By
> modifying the CGIHTTPServer.py code to read two extra
> bytes, we "fix" the problem, but then of course the
> server hangs when accessed from Netscape which doesn't
> supply the extra two bytes.
>
> My questions:
>
> Is this a known problem for IE and maybe other browsers?
>
> Is it legal for a browser to do this, in which case is
> it something CGIHTTPServer.py should be handling?
>
> Can anyone point me to other code (haven't been able to
> find the relevant lines in Zope yet) in web servers which
> might already handle this in a general and safe manner?
>
> Thanks for any pointers.  I've tried a few Google searches
> so far without any bites, but it seems pretty obvious that
> this is something that is known and solved, if I knew
> where to look, and it seems likely the fix should be added
> to CGIHTTPServer.py.
>
> -Peter