[New-bugs-announce] [issue32541] cgi.FieldStorage constructor assumes all lines terminate with \n

Ian Craggs report at bugs.python.org
Fri Jan 12 13:41:57 EST 2018

New submission from Ian Craggs <icraggs at gmail.com>:

Using cgi.FieldStorage in an HTTP server in a subclass of BaseHTTPRequestHandler, parsing the request with:

form = cgi.FieldStorage(fp=self.rfile,

This has been working fine with clients using the Python requests library. Now processing requests from a Java library (org.apache.cxf.jaxrs.client.WebClient), the final line in a multipart request does not include the (\r)\n, which causes the final read to hang until a socket timeout.  The read in question is in cgi.py, read_lines_to_outerboundary:

line = self.fp.readline(1<<16) # bytes

(line 824 in Python 3.6.2).  I changed this read to not assume the termination of the final line with \n:

    def read_line(self, last_boundary):
        line = self.fp.readline(len(last_boundary))
        if line != last_boundary and not line.endswith(b"\n"):
            line += self.fp.readline((1<<16) - len(last_boundary))
        return line

and the request worked.  The Java library is being used in tests against our production web server so I assume that is working correctly.  

Perhaps I am misusing the FieldStorage class, I don't know, I'm not expert on this.

components: Library (Lib), macOS
messages: 309868
nosy: Ian Craggs, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: cgi.FieldStorage constructor assumes all lines terminate with \n
type: behavior
versions: Python 3.6

Python tracker <report at bugs.python.org>

More information about the New-bugs-announce mailing list