<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body text="#330033" bgcolor="#ffffff">
On 11/20/2010 3:38 AM, Éric Araujo wrote:
<blockquote cite="mid:4CE7B34D.4020309@netwok.org" type="cite">
<pre wrap="">Hello
</pre>
<blockquote type="cite">
<pre wrap="">cgitb.enable(0,"d:\temp")
</pre>
</blockquote>
<pre wrap="">
Isn’t that expanded to “d:<tab>emp”?
</pre>
</blockquote>
<br>
Oops. Yes, that fixes the problem with creation of the temp file,
thanks for catching that. I now get a complete report of the
original error in the temp file (below). I am a bit less confused
now... but it seems that there are still a number of issues. Here
is an enumeration of problems I was hard pressed to make before you
removed my confusion on this issue.<br>
<br>
1. cgitb should expect to report to a binary stdout, using whatever
encoding (possibly ASCII) that seems appropriate for the output that
in generates.<br>
<br>
2. Some appropriate documentation or API or both should be provided
to enable a script to set "binary" mode for stdout for CGI scripts.
<a
href="http://www.eggheadcafe.com/software/aspnet/36023550/cgi-python-3-write-raw-binary.aspxhttp://www.eggheadcafe.com/software/aspnet/36023550/cgi-python-3-write-raw-binary.aspx">This
link</a> demonstrates the confusion (wish I had found it earlier)
that is encountered by such lack. One must tell msvcrt the stream
is binary (I had figured that out early on), one must also sidestep
the use of the cp1252 default when printing binary, one must also
choose a proper text encoding corresponding to the HTTP headers
sent. My second email in this thread, sent a few hours after the
first, shows a convenient set of cures for all but msvcrt (as long
as only "write" is used for writing. "print" support could be
added, similarly). Likely something along this line is needed for
stdin as well, I haven't yet experimented with uploading binary
content to a CGI.<br>
<br>
One could speculate about having the Python runtime auto-detect CGI
mode, but I don't know of any foolproof technique for that, and the
selection of the "proper" text encoding depends on the details of
the CGI, so having instead an API or two that assists with doing
this sort of thing would be better; the need for documentation, at
least, seems imperative.<br>
<br>
3. subprocess documentation could be improved to point out that
when using subprocess.PIPE to talk to a Python subprocess, that the
communications will be in binary. Again, I don't know of any way to
autodetect the subprocess environment, but if it were possible to
select an appropriate encoding and use it consistently on both sides
of the PIPE, that would be a convenience to its use; if not
possible, documenting the issue, and providing an API to use to
easily select such encodings both in client and server, would be
helpful.<br>
<br>
While the layers are all there, and ".buffer" is documented for
TextIOWrapper, the use of sys.stdout.buffer and the fact that it has
a full set of operations isn't immediately obvious from the
reference material; perhaps it is in a tutorial I haven't found,
but... I was looking, and didn't find it.<br>
<br>
Of course, subprocess may launch non-Python programs; they will have
their own ideas of binary vs text encoding, so it is important that
it is convenient to match them on the Python side.<br>
<br>
It would be nice if subprocess had a mechanism for providing
no-deadlock stdout data to the parent prior to the child
terminating. A CGI implementation via subprocess shouldn't
accumulate all of stdout (or all of stderr, for that matter,
although less important). I don't (yet) know enough about Python
threading to know if this is possible, but it certainly would be
useful.<br>
<br>
4. http.server has a number of bugs and limitations.<br>
4a. _url_collapse_path_split seems inefficient (although I have to
benchmark it against what I think would be more efficient), and for
its only use within http.server it produces the wrong information,
so the information has to be recombined and resplit to make it
function properly, adding to the perception of inefficiency.<br>
4b. Detection of "executable" on Windows is simply wrong. Unix
execution bits do not exist.<br>
4c. is_cgi doesn't properly handle PATHINFO parts of the path, this
is the other half of 4a. The Python2.x CGIHTTPServer.py had this
right, but the introduction and use of _url_collapse_path_split
broke it.<br>
4d. Searching for a ? to find an explicit query string should use
.find('?') rather than .rfind('?') as there is no prohibition on
using '?' within a query string, AFAIK.<br>
4e. doesn't set the REQUEST_URI, HTTP_HOST, or HTTP_PORT
environment variables for the CGI.<br>
4f. Should not send the 200 response until it sees if the CGI sends
a Status: header.<br>
4g. Should not buffer all of stdout: subprocess.communicate is
inappropriate for a web server CGI interface. The data should
stream through to avoid consuming inordinate amounts of memory. The
only solution within the current limitations of subprocess is to
abandon stderr, force the CGI to do its own error logging, and use
shutil.copyfileobj to hook up p.stdout to self.wfile once the
Status: message processing has happened.<br>
4h. Doesn't seem to close p.stdin (I'm not sure if that is
necessary, it may happen when p is garbage collected, but effort was
made to close p.stdout and p.stderr, which seem similar.)<br>
<br>
<br>
<table summary="heading" width="100%" border="0" cellpadding="2"
cellspacing="0">
<tbody>
<tr bgcolor="#6622aa">
<td valign="bottom"><font color="#ffffff" face="helvetica,
arial"> <br>
<big><big><strong>TypeError</strong></big></big></font></td>
<td valign="bottom" align="right"><font color="#ffffff"
face="helvetica, arial">Python 3.2a4:
c:\python32\python.exe<br>
Sat Nov 20 09:28:41 2010</font></td>
</tr>
</tbody>
</table>
<p>A problem occurred in a Python script. Here is the sequence of
function calls leading up to the error, in the order they
occurred.</p>
<table width="100%" border="0" cellpadding="0" cellspacing="0">
<tbody>
<tr>
<td bgcolor="#d8bbff"><big> </big><a>d:\my\py\test12.py</a> in
<strong></strong>()</td>
</tr>
<tr>
<td><font color="#909090"><tt> <small> 4</small> import cgitb<br>
</tt></font></td>
</tr>
<tr>
<td><font color="#909090"><tt> <small> 5</small> sys.stdout.write("out")<br>
</tt></font></td>
</tr>
<tr>
<td><font color="#909090"><tt> <small> 6</small> fhb = open("fhb", "wb")<br>
</tt></font></td>
</tr>
<tr>
<td><font color="#909090"><tt> <small> 7</small> cgitb.enable(0,"d:\\temp")<br>
</tt></font></td>
</tr>
<tr>
<td bgcolor="#ffccee"><tt>=><small> 8</small> fhb.write("abcdef") # try writing non-binary to binary file. Expect an error, of course.<br>
</tt></td>
</tr>
<tr>
<td><small><font color="#909090"><strong>fhb</strong> =
<_io.BufferedWriter name='fhb'>, fhb.<strong>write</strong> =
<built-in method write of _io.BufferedWriter
object></font></small></td>
</tr>
</tbody>
</table>
<strong>TypeError</strong>: 'str' does not support the buffer
interface
<br>
<tt><small> </small> </tt>args =
("'str' does not support the buffer interface",)
<br>
<tt><small> </small> </tt>with_traceback =
<built-in method with_traceback of TypeError object>
<br>
</body>
</html>