<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>Can you please take a look at</div><div><a href="http://docs.python.org/dev/whatsnew/3.2.html#pep-3333-python-web-server-gateway-interface-v1-0-1">http://docs.python.org/dev/whatsnew/3.2.html#pep-3333-python-web-server-gateway-interface-v1-0-1</a></div><div>to see if it accurately recaps the resolution of the WSGI text/bytes issues.</div><div>I would appreciate any feedback, as it is likely that the whatsnew</div><div>document will be most people's first chance to hear the outcome</div><div>of the multi-year discussion.</div><div><br></div><div>Thanks,</div><div><br></div><div><br></div><div>Raymond</div><div><br></div><br><div><div>On Jan 6, 2011, at 3:50 PM, And Clover wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>On Tue, 2011-01-04 at 03:44 +0100, Victor Stinner wrote:<br><blockquote type="cite">What is this horrible encoding "bytes-as-unicode"?<br></blockquote><br>It is a unicode string decoded from bytes using ISO-8859-1. ISO-8859-1<br>is the encoding specified by the HTTP RFC, as well as having the happy<br>property of preserving every input byte. PEP 3333 requires it.<br><br><blockquote type="cite">os.environ is supposed to be correctly decoded and contain valid<br></blockquote>unicode characters.<br><br>It is not possible to ‘correctly’ decode to unicode for os.environ<br>because that decoding happens long before the web application (the<br>only party that knows what encoding should be in use) gets a look in.<br><br>Maybe the web application is using UTF-8, maybe it's using cp1252,<br>but if we let the server/gateway decide and do that decoding<br>before the application can do anything about it, we will get the wrong<br>encoding in *many* cases and the result will be permanent, unrecoverable<br>mangling of non-ASCII characters in submitted headers.<br><br><blockquote type="cite">If WSGI uses another encoding than the locale encoding (which is a bad<br></blockquote>idea),<br><br>It's an absolutely necessary idea. The locale encoding is nothing to do<br>with the web application's encoding. Windows applications need to be<br>able to use UTF-8 (which is never the ANSI code page), and web<br>applications in general need to be deployable to any server without<br>having to worry about the server's locale.<br><br>The locale-dependent status quo is that non-ASCII characters in URL<br>paths and other HTTP headers don't work for Python apps.<br><br>The recoding dances present in wsgiref's CGIHandler for 3.2 are<br>distasteful but completely necessary to normalise differences in<br>encodings used by various servers and platforms to generate their CGI<br>environment.<br><br><blockquote type="cite"> it should use os.environb and decodes keys and values using its<br></blockquote><blockquote type="cite">own encoding.<br></blockquote><br>Well yes, but:<br><br>(a) os.environb doesn't exist in previous Python 3.1, making it<br>impossible to implement WSGI before 3.2;<br>(b) a byte environment on Windows would have to be encoded<br>from the Unicode environment, with a server-specific encoding,<br>and then what encoding are you going to choose for the variables<br>that contain non-HTTP-sourced native Unicode strings (such as,<br>very commonly, Windows pathnames)?<br><br>The bytes-or-bytes-in-Unicode argument is something that has been<br>bounced around Web-SIG for literally *years*; this is what we ended up<br>with. Although I personally like bytes, frankly, a re-run of this<br>argument *again* whilst WSGI remains in perpetual stalemate does not<br>appeal. WSGI and wsgiref in Python 3.0-3.1 simply does not work. This<br>has long been an embarrassing situation for what is supposed to be a<br>leading<br>web language. Let us not perpetuate this sorry story to 3.2 as well.<br><br>-- <br>And Clover<br><a href="mailto:and@doxdesk.com">mailto:and@doxdesk.com</a> <a href="http://www.doxdesk.com">http://www.doxdesk.com</a><br><a href="skype:uknrbobince">skype:uknrbobince</a> gtalk:chat?jid=bobince@gmail.com<br><br><br>_______________________________________________<br>Python-Dev mailing list<br><a href="mailto:Python-Dev@python.org">Python-Dev@python.org</a><br>http://mail.python.org/mailman/listinfo/python-dev<br>Unsubscribe: http://mail.python.org/mailman/options/python-dev/raymond.hettinger%40gmail.com<br></div></blockquote></div><br></body></html>