[Web-SIG] WSGI 2.0 Round 2: requirements and call for interest

[Armin Ronacher]

Hi,

I just want to reply to this because I think many people seem to be 
missing why things are done in a certain way.  Especially if the appear 
to be odd.

On 05/01/2016 12:26, Cory Benfield wrote:
> 1. WSGI is prone to header injection vulnerabilities issues by
> designdue to the conversion of HTTP headers to CGI-style environment
> variables: if the server doesn’t specifically prevent it, X-Foo and
> X_Foo both become HTTP_X_Foo. I don’t believe it’s a good choice to
> destructively encode headers, expect applications to undo the damage
> somehow, and introduce security vulnerabilities in the process. If
> mimicking CGI is still considered a must-have — 1% of current Python web
> programmers may have heard about it, most of them from PEP 3333 — then
> that burden should be pushed onto the server, not the application.
Headers always will have to be encoded destructively if you want any 
form of generic processing.  We need header joining, we need to 
normalize the keys already at least to the extend of the HTTP 
specification.  I'm happy to not perform the conversion of dashes to 
underscores but you will work in environments where this conversion was 
already done so the spec will need to deal with that case anyways.

The WSGI spec currently also does not sufficiently explain how to join 
headers.  In particular the cookie header was written without header 
joining in mind which is why it needs to be joined differently than all 
other headers.  Header joining also comes up as a big topic in HTTP 2
so the spec will need to cover this.

> 2. More generally, I fail to see how mixing HTTP headers,
> server-related inputs, and environment variables in a dict adds
> values. It prevents iterating on each collection separately. It only
> makes sense if not offering more features than CGI is a design goal;
> in that case, this discussion doesn’t serve a purpose anyway. It
> would be nicer and possibly more secure if the application received
> separately:
I think this is largely a nice to have, not something that has any 
overall benefits.  I rather just clean up the actual stupid things such 
as CONTENT_TYPE and CONTENT_LENGTH which cause a lot more real world 
friction than just the names of keys in general.  This really should not 
turn into meaningless bikeshedding about what information should be 
called.  Also consider how much code out there already assumes CGI/WSGI 
variables so any move off that really should have good reasons or we all 
will just waste enormous amounts just to transpose between the two 
representations.

> a. Configuration information, which servers could read from
> environment variables by default for backwards compatibility, but could
> also get through more secure channels and restrict to what the
> application needs in order to better isolate it from the entire OS.
What WSGI traditionally lacked was a setup phase where data could be 
passed to the application that was server specific but not request 
bound.  For instance there is no reason an application cannot get hold 
of wsgi.errors before a request comes in.  I would like to see this 
fixed in a new specification.

> 3. Stop pretending that HTTP is a unicode protocol, or at least stop
> ignoring reality when doing so. WSGI enforces ISO-8859-1-decoded str
> objects in the environ, which is just wrong. It’s all the more a
> surprising choice since this change was driven by Python 3, that UTF-8
> is the correct choice, and that Python 3 defaults to UTF-8. Django has
> to re-encode and re-decode before doing anything with HTTP headers:
I agree with this but you will have to have that fight with others.  I 
said many times before that values should never have been unicode values 
in the first place but certain decisions in the Python 3 standard 
library at the time prevented that.  In particular until 3.2 or so it 
was impossible to parse byte URLs.

> 5. Improve request / response length handling and connection closure.
> Armin and Graham have talked about in the past and know the topic
> better than I do. There’s also a rejected PEP by Armin which made
> sense to me.
I think last time I discussed that with Graham it was not clear what the 
solution is in the context of WSGI.  The idea that there is a 
content-length is laughable in the context of a real application where 
the server is performing conversions on the input and output stream.  We 
would need many more than just one content length and an automatically 
terminated input stream.

However at that point you will quickly realize that you can't have it 
both ways and you either have a WSGI like protocol, or raw access to 
sockets but certainly not both.  This topic has caused a lot of 
bikeshedding in the past and I fail to see how it will be differently 
this time.

My current thinking is that the most realistic approach to most of those 
problems will be the concept of framing on both the input and output 
side.  That's somewhat compatible with both chunked transports well as 
websockets.  But if we do go down this road we will most likely have to 
standardize on a library that implements WSGI as the complexity of 
dealing with this sort of stuff is significantly higher than what we had 
to do in the past.


Regards,
Armin