From davidgshi at yahoo.co.uk  Wed Mar  4 22:28:21 2009
From: davidgshi at yahoo.co.uk (David Shi)
Date: Wed, 4 Mar 2009 21:28:21 +0000 (GMT)
Subject: [Web-SIG] Use both Python and Javascript in html webpages
Message-ID: <829575.32812.qm@web26304.mail.ukl.yahoo.com>

Can we use both Python and Javascript in html webpages??? Any demo on this?
?
Regards.
?
David


      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/web-sig/attachments/20090304/7e42d966/attachment.htm>

From pstradomski at gmail.com  Wed Mar  4 23:02:21 2009
From: pstradomski at gmail.com (=?utf-8?q?Pawe=C5=82_Stradomski?=)
Date: Wed, 4 Mar 2009 23:02:21 +0100
Subject: [Web-SIG] Use both Python and Javascript in html webpages
In-Reply-To: <829575.32812.qm@web26304.mail.ukl.yahoo.com>
References: <829575.32812.qm@web26304.mail.ukl.yahoo.com>
Message-ID: <200903042302.21264.pstradomski@gmail.com>

W li?cie David Shi z dnia ?roda 04 marca 2009:
> Can we use both Python and Javascript in html webpages??? Any demo on this?
> ?
Looks like a large confusion... you can use python on the server side and 
javascript on the client side (so basically only javascript is used in the 
html webpage itself).

OK. I guess you could use IronPython in silverlight.

-- 
Pawe? Stradomski

From alan at xhaus.com  Thu Mar  5 12:58:17 2009
From: alan at xhaus.com (Alan Kennedy)
Date: Thu, 5 Mar 2009 11:58:17 +0000
Subject: [Web-SIG] Use both Python and Javascript in html webpages
In-Reply-To: <829575.32812.qm@web26304.mail.ukl.yahoo.com>
References: <829575.32812.qm@web26304.mail.ukl.yahoo.com>
Message-ID: <4a951aa00903050358w69d7eb22nd5ac4ae1f6178960@mail.gmail.com>

[David]
> Can we use both Python and Javascript in html webpages??? Any demo on this?

If you're willing to write rpython, PyPy can compile it to javascript
which run can in a browser.

http://codespeak.net/pypy/dist/pypy/doc/js/using.html

HTH,

Alan.

From timothy.soehnlin at gmail.com  Fri Mar  6 12:47:48 2009
From: timothy.soehnlin at gmail.com (Timothy Soehnlin)
Date: Fri, 6 Mar 2009 06:47:48 -0500
Subject: [Web-SIG] Web-SIG Digest, Vol 64, Issue 2
In-Reply-To: <mailman.48.1236337225.2799.web-sig@python.org>
References: <mailman.48.1236337225.2799.web-sig@python.org>
Message-ID: <5817f2f30903060347o56652894pad0e09dde632884@mail.gmail.com>

If you are using Alchemy <http://labs.adobe.com/technologies/alchemy/> for
Flash 10, I believe someone has already ported python (perhaps internally at
Adobe, not public yet) to run inside of the Flash VM.  With the
ExternalInterface classes in the Flash VM you could allow JavaScript and
Python to coexist in the browser space.

2009/3/6 <web-sig-request at python.org>

> Send Web-SIG mailing list submissions to
>        web-sig at python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>        http://mail.python.org/mailman/listinfo/web-sig
> or, via email, send a message with subject or body 'help' to
>        web-sig-request at python.org
>
> You can reach the person managing the list at
>        web-sig-owner at python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Web-SIG digest..."
>
> Today's Topics:
>
>   1. Re: Use both Python and Javascript in html webpages (Alan Kennedy)
>
>
> ---------- Forwarded message ----------
> From: Alan Kennedy <alan at xhaus.com>
> To: web-sig at python.org
> Date: Thu, 5 Mar 2009 11:58:17 +0000
> Subject: Re: [Web-SIG] Use both Python and Javascript in html webpages
> [David]
> > Can we use both Python and Javascript in html webpages?   Any demo on
> this?
>
> If you're willing to write rpython, PyPy can compile it to javascript
> which run can in a browser.
>
> http://codespeak.net/pypy/dist/pypy/doc/js/using.html
>
> HTH,
>
> Alan.
>
>
> _______________________________________________
> Web-SIG mailing list
> Web-SIG at python.org
> http://mail.python.org/mailman/listinfo/web-sig
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/web-sig/attachments/20090306/90f228c4/attachment.htm>

From kevinar18 at hotmail.com  Thu Mar 19 02:48:22 2009
From: kevinar18 at hotmail.com (Kevin Ar18)
Date: Wed, 18 Mar 2009 21:48:22 -0400
Subject: [Web-SIG] PostgreSQL Admin
Message-ID: <SNT110-W37C1D88DDC8A7C646D8434AA960@phx.gbl>




_________________________________________________________________
Hotmail? is up to 70% faster. Now good news travels really fast. 
http://windowslive.com/online/hotmail?ocid=TXT_TAGLM_WL_HM_70faster_032009

From kevinar18 at hotmail.com  Tue Mar 24 22:47:54 2009
From: kevinar18 at hotmail.com (Kevin Ar18)
Date: Tue, 24 Mar 2009 17:47:54 -0400
Subject: [Web-SIG] Browser email & SQL Admin made with Python?
Message-ID: <SNT110-W598C3CBAD55461B914CAC9AA930@phx.gbl>


If you want the details, read this paragraph:

I would like to have some kind of app that allows me to manage a list of contacts in a database like fashion (where I can add new table columns and add new data to people).  Yet, at the same time, I would like to integrate this with other things over time.  For example, I would like to integrate the database with a web based email system.  Instead of having one inbox, I'd have one inbox per contact in my database. In that way I can look at the contact and see only emails related to that contact.... or read several emails together as an ongoing conversation. Or setup a notification if someone never responds to the email, so I can try again.  And, of course, over time, I may want to add more features....

 

So, I figured I could use a Python web framework.  Then take a pre-built sql admin app and a pre-built web based email system and build from there.

 

Are there any "PostgreSQL admin" apps or "email apps" already made using Python or am I going to have to build a sql admin and a browser based email all from scratch :( ?

 

Is there some other way that I can build what I need (with the option to exptend it in Python as I need)?  like a CMS?

 

Would it help if I explained the concept in more detail or if I asked this in a different Python mailing list?

_________________________________________________________________
Get quick access to your favorite MSN content with Internet Explorer 8. 
http://ie8.msn.com/microsoft/internet-explorer-8/en-us/ie8.aspx?ocid=B037MSN55C0701A
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/web-sig/attachments/20090324/3d197110/attachment.htm>

From alan at xhaus.com  Fri Mar 27 21:30:52 2009
From: alan at xhaus.com (Alan Kennedy)
Date: Fri, 27 Mar 2009 15:30:52 -0500
Subject: [Web-SIG] WSGI Open Space @ PyCon.
Message-ID: <4a951aa00903271330w48055728i582263fcf67687e5@mail.gmail.com>

Dear all,

For those of you at PyCon, there is a WSGI Open Space @ 5pm today (Friday).

The sub-title of the open space is "Does WSGI need revision"?

An example: Philip Jenvey (http://dunderboss.blogspot.com/) raised the
need for something akin to what Java folks call "Lifecycle methods",
so that WSGI apps can do initialization and finalization.

http://java.sun.com/j2ee/tutorial/1_3-fcs/doc/Servlets4.html

I'm sure there are plenty of other topics that could be discussed as well.

See you @5pm.

Alan.

From graham.dumpleton at gmail.com  Fri Mar 27 22:33:07 2009
From: graham.dumpleton at gmail.com (Graham Dumpleton)
Date: Sat, 28 Mar 2009 08:33:07 +1100
Subject: [Web-SIG] WSGI Open Space @ PyCon.
In-Reply-To: <4a951aa00903271330w48055728i582263fcf67687e5@mail.gmail.com>
References: <4a951aa00903271330w48055728i582263fcf67687e5@mail.gmail.com>
Message-ID: <88e286470903271433s10190ccfrf0b6fd018b764e0e@mail.gmail.com>

2009/3/28 Alan Kennedy <alan at xhaus.com>:
> Dear all,
>
> For those of you at PyCon, there is a WSGI Open Space @ 5pm today (Friday).
>
> The sub-title of the open space is "Does WSGI need revision"?
>
> An example: Philip Jenvey (http://dunderboss.blogspot.com/) raised the
> need for something akin to what Java folks call "Lifecycle methods",
> so that WSGI apps can do initialization and finalization.
>
> http://java.sun.com/j2ee/tutorial/1_3-fcs/doc/Servlets4.html
>
> I'm sure there are plenty of other topics that could be discussed as well.
>
> See you @5pm.

Please, whatever you do, do not go making anything like this, or even
a standard request/response object a part of the WSGI standard.

Create a new specification for this 'application level' stuff which is
distinct from WSGI and leave WSGI as being the 'server gateway
interface' as it is really meant to be.

This should go as far as coming up with a better middleware
abstraction for the application layer and discouraging people from
using WSGI middleware as they exist now.

All these new components, although the reference implementation may
host on top of WSGI, should other wise hide WSGI thereby allowing them
to be hosted on top of an alternate interface or a newer revision of
WSGI, such as the minimal revision talked about for WSGI 2.0.

If this stuff is all pushed into the WSGI specification then it will
be a backward step as far as improving the situation for Python as far
as web hosting availability.

I have been trying to put together a blog entry saying just this and
other things about the role of WSGI, but just haven't had the time.
Since sprint almost starting, probably will not get a chance now.

Graham

From mark.mchristensen at gmail.com  Sat Mar 28 07:14:08 2009
From: mark.mchristensen at gmail.com (Mark Ramm)
Date: Sat, 28 Mar 2009 01:14:08 -0500
Subject: [Web-SIG] WSGI Open Space @ PyCon.
In-Reply-To: <88e286470903271433s10190ccfrf0b6fd018b764e0e@mail.gmail.com>
References: <4a951aa00903271330w48055728i582263fcf67687e5@mail.gmail.com>
	<88e286470903271433s10190ccfrf0b6fd018b764e0e@mail.gmail.com>
Message-ID: <ec2d68db0903272314u6aefd01dm1c3146c22b6e28b0@mail.gmail.com>

My thought is that we should do a couple things to the wsgi standard,
and then anything like the lifecycle methods gets addresse,d it should
be pushed into a "container" standard or something.

I think Robert Brewer's WSGI Service Bus proposal that he made a
couple years ago at PyCon needs a new name, but it does provide a good
start on the lifecycle stuff.

As for WSGI itself, we should make a couple of smaller changes which I
think will likely be a bit easier to quantify and agree on. I'm sure
lots more folks from yesterday's discussion will chip in here, but
this is my take on the things we discussed.

1) We should drop the start_response callable, and return a three
member tupple from the wsgi callable:

   def wsgi2app(environ):
         ....
         return (status_code, headers, response_iterator)

2) We should turn wsgi.input into an iterator rather than a somewhat
file-like object.   WSGI middleware that reads part of the wsgi.input
iterator should make sure to restore it using itertools.chain or
replace it with whatever.  If there's a content length specified from
the server the middleware should be responsible for maintaining or
deleting that information as nessisary.   Content length of 0 is
allowed and means there's no data, whereas an unspecified or content
length, indicates that the value is unknown.  This will create a good
symmetry between the input and output methods, and seems like a good
comprimise between flexibility for middleware creators, and ease of
use for consumers.


3) The server should encode the headers and include explicit
information about the encoding in the wsgi environ variable.  So that
any assumptions about what they bytes in the headers represent is made
explicit.

I think we're all very sold on item 1, and items 2 and 3 require more
thinking, but seemed reasonable to those present at the discussion
this afternoon.    Hopefully we'll be meeting again on Saturday and
will be able to continue to think through this stuff and push this all
forward some more.

I'm sure there also be several other minor tweeks to the spec like:

* Not de-encoding encoded slashes in path strings, so that
applications can tell the difference between path separators and
encoded slashes.
* adding a "ClientWentAway" exception that indicates that wsgi.imput
has not been officially exausted, but that the client went away before
wsgi.input was fully populated.

I'm sure there are more.   It might also be interesting to look at
Rack and Jack the ruby and javascript implementations of the WSGI
idea:

http://jackjs.org/
http://rack.rubyforge.org/doc/files/SPEC.html

--Mark Ramm


On Fri, Mar 27, 2009 at 5:33 PM, Graham Dumpleton
<graham.dumpleton at gmail.com> wrote:
> 2009/3/28 Alan Kennedy <alan at xhaus.com>:
>> Dear all,
>>
>> For those of you at PyCon, there is a WSGI Open Space @ 5pm today (Friday).
>>
>> The sub-title of the open space is "Does WSGI need revision"?
>>
>> An example: Philip Jenvey (http://dunderboss.blogspot.com/) raised the
>> need for something akin to what Java folks call "Lifecycle methods",
>> so that WSGI apps can do initialization and finalization.
>>
>> http://java.sun.com/j2ee/tutorial/1_3-fcs/doc/Servlets4.html
>>
>> I'm sure there are plenty of other topics that could be discussed as well.
>>
>> See you @5pm.
>
> Please, whatever you do, do not go making anything like this, or even
> a standard request/response object a part of the WSGI standard.
>
> Create a new specification for this 'application level' stuff which is
> distinct from WSGI and leave WSGI as being the 'server gateway
> interface' as it is really meant to be.
>
> This should go as far as coming up with a better middleware
> abstraction for the application layer and discouraging people from
> using WSGI middleware as they exist now.
>
> All these new components, although the reference implementation may
> host on top of WSGI, should other wise hide WSGI thereby allowing them
> to be hosted on top of an alternate interface or a newer revision of
> WSGI, such as the minimal revision talked about for WSGI 2.0.
>
> If this stuff is all pushed into the WSGI specification then it will
> be a backward step as far as improving the situation for Python as far
> as web hosting availability.
>
> I have been trying to put together a blog entry saying just this and
> other things about the role of WSGI, but just haven't had the time.
> Since sprint almost starting, probably will not get a chance now.
>
> Graham
> _______________________________________________
> Web-SIG mailing list
> Web-SIG at python.org
> Web SIG: http://www.python.org/sigs/web-sig
> Unsubscribe: http://mail.python.org/mailman/options/web-sig/mark.mchristensen%40gmail.com
>



-- 
Mark Ramm-Christensen
email: mark at compoundthinking dot com
blog: www.compoundthinking.com/blog

From graham.dumpleton at gmail.com  Sat Mar 28 07:53:25 2009
From: graham.dumpleton at gmail.com (Graham Dumpleton)
Date: Sat, 28 Mar 2009 17:53:25 +1100
Subject: [Web-SIG] WSGI Open Space @ PyCon.
In-Reply-To: <ec2d68db0903272314u6aefd01dm1c3146c22b6e28b0@mail.gmail.com>
References: <4a951aa00903271330w48055728i582263fcf67687e5@mail.gmail.com>
	<88e286470903271433s10190ccfrf0b6fd018b764e0e@mail.gmail.com>
	<ec2d68db0903272314u6aefd01dm1c3146c22b6e28b0@mail.gmail.com>
Message-ID: <88e286470903272353r58eba4dt2f89ceab4a33644@mail.gmail.com>

2009/3/28 Mark Ramm <mark.mchristensen at gmail.com>:
> My thought is that we should do a couple things to the wsgi standard,
> and then anything like the lifecycle methods gets addresse,d it should
> be pushed into a "container" standard or something.
>
> I think Robert Brewer's WSGI Service Bus proposal that he made a
> couple years ago at PyCon needs a new name, but it does provide a good
> start on the lifecycle stuff.

>From memory, my concern over that specification was that it sort of
assumed that applications were all preloaded. I am not sure how well
it would work where lazy loading is performed and where there are
multiple WSGI applications running in a interpreter but where they
weren't themselves mounted within a WSGI application, but through
external mechanisms dictated by the WSGI hosting mechanism.

> As for WSGI itself, we should make a couple of smaller changes which I
> think will likely be a bit easier to quantify and agree on. I'm sure
> lots more folks from yesterday's discussion will chip in here, but
> this is my take on the things we discussed.
>
> 1) We should drop the start_response callable, and return a three
> member tupple from the wsgi callable:
>
> ? def wsgi2app(environ):
> ? ? ? ? ....
> ? ? ? ? return (status_code, headers, response_iterator)
>
> 2) We should turn wsgi.input into an iterator rather than a somewhat
> file-like object. ? WSGI middleware that reads part of the wsgi.input
> iterator should make sure to restore it using itertools.chain or
> replace it with whatever. ?If there's a content length specified from
> the server the middleware should be responsible for maintaining or
> deleting that information as nessisary. ? Content length of 0 is
> allowed and means there's no data, whereas an unspecified or content
> length, indicates that the value is unknown. ?This will create a good
> symmetry between the input and output methods, and seems like a good
> comprimise between flexibility for middleware creators, and ease of
> use for consumers.

The problem with an iterator/generator is how do you control the size
of the chunks of data returned. An iterator also probably isn't going
to make chunked request content any easier to handle.

It may be easier to change how people use the wsgi.input that exists
now. First off allow one to say:

  wsgi.input.read()

to get all input, rather than passing CONTENT_LENGTH as argument.

For consume all data in chunks until exhausted, require a proper eof
indicator in the form of an empty string read, then can say:

  s = wsgi.input.read(BLOCKSIZE)
  while s:
    # do something with 's'
    s = wsgi.input.read(BLOCKSIZE)

That way you don't have to make around with checking how much you have read.

This does require that an exception be raised if client closes
connection before all data expected was read.

The question thus is, what would be the actual benefits of changing to
an iterator/generator.

> 3) The server should encode the headers and include explicit
> information about the encoding in the wsgi environ variable. ?So that
> any assumptions about what they bytes in the headers represent is made
> explicit.

That could be fun. For Apache/mod_wsgi at least you are in control of
the conversion. In Python 3.0 and CGI/WSGI the os.environ variables
are already unicode strings because they were converted by Python. How
this is done varies between UNIX and Windows platforms.

> I think we're all very sold on item 1, and items 2 and 3 require more
> thinking, but seemed reasonable to those present at the discussion
> this afternoon. ? ?Hopefully we'll be meeting again on Saturday and
> will be able to continue to think through this stuff and push this all
> forward some more.
>
> I'm sure there also be several other minor tweeks to the spec like:

Yeah, like defining how wsgi.file_wrapper should behave where response
Content-Length is defined but wrapped file actually provides more
content than that.

> * Not de-encoding encoded slashes in path strings, so that
> applications can tell the difference between path separators and
> encoded slashes.

When sitting on top of Apache, whether it be mod_wsgi, fastcgi, scgi,
ajp or CGI, you don't really have much choice, you get what Apache
gives you.

> * adding a "ClientWentAway" exception that indicates that wsgi.imput
> has not been officially exausted, but that the client went away before
> wsgi.input was fully populated.

The problem with an exception is what namespace do you put it in. You
almost need to have the type as part of the WSGI environment. You may
just be better standardising it by saying that an IOError must be
raised and leave it at that. At the moment most stuff doesn't even pay
attention to the fact that an exception could occur for some WSGI
adapters.

> I'm sure there are more. ? It might also be interesting to look at
> Rack and Jack the ruby and javascript implementations of the WSGI
> idea:
>
> http://jackjs.org/
> http://rack.rubyforge.org/doc/files/SPEC.html
>
> --Mark Ramm
>
>
> On Fri, Mar 27, 2009 at 5:33 PM, Graham Dumpleton
> <graham.dumpleton at gmail.com> wrote:
>> 2009/3/28 Alan Kennedy <alan at xhaus.com>:
>>> Dear all,
>>>
>>> For those of you at PyCon, there is a WSGI Open Space @ 5pm today (Friday).
>>>
>>> The sub-title of the open space is "Does WSGI need revision"?
>>>
>>> An example: Philip Jenvey (http://dunderboss.blogspot.com/) raised the
>>> need for something akin to what Java folks call "Lifecycle methods",
>>> so that WSGI apps can do initialization and finalization.
>>>
>>> http://java.sun.com/j2ee/tutorial/1_3-fcs/doc/Servlets4.html
>>>
>>> I'm sure there are plenty of other topics that could be discussed as well.
>>>
>>> See you @5pm.
>>
>> Please, whatever you do, do not go making anything like this, or even
>> a standard request/response object a part of the WSGI standard.
>>
>> Create a new specification for this 'application level' stuff which is
>> distinct from WSGI and leave WSGI as being the 'server gateway
>> interface' as it is really meant to be.
>>
>> This should go as far as coming up with a better middleware
>> abstraction for the application layer and discouraging people from
>> using WSGI middleware as they exist now.
>>
>> All these new components, although the reference implementation may
>> host on top of WSGI, should other wise hide WSGI thereby allowing them
>> to be hosted on top of an alternate interface or a newer revision of
>> WSGI, such as the minimal revision talked about for WSGI 2.0.
>>
>> If this stuff is all pushed into the WSGI specification then it will
>> be a backward step as far as improving the situation for Python as far
>> as web hosting availability.
>>
>> I have been trying to put together a blog entry saying just this and
>> other things about the role of WSGI, but just haven't had the time.
>> Since sprint almost starting, probably will not get a chance now.
>>
>> Graham
>> _______________________________________________
>> Web-SIG mailing list
>> Web-SIG at python.org
>> Web SIG: http://www.python.org/sigs/web-sig
>> Unsubscribe: http://mail.python.org/mailman/options/web-sig/mark.mchristensen%40gmail.com
>>
>
>
>
> --
> Mark Ramm-Christensen
> email: mark at compoundthinking dot com
> blog: www.compoundthinking.com/blog
>

From mark.mchristensen at gmail.com  Sat Mar 28 13:18:44 2009
From: mark.mchristensen at gmail.com (Mark Ramm)
Date: Sat, 28 Mar 2009 08:18:44 -0400
Subject: [Web-SIG] WSGI Open Space @ PyCon.
In-Reply-To: <88e286470903272353r58eba4dt2f89ceab4a33644@mail.gmail.com>
References: <4a951aa00903271330w48055728i582263fcf67687e5@mail.gmail.com>
	<88e286470903271433s10190ccfrf0b6fd018b764e0e@mail.gmail.com>
	<ec2d68db0903272314u6aefd01dm1c3146c22b6e28b0@mail.gmail.com>
	<88e286470903272353r58eba4dt2f89ceab4a33644@mail.gmail.com>
Message-ID: <ec2d68db0903280518k6f800f8x15ffadaaa49ae474@mail.gmail.com>

On Sat, Mar 28, 2009 at 2:53 AM, Graham Dumpleton
<graham.dumpleton at gmail.com> wrote:
> 2009/3/28 Mark Ramm <mark.mchristensen at gmail.com>:
>> My thought is that we should do a couple things to the wsgi standard,
>> and then anything like the lifecycle methods gets addresse,d it should
>> be pushed into a "container" standard or something.
>>
>> I think Robert Brewer's WSGI Service Bus proposal that he made a
>> couple years ago at PyCon needs a new name, but it does provide a good
>> start on the lifecycle stuff.
>
> From memory, my concern over that specification was that it sort of
> assumed that applications were all preloaded. I am not sure how well
> it would work where lazy loading is performed and where there are
> multiple WSGI applications running in a interpreter but where they
> weren't themselves mounted within a WSGI application, but through
> external mechanisms dictated by the WSGI hosting mechanism.
>
>> As for WSGI itself, we should make a couple of smaller changes which I
>> think will likely be a bit easier to quantify and agree on. I'm sure
>> lots more folks from yesterday's discussion will chip in here, but
>> this is my take on the things we discussed.
>>
>> 1) We should drop the start_response callable, and return a three
>> member tupple from the wsgi callable:
>>
>> ? def wsgi2app(environ):
>> ? ? ? ? ....
>> ? ? ? ? return (status_code, headers, response_iterator)
>>
>> 2) We should turn wsgi.input into an iterator rather than a somewhat
>> file-like object. ? WSGI middleware that reads part of the wsgi.input
>> iterator should make sure to restore it using itertools.chain or
>> replace it with whatever. ?If there's a content length specified from
>> the server the middleware should be responsible for maintaining or
>> deleting that information as nessisary. ? Content length of 0 is
>> allowed and means there's no data, whereas an unspecified or content
>> length, indicates that the value is unknown. ?This will create a good
>> symmetry between the input and output methods, and seems like a good
>> comprimise between flexibility for middleware creators, and ease of
>> use for consumers.
>
> The problem with an iterator/generator is how do you control the size
> of the chunks of data returned. An iterator also probably isn't going
> to make chunked request content any easier to handle.
>
> It may be easier to change how people use the wsgi.input that exists
> now. First off allow one to say:
>
> ?wsgi.input.read()
>
> to get all input, rather than passing CONTENT_LENGTH as argument.
>
> For consume all data in chunks until exhausted, require a proper eof
> indicator in the form of an empty string read, then can say:
>
> ?s = wsgi.input.read(BLOCKSIZE)
> ?while s:
> ? ?# do something with 's'
> ? ?s = wsgi.input.read(BLOCKSIZE)
>
> That way you don't have to make around with checking how much you have read.
>
> This does require that an exception be raised if client closes
> connection before all data expected was read.
>
> The question thus is, what would be the actual benefits of changing to
> an iterator/generator.
>
>> 3) The server should encode the headers and include explicit
>> information about the encoding in the wsgi environ variable. ?So that
>> any assumptions about what they bytes in the headers represent is made
>> explicit.
>
> That could be fun. For Apache/mod_wsgi at least you are in control of
> the conversion. In Python 3.0 and CGI/WSGI the os.environ variables
> are already unicode strings because they were converted by Python. How
> this is done varies between UNIX and Windows platforms.
>
>> I think we're all very sold on item 1, and items 2 and 3 require more
>> thinking, but seemed reasonable to those present at the discussion
>> this afternoon. ? ?Hopefully we'll be meeting again on Saturday and
>> will be able to continue to think through this stuff and push this all
>> forward some more.
>>
>> I'm sure there also be several other minor tweeks to the spec like:
>
> Yeah, like defining how wsgi.file_wrapper should behave where response
> Content-Length is defined but wrapped file actually provides more
> content than that.
>
>> * Not de-encoding encoded slashes in path strings, so that
>> applications can tell the difference between path separators and
>> encoded slashes.
>
> When sitting on top of Apache, whether it be mod_wsgi, fastcgi, scgi,
> ajp or CGI, you don't really have much choice, you get what Apache
> gives you.

Which is fine, I guess, but it does make it impossible to tell the
difference between real slashes and encoded ones in WSGI application
code.  I would love it if there were some way around that.

>> * adding a "ClientWentAway" exception that indicates that wsgi.imput
>> has not been officially exhausted, but that the client went away before
>> wsgi.input was fully populated.
>
> The problem with an exception is what namespace do you put it in. You
> almost need to have the type as part of the WSGI environment. You may
> just be better standardising it by saying that an IOError must be
> raised and leave it at that. At the moment most stuff doesn't even pay
> attention to the fact that an exception could occur for some WSGI
> adapters.

That would be fine with me.   The issue is definitely not the
exception name, but the fact that one can be raised/caught in a
standard way.

From fumanchu at aminus.org  Sat Mar 28 16:04:39 2009
From: fumanchu at aminus.org (Robert Brewer)
Date: Sat, 28 Mar 2009 08:04:39 -0700
Subject: [Web-SIG] WSGI Open Space @ PyCon.
In-Reply-To: <4a951aa00903271330w48055728i582263fcf67687e5@mail.gmail.com>
References: <4a951aa00903271330w48055728i582263fcf67687e5@mail.gmail.com>
Message-ID: <F1962646D3B64642B7C9A06068EE1E6407B53581@ex10.hostedexchange.local>

Alan Kennedy wrote:
> For those of you at PyCon, there is a WSGI Open Space @ 5pm today
> (Friday).
> 
> The sub-title of the open space is "Does WSGI need revision"?

Hi all,

We had a good meeting but it was too short. We plan on having another
Open Space meeting at 5pm today (Saturday) to continue the discussion.

Those present at the first meeting:

 * Mark Ramm (TG)
 * Frank Wierzbicki (Jython)
 * Mike Orr (Pylons)
 * Bob Brewer (CherryPy)
 * Ian Bicking (Paste, etc)
 * Eric Larson (CherryPy, WSGI advocate)
 * Alan Kennedy (WSGI gateway servlets/Jython)
 * Michael Twomey (WSGI user + HTTP servers + Twisted)
 * Jorge Vargas (TG)
 * Ian Charnas (TG CMS)
 * Phil Jenvey (Pylons + Jython)
 * 8+ others, mostly lurking

Topic: Unicode values in the WSGI environ
-----------------------------------------

Request:

We discussed any blockers to this. Request headers are pretty easy since
the spec requires falling back to ISO-8859-1. HOST might be
IDNA-encoded, but that can be consistent, too. What to do with
SCRIPT_NAME and PATH_INFO is more difficult, since there is no consensus
on encoding--there's percent encoding, of course, but that doesn't cover
multi-byte character encodings. The request body (wsgi.input) would not
be decoded.

The general consensus was that we could specify the decoding of the
request metadata (request line and headers) to be the responsibility of
WSGI servers, and leave it at that--different servers may offer
different means of configuring the decoding. There are pitfalls to this
approach, which the spec should address; in particular, some decoding
strategies may not be reversible. In addition, apps should at least know
which encoding the server chose via a new WSGI environ entry. Name
suggestions welcome.

There was some discussion, but no agreement, on including both unicode
and str (str and bytes in Python 3.x) versions of these values in the
environ.


Response:

I *think* the general consensus was that applications could return
unicode status and headers. But we also noted that servers should be
able to encode any bytes using IDNA/ISO-8859-1/RFC-2047/utf-8 where
appropriate. Not sure where this ended up.


Topic: Return a tuple of (status, headers, body)
------------------------------------------------

That is, get rid of the start_response callable. The general consensus
was that this is a simple, but powerful improvement, which Rack/Jack
have demonstrated already. The "simplest possible application object"
would change from this:

  def simple_app(environ, start_response):
      """Simplest possible application object"""
      status = '200 OK'
      response_headers = [('Content-type','text/plain')]
      start_response(status, response_headers)
      return ['Hello world!\n']

...to this:

  def simple_app(environ):
      """Simplest possible application object"""
      status = '200 OK'
      response_headers = [('Content-type','text/plain')]
      body = ['Hello world!\n']
      return (status, response_headers, body)


Topic: wsgi.input
-----------------

Some authors have found problems with the file-like design of
wsgi.input, specifically, that the rfile is not rewindable/seekable; if
a middleware component reads part of the stream, things could get ugly
if a later app tries to read the full stream.

Discussion centered around replacing the current file-like wsgi.input
with an iterable. Origin servers would be responsible for chunking the
request body, and each consumer would be required to re-stream each
chunk.

This topic didn't seem to have the strong consensus that the previous
ones did. My personal feeling was that we need more time to tease out
the new problems this approach would raise (perhaps with an
implementation or two).

Making this change would, however, solve a related issue: how apps can
safely read the full wsgi.input when the client did not supply a
Content-Length (i.e. the servers would handle all that).


Other topics raised but deferred
--------------------------------

 * Standard Request/Response objects. There was one call for, and many
against, this.
 * Lots of little changes: the server's supported HTTP version,
file_wrapper edge cases, etc.
 * Python 3, and the scheduling of WSGI improvements
 * Asynchronous WSGI support. Mostly non-existent. Fix it? Fork it? Drop
it?
 * Lifecycle methods (start/stop/etc event API driven by the container)
 * Remove app_iter.close?


Robert Brewer
fumanchu at aminus.org


From brandon at rhodesmill.org  Sat Mar 28 20:24:33 2009
From: brandon at rhodesmill.org (Brandon Craig Rhodes)
Date: Sat, 28 Mar 2009 15:24:33 -0400
Subject: [Web-SIG] thoughts on an iterator
Message-ID: <874oxdxxfy.fsf@rhodesmill.org>

Graham, I confess that it was I who brought up the idea of a wsgi.input
iterator at the WSGI Open Space yesterday evening. :-) The discussion
seemed to be assuming a file-like input object that could be read from
by a piece of middleware, then "backed up" or "rewound" before passing
it down to the next layer.  This seemed to have problems: it doesn't
support the case where the middleware wants to alter the input or pass
it piecemeal down to the client as it comes in, and it also means that
the *entire* input stream has to be kept around in memory for the
lifetime of the whole request in case the client reading it is not the
"real client" at the bottom of the stack, and a request is coming that
will ask for the whole thing to be replayed.

So, I suggested placing the responsibility for rewind and buffering on
the middleware.  You want to read 2k of the input to make a middleware
decision before invoking the next layer down?  Then read it, and pass
along a fresh iterator that first yields that 2k, then starts yielding
everything from the partially-read iterator.  Or, you can pass along a
filter iterator that scans or changes the entire stream as it reads it
from the upstream iterator.

But, having through more about the idea, I think that your criticisms,
Graham, are exactly on-target.  Iterators don't give enough control to
the reader to ask about the chunks (lines or blocks) that get delivered
as they read.  So at the very least we should indeed be looking at a
file-like object; it's still easy to construct a file-like object that's
really streaming from another file as it comes in, and we could even
provide shortcuts to build files from inline iterators or something.

And, the idea that each piece of middleware does its *own* buffering
might be a bad one too.  One might naively store everything in RAM,
another might put blocks on disk, another might run you out of /tmp
space trying to do the same thing - even storing duplicates of the same
data if we're not careful!  The same 1MB initial block could wind up on
disk two or three times if each piece of middleware thinks it's the one
with it cached to pass along to the bottom layer that's reading 16k
blocks at a time.

So what's left of my suggestion?  I suggest that we *not* commit to
unlimited rewinding of the input stream; that was my single real
insight, and an uncontrollable iterator design gives up too much in
order to achieve that.  A file-like object is more appropriate, but we
either need to make middleware do its own caching of partially-consumed
data, *or* we need some way for middleware to signal whether it needs
the older data kept.

Could "input.bookmark()" signal that everything from this point on in
the stream needs to be retained, in memory or on disk, to be rewound to
later?  And the data be released only after the bookmark is deleted?

   b = input.bookmark()
   input.read()...

   input2 = b.file()
   del b

Or, we could allow the "input" object to support cloning, where all data
is cached from the clone-that's-read-least-far to the one that's read
the farthest:

   c = input.clone()
   input.read(100)
   # 100 bytes are now cached by the framework, in RAM or on disk or on
   # a USB keyfob or wherever this framework puts it. (Django will write
   # their own caching that's different from everyone else's).
   c.read(100)
   # the bytes are released
   del c
   # Now that there's just one active clone, no buffering takes place.

That way one could "read ahead" on your own input, while passing the
complete stream back down to the next level.  This has the disadvantage
that if a middleware piece wants to keep the first 100MB and last 100MB
from a stream but throw out the middle, it's got no way to do so without
dropping back to its own caching scheme that the framework can't
coordinate with other schemes; but it seems to cover the majority of
cases that I can think of.

Anyway: no unlimited caching, no unlimited rewind; that's my argument.
Iterators were just one way of cleaning getting there, but probably, in
the light of the next day, not a powerful enough way.

-- 
Brandon Craig Rhodes   brandon at rhodesmill.org   http://rhodesmill.org/brandon

From fumanchu at aminus.org  Sat Mar 28 22:42:40 2009
From: fumanchu at aminus.org (Robert Brewer)
Date: Sat, 28 Mar 2009 14:42:40 -0700
Subject: [Web-SIG] thoughts on an iterator
In-Reply-To: <874oxdxxfy.fsf@rhodesmill.org>
References: <874oxdxxfy.fsf@rhodesmill.org>
Message-ID: <F1962646D3B64642B7C9A06068EE1E6407B535E3@ex10.hostedexchange.local>

Brandon Craig Rhodes wrote:
> Graham, I confess that it was I who brought up the idea of a
wsgi.input
> iterator at the WSGI Open Space yesterday evening. :-) The discussion
> seemed to be assuming a file-like input object that could be read from
> by a piece of middleware, then "backed up" or "rewound" before passing
> it down to the next layer.  This seemed to have problems: it doesn't
> support the case where the middleware wants to alter the input or pass
> it piecemeal down to the client as it comes in, and it also means that
> the *entire* input stream has to be kept around in memory for the
> lifetime of the whole request in case the client reading it is not the
> "real client" at the bottom of the stack, and a request is coming that
> will ask for the whole thing to be replayed.
> 
> So, I suggested placing the responsibility for rewind and buffering on
> the middleware.  You want to read 2k of the input to make a middleware
> decision before invoking the next layer down?  Then read it, and pass
> along a fresh iterator that first yields that 2k, then starts yielding
> everything from the partially-read iterator.  Or, you can pass along a
> filter iterator that scans or changes the entire stream as it reads it
> from the upstream iterator.
> 
> But, having through more about the idea, I think that your criticisms,
> Graham, are exactly on-target.  Iterators don't give enough control to
> the reader to ask about the chunks (lines or blocks) that get
delivered
> as they read.  So at the very least we should indeed be looking at a
> file-like object;

Hmmmm. Graham brought up chunked requests which I don't think have much
bearing on this issue--the server/app can't rely on the client-specified
chunk sizes either way (or you enable a Denial of Service attack). I
don't see much difference between the file approach and the iterator
approach, other than moving the read chunk size from the app (or more
likely, the cgi module) to the server. That may be what kills this
proposal: cgi.FieldStorage expects a file pointer and I doubt we want to
either rewrite the entire cgi module to support iterators, or re-package
the iterator up as a file.

> it's still easy to construct a file-like object that's
> really streaming from another file as it comes in, and we could even
> provide shortcuts to build files from inline iterators or something.

Right; either approach can be re-streamed pretty easily.

> And, the idea that each piece of middleware does its *own* buffering
> might be a bad one too.  One might naively store everything in RAM,
> another might put blocks on disk, another might run you out of /tmp
> space trying to do the same thing - even storing duplicates of the
same
> data if we're not careful!  The same 1MB initial block could wind up
on
> disk two or three times if each piece of middleware thinks it's the
one
> with it cached to pass along to the bottom layer that's reading 16k
> blocks at a time.

Any middleware which did so would pretty quickly get fixed or abandoned.
I don't think that's a strong argument given that we have many
developers with experience in this area from existing middleware.

> So what's left of my suggestion?  I suggest that we *not* commit to
> unlimited rewinding of the input stream; that was my single real
> insight, and an uncontrollable iterator design gives up too much in
> order to achieve that.  A file-like object is more appropriate, but we
> either need to make middleware do its own caching of
partially-consumed
> data, *or* we need some way for middleware to signal whether it needs
> the older data kept.
> 
> Could "input.bookmark()" signal that everything from this point on in
> the stream needs to be retained, in memory or on disk, to be rewound
to
> later?  And the data be released only after the bookmark is deleted?
> 
>    b = input.bookmark()
>    input.read()...
> 
>    input2 = b.file()
>    del b
> 
> Or, we could allow the "input" object to support cloning, where all
> data
> is cached from the clone-that's-read-least-far to the one that's read
> the farthest:
> 
>    c = input.clone()
>    input.read(100)
>    # 100 bytes are now cached by the framework, in RAM or on disk or
on
>    # a USB keyfob or wherever this framework puts it. (Django will
> write
>    # their own caching that's different from everyone else's).
>    c.read(100)
>    # the bytes are released
>    del c
>    # Now that there's just one active clone, no buffering takes place.
> 
> That way one could "read ahead" on your own input, while passing the
> complete stream back down to the next level.  This has the
disadvantage
> that if a middleware piece wants to keep the first 100MB and last
100MB
> from a stream but throw out the middle, it's got no way to do so
> without
> dropping back to its own caching scheme that the framework can't
> coordinate with other schemes; but it seems to cover the majority of
> cases that I can think of.

Those seem like strategies for individual middleware components to
implement, not necessary to burden the general case with it.

> Anyway: no unlimited caching, no unlimited rewind; that's my argument.
> Iterators were just one way of cleaning getting there, but probably,
in
> the light of the next day, not a powerful enough way.

I'd vote to stick with the file-like approach for no other reason than
that FieldStorage expects one.


Robert Brewer
fumanchu at aminus.org


From fumanchu at aminus.org  Sun Mar 29 06:10:49 2009
From: fumanchu at aminus.org (Robert Brewer)
Date: Sat, 28 Mar 2009 21:10:49 -0700
Subject: [Web-SIG] WSGI Open Space @ PyCon.
In-Reply-To: <F1962646D3B64642B7C9A06068EE1E6407B53581@ex10.hostedexchange.local>
References: <4a951aa00903271330w48055728i582263fcf67687e5@mail.gmail.com>
	<F1962646D3B64642B7C9A06068EE1E6407B53581@ex10.hostedexchange.local>
Message-ID: <F1962646D3B64642B7C9A06068EE1E6407B5361E@ex10.hostedexchange.local>

Hi all,

We had a good second meeting and answered more issues. My understanding
is that there is another BoF scheduled for tomorrow (Sunday). Check the
Open Space board for details.

Those present at the second meeting:

 * Mark Ramm (TG)
 * Mike Orr (Pylons)
 * Bob Brewer (CherryPy)
 * Ian Bicking (Paste, etc)
 * Alan Kennedy (WSGI gateway servlets/Jython)
 * Rick Copeland (TG)
 * James Bennett (Django)
 * Gary Poster (Launchpad)
 * Chris McDonough (Zope, repoze, etc)
 * Garrett Smith (async WSGI server and middleware)
 * Kumar McMillan (Pylons)
 * Alex Morega (WSGI user)
 * Andrew Sawyer (lurker)
 * Marcus Cavanaugh (Pylons)
 * David Reed (used to be Twisted.web2 maintainer)
 * 8+ others, mostly lurking


Revisited Topic: Unicode values in the WSGI environ
---------------------------------------------------

Consensus: Response status and headers MUST BE unicode. Doing otherwise
(handling both unicode and byte string) would unnecessarily complicate
the construction of middleware components. Origin HTTP servers MUST
decode these to the appropriate bytestrings (all ISO-8859-1?) before
writing them out to the socket.


Revisited Topic: wsgi.input
---------------------------

I raised the issue that, if wsgi.input were an iterable, many apps would
just have to take the extra step of wrapping it in a file-like object
anyway to pass to cgi.Fieldstorage. Others reopened the desire to allow
the app to determine the size of each read().

We didn't reach consensus, IMO. Alan argued for an iterable to more
easily support asynchronous servers. The counter-argument was that
servers could use non-blocking sockets to allow apps which read() to
yield in the case of no immediate data rather than block indefinitely.
If a file-like object were retained, it would help to publish a
chainable file example to help middleware re-stream files they read any
part of.


Response iterable type
----------------------

The current spec says "all strings referred to in this specification
must be of type str or StringType". James asked if this could be
loosened to str-like objects. Perhaps we could replace strict typing
with an ABC requirement? General consensus: -0.


Continuing deferred issues
--------------------------

 * Lots of little changes: the server's supported HTTP version,
   file_wrapper edge cases, etc.
 * Python 3, and the scheduling of WSGI improvements
 * Asynchronous WSGI support. Mostly non-existent. Fix it? Fork it?
   Drop it?
 * Lifecycle methods (start/stop/etc event API driven by the container)
 * Remove app_iter.close?


Robert Brewer
fumanchu at aminus.org


From noah.gift at gmail.com  Sun Mar 29 07:14:23 2009
From: noah.gift at gmail.com (Noah Gift)
Date: Sun, 29 Mar 2009 18:14:23 +1300
Subject: [Web-SIG] WSGI Open Space @ PyCon.
In-Reply-To: <F1962646D3B64642B7C9A06068EE1E6407B5361E@ex10.hostedexchange.local>
References: <4a951aa00903271330w48055728i582263fcf67687e5@mail.gmail.com>
	<F1962646D3B64642B7C9A06068EE1E6407B53581@ex10.hostedexchange.local>
	<F1962646D3B64642B7C9A06068EE1E6407B5361E@ex10.hostedexchange.local>
Message-ID: <e91cc0270903282214x333c2624lfd9fbdbd0ae68313@mail.gmail.com>

On Sun, Mar 29, 2009 at 5:10 PM, Robert Brewer <fumanchu at aminus.org> wrote:

> Hi all,
>
> We had a good second meeting and answered more issues. My understanding
> is that there is another BoF scheduled for tomorrow (Sunday). Check the
> Open Space board for details.
>
> Those present at the second meeting:
>
>  * Mark Ramm (TG)
>  * Mike Orr (Pylons)
>  * Bob Brewer (CherryPy)
>  * Ian Bicking (Paste, etc)
>  * Alan Kennedy (WSGI gateway servlets/Jython)
>  * Rick Copeland (TG)
>  * James Bennett (Django)
>  * Gary Poster (Launchpad)
>  * Chris McDonough (Zope, repoze, etc)
>  * Garrett Smith (async WSGI server and middleware)
>  * Kumar McMillan (Pylons)
>  * Alex Morega (WSGI user)
>  * Andrew Sawyer (lurker)
>  * Marcus Cavanaugh (Pylons)
>  * David Reed (used to be Twisted.web2 maintainer)
>  * 8+ others, mostly lurking
>
>
> Revisited Topic: Unicode values in the WSGI environ
> ---------------------------------------------------
>
> Consensus: Response status and headers MUST BE unicode. Doing otherwise
> (handling both unicode and byte string) would unnecessarily complicate
> the construction of middleware components. Origin HTTP servers MUST
> decode these to the appropriate bytestrings (all ISO-8859-1?) before
> writing them out to the socket.
>
>
> Revisited Topic: wsgi.input
> ---------------------------
>
> I raised the issue that, if wsgi.input were an iterable, many apps would
> just have to take the extra step of wrapping it in a file-like object
> anyway to pass to cgi.Fieldstorage. Others reopened the desire to allow
> the app to determine the size of each read().
>
> We didn't reach consensus, IMO. Alan argued for an iterable to more
> easily support asynchronous servers.


+1 on the iterator, although I might just like the idea and might be missing
something important.  It seems like there are a lot of powerful things being
developed with generators in mind, and there are some nifty things you can
do with them like the contextlib example:
http://docs.python.org/library/contextlib.html#contextlib.closing

Glad to hear a wide range of people showed, even a Django person :)


> The counter-argument was that
> servers could use non-blocking sockets to allow apps which read() to
> yield in the case of no immediate data rather than block indefinitely.
> If a file-like object were retained, it would help to publish a
> chainable file example to help middleware re-stream files they read any
> part of.
>
>
> Response iterable type
> ----------------------
>
> The current spec says "all strings referred to in this specification
> must be of type str or StringType". James asked if this could be
> loosened to str-like objects. Perhaps we could replace strict typing
> with an ABC requirement? General consensus: -0.
>
>
> Continuing deferred issues
> --------------------------
>
>  * Lots of little changes: the server's supported HTTP version,
>   file_wrapper edge cases, etc.
>  * Python 3, and the scheduling of WSGI improvements
>  * Asynchronous WSGI support. Mostly non-existent. Fix it? Fork it?
>   Drop it?
>  * Lifecycle methods (start/stop/etc event API driven by the container)
>  * Remove app_iter.close?
>
>
> Robert Brewer
> fumanchu at aminus.org
>
> _______________________________________________
> Web-SIG mailing list
> Web-SIG at python.org
> Web SIG: http://www.python.org/sigs/web-sig
> Unsubscribe:
> http://mail.python.org/mailman/options/web-sig/noah.gift%40gmail.com
>



-- 
Cheers,

Noah
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/web-sig/attachments/20090329/65ef3f75/attachment.htm>

From fumanchu at aminus.org  Sun Mar 29 16:57:53 2009
From: fumanchu at aminus.org (Robert Brewer)
Date: Sun, 29 Mar 2009 07:57:53 -0700
Subject: [Web-SIG] WSGI Open Space @ PyCon.
In-Reply-To: <F1962646D3B64642B7C9A06068EE1E6407B5361E@ex10.hostedexchange.local>
References: <4a951aa00903271330w48055728i582263fcf67687e5@mail.gmail.com><F1962646D3B64642B7C9A06068EE1E6407B53581@ex10.hostedexchange.local>
	<F1962646D3B64642B7C9A06068EE1E6407B5361E@ex10.hostedexchange.local>
Message-ID: <F1962646D3B64642B7C9A06068EE1E6407B53650@ex10.hostedexchange.local>

I wrote:
> We had a good second meeting and answered more issues. My
understanding
> is that there is another BoF scheduled for tomorrow (Sunday). Check
the
> Open Space board for details.

My mistake. I'll put up an Open Space reservation for 5pm today ASAP.


Robert Brewer
fumanchu at aminus.org


From fumanchu at aminus.org  Mon Mar 30 01:21:36 2009
From: fumanchu at aminus.org (Robert Brewer)
Date: Sun, 29 Mar 2009 16:21:36 -0700
Subject: [Web-SIG] WSGI Open Space @ PyCon.
In-Reply-To: <F1962646D3B64642B7C9A06068EE1E6407B5361E@ex10.hostedexchange.local>
References: <4a951aa00903271330w48055728i582263fcf67687e5@mail.gmail.com><F1962646D3B64642B7C9A06068EE1E6407B53581@ex10.hostedexchange.local>
	<F1962646D3B64642B7C9A06068EE1E6407B5361E@ex10.hostedexchange.local>
Message-ID: <F1962646D3B64642B7C9A06068EE1E6407B536EA@ex10.hostedexchange.local>

We had a smaller third meeting and answered more issues.

Those present at the third meeting:

 * Mark Ramm (TG)
 * Mike Orr (Pylons)
 * Bob Brewer (CherryPy)
 * Glyph Lefkowitz (Twisted)
 * David Reid (Twisted)
 * Jean-Paul Calderone (Twisted)

Continuing Topic: string type for PATH_INFO and SCRIPT_NAME
-----------------------------------------------------------

Much discussion on how to safely decode the Request-URI. Several options
were put forth, including schemes where both unicode and bytes are stuck
in the environ. Final rough consensus was that, even though request
headers MUST be unicode in the environ, SCRIPT_NAME and PATH_INFO
probably MUST be byte strings in order to not "guess wrong" about their
encoding. In addition, a new environ key which indicates whether
%2F-slashes were decoded improperly or not would be beneficial.


Continuing Topic: wsgi.input
----------------------------

Glyph: iterable is good; file-like is also OK.

Big issue: need a way for the app to tell the server that it is waiting
on output from some other source, possibly running in the same event
loop.

      _______ Reactor ______
     /                      \
+--------+  +--------+  +----------+
|  IMAP  |==|  App   |==|  Server  |
+--------+  +--------+  +----------+

Yielding an empty string (as WSGI 1.0 does) does not provide enough
information; the app needs a way to yield a token which tells the server
"don't call my next() method again until my other source has given me
more input on which to operate."


Asynchronous WSGI support
-------------------------

Mostly non-existent. Fix it? Fork it? Drop it? Glyph seemed to think
we're really close if we fix wsgi.input.


Response value type
-------------------

Glyph suggested as the response tuple grows (e.g. by adding a "close"
method), we should more consider returning an object with .status,
.headers, .body, and .close attributes. Packing/unpacking tuples becomes
tedious. Everyone agreed. If changing to an object is not possible, then
a tuple should not have a variable length; that is, no members would be
optional. Returning a dict would be another option (which would allow
optional keys).


Continuing deferred issues
--------------------------

 * Lots of little changes: the server's supported HTTP version,
   file_wrapper edge cases, etc.
 * Python 3, and the scheduling of WSGI improvements (version roadmap)
 * Lifecycle methods (start/stop/etc event API driven by the container)


Robert Brewer
fumanchu at aminus.org


From foom at fuhm.net  Mon Mar 30 01:30:01 2009
From: foom at fuhm.net (James Y Knight)
Date: Sun, 29 Mar 2009 19:30:01 -0400
Subject: [Web-SIG] WSGI Open Space @ PyCon.
In-Reply-To: <F1962646D3B64642B7C9A06068EE1E6407B536EA@ex10.hostedexchange.local>
References: <4a951aa00903271330w48055728i582263fcf67687e5@mail.gmail.com><F1962646D3B64642B7C9A06068EE1E6407B53581@ex10.hostedexchange.local>
	<F1962646D3B64642B7C9A06068EE1E6407B5361E@ex10.hostedexchange.local>
	<F1962646D3B64642B7C9A06068EE1E6407B536EA@ex10.hostedexchange.local>
Message-ID: <6FC1C4FD-E610-4BFB-9237-A8A1905FFFD0@fuhm.net>

On Mar 29, 2009, at 7:21 PM, Robert Brewer wrote:
> In addition, a new environ key which indicates whether
> %2F-slashes were decoded improperly or not would be beneficial.

Of course it's not "improper": the CGI spec (which WSGI incorporates  
by reference) REQUIRES that decoding.

James

From exarkun at divmod.com  Mon Mar 30 02:11:46 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Sun, 29 Mar 2009 19:11:46 -0500
Subject: [Web-SIG] WSGI Open Space @ PyCon.
In-Reply-To: <F1962646D3B64642B7C9A06068EE1E6407B536EA@ex10.hostedexchange.local>
Message-ID: <20090330001146.24697.917246188.divmod.quotient.1317@henry.divmod.com>

On Sun, 29 Mar 2009 16:21:36 -0700, Robert Brewer <fumanchu at aminus.org> wrote:
>We had a smaller third meeting and answered more issues.

Hi all,

First, thanks for writing up these reports.

> [snip]
>
>Asynchronous WSGI support
>-------------------------
>
>Mostly non-existent. Fix it? Fork it? Drop it? Glyph seemed to think
>we're really close if we fix wsgi.input.
>

Those who were present know that I couldn't understand Glyph's proposed
solution.  After the meeting we worked on some details and wrote some
code (completely non-working, not even syntactically correct).

The result looks something like this:

    @asyncApp
    def application(environ):
        yield "foo"
        yield "bar"
        folders = yield imapFolders()
        for folder in folders:
            yield "Folder:", folder

Where `asyncApp` and `imapFolders` are two functions provided by the
async WSGI container (maybe they're generalizable and can be shared
between different async WSGI containers, or maybe not, I'm not sure
yet).

In Twisted's case, `asyncApp` ends up as about 20 lines, which doesn't
seem too bad (I can share it if anyone is interested in the specifics,
it's mostly just gross generator/Deferred mangling stuff).

Jean-Paul

From graham.dumpleton at gmail.com  Mon Mar 30 02:13:45 2009
From: graham.dumpleton at gmail.com (Graham Dumpleton)
Date: Mon, 30 Mar 2009 11:13:45 +1100
Subject: [Web-SIG] WSGI Open Space @ PyCon.
In-Reply-To: <F1962646D3B64642B7C9A06068EE1E6407B536EA@ex10.hostedexchange.local>
References: <4a951aa00903271330w48055728i582263fcf67687e5@mail.gmail.com>
	<F1962646D3B64642B7C9A06068EE1E6407B53581@ex10.hostedexchange.local>
	<F1962646D3B64642B7C9A06068EE1E6407B5361E@ex10.hostedexchange.local>
	<F1962646D3B64642B7C9A06068EE1E6407B536EA@ex10.hostedexchange.local>
Message-ID: <88e286470903291713r2e0e1916j26e872baaf81a8bd@mail.gmail.com>

2009/3/30 Robert Brewer <fumanchu at aminus.org>:
> We had a smaller third meeting and answered more issues.
>
> Those present at the third meeting:
>
> ?* Mark Ramm (TG)
> ?* Mike Orr (Pylons)
> ?* Bob Brewer (CherryPy)
> ?* Glyph Lefkowitz (Twisted)
> ?* David Reid (Twisted)
> ?* Jean-Paul Calderone (Twisted)
>
> Continuing Topic: string type for PATH_INFO and SCRIPT_NAME
> -----------------------------------------------------------
>
> Much discussion on how to safely decode the Request-URI. Several options
> were put forth, including schemes where both unicode and bytes are stuck
> in the environ. Final rough consensus was that, even though request
> headers MUST be unicode in the environ, SCRIPT_NAME and PATH_INFO
> probably MUST be byte strings in order to not "guess wrong" about their
> encoding.

Which may be a problem if want to support CGI/WSGI bridges as in
Python 3.0 os.environ is unicode and so conversion already done. The
issue is whether one can convert it back to bytes safely. Presumably
the encoding applied is known from somewhere.

> In addition, a new environ key which indicates whether
> %2F-slashes were decoded improperly or not would be beneficial.

Someone want to define 'decoded improperly'? :-)

But then, see that subsequent followup has in part answer that already.

> Continuing Topic: wsgi.input
> ----------------------------
>
> Glyph: iterable is good; file-like is also OK.

Any mention of chunked request input and how to handle it if file like
object still used?

Still believe we must have requirement for empty string as EOS
sentinel and require changes to how wsgi.input is consumed to read
until empty string, rather than only reading what content length says.

> Big issue: need a way for the app to tell the server that it is waiting
> on output from some other source, possibly running in the same event
> loop.
>
> ? ? ?_______ Reactor ______
> ? ? / ? ? ? ? ? ? ? ? ? ? ?\
> +--------+ ?+--------+ ?+----------+
> | ?IMAP ?|==| ?App ? |==| ?Server ?|
> +--------+ ?+--------+ ?+----------+
>
> Yielding an empty string (as WSGI 1.0 does) does not provide enough
> information; the app needs a way to yield a token which tells the server
> "don't call my next() method again until my other source has given me
> more input on which to operate."

Which almost sounds like you want to allow a specific implementation
to supply a special class instance via WSGI environment, an instance
of which could be passed back instead of a string from the iterable.
So, like wsgi.file_wrapper(), but instead of replacing the whole
iterable, it becomes one element of it.

This could actually be generalised rather than being specific to this
specific use case scenario. For example, the WSGI environment key may
be 'wsgi.script_control', sort of akin to Script-Control response
header in CGI 1.2. The arguments to this when called could be a string
giving what is being controlled and a tuple or dictionary the meaning
of which is specific to what is being controlled.

The problem with this is what happens if a WSGI middleware tries to do
something with it. If the separate change is made to allow string like
objects to be returned instead of only string objects, then its string
like behaviour could be to appear like an empty string. Thus a
middleware would see it as an empty string. Of course, the WSGI
middleware then may suppress it and not pass it back down the line.

As with wsgi.file_wrapper, the problem is that only the WSGI adapter
really knows what the type of the iterable instance being returned is,
a WSGI middleware can't work it out, except maybe by creating a dummy
instance of one and comparing the types. Even then, that may not be
guaranteed.

This therefore makes it hard for a WSGI middleware to even detect such
special control/meta elements and simply pass them through. That also
wouldn't work anyway, as a WSGI middleware could be trying to combine
together all the strings into one big string to stick a content length
on it. In that case it isn't going to even know that a special control
element is saying that it should stop trying to do that and instead
flush out what it has already accumulated so that underlying server
can do other stuff while waiting for file descriptor to be ready.

A WSGI middleware doing this sort of thing is going to screw you up
even if empty string as might be getting used for this purpose now. In
short, I can't see how you can do it this way, as you have no
guarantee that a WSGI middleware will not hold on to it. Thus never
gets back to underlying WSGI adapter.

> Asynchronous WSGI support
> -------------------------
>
> Mostly non-existent. Fix it? Fork it? Drop it? Glyph seemed to think
> we're really close if we fix wsgi.input.

Which fix/change to wsgi.input are we talking about here?

> Response value type
> -------------------
>
> Glyph suggested as the response tuple grows (e.g. by adding a "close"
> method), we should more consider returning an object with .status,
> .headers, .body, and .close attributes. Packing/unpacking tuples becomes
> tedious. Everyone agreed. If changing to an object is not possible, then
> a tuple should not have a variable length; that is, no members would be
> optional. Returning a dict would be another option (which would allow
> optional keys).

I'll have something more to say about this another time. :-)

> Continuing deferred issues
> --------------------------
>
> ?* Lots of little changes: the server's supported HTTP version,
> ? file_wrapper edge cases, etc.
> ?* Python 3, and the scheduling of WSGI improvements (version roadmap)
> ?* Lifecycle methods (start/stop/etc event API driven by the container)

Graham

From ionel.mc at gmail.com  Mon Mar 30 13:01:56 2009
From: ionel.mc at gmail.com (Ionel Maries Cristian)
Date: Mon, 30 Mar 2009 14:01:56 +0300
Subject: [Web-SIG] WSGI Open Space @ PyCon.
In-Reply-To: <88e286470903291713r2e0e1916j26e872baaf81a8bd@mail.gmail.com>
References: <4a951aa00903271330w48055728i582263fcf67687e5@mail.gmail.com>
	<F1962646D3B64642B7C9A06068EE1E6407B53581@ex10.hostedexchange.local>
	<F1962646D3B64642B7C9A06068EE1E6407B5361E@ex10.hostedexchange.local>
	<F1962646D3B64642B7C9A06068EE1E6407B536EA@ex10.hostedexchange.local>
	<88e286470903291713r2e0e1916j26e872baaf81a8bd@mail.gmail.com>
Message-ID: <b322b4e60903300401p287761dax927e59f6713df0df@mail.gmail.com>

On Mon, Mar 30, 2009 at 03:13, Graham Dumpleton
<graham.dumpleton at gmail.com> wrote:
[...]

> The problem with this is what happens if a WSGI middleware tries to do
> something with it. If the separate change is made to allow string like
> objects to be returned instead of only string objects, then its string
> like behaviour could be to appear like an empty string. Thus a
> middleware would see it as an empty string. Of course, the WSGI
> middleware then may suppress it and not pass it back down the line.
>
> As with wsgi.file_wrapper, the problem is that only the WSGI adapter
> really knows what the type of the iterable instance being returned is,
> a WSGI middleware can't work it out, except maybe by creating a dummy
> instance of one and comparing the types. Even then, that may not be
> guaranteed.
>
> This therefore makes it hard for a WSGI middleware to even detect such
> special control/meta elements and simply pass them through. That also
> wouldn't work anyway, as a WSGI middleware could be trying to combine
> together all the strings into one big string to stick a content length
> on it. In that case it isn't going to even know that a special control
> element is saying that it should stop trying to do that and instead
> flush out what it has already accumulated so that underlying server
> can do other stuff while waiting for file descriptor to be ready.
>
> A WSGI middleware doing this sort of thing is going to screw you up
> even if empty string as might be getting used for this purpose now. In
> short, I can't see how you can do it this way, as you have no
> guarantee that a WSGI middleware will not hold on to it. Thus never
> gets back to underlying WSGI adapter.
>

That's wrong - middleware should not eat up response chunks.

To quote the wsgi spec:
       To put this requirement another way, a middleware component
       must yield at least one value each time its underlying application
       yields a value. If the middleware cannot yield any other value,
       it must yield an empty string.
See:
http://www.python.org/dev/peps/pep-0333/#middleware-handling-of-block-boundaries


-- ionel

From graham.dumpleton at gmail.com  Mon Mar 30 13:14:23 2009
From: graham.dumpleton at gmail.com (Graham Dumpleton)
Date: Mon, 30 Mar 2009 22:14:23 +1100
Subject: [Web-SIG] WSGI Open Space @ PyCon.
In-Reply-To: <b322b4e60903300401p287761dax927e59f6713df0df@mail.gmail.com>
References: <4a951aa00903271330w48055728i582263fcf67687e5@mail.gmail.com>
	<F1962646D3B64642B7C9A06068EE1E6407B53581@ex10.hostedexchange.local>
	<F1962646D3B64642B7C9A06068EE1E6407B5361E@ex10.hostedexchange.local>
	<F1962646D3B64642B7C9A06068EE1E6407B536EA@ex10.hostedexchange.local>
	<88e286470903291713r2e0e1916j26e872baaf81a8bd@mail.gmail.com>
	<b322b4e60903300401p287761dax927e59f6713df0df@mail.gmail.com>
Message-ID: <88e286470903300414y2139204w5de73eea8d179045@mail.gmail.com>

2009/3/30 Ionel Maries Cristian <ionel.mc at gmail.com>:
> On Mon, Mar 30, 2009 at 03:13, Graham Dumpleton
> <graham.dumpleton at gmail.com> wrote:
> [...]
>
>> The problem with this is what happens if a WSGI middleware tries to do
>> something with it. If the separate change is made to allow string like
>> objects to be returned instead of only string objects, then its string
>> like behaviour could be to appear like an empty string. Thus a
>> middleware would see it as an empty string. Of course, the WSGI
>> middleware then may suppress it and not pass it back down the line.
>>
>> As with wsgi.file_wrapper, the problem is that only the WSGI adapter
>> really knows what the type of the iterable instance being returned is,
>> a WSGI middleware can't work it out, except maybe by creating a dummy
>> instance of one and comparing the types. Even then, that may not be
>> guaranteed.
>>
>> This therefore makes it hard for a WSGI middleware to even detect such
>> special control/meta elements and simply pass them through. That also
>> wouldn't work anyway, as a WSGI middleware could be trying to combine
>> together all the strings into one big string to stick a content length
>> on it. In that case it isn't going to even know that a special control
>> element is saying that it should stop trying to do that and instead
>> flush out what it has already accumulated so that underlying server
>> can do other stuff while waiting for file descriptor to be ready.
>>
>> A WSGI middleware doing this sort of thing is going to screw you up
>> even if empty string as might be getting used for this purpose now. In
>> short, I can't see how you can do it this way, as you have no
>> guarantee that a WSGI middleware will not hold on to it. Thus never
>> gets back to underlying WSGI adapter.
>>
>
> That's wrong - middleware should not eat up response chunks.
>
> To quote the wsgi spec:
> ? ? ? To put this requirement another way, a middleware component
> ? ? ? must yield at least one value each time its underlying application
> ? ? ? yields a value. If the middleware cannot yield any other value,
> ? ? ? it must yield an empty string.
> See:
> http://www.python.org/dev/peps/pep-0333/#middleware-handling-of-block-boundaries

In practice I doubt that that is being applied consistently and you
thus can't rely on it.

Even so, any WSGI middleware still needs a way of detecting these
control/meta elements and know it has to pass them through straight
away. They can't just pass back an empty string in place of them as
that defeats the purpose of them.

Graham

From ionel.mc at gmail.com  Mon Mar 30 14:15:26 2009
From: ionel.mc at gmail.com (Ionel Maries Cristian)
Date: Mon, 30 Mar 2009 15:15:26 +0300
Subject: [Web-SIG] WSGI Open Space @ PyCon.
In-Reply-To: <88e286470903300414y2139204w5de73eea8d179045@mail.gmail.com>
References: <4a951aa00903271330w48055728i582263fcf67687e5@mail.gmail.com>
	<F1962646D3B64642B7C9A06068EE1E6407B53581@ex10.hostedexchange.local>
	<F1962646D3B64642B7C9A06068EE1E6407B5361E@ex10.hostedexchange.local>
	<F1962646D3B64642B7C9A06068EE1E6407B536EA@ex10.hostedexchange.local>
	<88e286470903291713r2e0e1916j26e872baaf81a8bd@mail.gmail.com>
	<b322b4e60903300401p287761dax927e59f6713df0df@mail.gmail.com>
	<88e286470903300414y2139204w5de73eea8d179045@mail.gmail.com>
Message-ID: <b322b4e60903300515i73864a44h62940cb08bf39ca8@mail.gmail.com>

On Mon, Mar 30, 2009 at 14:14, Graham Dumpleton
<graham.dumpleton at gmail.com> wrote:
> 2009/3/30 Ionel Maries Cristian <ionel.mc at gmail.com>:
>> On Mon, Mar 30, 2009 at 03:13, Graham Dumpleton
>> <graham.dumpleton at gmail.com> wrote:
>> [...]
>>
>>> The problem with this is what happens if a WSGI middleware tries to do
>>> something with it. If the separate change is made to allow string like
>>> objects to be returned instead of only string objects, then its string
>>> like behaviour could be to appear like an empty string. Thus a
>>> middleware would see it as an empty string. Of course, the WSGI
>>> middleware then may suppress it and not pass it back down the line.
>>>
>>> As with wsgi.file_wrapper, the problem is that only the WSGI adapter
>>> really knows what the type of the iterable instance being returned is,
>>> a WSGI middleware can't work it out, except maybe by creating a dummy
>>> instance of one and comparing the types. Even then, that may not be
>>> guaranteed.
>>>
>>> This therefore makes it hard for a WSGI middleware to even detect such
>>> special control/meta elements and simply pass them through. That also
>>> wouldn't work anyway, as a WSGI middleware could be trying to combine
>>> together all the strings into one big string to stick a content length
>>> on it. In that case it isn't going to even know that a special control
>>> element is saying that it should stop trying to do that and instead
>>> flush out what it has already accumulated so that underlying server
>>> can do other stuff while waiting for file descriptor to be ready.
>>>
>>> A WSGI middleware doing this sort of thing is going to screw you up
>>> even if empty string as might be getting used for this purpose now. In
>>> short, I can't see how you can do it this way, as you have no
>>> guarantee that a WSGI middleware will not hold on to it. Thus never
>>> gets back to underlying WSGI adapter.
>>>
>>
>> That's wrong - middleware should not eat up response chunks.
>>
>> To quote the wsgi spec:
>> ? ? ? To put this requirement another way, a middleware component
>> ? ? ? must yield at least one value each time its underlying application
>> ? ? ? yields a value. If the middleware cannot yield any other value,
>> ? ? ? it must yield an empty string.
>> See:
>> http://www.python.org/dev/peps/pep-0333/#middleware-handling-of-block-boundaries
>
> In practice I doubt that that is being applied consistently and you
> thus can't rely on it.
>
> Even so, any WSGI middleware still needs a way of detecting these
> control/meta elements and know it has to pass them through straight
> away. They can't just pass back an empty string in place of them as
> that defeats the purpose of them.
>
> Graham
>

Meta/control elements could be saved somewhere on a special object
in the environ and empty strings can just be a notification we need to
check that object for the meta/control element.
This way should penetrate all middleware that comply to the rules in the
wsgi spec.

There is not so much middleware that breaks this - I only know of error
capturing middleware - but those are acceptable offenders. Either way,
it's easy to fix.

-- ionel

From maluke at gmail.com  Mon Mar 30 14:43:47 2009
From: maluke at gmail.com (Sergey Schetinin)
Date: Mon, 30 Mar 2009 15:43:47 +0300
Subject: [Web-SIG] WSGI Open Space @ PyCon.
In-Reply-To: <b322b4e60903300515i73864a44h62940cb08bf39ca8@mail.gmail.com>
References: <4a951aa00903271330w48055728i582263fcf67687e5@mail.gmail.com>
	<F1962646D3B64642B7C9A06068EE1E6407B53581@ex10.hostedexchange.local>
	<F1962646D3B64642B7C9A06068EE1E6407B5361E@ex10.hostedexchange.local>
	<F1962646D3B64642B7C9A06068EE1E6407B536EA@ex10.hostedexchange.local>
	<88e286470903291713r2e0e1916j26e872baaf81a8bd@mail.gmail.com>
	<b322b4e60903300401p287761dax927e59f6713df0df@mail.gmail.com>
	<88e286470903300414y2139204w5de73eea8d179045@mail.gmail.com>
	<b322b4e60903300515i73864a44h62940cb08bf39ca8@mail.gmail.com>
Message-ID: <116315680903300543k29309049u4e6646ac29fc497e@mail.gmail.com>

(Sorry if this is a duplicate message).

> Topic: Return a tuple of (status, headers, body)
> ------------------------------------------------
>
> That is, get rid of the start_response callable. The general consensus
> was that this is a simple, but powerful improvement, which Rack/Jack
> have demonstrated already. The "simplest possible application object"
> would change from this:
>
>   def simple_app(environ, start_response):
>       """Simplest possible application object"""
>       status = '200 OK'
>       response_headers = [('Content-type','text/plain')]
>       start_response(status, response_headers)
>       return ['Hello world!\n']
>
> ...to this:
>
>   def simple_app(environ):
>       """Simplest possible application object"""
>       status = '200 OK'
>       response_headers = [('Content-type','text/plain')]
>       body = ['Hello world!\n']
>       return (status, response_headers, body)

Did you consider a variation that eliminates the start_response but
returns status and headers as first item of the iterable? Considering
how response is usually generated it could save some code in some
cases and wouldn't add any overhead in others, it also has a pleasant
similarity to the HTTP response format:

  def simple_app(environ):
      """Simplest possible application object"""
      status = '200 OK'
      response_headers = [('Content-type','text/plain')]
      yield status, response_headers
      yield 'Hello world!\n'

Anyway, whichever way will be accepted they are easy to adapt to one
another with a decorator.

From pje at telecommunity.com  Mon Mar 30 17:45:31 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 30 Mar 2009 11:45:31 -0400
Subject: [Web-SIG] WSGI Open Space @ PyCon.
In-Reply-To: <116315680903300543k29309049u4e6646ac29fc497e@mail.gmail.co
 m>
References: <4a951aa00903271330w48055728i582263fcf67687e5@mail.gmail.com>
	<F1962646D3B64642B7C9A06068EE1E6407B53581@ex10.hostedexchange.local>
	<F1962646D3B64642B7C9A06068EE1E6407B5361E@ex10.hostedexchange.local>
	<F1962646D3B64642B7C9A06068EE1E6407B536EA@ex10.hostedexchange.local>
	<88e286470903291713r2e0e1916j26e872baaf81a8bd@mail.gmail.com>
	<b322b4e60903300401p287761dax927e59f6713df0df@mail.gmail.com>
	<88e286470903300414y2139204w5de73eea8d179045@mail.gmail.com>
	<b322b4e60903300515i73864a44h62940cb08bf39ca8@mail.gmail.com>
	<116315680903300543k29309049u4e6646ac29fc497e@mail.gmail.com>
Message-ID: <20090330154307.D50943A40AF@sparrow.telecommunity.com>

At 03:43 PM 3/30/2009 +0300, Sergey Schetinin wrote:
>Did you consider a variation that eliminates the start_response but
>returns status and headers as first item of the iterable? Considering
>how response is usually generated it could save some code in some
>cases and wouldn't add any overhead in others, it also has a pleasant
>similarity to the HTTP response format:
>
>   def simple_app(environ):
>       """Simplest possible application object"""
>       status = '200 OK'
>       response_headers = [('Content-type','text/plain')]
>       yield status, response_headers
>       yield 'Hello world!\n'


I thought about this, and also a variant where status is in the 
headers, as a 'Status' header ala CGI, with a default of '200 
OK'.  In which case, the following would both be valid apps:

     def simple_app(environ):
         return [('Content-type','text/plain')], "Hello world!\n"

     def simple_app(environ):
         yield [('Content-type','text/plain')]
         yield "Hello world!\n"

However, the main reason I suggested the fixed 3-tuple return is to 
*discourage* the use of generators as response bodies, because a lot 
of people think that they should use yield to build up the output, 
ala 'print' in CGI.  This is really bad for performance, given that 
iterator yielding is supposed to be used to flush the output channel.

Unless you're transmitting a large file, you really should just be 
returning one string with the entire body in the common case.  As you 
point out, it's not that difficult to wrap something with a decorator 
to use one of these simple styles if you need to. 


From ionrock at gmail.com  Mon Mar 30 17:46:29 2009
From: ionrock at gmail.com (Eric Larson)
Date: Mon, 30 Mar 2009 10:46:29 -0500
Subject: [Web-SIG] WSGI Open Space @ PyCon.
In-Reply-To: <F1962646D3B64642B7C9A06068EE1E6407B53581@ex10.hostedexchange.local>
References: <4a951aa00903271330w48055728i582263fcf67687e5@mail.gmail.com>
	<F1962646D3B64642B7C9A06068EE1E6407B53581@ex10.hostedexchange.local>
Message-ID: <6e46dd1e0903300846s59dd2cf2g9f46eb4892347cba@mail.gmail.com>

Hi,

On Sat, Mar 28, 2009 at 10:04 AM, Robert Brewer <fumanchu at aminus.org> wrote:
> Alan Kennedy wrote:
>> For those of you at PyCon, there is a WSGI Open Space @ 5pm today
>> (Friday).
>>
>> The sub-title of the open space is "Does WSGI need revision"?
>
> Hi all,
>
> We had a good meeting but it was too short. We plan on having another
> Open Space meeting at 5pm today (Saturday) to continue the discussion.
>
> Those present at the first meeting:
>
> ?* Mark Ramm (TG)
> ?* Frank Wierzbicki (Jython)
> ?* Mike Orr (Pylons)
> ?* Bob Brewer (CherryPy)
> ?* Ian Bicking (Paste, etc)
> ?* Eric Larson (CherryPy, WSGI advocate)
> ?* Alan Kennedy (WSGI gateway servlets/Jython)
> ?* Michael Twomey (WSGI user + HTTP servers + Twisted)
> ?* Jorge Vargas (TG)
> ?* Ian Charnas (TG CMS)
> ?* Phil Jenvey (Pylons + Jython)
> ?* 8+ others, mostly lurking
>
> Topic: Unicode values in the WSGI environ
> -----------------------------------------
>
> Request:
>
> We discussed any blockers to this. Request headers are pretty easy since
> the spec requires falling back to ISO-8859-1. HOST might be
> IDNA-encoded, but that can be consistent, too. What to do with
> SCRIPT_NAME and PATH_INFO is more difficult, since there is no consensus
> on encoding--there's percent encoding, of course, but that doesn't cover
> multi-byte character encodings. The request body (wsgi.input) would not
> be decoded.
>
> The general consensus was that we could specify the decoding of the
> request metadata (request line and headers) to be the responsibility of
> WSGI servers, and leave it at that--different servers may offer
> different means of configuring the decoding. There are pitfalls to this
> approach, which the spec should address; in particular, some decoding
> strategies may not be reversible. In addition, apps should at least know
> which encoding the server chose via a new WSGI environ entry. Name
> suggestions welcome.
>
> There was some discussion, but no agreement, on including both unicode
> and str (str and bytes in Python 3.x) versions of these values in the
> environ.
>
>
> Response:
>
> I *think* the general consensus was that applications could return
> unicode status and headers. But we also noted that servers should be
> able to encode any bytes using IDNA/ISO-8859-1/RFC-2047/utf-8 where
> appropriate. Not sure where this ended up.
>
>
> Topic: Return a tuple of (status, headers, body)
> ------------------------------------------------
>
> That is, get rid of the start_response callable. The general consensus
> was that this is a simple, but powerful improvement, which Rack/Jack
> have demonstrated already. The "simplest possible application object"
> would change from this:
>
> ?def simple_app(environ, start_response):
> ? ? ?"""Simplest possible application object"""
> ? ? ?status = '200 OK'
> ? ? ?response_headers = [('Content-type','text/plain')]
> ? ? ?start_response(status, response_headers)
> ? ? ?return ['Hello world!\n']
>
> ...to this:
>
> ?def simple_app(environ):
> ? ? ?"""Simplest possible application object"""
> ? ? ?status = '200 OK'
> ? ? ?response_headers = [('Content-type','text/plain')]
> ? ? ?body = ['Hello world!\n']
> ? ? ?return (status, response_headers, body)
>
>

Seeing as this tuple idea is low hanging fruit, I went ahead and
created a small bit of middleware for making the conversion.

http://bitbucket.org/elarson/pack/wiki/Home

Hope it helps!

Eric Larson

> Topic: wsgi.input
> -----------------
>
> Some authors have found problems with the file-like design of
> wsgi.input, specifically, that the rfile is not rewindable/seekable; if
> a middleware component reads part of the stream, things could get ugly
> if a later app tries to read the full stream.
>
> Discussion centered around replacing the current file-like wsgi.input
> with an iterable. Origin servers would be responsible for chunking the
> request body, and each consumer would be required to re-stream each
> chunk.
>
> This topic didn't seem to have the strong consensus that the previous
> ones did. My personal feeling was that we need more time to tease out
> the new problems this approach would raise (perhaps with an
> implementation or two).
>
> Making this change would, however, solve a related issue: how apps can
> safely read the full wsgi.input when the client did not supply a
> Content-Length (i.e. the servers would handle all that).
>
>
> Other topics raised but deferred
> --------------------------------
>
> ?* Standard Request/Response objects. There was one call for, and many
> against, this.
> ?* Lots of little changes: the server's supported HTTP version,
> file_wrapper edge cases, etc.
> ?* Python 3, and the scheduling of WSGI improvements
> ?* Asynchronous WSGI support. Mostly non-existent. Fix it? Fork it? Drop
> it?
> ?* Lifecycle methods (start/stop/etc event API driven by the container)
> ?* Remove app_iter.close?
>
>
> Robert Brewer
> fumanchu at aminus.org
>
> _______________________________________________
> Web-SIG mailing list
> Web-SIG at python.org
> Web SIG: http://www.python.org/sigs/web-sig
> Unsubscribe: http://mail.python.org/mailman/options/web-sig/ionrock%40gmail.com
>

From maluke at gmail.com  Mon Mar 30 18:32:58 2009
From: maluke at gmail.com (Sergey Schetinin)
Date: Mon, 30 Mar 2009 19:32:58 +0300
Subject: [Web-SIG] WSGI Open Space @ PyCon.
In-Reply-To: <6e46dd1e0903300846s59dd2cf2g9f46eb4892347cba@mail.gmail.com>
References: <4a951aa00903271330w48055728i582263fcf67687e5@mail.gmail.com>
	<F1962646D3B64642B7C9A06068EE1E6407B53581@ex10.hostedexchange.local>
	<6e46dd1e0903300846s59dd2cf2g9f46eb4892347cba@mail.gmail.com>
Message-ID: <116315680903300932i76519c45qe4aa9a0c3d1cc35b@mail.gmail.com>

On Mon, Mar 30, 2009 at 18:46, Eric Larson <ionrock at gmail.com> wrote:
> Seeing as this tuple idea is low hanging fruit, I went ahead and
> created a small bit of middleware for making the conversion.
>
> http://bitbucket.org/elarson/pack/wiki/Home
>
> Hope it helps!

If I'm not missing some quirk about some older Python version, it can
be done simpler:


def wsgi2to1(app):
    """
       Adapts WSGI-2 app to WSGI-1.0 interface
    """
    def wrapped(env, sr):
        status, headers, body = app(env)
        sr(status, headers)
        return body
    #@@ peak.util.decorators.rewrap
    return wrapped




# this implements the new interface
def hello_app(env):
    return ('200 OK', [('Content-Type', 'text/plain')], ['hello world'])

# this one conforms to the old one
hello_app_1 = wsgi2to1(hello_app)

From ianb at colorstudy.com  Mon Mar 30 18:42:10 2009
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 30 Mar 2009 11:42:10 -0500
Subject: [Web-SIG] thoughts on an iterator
In-Reply-To: <F1962646D3B64642B7C9A06068EE1E6407B535E3@ex10.hostedexchange.local>
References: <874oxdxxfy.fsf@rhodesmill.org>
	<F1962646D3B64642B7C9A06068EE1E6407B535E3@ex10.hostedexchange.local>
Message-ID: <b654cd2e0903300942m32b1f38dhc3c2170bad46e488@mail.gmail.com>

2009/3/28 Robert Brewer <fumanchu at aminus.org>:
> Hmmmm. Graham brought up chunked requests which I don't think have much
> bearing on this issue--the server/app can't rely on the client-specified
> chunk sizes either way (or you enable a Denial of Service attack). I
> don't see much difference between the file approach and the iterator
> approach, other than moving the read chunk size from the app (or more
> likely, the cgi module) to the server. That may be what kills this
> proposal: cgi.FieldStorage expects a file pointer and I doubt we want to
> either rewrite the entire cgi module to support iterators, or re-package
> the iterator up as a file.

There are some alternate implementations of the cgi POST-parsing
functionality, some of which might be more amenable to using an
iterator.  Or for that matter, none of us have probably read the cgi
module with this in mind.  With a quick look, it'll be slightly tricky
because it uses .readline a lot, but there's just not that much code
involved so it can't be too hard.

For clarity, I think everyone has been discussing an *iterator*, not
an iterable; an iterable would have a lot of unnecessary overhead, but
I've seen both terms used.

I don't agree with Graham's objection, as I think the reason to read
specific-sized chunks is that you don't want to read too much data
into memory at one time.  But the server is free to chunk the iterator
to avoid too much data, and once the strings are in memory the
consumer really isn't any better off reading a smaller chunk than what
is available.

This also means I can stop making up entirely random chunk sizes in
applications.  Applications have no real information to inform this
chunking.  If the string is already in memory, the chunking actually
is counterproductive.

-- 
Ian Bicking  |  http://blog.ianbicking.org

From alan at xhaus.com  Mon Mar 30 19:36:21 2009
From: alan at xhaus.com (Alan Kennedy)
Date: Mon, 30 Mar 2009 12:36:21 -0500
Subject: [Web-SIG] thoughts on an iterator
In-Reply-To: <F1962646D3B64642B7C9A06068EE1E6407B535E3@ex10.hostedexchange.local>
References: <874oxdxxfy.fsf@rhodesmill.org>
	<F1962646D3B64642B7C9A06068EE1E6407B535E3@ex10.hostedexchange.local>
Message-ID: <4a951aa00903301036h3ff89015y48b806156d7609e9@mail.gmail.com>

Hi all,

It was great to meet (nearly) everybody at PyCon; I look forward to
the next time.

I particularly want to thank Robert for being so meticulous about
recording and reporting the discussions; a necessary part of moving
forward, IMO.

[Robert]
> Hmmmm. Graham brought up chunked requests which I don't think have much
> bearing on this issue--the server/app can't rely on the client-specified
> chunk sizes either way (or you enable a Denial of Service attack). I
> don't see much difference between the file approach and the iterator
> approach, other than moving the read chunk size from the app (or more
> likely, the cgi module) to the server. That may be what kills this
> proposal: cgi.FieldStorage expects a file pointer and I doubt we want to
> either rewrite the entire cgi module to support iterators, or re-package
> the iterator up as a file.

I recommend that any discussion of file-like vs. iterator for input
should be informed by this discussion between myself and PJE back when
the spec was being written.

http://mail.python.org/pipermail/web-sig/2004-September/000885.html

Most relevant quote

[PJE]
> Aha!  There's the problem.  The 'read()' protocol is what's wrong.  If
> 'wsgi.input' were an *iterator* instead of a file-like object, it would be
> fairly straightforward for async servers to implement "would block" reads
> as yielding empty strings.  And, servers could actually support streaming
> input via chunked encoding, because they could just yield blocks once
> they've arrived.
>
> The downside to making 'wsgi.input' an iterator is that you lose control
> over how much data to read at a time: the upstream server or middleware
> determines how much data you get.  But, it's quite possible to make a
> buffering, file-like wrapper over such an iterator, if that's what you
> really need, and your code is synchronous.  (This will slightly increase
> the coding burden for interfacing applications and frameworks that expect
> to have a readable stream for CGI input.)  For asynchronous code, you're
> just going to invoke some sort of callback with each block, and it's the
> callback's job to deal with it.
>
> What does everybody think?  If combined with a "pause iterating me until
> there's input data available" extension API, this would let the input
> stream be non-blocking, and solve the chunked-encoding input issue all in
> one change to the protocol.  Or am I missing something here?

http://mail.python.org/pipermail/web-sig/2004-September/000890.html

I'd also be interested in the Twisted folk's take on that discussion.

All the best,

Alan.