From pje at telecommunity.com Fri Jun 2 21:13:53 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 02 Jun 2006 15:13:53 -0400 Subject: [Web-SIG] wsgiref problems & quibbles In-Reply-To: <20060504065438.GA9574@caltech.edu> Message-ID: <5.1.1.6.0.20060602145750.01f48760@mail.telecommunity.com> At 11:54 PM 5/3/2006 -0700, Titus Brown wrote: >Hi, Phillip, > >so I threw together a few WSGI apps recently, and used wsgiref to serve >them. I ran into two problems, and had a few quibbles to raise as well. > >=== > >First, a patch to fix the existing unit tests is attached. Thanks. >=== > >Second, there's a bug in handlers.BaseHandler.finish_response, line 132. >The line: > > if not self.result_is_file() and not self.sendfile(): > >should read > > if not self.result_is_file() or not self.sendfile(): > ^^ > >Otherwise use of wsgi.file_wrapper fails to return any data. This has been changed in the SVN version now; thanks. >I also can't find any place in the code where 'sendfile' is called to >actually send the file. Should that 'if' statement have an 'else'?? No. See the docstring for the 'sendfile()' method. 'self.sendfile()' is supposed to send the file (if it can) and return True, otherwise False if it can't actually send the file. The default implementation returns False. >And now for the few miscellaneous quibbles ;), mostly in re WSGIServer: > > * the __init__ function is inherited, ultimately, from > SocketServer.TCPServer via BaseHTTPServer. This gives it three > uglification problems: > > - the host, port are passed in as a tuple; > - the request handler must be explicitly specified; > - the WSGI app object must be set elsewhere. > > i.e. to instantiate WSGIServer and set it up, you need to call > > s = wsgiref.simple_server.WSGIServer((host, port), > wsgiref.simple_server.RequestHandler) > s.set_app(app) > > This seems *very* counterintuitive and overcomplex to me, and > it seems to be entirely for the benefit of complying with an > interface inherited from code that WSGI newbies probably won't > be using anyway. > > My suggestion is to eliminate this complexity and simply make it > > s = wsgiref.WSGIServer(host, port, app) > > or (if you still insist on the performance enhancement of > keeping it under a separate module ;) > > s = wsgiref.simple_server.WSGIServer(host, port, app) > > What do you think? I think that to avoid breaking compatibility with existing programs that use WSGIServer I could see supporting WSGIServer((host,port),app), (with a default app of None) but not (host, port, app) unless another constructor was added. > * the get_app/set_app functions must be due to support for python > 2.1; they're obviously unnecessary now, what with properties in 2.2 > and up. Actually, I don't remember now why I did that. I'm not even sure that simple_server supports Python 2.1, although most of the other wsgiref modules are intended to support it. > Should they be changed to properties, for inclusion > in the stdlib? I don't see a benefit to doing so. The internal interface between the handler and server objects is such that get_app() is used now, and of course existing code using wsgiref uses set_app(). New code is likely to use the constructor instead, so I don't see any benefit coming from adding properties. > * the simple_server code is completely untested by the unit tests, > it seems. Do you have any thoughts on how to instrument it to > be tested? Well, for unit testing purposes it seems to me that the request handler can be tested without having an actual connection. Similarly, you can unit test the server's setup_environ() and other methods added. (I'm assuming here that "unit" testing would mean testing that modicum of functionality which wsgiref adds to the BaseHTTP stuff.) > (The easiest way for me to test it would be to install > wsgi_intercept and use urllib2 to run various mock HTTP tests. That sounds more like you want integration testing; I don't really see a way to do that. > I doubt people want wsgi_intercept in the stdlib, tho, even > if it's only used for tests...) Why not? Is there something wrong with it? :) > * finally, there are a lot of blank lines loitering about in your > python files. Some of them are for code spacing, some of them > are just hanging out at the bottom of files like > simple_server.py. Do you want to leave these in? I'd like to, but Guido wouldn't. :) I'll let them die a natural death once the code is in the stdlib. (All of them, including the trailing ones, are to space out code to nicely paged screenfuls in my text editor, so I can navigate easily.) From pje at telecommunity.com Wed Jun 7 00:49:45 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 06 Jun 2006 18:49:45 -0400 Subject: [Web-SIG] wsgiref doc draft; reviews/patches wanted Message-ID: <5.1.1.6.0.20060606184324.0200b360@mail.telecommunity.com> I've finished my draft for the wsgiref documentation (including stuff I swiped from AMK's draft; thanks AMK!), and am looking for comments before I add it to the stdlib documentation. Source: http://svn.eby-sarna.com/svnroot/wsgiref/docs PDF: http://peak.telecommunity.com/wsgiref.pdf HTML: http://peak.telecommunity.com/wsgiref_docs/ My current plan is to make a hopefully-final release of the standalone version of wsgiref on PyPI, then clone that version for inclusion in the stdlib. The latest version of wsgiref in the eby-sarna SVN includes a new ``make_server()`` convenience function (addressing Titus' concerns about the constructor signatures while retaining backward compatibility) and it adds a ``wsgiref.validate`` module based on paste.lint. In addition to those two new features, tests were added for the new validate module and for WSGIServer. The test suite and directory layout of the package were also simplified and consolidated to make merging to the stdlib easier. Feedback welcomed. From pje at telecommunity.com Wed Jun 7 17:41:59 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 07 Jun 2006 11:41:59 -0400 Subject: [Web-SIG] [Python-Dev] wsgiref doc draft; reviews/patches wanted In-Reply-To: <20060607123830.GB9578@localhost.localdomain> References: <5.1.1.6.0.20060606184324.0200b360@mail.telecommunity.com> <5.1.1.6.0.20060606184324.0200b360@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060607114108.01e92b40@mail.telecommunity.com> At 08:38 AM 6/7/2006 -0400, A.M. Kuchling wrote: >On Tue, Jun 06, 2006 at 06:49:45PM -0400, Phillip J. Eby wrote: > > Source: http://svn.eby-sarna.com/svnroot/wsgiref/docs > >Minor correction: svn://svn.eby-sarna.com/svnroot/wsgiref/docs >(at least, http didn't work for me). Oops... I meant: http://svn.eby-sarna.com/wsgiref/docs/ Kind of garbled up the svn: and http: URLs; the HTTP one is for ViewCVS. >The docs look good, and I think they'd be ready to go in. > >--amk >_______________________________________________ >Python-Dev mailing list >Python-Dev at python.org >http://mail.python.org/mailman/listinfo/python-dev >Unsubscribe: >http://mail.python.org/mailman/options/python-dev/pje%40telecommunity.com From joe.gregorio at gmail.com Wed Jun 7 20:56:27 2006 From: joe.gregorio at gmail.com (Joe Gregorio) Date: Wed, 7 Jun 2006 14:56:27 -0400 Subject: [Web-SIG] wsgiref doc draft; reviews/patches wanted In-Reply-To: <5.1.1.6.0.20060606184324.0200b360@mail.telecommunity.com> References: <5.1.1.6.0.20060606184324.0200b360@mail.telecommunity.com> Message-ID: <3f1451f50606071156h9e612e3y602918973349a61f@mail.gmail.com> Phillip, 1. It's not really clear from the abstract 'what' this library provides. You might want to consider moving the text from 1.1 up to the same level as the abstract. 2. In section 1.1 you might want to consider dropping the sentence: "Only authors of web servers and programming frameworks need to know every detail..." It doesn't offer any concrete information and just indirectly makes WSGI look complicated. 3. From the abstract: "Having a standard interface makes it easy to use a WSGI-supporting application with a number of different web servers." is a little akward, how about: "Having a standard interface makes it easy to use an application that supports WSGI with a number of different web servers." 4. I believe the order of submodules presented is important and think that they should be listed with 'handlers' and 'simple_server' first: wsgiref.handlers - server/gateway base classes wsgiref.simple_server - a simple WSGI HTTP server wsgiref.util - WSGI environment utilities wsgiref.headers - WSGI response header tools wsgiref.validate - WSGI conformance checker 5. You might consider moving 'headers' into 'util'. Of course, you could go all the way in simplifying and move 'validate' in there too. wsgiref.handlers - server/gateway base classes wsgiref.simple_server - a simple WSGI HTTP server wsgiref.util - WSGI environment utilities Besides those nits it looks very good and will be a fine addition to the core library. -joe On 6/6/06, Phillip J. Eby wrote: > I've finished my draft for the wsgiref documentation (including stuff I > swiped from AMK's draft; thanks AMK!), and am looking for comments before I > add it to the stdlib documentation. > > Source: http://svn.eby-sarna.com/svnroot/wsgiref/docs > PDF: http://peak.telecommunity.com/wsgiref.pdf > HTML: http://peak.telecommunity.com/wsgiref_docs/ > > My current plan is to make a hopefully-final release of the standalone > version of wsgiref on PyPI, then clone that version for inclusion in the > stdlib. > > The latest version of wsgiref in the eby-sarna SVN includes a new > ``make_server()`` convenience function (addressing Titus' concerns about > the constructor signatures while retaining backward compatibility) and it > adds a ``wsgiref.validate`` module based on paste.lint. > > In addition to those two new features, tests were added for the new > validate module and for WSGIServer. The test suite and directory layout of > the package were also simplified and consolidated to make merging to the > stdlib easier. > > Feedback welcomed. > > _______________________________________________ > Web-SIG mailing list > Web-SIG at python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: http://mail.python.org/mailman/options/web-sig/joe.gregorio%40gmail.com > -- Joe Gregorio http://bitworking.org From pje at telecommunity.com Fri Jun 9 18:01:46 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 09 Jun 2006 12:01:46 -0400 Subject: [Web-SIG] [Python-Dev] wsgiref doc draft; reviews/patches wanted In-Reply-To: <3f1451f50606071156h9e612e3y602918973349a61f@mail.gmail.com > References: <5.1.1.6.0.20060606184324.0200b360@mail.telecommunity.com> <5.1.1.6.0.20060606184324.0200b360@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060609115435.00a04748@mail.telecommunity.com> At 02:56 PM 6/7/2006 -0400, Joe Gregorio wrote: >Phillip, > >1. It's not really clear from the abstract 'what' this library >provides. You might want > to consider moving the text from 1.1 up to the same level as the abstract. Done. >2. In section 1.1 you might want to consider dropping the sentence: >"Only authors > of web servers and programming frameworks need to know every detail..." > It doesn't offer any concrete information and just indirectly > makes WSGI look complicated. That bit was taken from AMK's draft; I'm going to trust his intuition here as to why he thought it was desirable to say this. >3. From the abstract: "Having a standard interface makes it easy to use a > WSGI-supporting application with a number of different web servers." > > is a little akward, how about: > > "Having a standard interface makes it easy to use an application > that supports WSGI with a number of different web servers." Done. >4. I believe the order of submodules presented is important and think that > they should be listed with 'handlers' and 'simple_server' first: I agree that the order is important, but I intentionally chose the current order to be a gentle slope of complexity, from the near-trivial functions on up to the server/handler framework last. I'm not sure what ordering principle you're suggesting to use instead. >5. You might consider moving 'headers' into 'util'. Of course, you could > go all the way in simplifying and move 'validate' in there too. Not and maintain backward compatibility. There is, after all, code in the field using these modules for about a year and a half now. From joe.gregorio at gmail.com Fri Jun 9 18:40:33 2006 From: joe.gregorio at gmail.com (Joe Gregorio) Date: Fri, 9 Jun 2006 12:40:33 -0400 Subject: [Web-SIG] [Python-Dev] wsgiref doc draft; reviews/patches wanted In-Reply-To: <5.1.1.6.0.20060609115435.00a04748@mail.telecommunity.com> References: <5.1.1.6.0.20060606184324.0200b360@mail.telecommunity.com> <5.1.1.6.0.20060609115435.00a04748@mail.telecommunity.com> Message-ID: <3f1451f50606090940y29196043i99d10daddce2c297@mail.gmail.com> On 6/9/06, Phillip J. Eby wrote: > >4. I believe the order of submodules presented is important and think that > > they should be listed with 'handlers' and 'simple_server' first: > > I agree that the order is important, but I intentionally chose the current > order to be a gentle slope of complexity, from the near-trivial functions > on up to the server/handler framework last. I'm not sure what ordering > principle you're suggesting to use instead. When I first hit the documentation I was confused by the order. This is wsgiref, a reference implementation of wsgi and I expected wsgi servers and middleware, actual implementations of WSGI, to be the most prominent part of the library and thus presented first. The utility functions would come afterward, after you got the basic wsgi pieces in place. -joe -- Joe Gregorio http://bitworking.org From pje at telecommunity.com Fri Jun 9 18:48:23 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 09 Jun 2006 12:48:23 -0400 Subject: [Web-SIG] FYI: wsgiref is now checked in Message-ID: <5.1.1.6.0.20060609124432.02e7c348@mail.telecommunity.com> The checked-in code substantially matches the public 0.1 release of wsgiref. There are some minor changes to the docs and the test module, but these have also been made in the SVN trunk of wsgiref's home repository, so that future releases don't diverge too much. The plan is to continue to maintain the standalone version and update the stdlib from it as appropriate, although I don't know of anything that would be changing any time soon. The checkin includes a wsgiref.egg-info file, so if you have a program that uses setuptools to depend on wsgiref being installed, setuptools under Python 2.5 should detect that the stdlib already includes wsgiref. Thanks for all the feedback and assistance and code contributions. From pje at telecommunity.com Mon Jun 12 06:00:01 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 12 Jun 2006 00:00:01 -0400 Subject: [Web-SIG] [Python-Dev] FYI: wsgiref is now checked in In-Reply-To: <1f7befae0606101222j7be21daas987e7f24bf5e1445@mail.gmail.co m> References: <5.1.1.6.0.20060609124432.02e7c348@mail.telecommunity.com> <5.1.1.6.0.20060609124432.02e7c348@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060611235632.03753c30@sparrow.telecommunity.com> At 03:22 PM 6/10/2006 -0400, Tim Peters wrote: >This may be because compare_generic_iter() uses `assert` statements, >and those vanish under -O. If so, a test shouldn't normally use >`assert`. On rare occasions it's appropriate, like test_struct's: > > if x < 0: > expected += 1L << self.bitsize > assert expected > 0 > >That isn't testing any of struct's functionality, it's documenting and >verifying a fundamental _belief_ of the test author's: the test >itself is buggy if that assert ever triggers. Or, IOW, it's being >used for what an assert statement should be used for :-) Thanks for the bug report; I've fixed these problems in the standalone version (0.1.2 on the cheeseshop) and in the Python 2.5 trunk. Web-SIG folks take note: wsgiref.validate is based on paste.lint, so paste.lint has the same problem. That is, errors won't be raised if the code is run with -O. As a side effect of fixing the problems, I found that some of the wsgiref.validate (aka paste.lint) asserts have improperly computed messages. Instead of getting an explanation of the problem, you'll instead get a different error at the assert. I fixed these in wsgiref.validate, but the underlying problems presumably still exist in paste.lint.