From ianb at colorstudy.com Wed Nov 1 00:42:43 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 31 Oct 2006 17:42:43 -0600 Subject: [Web-SIG] wsgi.url_vars feedback In-Reply-To: <435DF58A933BA74397B42CDEB8145A86064953D9@ex9.hostedexchange.local> References: <435DF58A933BA74397B42CDEB8145A86064953D9@ex9.hostedexchange.local> Message-ID: <4547DF73.7070200@colorstudy.com> Robert Brewer wrote: > Ian Bicking wrote: >> Having thought about it, I think storing a tuple of >> (args, kwargs) is the best way to do this, since it's >> most explicit. Consumers can deal with args specially, >> ignore them, or raise an error, as they see fit -- >> there are reasons to do each of these. Hiding args >> in kwargs makes this choice more implicit, and probably >> more error prone as a result. >> >> One little question: if a dispatcher can never produce >> one of the kinds of information (which happens for some >> of them), should they put in an empty list/tuple or >> empty dict, or should they put in None for that item? >> I'm currently saying they must put in a list/tuple or dict. > > I would've thought they'd just leave out the entry altogether. You can't omit something from a two-tuple; it would become ambiguous one-tuple of course! -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ianb at colorstudy.com Wed Nov 1 00:42:48 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 31 Oct 2006 17:42:48 -0600 Subject: [Web-SIG] wsgi.url_vars feedback In-Reply-To: <5.1.1.6.0.20061031174422.026b9108@sparrow.telecommunity.com> References: <5.1.1.6.0.20061024125257.02d04008@sparrow.telecommunity.com> <5.1.1.6.0.20061024125257.02d04008@sparrow.telecommunity.com> <5.1.1.6.0.20061031174422.026b9108@sparrow.telecommunity.com> Message-ID: <4547DF78.2090005@colorstudy.com> Phillip J. Eby wrote: > At 04:17 PM 10/31/2006 -0600, Ian Bicking wrote: >> One little question: if a dispatcher can never produce one of the >> kinds of information (which happens for some of them), should they put >> in an empty list/tuple or empty dict, or should they put in None for >> that item? I'm currently saying they must put in a list/tuple or dict. > > This is the correct choice, IMO. I think the spec should be explicit, > however, that these values should be usable with * and ** (or apply()), > as that will help clarify the meaning/rationale of the values. Sure; I've said list/tuple and dict in the spec, but I don't know if there's a reason to be quite as strict as WSGI in the types. *args is fairly flexible, anything that can be coerced to a tuple seems to work. **kwargs is more picky -- the keys must be strings (which I've mentioned, but not very strictly) and it must be a dict object (dict-like objects are not accepted, which I don't quite understand). I suppose I should say that; or conversely the spec should specifically tell consumers to use **dict(kwargs) to guard against dict-like objects. Anyway, I'll just go with **-safe. >> Anyway, I've updated the spec: >> >> http://wsgi.org/wsgi/Specifications/url_vars >> http://wsgi.org/wsgi/Specifications/url_vars?action=diff >> >> Is everyone happy with this version? > > I still think it should be url_args rather than url_vars -- I don't see > any reason why they could be considered "variables" rather than > arguments. :) But other than that, and the desire to see clarification > about */** as an intended/supported use case, I give it a +1. I was thinking about variables similar to GET and POST variables, which seems to be the most common way to refer to them. (Of course GET variables are actually query string variables, which is orthogonal to the request method, but so it goes.) -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From luke.arno at gmail.com Wed Nov 1 08:33:43 2006 From: luke.arno at gmail.com (Luke Arno) Date: Wed, 1 Nov 2006 02:33:43 -0500 Subject: [Web-SIG] wsgi.url_vars feedback In-Reply-To: <4547DF78.2090005@colorstudy.com> References: <5.1.1.6.0.20061024125257.02d04008@sparrow.telecommunity.com> <5.1.1.6.0.20061031174422.026b9108@sparrow.telecommunity.com> <4547DF78.2090005@colorstudy.com> Message-ID: I feel even more +1 on ['wsgi.url_vars'] now. I implemented as currently discussed in selector and it works out very well. Selector now supports positional args in path expressions. I even made some experimental little decorators so that url_vars can go into method signatures. I don't know that I will ever use them (bit clever). Info is on the selector page (badly in need of work though it may be): http://lukearno.com/projects/selector/ (Selector is now LGPL by popular demand, BTW.) Cheers, - Luke From pje at telecommunity.com Wed Nov 1 16:54:39 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 01 Nov 2006 10:54:39 -0500 Subject: [Web-SIG] wsgi.url_vars feedback In-Reply-To: <4547DF78.2090005@colorstudy.com> References: <5.1.1.6.0.20061031174422.026b9108@sparrow.telecommunity.com> <5.1.1.6.0.20061024125257.02d04008@sparrow.telecommunity.com> <5.1.1.6.0.20061024125257.02d04008@sparrow.telecommunity.com> <5.1.1.6.0.20061031174422.026b9108@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061101104941.042fde78@sparrow.telecommunity.com> At 05:42 PM 10/31/2006 -0600, Ian Bicking wrote: >Phillip J. Eby wrote: >>At 04:17 PM 10/31/2006 -0600, Ian Bicking wrote: >>>Anyway, I've updated the spec: >>> >>>http://wsgi.org/wsgi/Specifications/url_vars >>>http://wsgi.org/wsgi/Specifications/url_vars?action=diff >>> >>>Is everyone happy with this version? >>I still think it should be url_args rather than url_vars -- I don't see >>any reason why they could be considered "variables" rather than >>arguments. :) But other than that, and the desire to see clarification >>about */** as an intended/supported use case, I give it a +1. > >I was thinking about variables similar to GET and POST variables, which >seems to be the most common way to refer to them. (Of course GET >variables are actually query string variables, which is orthogonal to the >request method, but so it goes.) That would actually be another reason *not* to use "vars", because these aren't the same things. They're something that's framework-specific, not part of the nature of HTTP. I would actually suggest that url_params would be better, except that URLs *do* have something called parameters. If you *must* have vars, please call it wsgi.routing_vars or wsgi.dispatch_vars instead, so at least it will be clearly distinguishable from other URL variables. In fact, wsgi.dispatch_arguments would probably be the *most* descriptive name for what these actually are. Note that there isn't any requirement here that the data even comes from the URL! From ianb at colorstudy.com Wed Nov 1 18:03:45 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 01 Nov 2006 11:03:45 -0600 Subject: [Web-SIG] Proposal: Handling POST forms in WSGI In-Reply-To: <453A8B67.4070409@colorstudy.com> References: <453A8B67.4070409@colorstudy.com> Message-ID: <4548D371.8010601@colorstudy.com> I've updated this specification in response to input, mostly changing language but not the meat of the specification itself: http://wsgi.org/wsgi/Specifications/handling_post_forms http://wsgi.org/wsgi/Specifications/handling_post_forms?action=diff One thing that occurred to me is that wsgi.input might be a better container for the data than an environ key, like: fs = getattr(environ['wsgi.input'], 'cgi_FieldStorage', None) if fs is None: # parse and replace wsgi.input... I actually have reason to want to generalize this, maybe like: def parse_form(environ): # Check for POST, CONTENT_TYPE... decoded = getattr(environ['wsgi.input'], 'decoded', None) if decoded is not None: decoded_type, decoded_value = decoded if decoded_type == 'cgi.FieldStorage': return decoded_value decoded_value = cgi.FieldStorage(...) input = FakeInput(None, 'cgi.FieldStorage', decoded_value) environ['wsgi.input'] = input return decoded_value class FakeInput(object): def __init__(self, file_body, decoded_type, decoded_value): self._body = file_body self.decoded = (decoded_type, decoded_value) def read(self, *args): if self._body is None: raise EOFError("wsgi.input has already been consumed to produce %r" % self.decoded) return self._body.read(*args) # other file methods... This allows for internal WSGI requests with bodies like text/json to totally avoid serializing and deserializing. In some cases there'd be exceptions raised (like this case), but in other cases you can lazily serialize to deal with intermediaries. I don't know why this didn't occur to me, since I've actually been doing this in WSGIRemote, but I like it more than my current proposal. If it sounds like a good idea to other people I'll drop the current proposal and create this more generalized proposal with a section on POST forms. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ianb at colorstudy.com Wed Nov 1 18:10:55 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 01 Nov 2006 11:10:55 -0600 Subject: [Web-SIG] wsgi.url_vars feedback In-Reply-To: <5.1.1.6.0.20061101104941.042fde78@sparrow.telecommunity.com> References: <5.1.1.6.0.20061031174422.026b9108@sparrow.telecommunity.com> <5.1.1.6.0.20061024125257.02d04008@sparrow.telecommunity.com> <5.1.1.6.0.20061024125257.02d04008@sparrow.telecommunity.com> <5.1.1.6.0.20061031174422.026b9108@sparrow.telecommunity.com> <5.1.1.6.0.20061101104941.042fde78@sparrow.telecommunity.com> Message-ID: <4548D51F.7080007@colorstudy.com> Phillip J. Eby wrote: > At 05:42 PM 10/31/2006 -0600, Ian Bicking wrote: >> Phillip J. Eby wrote: >>> At 04:17 PM 10/31/2006 -0600, Ian Bicking wrote: >>>> Anyway, I've updated the spec: >>>> >>>> http://wsgi.org/wsgi/Specifications/url_vars >>>> http://wsgi.org/wsgi/Specifications/url_vars?action=diff >>>> >>>> Is everyone happy with this version? >>> I still think it should be url_args rather than url_vars -- I don't >>> see any reason why they could be considered "variables" rather than >>> arguments. :) But other than that, and the desire to see >>> clarification about */** as an intended/supported use case, I give it >>> a +1. >> >> I was thinking about variables similar to GET and POST variables, >> which seems to be the most common way to refer to them. (Of course >> GET variables are actually query string variables, which is orthogonal >> to the request method, but so it goes.) > > That would actually be another reason *not* to use "vars", because these > aren't the same things. They're something that's framework-specific, > not part of the nature of HTTP. I would actually suggest that > url_params would be better, except that URLs *do* have something called > parameters. > > If you *must* have vars, please call it wsgi.routing_vars or > wsgi.dispatch_vars instead, so at least it will be clearly > distinguishable from other URL variables. In fact, > wsgi.dispatch_arguments would probably be the *most* descriptive name > for what these actually are. Note that there isn't any requirement here > that the data even comes from the URL! I don't really care that much, somehow url_vars just seemed the most natural choice and I hardly even thought about it. Of course "URLs" don't even exist (though I'm still fuzzy on why), so obviously the terminology is a bit crude. And really this is typically from the path, not the entire URL. Probably the name I like the most of what you've suggested would be "wsgi.routing_args", since that makes me think of "things that were captured on the way here" ("routing" more so than "dispatch"). Actually, I don't really feel any strong difference between args and vars, though I suppose now that it's a tuple of positional and named values then "args" makes more sense. This would break Luke's quick implementation, but hopefully he won't mind. It hasn't been moved to "accepted" yet, so he was warned ;) -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ianb at colorstudy.com Wed Nov 1 18:20:12 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 01 Nov 2006 11:20:12 -0600 Subject: [Web-SIG] Specification process Message-ID: <4548D74C.2090202@colorstudy.com> So I've got a couple specs out there that build on WSGI, and hopefully other people will be introducing more. These specs are (or should be) fairly light and optional, suggesting useful compatibilities between WSGI components rather than anything that changes WSGI itself. So I don't think we need a very heavy process, but we need some process. Does anyone have a suggestion? A timespan plus a quorum (minimum number of people that have to provide input or interest) plus some notion of "acceptance"? This isn't a big group, so maybe just consensus for now, which doesn't mean everyone has to support a spec, but only that there is support and no one blocks the spec. (Where blocking vs. having reservations is kind of like -1 vs -0 in our Python voting conventions.) Like I said, I don't want to make this a big complicated process, I just want to avoid a totally ambiguous process. People are always free to ignore these specifications, so it's really just as much a matter of getting a couple people to agree on some useful compatibility, and not everything has to be universally applicable or useful. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From luke.arno at gmail.com Wed Nov 1 18:22:37 2006 From: luke.arno at gmail.com (Luke Arno) Date: Wed, 1 Nov 2006 12:22:37 -0500 Subject: [Web-SIG] wsgi.url_vars feedback In-Reply-To: <4548D51F.7080007@colorstudy.com> References: <5.1.1.6.0.20061024125257.02d04008@sparrow.telecommunity.com> <5.1.1.6.0.20061031174422.026b9108@sparrow.telecommunity.com> <5.1.1.6.0.20061101104941.042fde78@sparrow.telecommunity.com> <4548D51F.7080007@colorstudy.com> Message-ID: On 11/1/06, Ian Bicking wrote: > Probably the name I like the most of what you've suggested would be > "wsgi.routing_args", since that makes me think of "things that were > captured on the way here" ("routing" more so than "dispatch"). > Actually, I don't really feel any strong difference between args and > vars, though I suppose now that it's a tuple of positional and named > values then "args" makes more sense. I kind of like something with "dispatch" in it but I don't really care. A brief name would be nice. path_args dispatch_args routing_args captures url_captures path_captures ... > This would break Luke's quick implementation, but hopefully he won't > mind. It hasn't been moved to "accepted" yet, so he was warned ;) Not a problem at all. I figured as much. :) - Luke From pje at telecommunity.com Wed Nov 1 18:33:37 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 01 Nov 2006 12:33:37 -0500 Subject: [Web-SIG] wsgi.url_vars feedback In-Reply-To: <4548D51F.7080007@colorstudy.com> References: <5.1.1.6.0.20061101104941.042fde78@sparrow.telecommunity.com> <5.1.1.6.0.20061031174422.026b9108@sparrow.telecommunity.com> <5.1.1.6.0.20061024125257.02d04008@sparrow.telecommunity.com> <5.1.1.6.0.20061024125257.02d04008@sparrow.telecommunity.com> <5.1.1.6.0.20061031174422.026b9108@sparrow.telecommunity.com> <5.1.1.6.0.20061101104941.042fde78@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061101122547.02b101e8@sparrow.telecommunity.com> At 11:10 AM 11/1/2006 -0600, Ian Bicking wrote: >I don't really care that much, somehow url_vars just seemed the most >natural choice and I hardly even thought about it. Of course "URLs" don't >even exist (though I'm still fuzzy on why), so obviously the terminology >is a bit crude. And really this is typically from the path, not the >entire URL. > >Probably the name I like the most of what you've suggested would be >"wsgi.routing_args", since that makes me think of "things that were >captured on the way here" ("routing" more so than "dispatch"). routing_args is fine by me; its *purpose* is much more "discoverable" than url_vars is. When I see url_vars I think that maybe it's a collection of CGI vars related to the URL, like PATH_INFO, SCRIPT_NAME, etc.! > Actually, I don't really feel any strong difference between args and > vars, though I suppose now that it's a tuple of positional and named > values then "args" makes more sense. Yep, that was my reasoning; I think its most valuable use is as arguments to a callable, such that a dispatcher or routing system would just invoke a callable with some framework-defined arguments, combined with the * and ** from the routing arguments. Of course, at that point it becomes pretty clear that the thing doing the routing and the thing being called are part of the same framework anyway, and there's no real point to having the spec, but oh well. :) (Not that there aren't still *some* cross-framework use cases for this spec, but they seem a bit obscure to me.) From pje at telecommunity.com Wed Nov 1 19:01:36 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 01 Nov 2006 13:01:36 -0500 Subject: [Web-SIG] Specification process In-Reply-To: <4548D74C.2090202@colorstudy.com> Message-ID: <5.1.1.6.0.20061101123435.02be0278@sparrow.telecommunity.com> At 11:20 AM 11/1/2006 -0600, Ian Bicking wrote: >So I've got a couple specs out there that build on WSGI, and hopefully >other people will be introducing more. These specs are (or should be) >fairly light and optional, suggesting useful compatibilities between >WSGI components rather than anything that changes WSGI itself. So I >don't think we need a very heavy process, but we need some process. > >Does anyone have a suggestion? My suggestion is that WSGI extensions should be proposed using a proposer-owned namespace (e.g. "selector.vars" in the url_vars case), and simply published by the proposer, with an announcement to the Web-SIG For actualy *standardization*, I think that there should be some *positive* action from other people required. Standardization implies that something is useful and reasonably non-problematic, but I'm not convinced that e.g. the POST parsing stuff that's currently on the table is non-problematic. I'd want to see actual *implementations* of the proposal in a couple of widely-used frameworks before I'd consider it standards-worthy. Frankly, I'd like to err on the side of TOO MUCH process for WSGI extensions than too little. There is very little need for many of the extensions that get proposed, and often the proposals come with too high a price tag attached. Meanwhile, if a proposal is something useful, it can *easily* become a defacto standard. In fact, I think that further protocol standardization in WSGI is mostly bad design at the community level. We should encourage sharing *implementations* first, before sharing interfaces. So, I'm at a very strong -1 on having a simple process for defining WSGI extensions; the whole point of namespaced environ variables is to allow you to define your *own*; there is really no reason for everything to be named 'wsgi.something'. >Like I said, I don't want to make this a big complicated process, I do. :) Well, actually, I suggest that there be a *really* simple process: just publish your own specs under your own package namespace, announce it to Web-SIG, and let other people use it if they like it. But if you want to play in the wsgi.* namespace, it *should* be a big complicated process. >People are always free to >ignore these specifications, so it's really just as much a matter of >getting a couple people to agree on some useful compatibility, Great. But let's please do it *outside* the wsgi.* namespace, and remember that if you share *implementations*, you don't need to duplicate any code, or have as much discussion about how stuff works inside. People will either like it or not, and use it or not. That's as low of a process bar as can be managed. It's probably the case that some people don't feel that their extension is as valued by the community if it doesn't go in wsgi.*. But that's just their lack of self-esteem talking. If I ever come up with WSGI extensions, they're going to go under my name or a framework name. If two or three other frameworks copy me, *then* I might talk about proposing adding it to the wsgi.* space, and I still wouldn't expect the proposal to get a free pass from the Web-SIG just because it's me. To be honest, I don't think that the routing_args thing is *really* worthy of a wsgi.* namespace, it's just that with its problems fixed, it's not so much of a threat that it's worth fighting over. And to some extent, I'm giving it a pass because it's you. ;) But if this is just the start of a wave of people demanding that *their* extensions get in too, then I'm all in favor of slamming the door on it to help keep the rest out. IMO, wsgi.* standardization needs a PEP-sized or stdlib-inclusion-sized bar to it, since everyone is perfectly free to use their own namespaces for extensions, even ones they intend to offer as published interfaces for others to conform to. >and not >everything has to be universally applicable or useful. If you're going to put it in 'wsgi.*', I think it must at least "do no harm". I'm okay with the wsgi.routing_args thing, because at worst it's superfluous. The POST stuff, I still think is dangerously likely to be overused. Overenthusiasm for standardization is dangerous at this stage of WSGI's adoption. WSGI itself is still not well-understood by the Python developer audience at large; for example, a recent IBM Developerworks article about WSGI contained some *serious* errors in its example middleware, which was terribly non-compliant, as the author would've seen if he'd run it with wsgiref.validate or paste.lint on both sides of it. I guess what I'm saying is, it's bloody hard enough sometimes just to get people to write things that actually conform to the spec we have now, without adding a bunch of new ones. (Hell, we should probably require that proposed "standard" extensions include patches to wsgiref.validate that verify correct operation of the extensions!) From fumanchu at amor.org Wed Nov 1 19:40:11 2006 From: fumanchu at amor.org (Robert Brewer) Date: Wed, 1 Nov 2006 10:40:11 -0800 Subject: [Web-SIG] wsgi.url_vars feedback Message-ID: <435DF58A933BA74397B42CDEB8145A86065004DB@ex9.hostedexchange.local> Ian Bicking wrote: > One little question: if a dispatcher can never produce > one of the kinds of information (which happens for some > of them), should they put in an empty list/tuple or > empty dict, or should they put in None for that item? > I'm currently saying they must put in a list/tuple or dict. and I replied: > I would've thought they'd just leave out the entry altogether. and Ian answered: > You can't omit something from a two-tuple; it would become > ambiguous one-tuple of course! Of course. I didn't read your proposal well enough, I guess. But it still begs the question: if you expect various dispatchers can never produce either args or kwargs, then why not go with a pair of independent entries: wsgi.dispatch_args and wsgi.dispatch_kwargs? That would make the omission of one or the other much more natural for both the sender and receiver. Robert Brewer System Architect Amor Ministries fumanchu at amor.org From ianb at colorstudy.com Wed Nov 1 19:47:58 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 01 Nov 2006 12:47:58 -0600 Subject: [Web-SIG] Specification process In-Reply-To: <5.1.1.6.0.20061101123435.02be0278@sparrow.telecommunity.com> References: <5.1.1.6.0.20061101123435.02be0278@sparrow.telecommunity.com> Message-ID: <4548EBDE.7050202@colorstudy.com> Phillip J. Eby wrote: > At 11:20 AM 11/1/2006 -0600, Ian Bicking wrote: >> So I've got a couple specs out there that build on WSGI, and hopefully >> other people will be introducing more. These specs are (or should be) >> fairly light and optional, suggesting useful compatibilities between >> WSGI components rather than anything that changes WSGI itself. So I >> don't think we need a very heavy process, but we need some process. >> >> Does anyone have a suggestion? > > My suggestion is that WSGI extensions should be proposed using a > proposer-owned namespace (e.g. "selector.vars" in the url_vars case), > and simply published by the proposer, with an announcement to the Web-SIG That's a bit awkward. There's no suggestion of any even light authority to such a specification. Just a "hey, here's what I'm using". I think we can do better than that. If you are really uncomfortable with "wsgi." we can just use another namespace. I don't want to use a namespace owned by a package; I feel strongly about that. These specs are neutral ground. But if we just specify a process associated with some new namespace (maybe "wsgiorg") that's fine with me. If that means we can use a lighter process, I will eagerly use such a namespace. > For actualy *standardization*, I think that there should be some > *positive* action from other people required. Standardization implies > that something is useful and reasonably non-problematic, but I'm not > convinced that e.g. the POST parsing stuff that's currently on the table > is non-problematic. I'd want to see actual *implementations* of the > proposal in a couple of widely-used frameworks before I'd consider it > standards-worthy. By suggesting a quorum, which implies some minimum number of people who support a proposal, I was suggesting that some positive action be required. This is largely intended to avoid specs going forward simply because no one cares or is paying attention. > Frankly, I'd like to err on the side of TOO MUCH process for WSGI > extensions than too little. There is very little need for many of the > extensions that get proposed, and often the proposals come with too high > a price tag attached. Meanwhile, if a proposal is something useful, it > can *easily* become a defacto standard. > > In fact, I think that further protocol standardization in WSGI is mostly > bad design at the community level. We should encourage sharing > *implementations* first, before sharing interfaces. There are implementations associated with both specifications. There's no implementors of the specifications themselves, because the specifications didn't yet exist. But there are things that should use the specification, if a specification existed. > So, I'm at a very strong -1 on having a simple process for defining WSGI > extensions; the whole point of namespaced environ variables is to allow > you to define your *own*; there is really no reason for everything to be > named 'wsgi.something'. > > >> Like I said, I don't want to make this a big complicated process, > > I do. :) Well, actually, I suggest that there be a *really* simple > process: just publish your own specs under your own package namespace, > announce it to Web-SIG, and let other people use it if they like it. > > But if you want to play in the wsgi.* namespace, it *should* be a big > complicated process. > > >> People are always free to >> ignore these specifications, so it's really just as much a matter of >> getting a couple people to agree on some useful compatibility, > > Great. But let's please do it *outside* the wsgi.* namespace, and > remember that if you share *implementations*, you don't need to > duplicate any code, or have as much discussion about how stuff works > inside. People will either like it or not, and use it or not. That's > as low of a process bar as can be managed. I'm trying to propose specifications that can usefully have more than one implementation (url vars), or where the implementation may be embedded in a larger existing system that does not currently make it easy to introduce a new dependency or factoring of the code (post form). > It's probably the case that some people don't feel that their extension > is as valued by the community if it doesn't go in wsgi.*. But that's > just their lack of self-esteem talking. If I ever come up with WSGI > extensions, they're going to go under my name or a framework name. If > two or three other frameworks copy me, *then* I might talk about > proposing adding it to the wsgi.* space, and I still wouldn't expect the > proposal to get a free pass from the Web-SIG just because it's me. > > To be honest, I don't think that the routing_args thing is *really* > worthy of a wsgi.* namespace, it's just that with its problems fixed, > it's not so much of a threat that it's worth fighting over. And to some > extent, I'm giving it a pass because it's you. ;) But if this is just > the start of a wave of people demanding that *their* extensions get in > too, then I'm all in favor of slamming the door on it to help keep the > rest out. > > IMO, wsgi.* standardization needs a PEP-sized or stdlib-inclusion-sized > bar to it, since everyone is perfectly free to use their own namespaces > for extensions, even ones they intend to offer as published interfaces > for others to conform to. > > >> and not >> everything has to be universally applicable or useful. > > If you're going to put it in 'wsgi.*', I think it must at least "do no > harm". I'm okay with the wsgi.routing_args thing, because at worst it's > superfluous. The POST stuff, I still think is dangerously likely to be > overused. I call "consenting adults" on you here! But anyway, I'm now more interested in the other direction I mentioned, attaching an attribute to wsgi.input that gives the decoded form. > Overenthusiasm for standardization is dangerous at this stage of WSGI's > adoption. WSGI itself is still not well-understood by the Python > developer audience at large; for example, a recent IBM Developerworks > article about WSGI contained some *serious* errors in its example > middleware, which was terribly non-compliant, as the author would've > seen if he'd run it with wsgiref.validate or paste.lint on both sides of > it. > > I guess what I'm saying is, it's bloody hard enough sometimes just to > get people to write things that actually conform to the spec we have > now, without adding a bunch of new ones. (Hell, we should probably > require that proposed "standard" extensions include patches to > wsgiref.validate that verify correct operation of the extensions!) I think we are at a point when there are multiple people moving WSGI-based components forward from different directions, and we're ready to start cooperating at a higher level. Obviously there are problems that can arise, which is why I would encourage peer review in this form. And hey, we already have a rejected proposal! (It didn't actually get as far as the mailing list, but it's on the website.) So the system works, this isn't just about moving stuff through. But ultimately if this turns into a cabal of like-minded WSGI-oriented individuals agreeing on things they think are interesting and useful, I'm actually okay with that. I don't know if that reveals an inner dictatorial tendency, or just a desire for expedient forward movement that's otherwise hard to achieve when there's no umbrella project or hierarchy of concepts. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ianb at colorstudy.com Wed Nov 1 19:48:38 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 01 Nov 2006 12:48:38 -0600 Subject: [Web-SIG] wsgi.url_vars feedback In-Reply-To: <435DF58A933BA74397B42CDEB8145A86065004DB@ex9.hostedexchange.local> References: <435DF58A933BA74397B42CDEB8145A86065004DB@ex9.hostedexchange.local> Message-ID: <4548EC06.5020704@colorstudy.com> Robert Brewer wrote: > Ian Bicking wrote: >> One little question: if a dispatcher can never produce >> one of the kinds of information (which happens for some >> of them), should they put in an empty list/tuple or >> empty dict, or should they put in None for that item? >> I'm currently saying they must put in a list/tuple or dict. > > and I replied: >> I would've thought they'd just leave out the entry altogether. > > and Ian answered: >> You can't omit something from a two-tuple; it would become >> ambiguous one-tuple of course! > > Of course. I didn't read your proposal well enough, I guess. > > But it still begs the question: if you expect various dispatchers can > never produce either args or kwargs, then why not go with a pair of > independent entries: wsgi.dispatch_args and wsgi.dispatch_kwargs? That > would make the omission of one or the other much more natural for both > the sender and receiver. I thought briefly about that. Maybe I should have thought more about it; the little dance I had to do to merge both positional and keyword arguments into a tuple is a little silly (in the regex example in the spec). It does feel conceptually good to treat them as a combined whole. But conversely, if I was to expose these keys directly through a request object I'd do it as two separate attributes, because the tuple is awkward to work with. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ben at groovie.org Wed Nov 1 19:11:15 2006 From: ben at groovie.org (Ben Bangert) Date: Wed, 1 Nov 2006 10:11:15 -0800 Subject: [Web-SIG] wsgi.url_vars feedback In-Reply-To: <5.1.1.6.0.20061101122547.02b101e8@sparrow.telecommunity.com> References: <5.1.1.6.0.20061101104941.042fde78@sparrow.telecommunity.com> <5.1.1.6.0.20061031174422.026b9108@sparrow.telecommunity.com> <5.1.1.6.0.20061024125257.02d04008@sparrow.telecommunity.com> <5.1.1.6.0.20061024125257.02d04008@sparrow.telecommunity.com> <5.1.1.6.0.20061031174422.026b9108@sparrow.telecommunity.com> <5.1.1.6.0.20061101104941.042fde78@sparrow.telecommunity.com> <5.1.1.6.0.20061101122547.02b101e8@sparrow.telecommunity.com> Message-ID: <30FDB66E-EE16-4F37-9277-B145472243F0@groovie.org> On Nov 1, 2006, at 9:33 AM, Phillip J. Eby wrote: > Yep, that was my reasoning; I think its most valuable use is as > arguments > to a callable, such that a dispatcher or routing system would just > invoke a > callable with some framework-defined arguments, combined with the * > and ** > from the routing arguments. > > Of course, at that point it becomes pretty clear that the thing > doing the > routing and the thing being called are part of the same framework > anyway, > and there's no real point to having the spec, but oh well. :) > (Not that > there aren't still *some* cross-framework use cases for this spec, > but they > seem a bit obscure to me.) There's a point I think, at the very least it gives unifying points to several types of dispatchers. This means it could be rather easy to create a more general version of Luke's Selector that has pluggable URL lookup schemes since there's a consistent place to look for the information. +1 on the whole proposal, I look forward to adding it to Routes. Cheers, Ben From pje at telecommunity.com Wed Nov 1 20:35:12 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 01 Nov 2006 14:35:12 -0500 Subject: [Web-SIG] Specification process In-Reply-To: <4548EBDE.7050202@colorstudy.com> References: <5.1.1.6.0.20061101123435.02be0278@sparrow.telecommunity.com> <5.1.1.6.0.20061101123435.02be0278@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061101140021.0278c798@sparrow.telecommunity.com> At 12:47 PM 11/1/2006 -0600, Ian Bicking wrote: >Phillip J. Eby wrote: >>At 11:20 AM 11/1/2006 -0600, Ian Bicking wrote: >>>So I've got a couple specs out there that build on WSGI, and hopefully >>>other people will be introducing more. These specs are (or should be) >>>fairly light and optional, suggesting useful compatibilities between >>>WSGI components rather than anything that changes WSGI itself. So I >>>don't think we need a very heavy process, but we need some process. >>> >>>Does anyone have a suggestion? >>My suggestion is that WSGI extensions should be proposed using a >>proposer-owned namespace (e.g. "selector.vars" in the url_vars case), and >>simply published by the proposer, with an announcement to the Web-SIG > >That's a bit awkward. There's no suggestion of any even light authority >to such a specification. Just a "hey, here's what I'm using". Well, that's what it *is*. Standardization, IMO, implies considerably more rigor. I specifically *don't* want these things to get WSGI-class "authority" from a lightweight process. That is *precisely* what I don't want. If specs want to run with the big dogs, let 'em deal with the fleas. Compare for example, with the very successful TurboGears/CherryPy templating standard. That's working out quite nicely, with no "official" standardization or authority, under its *own* name. They didn't see a need to put it under a 'setuptools' namespace, just because setuptools' entry point system is the glue that allows the interfaces to hook in! Ultimately, granting authority cheaply just results in the authority being cheapened. > I think we can do better than that. Sure - if you want to put in the rigor to do it right. I don't want it implied that these "lightweight" specs are anything but lightweight. >If you are really uncomfortable with "wsgi." we can just use another >namespace. I don't want to use a namespace owned by a package; I feel >strongly about that. These specs are neutral ground. But if we just >specify a process associated with some new namespace (maybe "wsgiorg") >that's fine with me. If that means we can use a lighter process, I will >eagerly use such a namespace. How about "experimental.whatever"? :) >By suggesting a quorum, which implies some minimum number of people who >support a proposal, I was suggesting that some positive action be >required. This is largely intended to avoid specs going forward simply >because no one cares or is paying attention. Fair enough. >I'm trying to propose specifications that can usefully have more than one >implementation (url vars), To be honest, I can only think of some very obscure use cases for that. The only reason you need the routing_args in environ is if you have to pass through some other WSGI component between the routing and the place where the arguments get used. I haven't come up with a real world example where that would happen, because normally a piece of routing middleware is going to be directly invoking the thing whose arguments were just obtained. Does anybody actually have an *example* of this use case? Or is there some other use case I'm just not seeing? Honestly, it seems like standardization for standardization's sake to me, and that's what I'm opposed to. Now, I'm *not* opposed to having a published, peer-reviewed spec. That's great! Do lots more of them. What I *am* opposed to is treating what are effectively Internet-Draft specs as if they were RFCs. Let's keep the standardization bar high (RFC), and the experimentation/communication bar low (Internet-Drafts). >or where the implementation may be embedded in a larger existing system >that does not currently make it easy to introduce a new dependency or >factoring of the code (post form). I don't see how this works for newly-proposed standards, which presumably don't have implementations yet. We should encourage lightweight, decoupled implementations, and not reward embedding and spaghettiware. This is in fact why I generally dislike middleware that adds junk to environ, as this leads to WSGhettI code. :) Frankly, if an application knows that the middleware *exists*, then there is no point to having it. The point of middleware is to be *invisible* to applications, otherwise it's not middleware, it's a framework. And just because you "standardize" it, doesn't make it any less of a framework. It just makes it a framework with more overhead, in both procedure *and* code. :) >>>and not >>>everything has to be universally applicable or useful. >>If you're going to put it in 'wsgi.*', I think it must at least "do no >>harm". I'm okay with the wsgi.routing_args thing, because at worst it's >>superfluous. The POST stuff, I still think is dangerously likely to be >>overused. > >I call "consenting adults" on you here! The bar for entrance to wsgi.* should be as high as that for the standard library, and the "do no harm" rule applies there too. >I think we are at a point when there are multiple people moving WSGI-based >components forward from different directions, and we're ready to start >cooperating at a higher level. Obviously there are problems that can >arise, which is why I would encourage peer review in this form. +1 peer review. +1 proposals and discussion and interbreeding and all manner of good things. I'm only -1 on there being a low bar for getting to put 'wsgi' in the name of the result. wsgi.* specs should at minimum include code to allow wsgiref.validate to validate implementations, and reference implementation(s) suitable for inclusion in wsgiref. I think that this is a reasonable bar to set for something that's getting called "standard" and given any "authority". If you want to have "authority" for these fast-track experimental proposals, create another group name, and let its proposals gain whatever authority it deserves. Call it the "Python Web Interfaces Group" or something, and put the stuff in 'p_wig.*' or some such. :) Or how about web_sig? >And hey, we already have a rejected proposal! (It didn't actually get as >far as the mailing list, but it's on the website.) So the system works, >this isn't just about moving stuff through. > >But ultimately if this turns into a cabal of like-minded WSGI-oriented >individuals agreeing on things they think are interesting and useful, I'm >actually okay with that. I don't know if that reveals an inner >dictatorial tendency, or just a desire for expedient forward movement >that's otherwise hard to achieve when there's no umbrella project or >hierarchy of concepts. By all means, create a cabal! I'm totally in favor. Just don't try to overstate its actual authority. If the cabal produces good work, it will gain authority from its reputation, under its *own* name. In marketing, "brand extension" is often fatal. For a recent example, notice that Google wised up and bought YouTube, which was a better brand than "Google Video". Assuming your cabal does good work, it will actually have more impact through independence, whereas mixing everything in under the WSGI name just dilutes the brand for everybody. From ianb at colorstudy.com Wed Nov 1 21:53:44 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 01 Nov 2006 14:53:44 -0600 Subject: [Web-SIG] Specification process In-Reply-To: <5.1.1.6.0.20061101140021.0278c798@sparrow.telecommunity.com> References: <5.1.1.6.0.20061101123435.02be0278@sparrow.telecommunity.com> <5.1.1.6.0.20061101123435.02be0278@sparrow.telecommunity.com> <5.1.1.6.0.20061101140021.0278c798@sparrow.telecommunity.com> Message-ID: <45490958.40200@colorstudy.com> Phillip J. Eby wrote: > Compare for example, with the very successful TurboGears/CherryPy > templating standard. That's working out quite nicely, with no > "official" standardization or authority, under its *own* name. They > didn't see a need to put it under a 'setuptools' namespace, just because > setuptools' entry point system is the glue that allows the interfaces to > hook in! Well, they use [python.templating.engine], so I don't think it really fits your point ;) When there's multiple consumers and producers, and none of them is more "core" than any other, the namespaces get a big vague. If you use too "personal" of a namespace then people will (often rightly) see it as simply an expression of scope and a binding to the implementation. If you use too general of a namespace you can overstep your bounds, self-declaring an authority you don't have. Anyway, I want to set up another scope of authority, short of the formality of WSGI and the PEP process, but more authority than just personal namespaces. >> I'm trying to propose specifications that can usefully have more than >> one implementation (url vars), > > To be honest, I can only think of some very obscure use cases for that. > The only reason you need the routing_args in environ is if you have to > pass through some other WSGI component between the routing and the place > where the arguments get used. I haven't come up with a real world > example where that would happen, because normally a piece of routing > middleware is going to be directly invoking the thing whose arguments > were just obtained. > > Does anybody actually have an *example* of this use case? Or is there > some other use case I'm just not seeing? Honestly, it seems like > standardization for standardization's sake to me, and that's what I'm > opposed to. I already did; there's three producers at this point: Routes, Selector, and Django. Pylons and wsgicollection are both consumers, and other systems like CherryPy and paste.httpwrappers are likely to be consumers in the future. The proposal doesn't match any of these systems, but it can be compatible with the functionality and purpose of all of them. This seems like the exact situation where a common spec is called for -- the potential for compatibility exists, but isn't realized without some basic neutral documentation of a common layer. If compatibility *already* existed than the ad hoc approach would have worked. If compatibility wouldn't work, then the spec isn't useful. >> or where the implementation may be embedded in a larger existing >> system that does not currently make it easy to introduce a new >> dependency or factoring of the code (post form). > > I don't see how this works for newly-proposed standards, which > presumably don't have implementations yet. We should encourage > lightweight, decoupled implementations, and not reward embedding and > spaghettiware. This is in fact why I generally dislike middleware that > adds junk to environ, as this leads to WSGhettI code. :) > > Frankly, if an application knows that the middleware *exists*, then > there is no point to having it. The point of middleware is to be > *invisible* to applications, otherwise it's not middleware, it's a > framework. > > And just because you "standardize" it, doesn't make it any less of a > framework. It just makes it a framework with more overhead, in both > procedure *and* code. :) Sure, and I wouldn't bring it up if it wasn't an actual issue. > wsgi.* specs should at minimum include code to allow wsgiref.validate to > validate implementations, and reference implementation(s) suitable for > inclusion in wsgiref. I think that this is a reasonable bar to set for > something that's getting called "standard" and given any "authority". Is there any way to plug in validation? I like the idea of including validators. Maybe an entry point(s): [wsgiref.validate.environ] key_name = validator_func [wsgiref.validate.app_iter] any_name = validator_func [wsgiref.validate.start_response] any_name = validator_func The last would take the (status, headers, exc_info) values. Hmm... I guess also a validator for the start response writer, though there's nothing very interesting you can do with that. And maybe "*" for a keyname for wsgiref.validate.environ, which would validate the entire environment. > If you want to have "authority" for these fast-track experimental > proposals, create another group name, and let its proposals gain > whatever authority it deserves. Call it the "Python Web Interfaces > Group" or something, and put the stuff in 'p_wig.*' or some such. :) > Or how about web_sig? I'm fine with that. I think Web-SIG is a fine place for discussion, even if the discussion isn't universally applicable. Otherwise Web-SIG would be dead silent ;) So, maybe two namespace options: wsgiorg.* websig.* I personally prefer wsgiorg, since we can do this on wsgi.org, and I like what the name implies. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From luke.arno at gmail.com Wed Nov 1 22:18:51 2006 From: luke.arno at gmail.com (Luke Arno) Date: Wed, 1 Nov 2006 16:18:51 -0500 Subject: [Web-SIG] Specification process In-Reply-To: <45490958.40200@colorstudy.com> References: <5.1.1.6.0.20061101123435.02be0278@sparrow.telecommunity.com> <5.1.1.6.0.20061101140021.0278c798@sparrow.telecommunity.com> <45490958.40200@colorstudy.com> Message-ID: On 11/1/06, Ian Bicking wrote: > > So, maybe two namespace options: > > wsgiorg.* > websig.* > > I personally prefer wsgiorg, since we can do this on wsgi.org, and I > like what the name implies. +1 for wsgiorg as a neutral namespace for building compatibility. -1 for process until we have problems that process might resolve. Premature bureaucratization is a sin worse than premature optimization. For my part, it is enough to say "hey everybody, we have a few tools that do the same thing in different ways (all the various dispatchers for instance); do we want to put things in a common place in a common way so that these tools are interchangeable?" That is all we have really done hear and I think it has worked out fine. Creating process is already creating overhead when there is no trouble that we need the process to address. What additional value comes from lending "authority" to this convention (url vars)? Bah! ;) Cheers, - Luke From pje at telecommunity.com Wed Nov 1 22:44:12 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 01 Nov 2006 16:44:12 -0500 Subject: [Web-SIG] Specification process In-Reply-To: References: <45490958.40200@colorstudy.com> <5.1.1.6.0.20061101123435.02be0278@sparrow.telecommunity.com> <5.1.1.6.0.20061101140021.0278c798@sparrow.telecommunity.com> <45490958.40200@colorstudy.com> Message-ID: <5.1.1.6.0.20061101164111.0277a108@sparrow.telecommunity.com> At 04:18 PM 11/1/2006 -0500, Luke Arno wrote: >What additional value comes from lending >"authority" to this convention (url vars)? Bah! ;) Exactly my point. Let a thousand flowers bloom in other namespaces. :) From ianb at colorstudy.com Thu Nov 2 01:37:17 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 01 Nov 2006 18:37:17 -0600 Subject: [Web-SIG] Specification process In-Reply-To: References: <5.1.1.6.0.20061101123435.02be0278@sparrow.telecommunity.com> <5.1.1.6.0.20061101140021.0278c798@sparrow.telecommunity.com> <45490958.40200@colorstudy.com> Message-ID: <45493DBD.8020502@colorstudy.com> Luke Arno wrote: > On 11/1/06, Ian Bicking wrote: >> >> So, maybe two namespace options: >> >> wsgiorg.* >> websig.* >> >> I personally prefer wsgiorg, since we can do this on wsgi.org, and I >> like what the name implies. > > +1 for wsgiorg as a neutral namespace for building > compatibility. -1 for process until we have problems > that process might resolve. > > Premature bureaucratization is a sin worse than > premature optimization. For my part, it is enough to > say "hey everybody, we have a few tools that do the > same thing in different ways (all the various > dispatchers for instance); do we want to put things > in a common place in a common way so that these > tools are interchangeable?" That is all we have really > done hear and I think it has worked out fine. I'm not trying to add bureaucracy. Mostly I'm trying to create something where we can tell people where to take ideas for specs, and when someone does that they can tell when they are "done". How formal we have to be, I don't know -- I think we can start pretty simple with some rules of thumb. I just want to avoid an ambiguous process where the only way to finish is to self-declare yourself done. Of note is the fact that PEP 333 is still of status "Draft" -- I'd rather avoid that limbo. (Though I think in this case Phillip just can self-declare it whatever status seems right? In which case I'd suggest something more assertive than "Draft") > Creating process is already creating overhead when > there is no trouble that we need the process to > address. What additional value comes from lending > "authority" to this convention (url vars)? Bah! ;) The "authority" is that: 1) the spec won't change without good reason, so you can start implementing against it (once it is "approved") 2) it's useful in multiple contexts 3) having implemented either side, you can expect that maybe *someone* will care (either producing or consuming what you are looking for) 4) some eyes have been on it, and it's passed the sanity check (and hopefully better than just a sanity check) 5) someone won't implement something they think is the same, but isn't, because the spec documents the requirements sufficiently Going through the process is a marker for all these items. And really that's the only important part, if someone is able to claim all these things then maybe that's the only process that is important. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From luke.arno at gmail.com Thu Nov 2 03:12:28 2006 From: luke.arno at gmail.com (Luke Arno) Date: Wed, 1 Nov 2006 21:12:28 -0500 Subject: [Web-SIG] Specification process In-Reply-To: <45493DBD.8020502@colorstudy.com> References: <5.1.1.6.0.20061101123435.02be0278@sparrow.telecommunity.com> <5.1.1.6.0.20061101140021.0278c798@sparrow.telecommunity.com> <45490958.40200@colorstudy.com> <45493DBD.8020502@colorstudy.com> Message-ID: On 11/1/06, Ian Bicking wrote: > Luke Arno wrote: > > On 11/1/06, Ian Bicking wrote: > >> > >> So, maybe two namespace options: > >> > >> wsgiorg.* > >> websig.* > >> > >> I personally prefer wsgiorg, since we can do this on wsgi.org, and I > >> like what the name implies. > > > > +1 for wsgiorg as a neutral namespace for building > > compatibility. -1 for process until we have problems > > that process might resolve. > > > > Premature bureaucratization is a sin worse than > > premature optimization. For my part, it is enough to > > say "hey everybody, we have a few tools that do the > > same thing in different ways (all the various > > dispatchers for instance); do we want to put things > > in a common place in a common way so that these > > tools are interchangeable?" That is all we have really > > done hear and I think it has worked out fine. > > I'm not trying to add bureaucracy. Mostly I'm trying to create > something where we can tell people where to take ideas for specs, and > when someone does that they can tell when they are "done". How formal > we have to be, I don't know -- I think we can start pretty simple with > some rules of thumb. I just want to avoid an ambiguous process where > the only way to finish is to self-declare yourself done. > > Of note is the fact that PEP 333 is still of status "Draft" -- I'd > rather avoid that limbo. (Though I think in this case Phillip just can > self-declare it whatever status seems right? In which case I'd suggest > something more assertive than "Draft") > > > Creating process is already creating overhead when > > there is no trouble that we need the process to > > address. What additional value comes from lending > > "authority" to this convention (url vars)? Bah! ;) > > The "authority" is that: > > 1) the spec won't change without good reason, so you can start > implementing against it (once it is "approved") > 2) it's useful in multiple contexts > 3) having implemented either side, you can expect that maybe *someone* > will care (either producing or consuming what you are looking for) > 4) some eyes have been on it, and it's passed the sanity check (and > hopefully better than just a sanity check) > 5) someone won't implement something they think is the same, but isn't, > because the spec documents the requirements sufficiently > > Going through the process is a marker for all these items. And really > that's the only important part, if someone is able to claim all these > things then maybe that's the only process that is important. I agree that some repository of documents as to what conventions exist to enhance interop between wsgi components is highly valuable. I would rather see those documents present real information upon which developers can decide what to do rationally than an appeal to authority in order to influence conformity on a sociological basis. Hence my "bah" :) A document should describe the convention and its status in plain language and factual terms. In this case, I think we should just state that there is a general consensus and which implementations do and will support that. Validators are nice too. None of that requires formalization thus far. Some statement on wsgi.org as to our desire to create interoperable components and our general commitment to not break our APIs without good reason might be good. I hope that goes without saying and if not I tend to think that actions (should) speak louder than words. Do we want to draw up some kind of WSGI community constitution while we are at it? ;) ;) IMHO, YMMV, yada yada yada. Cheers, - Luke From luke.arno at gmail.com Thu Nov 2 20:48:12 2006 From: luke.arno at gmail.com (Luke Arno) Date: Thu, 2 Nov 2006 14:48:12 -0500 Subject: [Web-SIG] Specification process In-Reply-To: <454A2E59.6050304@colorstudy.com> References: <5.1.1.6.0.20061101123435.02be0278@sparrow.telecommunity.com> <5.1.1.6.0.20061101140021.0278c798@sparrow.telecommunity.com> <45490958.40200@colorstudy.com> <45493DBD.8020502@colorstudy.com> <454A2E59.6050304@colorstudy.com> Message-ID: I am only suggesting that we table this for now. Latter, we need to have process. I am all for these goals. However, if we create a standardization process, people will use it. Things will get heavy. If we identify a nice opportunity to create compatibility, like url vars or whatever we are calling it, count me in. Grab the low hanging fruit and leave the ladders at home for now. Lets look at the concrete example: url vars. Those of us implementing are willing to create compatibility. Breaking that compatibility is extremely undesirable by technical reality without any invented authority. I think all 5 of your requirements are already being met by natural means. If we come across a real case that calls out for a process then I will be the first to agree to adopt such a "necessary evil." Until then it is just an "evil." ;) I think we need articles, tutorials and improved docs to help spread the understanding of what WSGI component oriented development means. That is the missing piece, IMHO, not formality. As we see in REST vs. WS, the lack of formality can be quite nice. Cheers, - Luke On 11/2/06, Ian Bicking wrote: > Luke Arno wrote: > >> The "authority" is that: > >> > >> 1) the spec won't change without good reason, so you can start > >> implementing against it (once it is "approved") > >> 2) it's useful in multiple contexts > >> 3) having implemented either side, you can expect that maybe *someone* > >> will care (either producing or consuming what you are looking for) > >> 4) some eyes have been on it, and it's passed the sanity check (and > >> hopefully better than just a sanity check) > >> 5) someone won't implement something they think is the same, but isn't, > >> because the spec documents the requirements sufficiently > >> > >> Going through the process is a marker for all these items. And really > >> that's the only important part, if someone is able to claim all these > >> things then maybe that's the only process that is important. > > > > I agree that some repository of documents as > > to what conventions exist to enhance interop > > between wsgi components is highly valuable. > > I would rather see those documents present > > real information upon which developers can > > decide what to do rationally than an appeal to > > authority in order to influence conformity on a > > sociological basis. Hence my "bah" :) > > > > A document should describe the convention > > and its status in plain language and factual > > terms. In this case, I think we should just state > > that there is a general consensus and which > > implementations do and will support that. > > Validators are nice too. > > > > None of that requires formalization thus far. > > The list of features I give *is* important to me, and I think a lot of > other people. What process is required to get to that, I don't know -- > just stating these qualities may be sufficient. > > > Some statement on wsgi.org as to our desire > > to create interoperable components and our > > general commitment to not break our APIs > > without good reason might be good. I hope that > > goes without saying and if not I tend to think > > that actions (should) speak louder than words. > > > > Do we want to draw up some kind of WSGI > > community constitution while we are at it? ;) ;) > > Actually I think some of what Ben and I have chatted about on IRC does > go in that direction. "Constitution" is a bit much, of course -- one of > the good parts of WSGI components is that if you don't like something > you just don't use it, or you fork it, or whatever. We don't need > consensus. *But* the result can be confusing for consumers of > components. So we've discussed setting up some basic standards for > projects -- opt-in backward compatibility policies (which you'd only > adopt when you felt a piece had become "stable"), some security > policies, and maybe some indexing (which wsgi.org can do from a few > directions). > > > -- > Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org > From ianb at colorstudy.com Sun Nov 5 08:44:54 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 05 Nov 2006 01:44:54 -0600 Subject: [Web-SIG] ANN: WSGIRemote Message-ID: <454D9676.4090706@colorstudy.com> Hi all. I spent some time today with WSGIRemote, and while I don't have a real release I've got some docs and a stabler-than-before API, and it's registered so you can do "easy_install WSGIRemote==dev". I'm interested in feedback. The web page is at: http://pythonpaste.org/wsgiremote/ Now you ask: what is it? So, I've been thinking about how to have fine-grained WSGI services that do useful little tasks through RESTish APIs. If you use this technique, you'll get lots of little pieces and it's a pain to deploy, and while it's "scalable" in that you can move the pieces in between machines and processes, it doesn't scale down all that well since there's lots of HTTP requests and serialization/deserialization going on. Paste already has something of a solution for this with paste.recursive (http://pythonpaste.org/module-paste.recursive.html) which lets you do subrequests without going over HTTP. There's still serialization and deserialization. WSGIRemote handles this case by doing the serialization lazily, and letting the client see the unserialized data so that it can avoid serialization/parsing entirely. Anyway, that's the basic idea. The API should also be useful for plain ol' over-HTTP calls. I'm very interested in feedback, including just how the client API looks in general disregarding the WSGI integration. There's probably some higher level resourcy stuff that could be done, for instance. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From luke.arno at gmail.com Mon Nov 6 16:48:19 2006 From: luke.arno at gmail.com (Luke Arno) Date: Mon, 6 Nov 2006 10:48:19 -0500 Subject: [Web-SIG] routing_args (was url_vars) Message-ID: So, it seemed to me like we had agreement on this. Does anyone disagree? environ['routing_args'] == (list_or_tuple, a_dict) Should the spec get updated and marked as accepted? Cheers, - Luke From ianb at colorstudy.com Mon Nov 6 17:21:37 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 06 Nov 2006 10:21:37 -0600 Subject: [Web-SIG] routing_args (was url_vars) In-Reply-To: References: Message-ID: <454F6111.3000909@colorstudy.com> Luke Arno wrote: > So, it seemed to me like we had agreement on > this. Does anyone disagree? > > environ['routing_args'] == (list_or_tuple, a_dict) > > Should the spec get updated and marked as > accepted? I think so, with 'wsgiorg.routing_args'. I've just put it off because of the namespace discussion, but I think that's resolved too. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From luke.arno at gmail.com Mon Nov 6 18:09:57 2006 From: luke.arno at gmail.com (Luke Arno) Date: Mon, 6 Nov 2006 12:09:57 -0500 Subject: [Web-SIG] routing_args (was url_vars) In-Reply-To: <454F6111.3000909@colorstudy.com> References: <454F6111.3000909@colorstudy.com> Message-ID: On 11/6/06, Ian Bicking wrote: > Luke Arno wrote: > > So, it seemed to me like we had agreement on > > this. Does anyone disagree? > > > > environ['routing_args'] == (list_or_tuple, a_dict) > > > > Should the spec get updated and marked as > > accepted? > > I think so, with 'wsgiorg.routing_args'. I've just put it off because > of the namespace discussion, but I think that's resolved too. Yup. That was it. +1 - Luke From lcrees at gmail.com Tue Nov 7 01:54:55 2006 From: lcrees at gmail.com (L. C. Rees) Date: Mon, 06 Nov 2006 17:54:55 -0700 Subject: [Web-SIG] routing_args (was url_vars) Message-ID: <454FD95F.9060200@gmail.com> >>/ So, it seemed to me like we had agreement on />>/ this. Does anyone disagree? />>/ />>/ environ['routing_args'] == (list_or_tuple, a_dict) />>/ />>/ Should the spec get updated and marked as />>/ accepted? / > I think so, with 'wsgiorg.routing_args'. I've just put it off because > of the namespace discussion, but I think that's resolved too. +1 From wilk at flibuste.net Tue Nov 7 08:49:10 2006 From: wilk at flibuste.net (William Dode) Date: Tue, 7 Nov 2006 07:49:10 +0000 (UTC) Subject: [Web-SIG] routing_args (was url_vars) References: <454F6111.3000909@colorstudy.com> Message-ID: On 06-11-2006, Ian Bicking wrote: > Luke Arno wrote: >> So, it seemed to me like we had agreement on >> this. Does anyone disagree? >> >> environ['routing_args'] == (list_or_tuple, a_dict) >> >> Should the spec get updated and marked as >> accepted? > > I think so, with 'wsgiorg.routing_args'. I've just put it off because > of the namespace discussion, but I think that's resolved too. > when it's 'wsgiorg', don't you think that we should make an other pep or just a page in wsgi.org instead of updating the wsgi spec ? -- William Dod? - http://flibuste.net From ianb at colorstudy.com Tue Nov 7 19:14:19 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 07 Nov 2006 12:14:19 -0600 Subject: [Web-SIG] routing_args (was url_vars) In-Reply-To: References: <454F6111.3000909@colorstudy.com> Message-ID: <4550CCFB.4090805@colorstudy.com> William Dode wrote: > On 06-11-2006, Ian Bicking wrote: >> Luke Arno wrote: >>> So, it seemed to me like we had agreement on >>> this. Does anyone disagree? >>> >>> environ['routing_args'] == (list_or_tuple, a_dict) >>> >>> Should the spec get updated and marked as >>> accepted? >> I think so, with 'wsgiorg.routing_args'. I've just put it off because >> of the namespace discussion, but I think that's resolved too. >> > > when it's 'wsgiorg', don't you think that we should make an other pep or > just a page in wsgi.org instead of updating the wsgi spec ? These extensions won't go in PEP 333, and won't be PEPs at all. I've described it a little on this page: http://wsgi.org/wsgi/Specifications -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ben at groovie.org Tue Nov 7 19:21:13 2006 From: ben at groovie.org (Ben Bangert) Date: Tue, 7 Nov 2006 10:21:13 -0800 Subject: [Web-SIG] routing_args (was url_vars) In-Reply-To: References: Message-ID: <59424B7F-9B09-4E82-AD6A-47CA79606F86@groovie.org> On Nov 6, 2006, at 7:48 AM, Luke Arno wrote: > So, it seemed to me like we had agreement on > this. Does anyone disagree? > environ['routing_args'] == (list_or_tuple, a_dict) > > Should the spec get updated and marked as > accepted? > +1 here, when its finalized (how many more needed?) I'll include this in the next Routes release. Cheers, Ben From luke.arno at gmail.com Tue Nov 7 20:31:48 2006 From: luke.arno at gmail.com (Luke Arno) Date: Tue, 7 Nov 2006 14:31:48 -0500 Subject: [Web-SIG] routing_args accepted - selector updated Message-ID: I see that Ian has marked the proposed routing args convention as accepted. I have updated selector and now consider it a stable feature so feel free to use it. http://wsgi.org/wsgi/Specifications/routing_args https://lukearno.com/projects/selector/ Cheers, - Luke From sh at defuze.org Wed Nov 8 08:32:03 2006 From: sh at defuze.org (Sylvain Hellegouarch) Date: Wed, 08 Nov 2006 07:32:03 +0000 Subject: [Web-SIG] routing_args accepted - selector updated In-Reply-To: References: Message-ID: <455187F3.6020606@defuze.org> Luke Arno wrote: > I see that Ian has marked the proposed routing args > convention as accepted. I have updated selector and > now consider it a stable feature so feel free to use it. > > http://wsgi.org/wsgi/Specifications/routing_args > https://lukearno.com/projects/selector/ > Hello there, Sorry for being so late into the discussion. I must say I like that proposal and I completely sedond it. Except for one small point, the spec says: "The values in routing_args need not be strings (except for the keys of named_args). For instance, a dispatcher is allowed to parse /archive/2005/10/01 into ((), {'date': datetime.date(2005, 10, 1)})." This could be an issue for deployment if we consider that applications or other middleware will not be able to know what a middleware that implements this proposal has decided to do, map the values or not. Say I have the value '00001' in my URI, I don't want the dispatcher to assume it's an int and maps it to '1'. Maybe this value is an identifier for me and I need the complete value. There is a loss of information. What I would like to add to the proposal if it's not too late is the mean to tell the middleware whether or not I agree that it maps values automatically. Say: s = Selector(allow_mapping=False) Or something equivlent. Personally I would in fact disable that feature altogether and let the application decide what is best but I doubt you will appreciate that idea. In any case I will certainly use this proposal in the WSGI dispatcher of Amplee, my APP implementation. http://trac.defuze.org/browser/oss/amplee/amplee/handler/store/wsgi/__init__.py Thanks, - Sylvain From luke.arno at gmail.com Wed Nov 8 15:46:36 2006 From: luke.arno at gmail.com (Luke Arno) Date: Wed, 8 Nov 2006 09:46:36 -0500 Subject: [Web-SIG] routing_args accepted - selector updated In-Reply-To: <455187F3.6020606@defuze.org> References: <455187F3.6020606@defuze.org> Message-ID: On 11/8/06, Sylvain Hellegouarch wrote: > Luke Arno wrote: > > I see that Ian has marked the proposed routing args > > convention as accepted. I have updated selector and > > now consider it a stable feature so feel free to use it. > > > > http://wsgi.org/wsgi/Specifications/routing_args > > https://lukearno.com/projects/selector/ > > > > Hello there, > > Sorry for being so late into the discussion. I must say I like that > proposal and I completely sedond it. Except for one small point, the > spec says: > > "The values in routing_args need not be strings (except for the keys of > named_args). For instance, a dispatcher is allowed to parse > /archive/2005/10/01 into ((), {'date': datetime.date(2005, 10, 1)})." > > This could be an issue for deployment if we consider that applications > or other middleware will not be able to know what a middleware that > implements this proposal has decided to do, map the values or not. > > Say I have the value '00001' in my URI, I don't want the dispatcher to > assume it's an int and maps it to '1'. Maybe this value is an identifier > for me and I need the complete value. There is a loss of information. > > What I would like to add to the proposal if it's not too late is the > mean to tell the middleware whether or not I agree that it maps values > automatically. > > Say: > > s = Selector(allow_mapping=False) > > Or something equivlent. > > Personally I would in fact disable that feature altogether and let the > application decide what is best but I doubt you will appreciate that idea. > > In any case I will certainly use this proposal in the WSGI dispatcher of > Amplee, my APP implementation. > > http://trac.defuze.org/browser/oss/amplee/amplee/handler/store/wsgi/__init__.py > Hi Sylvain, Selector does no coercion and I have the same preference. Everything comes through as strings. Some dispatch strategies do entail coercion and it is a hurdle to making dispatchees portable but I don't seen the routing_args convention being adopted if there was a restriction on types. I agree with you that it would be nice for any dispatch tool that does coercion to make it easy to turn off. Maybe we should annotate the spec? Cheers, - Luke From ianb at colorstudy.com Wed Nov 8 17:48:05 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 08 Nov 2006 10:48:05 -0600 Subject: [Web-SIG] routing_args accepted - selector updated In-Reply-To: <455187F3.6020606@defuze.org> References: <455187F3.6020606@defuze.org> Message-ID: <45520A45.9080708@colorstudy.com> Sylvain Hellegouarch wrote: > Luke Arno wrote: >> I see that Ian has marked the proposed routing args >> convention as accepted. I have updated selector and >> now consider it a stable feature so feel free to use it. >> >> http://wsgi.org/wsgi/Specifications/routing_args >> https://lukearno.com/projects/selector/ >> > > Hello there, > > Sorry for being so late into the discussion. I must say I like that > proposal and I completely sedond it. Except for one small point, the > spec says: > > "The values in routing_args need not be strings (except for the keys of > named_args). For instance, a dispatcher is allowed to parse > /archive/2005/10/01 into ((), {'date': datetime.date(2005, 10, 1)})." > > This could be an issue for deployment if we consider that applications > or other middleware will not be able to know what a middleware that > implements this proposal has decided to do, map the values or not. > > Say I have the value '00001' in my URI, I don't want the dispatcher to > assume it's an int and maps it to '1'. Maybe this value is an identifier > for me and I need the complete value. There is a loss of information. My assumption was that, while non-strings are allowed, it would be something the dispatcher would handle when asked to do so explicitly. That is, a dispatcher shouldn't speculatively coerce values to integers or dates. Every dispatcher has some kind of setup/configure routine, which is where type coercions are likely to go. If this doesn't seem clear from the spec, this could be noted. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ianb at colorstudy.com Fri Nov 10 19:40:55 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 10 Nov 2006 12:40:55 -0600 Subject: [Web-SIG] Possible specs Message-ID: <4554C7B7.3070909@colorstudy.com> I brainstormed some ideas for wsgiorg specs and added them to the spec page, and also copied here. I offer them here to see if there's particular specifications that seem interesting, and might be worth pursuing sooner than other ones. * Ben Bangert suggested a simple session standard, focused solely on the session ID (persistence handled elsewhere). This is fairly modest but still useful. * Maybe a full session interface built on the session ID standard. * Often debugging tools open security holes (for example, paste.evalexception gives you a Python prompt on every exception). Authentication isn't really the right way to handle it, because debugging might involve logging in as various users. A specification could just define a key that indicates when these debugging tools should be allowed. This might get set by configuration, IP address, a cookie, etc. * Debugging mode is something that can be used in all sorts of places; to increase verbosity, annotate output pages, displaying errors in the browser, etc. Having a single key for turning on debugging mode would allow its consumption in lots of places. Not as strict as authenticating. * Some systems prefer that unexpected exceptions bubble up, like test frameworks. A key could define this case (modelled on paste.throw_errors) and thus disable exception catchers. * Logging is a tricky situation. The logging module allows for statically setting up logging systems, then configuring them at startup. This often isn't the best way to set up logging. Putting a logging.Logger instance right in the environment might be better. This requires some design and usage before setting on one spec. * Request object wrapping the environment. * Thread-local values are a common technique in web frameworks, allowing global objects or functions to return request-specific information. This pattern could be codified into one core system, using some feedback from existing systems (which have their advantages and flaws). * Configuration takes fairly common forms, usually a dict of some sort. It could be put somewhere standard. * Maybe Paste Deploy's entry points could be standardized. (Paste Deploy itself only consumes those entry points; other consumers are possible and packages implementing those entry points don't introduce any dependency on Paste Deploy) * A way to extend wsgiref.validate to add more validation, for all these new specs. (Probably this is an implementation, not a spec) * A way to describe custom keys, maybe associated with the validation. * Anchors for doing recursive calls, similar to paste.recursive. (paste.recursive is kind of an old module that is more complicated than it needs to be) * A place to put a database transaction manager * More user information than just REMOTE_USER; like wsgiorg.user_info? -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ben at groovie.org Fri Nov 10 20:53:21 2006 From: ben at groovie.org (Ben Bangert) Date: Fri, 10 Nov 2006 11:53:21 -0800 Subject: [Web-SIG] Possible specs In-Reply-To: <4554C7B7.3070909@colorstudy.com> References: <4554C7B7.3070909@colorstudy.com> Message-ID: <78396D27-AB2A-497F-9C95-16D796BEF192@groovie.org> On Nov 10, 2006, at 10:40 AM, Ian Bicking wrote: > * Ben Bangert suggested a simple session standard, focused solely > on the session ID (persistence handled elsewhere). This is fairly > modest > but still useful. I'll have to find that and take a look as I no longer recall what I was talking about. :) > * Debugging mode is something that can be used in all sorts of > places; to increase verbosity, annotate output pages, displaying > errors > in the browser, etc. Having a single key for turning on debugging mode > would allow its consumption in lots of places. Not as strict as > authenticating. I'd really like this. In addition to having a key that indicates the app is in debug mode, there should probably be one to indicate its in an automated testing mode as well for unit/function tests. The testing one should also provide a key where object ref's can be put for a WSGI unit testing harnass, like how paste.fixture's TestApp runs. > * Request object wrapping the environment. I'd imagine this would be a base interface? > * Configuration takes fairly common forms, usually a dict of some > sort. It could be put somewhere standard. Yep, should be a pretty minimal spec as well. > * More user information than just REMOTE_USER; like > wsgiorg.user_info? I think a base spec for user information could be modeled on the user data information thats in the OpenID 2.0 spec. The same user keys/ values are in the LID spec as well, modeling our environ dict for the wsgiorg.user_info on those spec's would make using them easy, and they account for all the common fields in most every auth system I've encountered. Overall, I think we should hit the low hanging fruit first. So having the simple keys be standardized as wsgiorg specs first, then we can hit the more complex ones like session id's and stuff that requires more of an interface. Cheers, Ben From luke.arno at gmail.com Fri Nov 10 22:06:12 2006 From: luke.arno at gmail.com (Luke Arno) Date: Fri, 10 Nov 2006 16:06:12 -0500 Subject: [Web-SIG] Possible specs In-Reply-To: <4554C7B7.3070909@colorstudy.com> References: <4554C7B7.3070909@colorstudy.com> Message-ID: On 11/10/06, Ian Bicking wrote: > I brainstormed some ideas for wsgiorg specs and added them to the spec > page, and also copied here. I offer them here to see if there's > particular specifications that seem interesting, and might be worth > pursuing sooner than other ones. Wow. This is a tidal wave. :) I haven't had a chance to read on the wiki but I will comment on the short descriptions. > * Ben Bangert suggested a simple session standard, focused solely > on the session ID (persistence handled elsewhere). This is fairly modest > but still useful. > Maybe. Pick a key. wsgiorg.session_id? Does anyone use more than one session at a time? I don't, but it seems wise to ask. > * Maybe a full session interface built on the session ID standard. Maybe. I favor the dict interface. Must be lazy-loadable. wsgiorg.session_loader = callable that takes environ wsgiorg.session = cached dict like object We need a method to invalidate (on the dict-like?) ... > * Often debugging tools open security holes (for example, > paste.evalexception gives you a Python prompt on every exception). > Authentication isn't really the right way to handle it, because > debugging might involve logging in as various users. A specification > could just define a key that indicates when these debugging tools should > be allowed. This might get set by configuration, IP address, a cookie, etc. I don't care because I don't use anything like that. (Nor do I want to.) > * Debugging mode is something that can be used in all sorts of > places; to increase verbosity, annotate output pages, displaying errors > in the browser, etc. Having a single key for turning on debugging mode > would allow its consumption in lots of places. Not as strict as > authenticating. Maybe. wsgiorg.log_level? If you change the log level, do you want this to effect the whole stack or just your own stuff? > * Some systems prefer that unexpected exceptions bubble up, like > test frameworks. A key could define this case (modelled on > paste.throw_errors) and thus disable exception catchers. -1 I don't need it. Too abstract. Usually, you just have one error handling middleware on the outside of the rest, no? > * Logging is a tricky situation. The logging module allows for > statically setting up logging systems, then configuring them at startup. > This often isn't the best way to set up logging. Putting a > logging.Logger instance right in the environment might be better. This > requires some design and usage before setting on one spec. Maybe a lazy logger loader that takes the level as an argument? Seems a little silly. > * Request object wrapping the environment. I use YARO request objects for this and it works out well. Do we need a standard? What if different middleware authors prefer different request objects? If they are competing for the same environ key that is not good. I had a long email conversation with someone who wanted to push for YARO to become standard but I am just not sure that makes sense. > * Thread-local values are a common technique in web frameworks, > allowing global objects or functions to return request-specific > information. This pattern could be codified into one core system, using > some feedback from existing systems (which have their advantages and flaws). -1 but I don't like using thread locals for such things. Save the magic for things that need it. :) > * Configuration takes fairly common forms, usually a dict of some > sort. It could be put somewhere standard. -1 Different components may need different configs. I have seen class instances and modules, not just dictionaries. > * Maybe Paste Deploy's entry points could be standardized. (Paste > Deploy itself only consumes those entry points; other consumers are > possible and packages implementing those entry points don't introduce > any dependency on Paste Deploy) -1 We would have to conform to your ABC to make use of this, right? > * A way to extend wsgiref.validate to add more validation, for all > these new specs. (Probably this is an implementation, not a spec) That makes sense if our standards are that involved. I don't see standardizable clarity on much that is so complex. > * A way to describe custom keys, maybe associated with the validation. Same. > * Anchors for doing recursive calls, similar to paste.recursive. > (paste.recursive is kind of an old module that is more complicated than > it needs to be) Is that really such a common pattern? It is clever but I have yet to find a case for it. Maybe I am just overlooking something. What do you usually use that for? > * A place to put a database transaction manager -1 Way too specific. > * More user information than just REMOTE_USER; like wsgiorg.user_info? User objects are like request objects and I don't see what we gain from making them fight for a key. Could this actually decrease real interop? I am worried that things are getting a little inventive here. This is how things get heavy. Lets try to stick to codifying those things that have a clear common right way. Most of these don't meet that, in my opinion. Pursuing many of these will lead to endless arguments as they are just unproven which means we should just let the diversity simmer for a while. Sessions *might* be doable. IMHO, YMMV, ... Cheers, - Luke From ianb at colorstudy.com Fri Nov 10 22:14:31 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 10 Nov 2006 15:14:31 -0600 Subject: [Web-SIG] Possible specs In-Reply-To: <78396D27-AB2A-497F-9C95-16D796BEF192@groovie.org> References: <4554C7B7.3070909@colorstudy.com> <78396D27-AB2A-497F-9C95-16D796BEF192@groovie.org> Message-ID: <4554EBB7.8010307@colorstudy.com> Ben Bangert wrote: > On Nov 10, 2006, at 10:40 AM, Ian Bicking wrote: > >> * Ben Bangert suggested a simple session standard, focused solely >> on the session ID (persistence handled elsewhere). This is fairly >> modest >> but still useful. > > I'll have to find that and take a look as I no longer recall what I > was talking about. :) It's in the web-sig archives somewhere, I think. It was quite a while ago. >> * Debugging mode is something that can be used in all sorts of >> places; to increase verbosity, annotate output pages, displaying >> errors >> in the browser, etc. Having a single key for turning on debugging mode >> would allow its consumption in lots of places. Not as strict as >> authenticating. > > I'd really like this. In addition to having a key that indicates the > app is in debug mode, there should probably be one to indicate its in > an automated testing mode as well for unit/function tests. The > testing one should also provide a key where object ref's can be put > for a WSGI unit testing harnass, like how paste.fixture's TestApp runs. Yes, I thought about that too. It can be a good to use as a guard against things like sending email. I don't know if the testing variables necessarily need standardization, but I suppose they could have it. There's no consumers right now besides paste.fixture.TestApp. There are other frameworks that could consume those variables, but none of them are very closely integrated with WSGI (yet?). I don't those other frameworks have any way of dealing with that kind of information. >> * Request object wrapping the environment. > > I'd imagine this would be a base interface? I'll expand on that in a separate message, as I have a fairly minimal spec in mind. >> * Configuration takes fairly common forms, usually a dict of some >> sort. It could be put somewhere standard. > > Yep, should be a pretty minimal spec as well. It's a little more vague when you have more structured configuration, e.g., in Paste Deploy situations you have the global and local configuration. OTOH, the common interface can be a flat dictionary, with optional attributes for further optional structure. >> * More user information than just REMOTE_USER; like >> wsgiorg.user_info? > > I think a base spec for user information could be modeled on the user > data information thats in the OpenID 2.0 spec. The same user keys/ > values are in the LID spec as well, modeling our environ dict for the > wsgiorg.user_info on those spec's would make using them easy, and > they account for all the common fields in most every auth system I've > encountered. Where in the spec are those values? I was mostly thinking it would be a dictionary similar to the WSGI environment, and the contents would be fairly vague. Maybe it would have values like user_info['openid.homepage'], all of which would of course be optional. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ianb at colorstudy.com Fri Nov 10 22:45:13 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 10 Nov 2006 15:45:13 -0600 Subject: [Web-SIG] Possible specs In-Reply-To: References: <4554C7B7.3070909@colorstudy.com> Message-ID: <4554F2E9.2050107@colorstudy.com> Luke Arno wrote: > On 11/10/06, Ian Bicking wrote: >> I brainstormed some ideas for wsgiorg specs and added them to the spec >> page, and also copied here. I offer them here to see if there's >> particular specifications that seem interesting, and might be worth >> pursuing sooner than other ones. > > Wow. This is a tidal wave. :) > > I haven't had a chance to read on the wiki but I will > comment on the short descriptions. This is a straight copy of the wiki, I haven't written anything up on any of these beyond what you see here. >> * Ben Bangert suggested a simple session standard, focused solely >> on the session ID (persistence handled elsewhere). This is fairly modest >> but still useful. >> > > Maybe. Pick a key. wsgiorg.session_id? > > Does anyone use more than one session at a > time? I don't, but it seems wise to ask. Since it is backend/persistence-neutral, it's up the app to figure out how to consume it; there could be multiple sessions for a given ID, or not. This just gives you a stable ID. It should probably be created on demand, as if you don't care about the session you shouldn't be generating a session ID (and cookie to go with). >> * Maybe a full session interface built on the session ID standard. > > Maybe. > > I favor the dict interface. Must be lazy-loadable. > > wsgiorg.session_loader = callable that takes environ > wsgiorg.session = cached dict like object > > We need a method to invalidate (on the dict-like?) This is more-or-less what paste.session does. It might be a little too minimal, though. For instance, there's no way to distinguish between read-only access and read-write access; hence no way to do locking or serialization (without adding some real concurrency problems), and no way to avoid rewriting the session on every request in case some contents have been changed. >> * Debugging mode is something that can be used in all sorts of >> places; to increase verbosity, annotate output pages, displaying errors >> in the browser, etc. Having a single key for turning on debugging mode >> would allow its consumption in lots of places. Not as strict as >> authenticating. > > Maybe. > > wsgiorg.log_level? If you change the log level, > do you want this to effect the whole stack or > just your own stuff? I often do a bunch of stuff in debugging mode that is more aggressive than just log_level, and shouldn't be done at all in normal mode. So a real on-off switch for debugging is useful. >> * Some systems prefer that unexpected exceptions bubble up, like >> test frameworks. A key could define this case (modelled on >> paste.throw_errors) and thus disable exception catchers. > > -1 I don't need it. Too abstract. > > Usually, you just have one error handling > middleware on the outside of the rest, no? Sometimes there's multiple pieces of middleware. For instance, I might have one on the outside, with another one in a subapplication (which is independently deployable, so needs some kind of wrapper). Also it is needed to avoid catching errors when running tests, where you want the exception to go up all the way to your test runner (which might have a debugger or special traceback formatter). >> * Logging is a tricky situation. The logging module allows for >> statically setting up logging systems, then configuring them at startup. >> This often isn't the best way to set up logging. Putting a >> logging.Logger instance right in the environment might be better. This >> requires some design and usage before setting on one spec. > > Maybe a lazy logger loader that takes the level > as an argument? Seems a little silly. I'm not really sure here, myself. I know people ask for it, and sometimes I'm pretty sure I want something like this, but I don't feel very solid about what best practice in logging is. I couldn't write this one. >> * Thread-local values are a common technique in web frameworks, >> allowing global objects or functions to return request-specific >> information. This pattern could be codified into one core system, using >> some feedback from existing systems (which have their advantages and >> flaws). > > -1 but I don't like using thread locals for such things. > Save the magic for things that need it. :) Thread local stuff can be a pain; I often curse it. I think it can be okay with the right patterns. >> * Configuration takes fairly common forms, usually a dict of some >> sort. It could be put somewhere standard. > > -1 Different components may need different configs. > I have seen class instances and modules, not > just dictionaries. > >> * Maybe Paste Deploy's entry points could be standardized. (Paste >> Deploy itself only consumes those entry points; other consumers are >> possible and packages implementing those entry points don't introduce >> any dependency on Paste Deploy) > > -1 We would have to conform to your ABC > to make use of this, right? No, not at all, there's no base class (not even an optional base class). The interface for apps is basically: def app_factory(global_conf, **app_conf): return wsgi app It's pretty neutral, with some notable details: - global_conf is basically inherited configuration. It may or may not apply to this application. The application factory can ignore it or not as it sees fit. app_conf is definitely intended for this application; you can do more strict checking on it, make sure there's no extra values or that certain required values are present, etc. - all values can be strings, and strings should be handled specially. That is, if you expect an integer and you get a string, you should coerce it to an integer. If you expect a list and you get a string, you should probably split the string in some fashion, or just turn it into a one-item list. If you expect a boolean and you get a string, you should convert it intelligently (not based on the truth/falsiness of the string itself). That's it. Middleware and servers have similar interfaces. The server interface is a little under-powered (it just serves a WSGI application forever); it could be extended some, or left unspecified for now. Middleware takes as a first argument the application being wrapped. Oh, and composites, which are a little harder -- they take as a first argument an object that can load more WSGI apps. That's used for dispatchers that can direct to multiple subapplications. That's more tightly coupled with Paste Deploy's app naming conventions, and it might be better to put explicit app loading into the configuration format and pass them in as keyword arguments to the dispatching app. >> * A way to extend wsgiref.validate to add more validation, for all >> these new specs. (Probably this is an implementation, not a spec) > > That makes sense if our standards are that involved. > I don't see standardizable clarity on much that is so > complex. WSGI isn't all that complex, but validation is very helpful when people get it wrong. It should be useful for any of these specs as well. It also makes the spec much more explicitly clear, because computers check things more reliably than humans ;) >> * Anchors for doing recursive calls, similar to paste.recursive. >> (paste.recursive is kind of an old module that is more complicated than >> it needs to be) > > Is that really such a common pattern? It is clever > but I have yet to find a case for it. Maybe I am just > overlooking something. What do you usually use > that for? Originally it was to support internal redirects and inclusion (which were part of the Webware API). Now I find it useful for doing internal subrequests when using web-based APIs. >> * A place to put a database transaction manager > > -1 Way too specific. There'd have to be a database transaction manager standard first, anyway. >> * More user information than just REMOTE_USER; like >> wsgiorg.user_info? > > User objects are like request objects and I don't see > what we gain from making them fight for a key. > > Could this actually decrease real interop? This shouldn't be a real user object, more likely a dictionary, just like the WSGI request is a dictionary (environ) and not an object. Objects are hard to standardize ;) > I am worried that things are getting a little inventive > here. This is how things get heavy. Lets try to stick > to codifying those things that have a clear common > right way. Most of these don't meet that, in my > opinion. Pursuing many of these will lead to endless > arguments as they are just unproven which means > we should just let the diversity simmer for a while. Most of them are taken from keys already in Paste (or in a couple cases, that I wish were in Paste but have been lazy about adding). > Sessions *might* be doable. Possibly; but sessions don't personally interest me a whole lot. It wouldn't be my first choice to write up, but if someone else writes it up then great ;) -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ianb at colorstudy.com Fri Nov 10 22:51:12 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 10 Nov 2006 15:51:12 -0600 Subject: [Web-SIG] (proto) request object spec Message-ID: <4554F450.3070901@colorstudy.com> To clarify my note about a standard request object in my last email, here's the basic framework of the spec I imagine: A request object conforming to this specification will have a .environ attribute. This will be the WSGI request environment. The request object will also not store any state internally. All state will be stored in the WSGI request environment. If the environment is extracted and the request object recreated, then the resulting new request object should be entirely equivalent to the original request object. This makes it easy to get the request object *you* want, given a request object you may not want. The state requirement means intermediaries may change the request object without any loss of information. If a request object stores information in a non-standard key, then that information may not be directly exposed in other request objects, but it's not hidden and it is recoverable. In addition to this very minimal set of requirements for a request object, the specification could have a kind of style guide for request objects. No request object would have to conform, and it would not be intended to ensure interoperability. The style guide would just be to increase familiarity of developers when they encounter a new request object. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From luke.arno at gmail.com Sat Nov 11 03:12:47 2006 From: luke.arno at gmail.com (Luke Arno) Date: Fri, 10 Nov 2006 21:12:47 -0500 Subject: [Web-SIG] Possible specs In-Reply-To: <4554F2E9.2050107@colorstudy.com> References: <4554C7B7.3070909@colorstudy.com> <4554F2E9.2050107@colorstudy.com> Message-ID: On 11/10/06, Ian Bicking wrote: > Luke Arno wrote: > > On 11/10/06, Ian Bicking wrote: > >> I brainstormed some ideas for wsgiorg specs and added them to the spec > >> page, and also copied here. I offer them here to see if there's > >> particular specifications that seem interesting, and might be worth > >> pursuing sooner than other ones. > > > > Wow. This is a tidal wave. :) > > > > I haven't had a chance to read on the wiki but I will > > comment on the short descriptions. > > This is a straight copy of the wiki, I haven't written anything up on > any of these beyond what you see here. Ah, I see. > >> * Ben Bangert suggested a simple session standard, focused solely > >> on the session ID (persistence handled elsewhere). This is fairly modest > >> but still useful. > >> > > > > Maybe. Pick a key. wsgiorg.session_id? > > > > Does anyone use more than one session at a > > time? I don't, but it seems wise to ask. > > Since it is backend/persistence-neutral, it's up the app to figure out > how to consume it; there could be multiple sessions for a given ID, or > not. This just gives you a stable ID. It should probably be created on > demand, as if you don't care about the session you shouldn't be > generating a session ID (and cookie to go with). > > > >> * Maybe a full session interface built on the session ID standard. > > > > Maybe. > > > > I favor the dict interface. Must be lazy-loadable. > > > > wsgiorg.session_loader = callable that takes environ > > wsgiorg.session = cached dict like object > > > > We need a method to invalidate (on the dict-like?) > > This is more-or-less what paste.session does. It might be a little too > minimal, though. For instance, there's no way to distinguish between > read-only access and read-write access; hence no way to do locking or > serialization (without adding some real concurrency problems), and no > way to avoid rewriting the session on every request in case some > contents have been changed. > > > >> * Debugging mode is something that can be used in all sorts of > >> places; to increase verbosity, annotate output pages, displaying errors > >> in the browser, etc. Having a single key for turning on debugging mode > >> would allow its consumption in lots of places. Not as strict as > >> authenticating. > > > > Maybe. > > > > wsgiorg.log_level? If you change the log level, > > do you want this to effect the whole stack or > > just your own stuff? > > I often do a bunch of stuff in debugging mode that is more aggressive > than just log_level, and shouldn't be done at all in normal mode. So a > real on-off switch for debugging is useful. I expect your development (like mine) is a little atypical. I am often working on various parts of the stack at the same time. Most web developers are just working on an app and would not like to see debugging info start pouring out of everything under it. That is probably not what you are suggesting. Still, I think that this is a low value target for community standardization (though for an individual who is all over the stack, it is a great personal convention.) > >> * Some systems prefer that unexpected exceptions bubble up, like > >> test frameworks. A key could define this case (modelled on > >> paste.throw_errors) and thus disable exception catchers. > > > > -1 I don't need it. Too abstract. > > > > Usually, you just have one error handling > > middleware on the outside of the rest, no? > > Sometimes there's multiple pieces of middleware. For instance, I might > have one on the outside, with another one in a subapplication (which is > independently deployable, so needs some kind of wrapper). Also it is > needed to avoid catching errors when running tests, where you want the > exception to go up all the way to your test runner (which might have a > debugger or special traceback formatter). This is probably just not relevant to my development style. Or maybe I just don't get it. > > >> * Logging is a tricky situation. The logging module allows for > >> statically setting up logging systems, then configuring them at startup. > >> This often isn't the best way to set up logging. Putting a > >> logging.Logger instance right in the environment might be better. This > >> requires some design and usage before setting on one spec. > > > > Maybe a lazy logger loader that takes the level > > as an argument? Seems a little silly. > > I'm not really sure here, myself. I know people ask for it, and > sometimes I'm pretty sure I want something like this, but I don't feel > very solid about what best practice in logging is. I couldn't write > this one. I am with you. I create loggers for their respective scopes and call them as needed. Log levels go in a config and that is the whole story. What else? I think people sometimes ask for structure because they are unsure when all they need is to go for it. > >> * Thread-local values are a common technique in web frameworks, > >> allowing global objects or functions to return request-specific > >> information. This pattern could be codified into one core system, using > >> some feedback from existing systems (which have their advantages and > >> flaws). > > > > -1 but I don't like using thread locals for such things. > > Save the magic for things that need it. :) > > Thread local stuff can be a pain; I often curse it. I think it can be > okay with the right patterns. If we codify it, I think it will be used. A lot. I recently looked over a sort of a contact manager app for somebody. There were metaclasses all over the place. :) > >> * Configuration takes fairly common forms, usually a dict of some > >> sort. It could be put somewhere standard. > > > > -1 Different components may need different configs. > > I have seen class instances and modules, not > > just dictionaries. > > > >> * Maybe Paste Deploy's entry points could be standardized. (Paste > >> Deploy itself only consumes those entry points; other consumers are > >> possible and packages implementing those entry points don't introduce > >> any dependency on Paste Deploy) > > > > -1 We would have to conform to your ABC > > to make use of this, right? > > No, not at all, there's no base class (not even an optional base class). Sorry, I was thinking of Paste Script. I can't keep it all straight sometimes. ;) > The interface for apps is basically: > > def app_factory(global_conf, **app_conf): > return wsgi app > > It's pretty neutral, with some notable details: > > - global_conf is basically inherited configuration. It may or may not > apply to this application. The application factory can ignore it or not > as it sees fit. app_conf is definitely intended for this application; > you can do more strict checking on it, make sure there's no extra values > or that certain required values are present, etc. > > - all values can be strings, and strings should be handled specially. > That is, if you expect an integer and you get a string, you should > coerce it to an integer. If you expect a list and you get a string, you > should probably split the string in some fashion, or just turn it into a > one-item list. If you expect a boolean and you get a string, you should > convert it intelligently (not based on the truth/falsiness of the string > itself). > > > That's it. Middleware and servers have similar interfaces. The server > interface is a little under-powered (it just serves a WSGI application > forever); it could be extended some, or left unspecified for now. > Middleware takes as a first argument the application being wrapped. Oh, > and composites, which are a little harder -- they take as a first > argument an object that can load more WSGI apps. That's used for > dispatchers that can direct to multiple subapplications. That's more > tightly coupled with Paste Deploy's app naming conventions, and it might > be better to put explicit app loading into the configuration format and > pass them in as keyword arguments to the dispatching app. This is all a bit meta for me. I guess I don't get it. :) > >> * A way to extend wsgiref.validate to add more validation, for all > >> these new specs. (Probably this is an implementation, not a spec) > > > > That makes sense if our standards are that involved. > > I don't see standardizable clarity on much that is so > > complex. > > WSGI isn't all that complex, but validation is very helpful when people > get it wrong. It should be useful for any of these specs as well. It > also makes the spec much more explicitly clear, because computers check > things more reliably than humans ;) I am not arguing against validation. I am just hoping that it's superfluous in the near term. > >> * Anchors for doing recursive calls, similar to paste.recursive. > >> (paste.recursive is kind of an old module that is more complicated than > >> it needs to be) > > > > Is that really such a common pattern? It is clever > > but I have yet to find a case for it. Maybe I am just > > overlooking something. What do you usually use > > that for? > > Originally it was to support internal redirects and inclusion (which > were part of the Webware API). Now I find it useful for doing internal > subrequests when using web-based APIs. That makes more sense. I have some forwarding stuff like that in YARO because it can be used to hide the WSGI interface a little. Other than that I just call the thing I want to forward to. (I don't like to use exceptions for normal flow control very much, though it does come up.) Either way, finding what to forward to is the trick, which is dispatch, and there are a whole lot of ways to skin that cat. > >> * A place to put a database transaction manager > > > > -1 Way too specific. > > There'd have to be a database transaction manager standard first, anyway. I admire your ambition. :) > >> * More user information than just REMOTE_USER; like > >> wsgiorg.user_info? > > > > User objects are like request objects and I don't see > > what we gain from making them fight for a key. > > > > Could this actually decrease real interop? > > This shouldn't be a real user object, more likely a dictionary, just > like the WSGI request is a dictionary (environ) and not an object. > Objects are hard to standardize ;) They are. I would be interested in this though I remain somewhat skeptical. > > I am worried that things are getting a little inventive > > here. This is how things get heavy. Lets try to stick > > to codifying those things that have a clear common > > right way. Most of these don't meet that, in my > > opinion. Pursuing many of these will lead to endless > > arguments as they are just unproven which means > > we should just let the diversity simmer for a while. > > Most of them are taken from keys already in Paste (or in a couple cases, > that I wish were in Paste but have been lazy about adding). This is exactly my point. You have made a big impact with Paste because it is very pioneering. While that is very commendable, I would like to see things play out on a wider stage before they get standardized. > > Sessions *might* be doable. > > Possibly; but sessions don't personally interest me a whole lot. It > wouldn't be my first choice to write up, but if someone else writes it > up then great ;) Yeah it is sort of boring but could simplify interop and is a pretty well understood thing. I'm not volunteering, of course. ;) Maybe, if I find time. I feel like a total wet blanket right now but I hope it is useful. Cheers, - Luke From luke.arno at gmail.com Sat Nov 11 03:35:12 2006 From: luke.arno at gmail.com (Luke Arno) Date: Fri, 10 Nov 2006 21:35:12 -0500 Subject: [Web-SIG] (proto) request object spec In-Reply-To: <4554F450.3070901@colorstudy.com> References: <4554F450.3070901@colorstudy.com> Message-ID: On 11/10/06, Ian Bicking wrote: > To clarify my note about a standard request object in my last email, > here's the basic framework of the spec I imagine: > > A request object conforming to this specification will have a .environ > attribute. This will be the WSGI request environment. > > The request object will also not store any state internally. All state > will be stored in the WSGI request environment. If the environment is > extracted and the request object recreated, then the resulting new > request object should be entirely equivalent to the original request object. > > This makes it easy to get the request object *you* want, given a request > object you may not want. The state requirement means intermediaries may > change the request object without any loss of information. If a request > object stores information in a non-standard key, then that information > may not be directly exposed in other request objects, but it's not > hidden and it is recoverable. If I understand you: if I tack some environ values on to my request object and the user changes those value on the request object, this must automatically be reflected in the environ. Is that right? If so, I disagree. If the user wants to change the value of something in the environ, she can do so. If not that is fine. I would rather say "no auto-magical updates!" > In addition to this very minimal set of requirements for a request > object, the specification could have a kind of style guide for request > objects. No request object would have to conform, and it would not be > intended to ensure interoperability. The style guide would just be to > increase familiarity of developers when they encounter a new request object. The whole point of a request object is to provide a more friendly and to-taste interface to cover the universal but decidedly (and necessarily) clunky WSGI interface. If we specify a style guide, many will feel compelled to conform to it. This seems counterproductive when the very purpose is to meet diverse preferences. Though the similarity of request objects out there (note that I called mine "Yet Another Request Object") gives the appearance of a prime candidate for standardization, the subtle (or not) differences are very important to people in this area. It is sort of like the chair. It is a lot like the chair, actually: you are in it all day and it needs to be adjustable and fit just right. If this were a good candidate for standardization, I think WSGI would look a whole lot different. Cheers, - Luke From ianb at colorstudy.com Sat Nov 11 04:25:55 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 10 Nov 2006 21:25:55 -0600 Subject: [Web-SIG] Possible specs In-Reply-To: References: <4554C7B7.3070909@colorstudy.com> <4554F2E9.2050107@colorstudy.com> Message-ID: <455542C3.2010705@colorstudy.com> Luke Arno wrote: >> >> * Debugging mode is something that can be used in all sorts of >> >> places; to increase verbosity, annotate output pages, displaying >> errors >> >> in the browser, etc. Having a single key for turning on debugging mode >> >> would allow its consumption in lots of places. Not as strict as >> >> authenticating. >> > >> > Maybe. >> > >> > wsgiorg.log_level? If you change the log level, >> > do you want this to effect the whole stack or >> > just your own stuff? >> >> I often do a bunch of stuff in debugging mode that is more aggressive >> than just log_level, and shouldn't be done at all in normal mode. So a >> real on-off switch for debugging is useful. > > I expect your development (like mine) is a little atypical. > I am often working on various parts of the stack at the > same time. Most web developers are just working on > an app and would not like to see debugging info start > pouring out of everything under it. That is probably not > what you are suggesting. Still, I think that this is a low > value target for community standardization (though for > an individual who is all over the stack, it is a great > personal convention.) I find it fairly widely useful. Right now I do environ['paste.config'].get('debug') to figure out if I'm in debug mode, and I find myself doing this at all sorts of levels, including directly in application code. Though one potential problem is if everyone starts listening for the same key, then the debugging information could become overwhelming. In the context of the Paste Deploy entry point, I might generally do it like: class NoDefault: pass # marker def make_some_middleware(global_conf, debug=NoDefault, ...): if debug is NoDefault: debug = global_conf.get('debug') debug = paste.deploy.converters.asbool(debug) ... And maybe this is good enough. >> >> * Some systems prefer that unexpected exceptions bubble up, like >> >> test frameworks. A key could define this case (modelled on >> >> paste.throw_errors) and thus disable exception catchers. >> > >> > -1 I don't need it. Too abstract. >> > >> > Usually, you just have one error handling >> > middleware on the outside of the rest, no? >> >> Sometimes there's multiple pieces of middleware. For instance, I might >> have one on the outside, with another one in a subapplication (which is >> independently deployable, so needs some kind of wrapper). Also it is >> needed to avoid catching errors when running tests, where you want the >> exception to go up all the way to your test runner (which might have a >> debugger or special traceback formatter). > > This is probably just not relevant to my development > style. Or maybe I just don't get it. Well, the general pattern is: def error_catcher(app): def replacement_app(environ, start_response): if environ.get('paste.throw_errors'): return app(environ, start_response) try: return app(environ, start_response) except: # handle and report on error return replacement_app In a testing environment I always want errors to bubble up, I don't want a 500 response if it can be helped; so in paste.fixture.TestApp I always set this. It's not a big deal, but it's fairly simple to explain and handle. Low hanging fruit. >> >> * Logging is a tricky situation. The logging module allows for >> >> statically setting up logging systems, then configuring them at >> startup. >> >> This often isn't the best way to set up logging. Putting a >> >> logging.Logger instance right in the environment might be better. This >> >> requires some design and usage before setting on one spec. >> > >> > Maybe a lazy logger loader that takes the level >> > as an argument? Seems a little silly. >> >> I'm not really sure here, myself. I know people ask for it, and >> sometimes I'm pretty sure I want something like this, but I don't feel >> very solid about what best practice in logging is. I couldn't write >> this one. > > I am with you. I create loggers for their respective > scopes and call them as needed. Log levels go in > a config and that is the whole story. What else? > > I think people sometimes ask for structure because > they are unsure when all they need is to go for it. Well, often the container knows more about how logging should work than the thing itself knows. Probably most of the time. Passing around a logger helps with this. >> >> * Thread-local values are a common technique in web frameworks, >> >> allowing global objects or functions to return request-specific >> >> information. This pattern could be codified into one core system, >> using >> >> some feedback from existing systems (which have their advantages and >> >> flaws). >> > >> > -1 but I don't like using thread locals for such things. >> > Save the magic for things that need it. :) >> >> Thread local stuff can be a pain; I often curse it. I think it can be >> okay with the right patterns. > > If we codify it, I think it will be used. A lot. > > I recently looked over a sort of a contact manager app > for somebody. There were metaclasses all over the > place. :) Personally I've come to believe that threadlocals should always be retrieved via a function call. Threadlocal proxy objects just cause too much confusion. But it's awfully nice to have access to them. And for things like configuration I find it almost essential, unless you do process-wide configuration, which I abhor far more than threadlocal variables. >> The interface for apps is basically: >> >> def app_factory(global_conf, **app_conf): >> return wsgi app >> >> It's pretty neutral, with some notable details: >> >> - global_conf is basically inherited configuration. It may or may not >> apply to this application. The application factory can ignore it or not >> as it sees fit. app_conf is definitely intended for this application; >> you can do more strict checking on it, make sure there's no extra values >> or that certain required values are present, etc. >> >> - all values can be strings, and strings should be handled specially. >> That is, if you expect an integer and you get a string, you should >> coerce it to an integer. If you expect a list and you get a string, you >> should probably split the string in some fashion, or just turn it into a >> one-item list. If you expect a boolean and you get a string, you should >> convert it intelligently (not based on the truth/falsiness of the string >> itself). >> >> >> That's it. Middleware and servers have similar interfaces. The server >> interface is a little under-powered (it just serves a WSGI application >> forever); it could be extended some, or left unspecified for now. >> Middleware takes as a first argument the application being wrapped. Oh, >> and composites, which are a little harder -- they take as a first >> argument an object that can load more WSGI apps. That's used for >> dispatchers that can direct to multiple subapplications. That's more >> tightly coupled with Paste Deploy's app naming conventions, and it might >> be better to put explicit app loading into the configuration format and >> pass them in as keyword arguments to the dispatching app. > > This is all a bit meta for me. I guess I don't get it. :) It gives you a consistent way to configure WSGI stacks, from a config file, database, or whatever. >> >> * A way to extend wsgiref.validate to add more validation, for >> all >> >> these new specs. (Probably this is an implementation, not a spec) >> > >> > That makes sense if our standards are that involved. >> > I don't see standardizable clarity on much that is so >> > complex. >> >> WSGI isn't all that complex, but validation is very helpful when people >> get it wrong. It should be useful for any of these specs as well. It >> also makes the spec much more explicitly clear, because computers check >> things more reliably than humans ;) > > I am not arguing against validation. I am just hoping > that it's superfluous in the near term. Testing is never superfluous ;) >> >> * Anchors for doing recursive calls, similar to paste.recursive. >> >> (paste.recursive is kind of an old module that is more complicated >> than >> >> it needs to be) >> > >> > Is that really such a common pattern? It is clever >> > but I have yet to find a case for it. Maybe I am just >> > overlooking something. What do you usually use >> > that for? >> >> Originally it was to support internal redirects and inclusion (which >> were part of the Webware API). Now I find it useful for doing internal >> subrequests when using web-based APIs. > > That makes more sense. I have some forwarding > stuff like that in YARO because it can be used > to hide the WSGI interface a little. Other than that > I just call the thing I want to forward to. (I don't > like to use exceptions for normal flow control very > much, though it does come up.) I'm not a big fan of forwarding and using exceptions to unwrap the middleware. But including content is much simpler. I think it can be as simple as: def middleware(app): def replacement(environ, start_response): anchors = environ.setdefault( 'x-wsgiorg.app_anchors', {}) anchors[environ['SCRIPT_NAME']] = (app, environ.copy()) return app(environ, start_response) return replacement This gives you the app and an indication of what the environ looks like when the app is typically reached. From here you can implement recursive calls fairly easily. Whether there should be support for multiple anchors, I'm not sure. I think it could be argued that the closest anchor is best to use, but the furthest one offers the most URI space (supposing there are multiple pieces of middleware like this in a stack). > Either way, finding what to forward to is the trick, > which is dispatch, and there are a whole lot of > ways to skin that cat. In this case there's no guessing, if you are selecting where to forward by URI. This technique respects all dispatching. >> >> * A place to put a database transaction manager >> > >> > -1 Way too specific. >> >> There'd have to be a database transaction manager standard first, anyway. > > I admire your ambition. :) I think it's reasonable, but something for db-sig and not web-sig. >> >> * More user information than just REMOTE_USER; like >> >> wsgiorg.user_info? >> > >> > User objects are like request objects and I don't see >> > what we gain from making them fight for a key. >> > >> > Could this actually decrease real interop? >> >> This shouldn't be a real user object, more likely a dictionary, just >> like the WSGI request is a dictionary (environ) and not an object. >> Objects are hard to standardize ;) > > They are. I would be interested in this though I > remain somewhat skeptical. Right now we're doing this in a project where we want to support embedding the app in Zope or having it be stand-alone. We want to keep it fairly loosely coupled from Zope, so we're trying not to pass in a user object. We haven't gone that far with it, but maybe later it'll feel more clear to me. This isn't low hanging fruit. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ianb at colorstudy.com Sat Nov 11 05:54:39 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 10 Nov 2006 22:54:39 -0600 Subject: [Web-SIG] (proto) request object spec In-Reply-To: References: <4554F450.3070901@colorstudy.com> Message-ID: <4555578F.9090501@colorstudy.com> Luke Arno wrote: > If I understand you: if I tack some environ values on > to my request object and the user changes those > value on the request object, this must automatically > be reflected in the environ. Is that right? > > If so, I disagree. If the user wants to change the value > of something in the environ, she can do so. If not that > is fine. I would rather say "no auto-magical updates!" I'm not really clear what you're thinking. I guess I mean something like this is bad: class Request(object): def __init__(self, environ): self.environ = environ self.params = parse_params(environ) Now the request has a parsed form of the parameters (e.g., query string), and if you recreate it then you'll have problems getting the parameters again. This particular case is the motivation for the form_vars spec. >> In addition to this very minimal set of requirements for a request >> object, the specification could have a kind of style guide for request >> objects. No request object would have to conform, and it would not be >> intended to ensure interoperability. The style guide would just be to >> increase familiarity of developers when they encounter a new request >> object. > > The whole point of a request object is to provide a more > friendly and to-taste interface to cover the universal but > decidedly (and necessarily) clunky WSGI interface. If > we specify a style guide, many will feel compelled to > conform to it. This seems counterproductive when the > very purpose is to meet diverse preferences. Well, if it's a style guide then no one is forcing you to follow it. But if you don't care one way or the other, you can conform to something just because. I would assume the style guide would address the really consistent/boring stuff, like form parameters and a few typical keys like method and path_info. If you want, you can provide multiple ways to get to the same thing, or one way that doesn't fit the style guide; which will be typical when backward compatibility is a concern. > Though the similarity of request objects out there (note > that I called mine "Yet Another Request Object") gives > the appearance of a prime candidate for standardization, > the subtle (or not) differences are very important to > people in this area. It is sort of like the chair. It is a lot > like the chair, actually: you are in it all day and it needs > to be adjustable and fit just right. > > If this were a good candidate for standardization, I > think WSGI would look a whole lot different. I thought this way as well, but when the scope of the request object is strictly confined to the WSGI environment I think it makes it a lot easier. Not everyone will want to use that; for variety of small reasons I think this is only going to be appealing to WSGIish frameworks. But it's opt-in, so whatever. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From luke.arno at gmail.com Sun Nov 12 03:05:12 2006 From: luke.arno at gmail.com (Luke Arno) Date: Sat, 11 Nov 2006 21:05:12 -0500 Subject: [Web-SIG] Possible specs In-Reply-To: <455542C3.2010705@colorstudy.com> References: <4554C7B7.3070909@colorstudy.com> <4554F2E9.2050107@colorstudy.com> <455542C3.2010705@colorstudy.com> Message-ID: Sorry, I've been out planting scuppernongs all day. On 11/10/06, Ian Bicking wrote: > Luke Arno wrote: > >> >> * Debugging mode is something that can be used in all sorts of > >> >> places; to increase verbosity, annotate output pages, displaying > >> errors > >> >> in the browser, etc. Having a single key for turning on debugging mode > >> >> would allow its consumption in lots of places. Not as strict as > >> >> authenticating. > >> > > >> > Maybe. > >> > > >> > wsgiorg.log_level? If you change the log level, > >> > do you want this to effect the whole stack or > >> > just your own stuff? > >> > >> I often do a bunch of stuff in debugging mode that is more aggressive > >> than just log_level, and shouldn't be done at all in normal mode. So a > >> real on-off switch for debugging is useful. > > > > I expect your development (like mine) is a little atypical. > > I am often working on various parts of the stack at the > > same time. Most web developers are just working on > > an app and would not like to see debugging info start > > pouring out of everything under it. That is probably not > > what you are suggesting. Still, I think that this is a low > > value target for community standardization (though for > > an individual who is all over the stack, it is a great > > personal convention.) > > I find it fairly widely useful. Right now I do > environ['paste.config'].get('debug') to figure out if I'm in debug mode, > and I find myself doing this at all sorts of levels, including directly > in application code. > > Though one potential problem is if everyone starts listening for the > same key, then the debugging information could become overwhelming. In > the context of the Paste Deploy entry point, I might generally do it like: > > class NoDefault: pass # marker > > def make_some_middleware(global_conf, debug=NoDefault, ...): > if debug is NoDefault: > debug = global_conf.get('debug') > debug = paste.deploy.converters.asbool(debug) > ... > > And maybe this is good enough. I don't think someone just working on an app wants that overhead. I bet most people would just want to set log levels in a config file. The scoping of those log levels _is_ case specific so perhaps we should just say that reusable components should take runtime configuration with respect to logging. Framework and application authors can develop their own schemes. mw = MyMiddleWare(app, log_level=WHATEVER) Anything beyond that requires a lot of assumptions about how other people develop. > >> >> * Some systems prefer that unexpected exceptions bubble up, like > >> >> test frameworks. A key could define this case (modelled on > >> >> paste.throw_errors) and thus disable exception catchers. > >> > > >> > -1 I don't need it. Too abstract. > >> > > >> > Usually, you just have one error handling > >> > middleware on the outside of the rest, no? > >> > >> Sometimes there's multiple pieces of middleware. For instance, I might > >> have one on the outside, with another one in a subapplication (which is > >> independently deployable, so needs some kind of wrapper). Also it is > >> needed to avoid catching errors when running tests, where you want the > >> exception to go up all the way to your test runner (which might have a > >> debugger or special traceback formatter). > > > > This is probably just not relevant to my development > > style. Or maybe I just don't get it. > > Well, the general pattern is: > > def error_catcher(app): > def replacement_app(environ, start_response): > if environ.get('paste.throw_errors'): > return app(environ, start_response) > try: > return app(environ, start_response) > except: > # handle and report on error > return replacement_app > > In a testing environment I always want errors to bubble up, I don't want > a 500 response if it can be helped; so in paste.fixture.TestApp I always > set this. > > It's not a big deal, but it's fairly simple to explain and handle. Low > hanging fruit. I just put such things in a usually-outermost wrapping context and test without it. I don't catch unexpected exceptions at anyplace else in the stack so I guess that is why this doesn't make sense to me. innermost_app = App() test_innermost(innermost_app) # expect exception inner_app = SomeMiddleWare(innermost_app) test_inner(inner_app) # expect different exception outer_app = ErrorMiddleWare(inner_app) test_outer(outer_app) # expect 500 Low hanging, perhaps, but not tasty to me :) I think we are just doing things a little differently here. If I am the only deviant then maybe your way should be a standard? > >> >> * Logging is a tricky situation. The logging module allows for > >> >> statically setting up logging systems, then configuring them at > >> startup. > >> >> This often isn't the best way to set up logging. Putting a > >> >> logging.Logger instance right in the environment might be better. This > >> >> requires some design and usage before setting on one spec. > >> > > >> > Maybe a lazy logger loader that takes the level > >> > as an argument? Seems a little silly. > >> > >> I'm not really sure here, myself. I know people ask for it, and > >> sometimes I'm pretty sure I want something like this, but I don't feel > >> very solid about what best practice in logging is. I couldn't write > >> this one. > > > > I am with you. I create loggers for their respective > > scopes and call them as needed. Log levels go in > > a config and that is the whole story. What else? > > > > I think people sometimes ask for structure because > > they are unsure when all they need is to go for it. > > Well, often the container knows more about how logging should work than > the thing itself knows. Probably most of the time. Passing around a > logger helps with this. Then I favor runtime configuration of the middleware or application (or library or whatever) mw = MiddleWare(app, logger=my_logger) Framework and application authors can decide what bits should share what loggers under what conditions. > >> >> * Thread-local values are a common technique in web frameworks, > >> >> allowing global objects or functions to return request-specific > >> >> information. This pattern could be codified into one core system, > >> using > >> >> some feedback from existing systems (which have their advantages and > >> >> flaws). > >> > > >> > -1 but I don't like using thread locals for such things. > >> > Save the magic for things that need it. :) > >> > >> Thread local stuff can be a pain; I often curse it. I think it can be > >> okay with the right patterns. > > > > If we codify it, I think it will be used. A lot. > > > > I recently looked over a sort of a contact manager app > > for somebody. There were metaclasses all over the > > place. :) > > Personally I've come to believe that threadlocals should always be > retrieved via a function call. Threadlocal proxy objects just cause too > much confusion. But it's awfully nice to have access to them. And for > things like configuration I find it almost essential, unless you do > process-wide configuration, which I abhor far more than threadlocal > variables. I don't find myself needing this but I believe you. Is a wsgi.org spec the right place to standardize this broader scoped pattern? > >> The interface for apps is basically: > >> > >> def app_factory(global_conf, **app_conf): > >> return wsgi app > >> > >> It's pretty neutral, with some notable details: > >> > >> - global_conf is basically inherited configuration. It may or may not > >> apply to this application. The application factory can ignore it or not > >> as it sees fit. app_conf is definitely intended for this application; > >> you can do more strict checking on it, make sure there's no extra values > >> or that certain required values are present, etc. > >> > >> - all values can be strings, and strings should be handled specially. > >> That is, if you expect an integer and you get a string, you should > >> coerce it to an integer. If you expect a list and you get a string, you > >> should probably split the string in some fashion, or just turn it into a > >> one-item list. If you expect a boolean and you get a string, you should > >> convert it intelligently (not based on the truth/falsiness of the string > >> itself). > >> > >> > >> That's it. Middleware and servers have similar interfaces. The server > >> interface is a little under-powered (it just serves a WSGI application > >> forever); it could be extended some, or left unspecified for now. > >> Middleware takes as a first argument the application being wrapped. Oh, > >> and composites, which are a little harder -- they take as a first > >> argument an object that can load more WSGI apps. That's used for > >> dispatchers that can direct to multiple subapplications. That's more > >> tightly coupled with Paste Deploy's app naming conventions, and it might > >> be better to put explicit app loading into the configuration format and > >> pass them in as keyword arguments to the dispatching app. > > > > This is all a bit meta for me. I guess I don't get it. :) > > It gives you a consistent way to configure WSGI stacks, from a config > file, database, or whatever. That much I do get. It seems clever but I have never needed to do it. Outside of Paste and Paste users I always see folks just building stacks in good old Python. > >> >> * A way to extend wsgiref.validate to add more validation, for > >> all > >> >> these new specs. (Probably this is an implementation, not a spec) > >> > > >> > That makes sense if our standards are that involved. > >> > I don't see standardizable clarity on much that is so > >> > complex. > >> > >> WSGI isn't all that complex, but validation is very helpful when people > >> get it wrong. It should be useful for any of these specs as well. It > >> also makes the spec much more explicitly clear, because computers check > >> things more reliably than humans ;) > > > > I am not arguing against validation. I am just hoping > > that it's superfluous in the near term. > > Testing is never superfluous ;) Anything can be superfluous in a pure utility realm, such as software. Those of us who fulfill ourselves in the act of creating software can objectify that satisfaction into its vehicles of utility and come to mistake them as intrinsically valuable. Pardon the philosophy. ;) > >> >> * Anchors for doing recursive calls, similar to paste.recursive. > >> >> (paste.recursive is kind of an old module that is more complicated > >> than > >> >> it needs to be) > >> > > >> > Is that really such a common pattern? It is clever > >> > but I have yet to find a case for it. Maybe I am just > >> > overlooking something. What do you usually use > >> > that for? > >> > >> Originally it was to support internal redirects and inclusion (which > >> were part of the Webware API). Now I find it useful for doing internal > >> subrequests when using web-based APIs. > > > > That makes more sense. I have some forwarding > > stuff like that in YARO because it can be used > > to hide the WSGI interface a little. Other than that > > I just call the thing I want to forward to. (I don't > > like to use exceptions for normal flow control very > > much, though it does come up.) > > I'm not a big fan of forwarding and using exceptions to unwrap the > middleware. But including content is much simpler. I think it can be > as simple as: > > def middleware(app): > def replacement(environ, start_response): > anchors = environ.setdefault( > 'x-wsgiorg.app_anchors', {}) > anchors[environ['SCRIPT_NAME']] = (app, environ.copy()) > return app(environ, start_response) > return replacement > > This gives you the app and an indication of what the environ looks like > when the app is typically reached. From here you can implement > recursive calls fairly easily. > > Whether there should be support for multiple anchors, I'm not sure. I > think it could be argued that the closest anchor is best to use, but the > furthest one offers the most URI space (supposing there are multiple > pieces of middleware like this in a stack). Fancy. Confusing. Only works under the right conditions, no? I think this is cool experimentation but I just don't think it is a good candidate for standardization. > > Either way, finding what to forward to is the trick, > > which is dispatch, and there are a whole lot of > > ways to skin that cat. > > In this case there's no guessing, if you are selecting where to forward > by URI. This technique respects all dispatching. > > > >> >> * A place to put a database transaction manager > >> > > >> > -1 Way too specific. > >> > >> There'd have to be a database transaction manager standard first, anyway. > > > > I admire your ambition. :) > > I think it's reasonable, but something for db-sig and not web-sig. > > > >> >> * More user information than just REMOTE_USER; like > >> >> wsgiorg.user_info? > >> > > >> > User objects are like request objects and I don't see > >> > what we gain from making them fight for a key. > >> > > >> > Could this actually decrease real interop? > >> > >> This shouldn't be a real user object, more likely a dictionary, just > >> like the WSGI request is a dictionary (environ) and not an object. > >> Objects are hard to standardize ;) > > > > They are. I would be interested in this though I > > remain somewhat skeptical. > > Right now we're doing this in a project where we want to support > embedding the app in Zope or having it be stand-alone. We want to keep > it fairly loosely coupled from Zope, so we're trying not to pass in a > user object. > > We haven't gone that far with it, but maybe later it'll feel more clear > to me. This isn't low hanging fruit. Actually, this and the sessions, boring as they may be, are the best candidates to grow convention on. (Maybe standards are best when boring; well understood problems are nice and boring.) Should this be tackled by choosing a set of new keys for environ (to go beyond REMOTE_USER with wsgiorg.user_roles and such) or should there be one new key with a dict in it (like wsgiorg.user_attribs)? There are many existing specs for user attributes that we could examine. The first step should probably be for someone to document those. Given the proliferation of such standards, I am still skeptical... ... but I am always skeptical ;) Cheers, - Luke From luke.arno at gmail.com Sun Nov 12 03:48:40 2006 From: luke.arno at gmail.com (Luke Arno) Date: Sat, 11 Nov 2006 21:48:40 -0500 Subject: [Web-SIG] (proto) request object spec In-Reply-To: <4555578F.9090501@colorstudy.com> References: <4554F450.3070901@colorstudy.com> <4555578F.9090501@colorstudy.com> Message-ID: On 11/10/06, Ian Bicking wrote: > Luke Arno wrote: > > If I understand you: if I tack some environ values on > > to my request object and the user changes those > > value on the request object, this must automatically > > be reflected in the environ. Is that right? > > > > If so, I disagree. If the user wants to change the value > > of something in the environ, she can do so. If not that > > is fine. I would rather say "no auto-magical updates!" > > I'm not really clear what you're thinking. I guess I mean something > like this is bad: > > class Request(object): > def __init__(self, environ): > self.environ = environ > self.params = parse_params(environ) > > Now the request has a parsed form of the parameters (e.g., query > string), and if you recreate it then you'll have problems getting the > parameters again. > > This particular case is the motivation for the form_vars spec. Usually, it is safer to have request object creation non-destructive to the environment, if that is what you are getting at. This is the current behavior of YARO. I have "make destructive parsing of form POST data an option" on my to-do list, though. It currently uses StringIO to replace wsgi.input and one might not always want that to happen (big upload). > >> In addition to this very minimal set of requirements for a request > >> object, the specification could have a kind of style guide for request > >> objects. No request object would have to conform, and it would not be > >> intended to ensure interoperability. The style guide would just be to > >> increase familiarity of developers when they encounter a new request > >> object. > > > > The whole point of a request object is to provide a more > > friendly and to-taste interface to cover the universal but > > decidedly (and necessarily) clunky WSGI interface. If > > we specify a style guide, many will feel compelled to > > conform to it. This seems counterproductive when the > > very purpose is to meet diverse preferences. > > Well, if it's a style guide then no one is forcing you to follow it. > But if you don't care one way or the other, you can conform to something > just because. I don't see the point. Sorry. > I would assume the style guide would address the really > consistent/boring stuff, like form parameters and a few typical keys > like method and path_info. Even if we all do agree on the names of a couple of things (and we probably can), what good does it do? Has learning the request object interface been a chronic problem for users? If so, is this not more likely a reflection of shortcomings in documentation rather than the need for a standard? > If you want, you can provide multiple ways to get to the same thing, or > one way that doesn't fit the style guide; which will be typical when > backward compatibility is a concern. One _can_ do this and sometimes must but it is ugly. > > Though the similarity of request objects out there (note > > that I called mine "Yet Another Request Object") gives > > the appearance of a prime candidate for standardization, > > the subtle (or not) differences are very important to > > people in this area. It is sort of like the chair. It is a lot > > like the chair, actually: you are in it all day and it needs > > to be adjustable and fit just right. > > > > If this were a good candidate for standardization, I > > think WSGI would look a whole lot different. > > I thought this way as well, but when the scope of the request object is > strictly confined to the WSGI environment I think it makes it a lot > easier. Not everyone will want to use that; for variety of small > reasons I think this is only going to be appealing to WSGIish > frameworks. But it's opt-in, so whatever. That is my point. There are many smallish reasons to prefer one request convenience interface to another. Why try to sanctify some particular preferences? We don't need it for interop because WSGI already gives us that. Cheers, - Luke From joe at bitworking.org Mon Nov 13 03:28:01 2006 From: joe at bitworking.org (Joe Gregorio) Date: Sun, 12 Nov 2006 21:28:01 -0500 Subject: [Web-SIG] wsgiref bug with HEAD request In-Reply-To: <45368B81.8090308@defuze.org> References: <45368B81.8090308@defuze.org> Message-ID: <3f1451f50611121828r201de504od8df368d3ef831db@mail.gmail.com> According to RFC 2616 the HEAD response MUST NOT return a message-body in the response: http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.4 -joe On 10/18/06, Sylvain Hellegouarch wrote: > All, > > It seems the default server from wsgiref (from wsgiref.simple_server > import make_server) seems not to respect Content-Length in case of HEAD > request. > > Since no body can be returned in a response to a HEAD request, the > content length is set to 0 by the server. In that case Content-Length is > therefore set by the application or a middleware. > > wsgiref server disregard the existing value and sets to 0 either way. > > Seems bogus to me or am I missing something here? > > - Sylvain > _______________________________________________ > Web-SIG mailing list > Web-SIG at python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: http://mail.python.org/mailman/options/web-sig/joe.gregorio%40gmail.com > -- Joe Gregorio http://bitworking.org From christian at dowski.com Mon Nov 13 03:51:43 2006 From: christian at dowski.com (Christian Wyglendowski) Date: Sun, 12 Nov 2006 21:51:43 -0500 Subject: [Web-SIG] wsgiref bug with HEAD request In-Reply-To: <3f1451f50611121828r201de504od8df368d3ef831db@mail.gmail.com> References: <45368B81.8090308@defuze.org> <3f1451f50611121828r201de504od8df368d3ef831db@mail.gmail.com> Message-ID: On 11/12/06, Joe Gregorio wrote: > > According to RFC 2616 the HEAD response MUST NOT > return a message-body in the response: > > http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.4 Right. I *think* the problem that Sylvain is bringing to light is that a different headers are being returned for HEAD and GET requests. From the section of RFC 2616 that you referenced above: "The metainformation contained in the HTTP headers in response to a HEAD request SHOULD be identical to the information sent in response to a GET request." So if an app/middleware sets what it deems to be the correct Content-Length for a resource in response to a HEAD request, it gets overridden by the server, which sets it to zero. Christian http://www.dowski.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/web-sig/attachments/20061112/3eae5067/attachment.htm From fumanchu at amor.org Mon Nov 13 04:45:21 2006 From: fumanchu at amor.org (Robert Brewer) Date: Sun, 12 Nov 2006 19:45:21 -0800 Subject: [Web-SIG] Possible specs References: <4554C7B7.3070909@colorstudy.com> Message-ID: <435DF58A933BA74397B42CDEB8145A86224C9E@ex9.hostedexchange.local> Ian Bicking wrote: > I brainstormed some ideas for wsgiorg specs and added them to the spec > page, and also copied here. I offer them here to see if there's > particular specifications that seem interesting, and might be worth > pursuing sooner than other ones. > ... and Luke Arno replied: > Could this actually decrease real interop? > I am worried that things are getting a little inventive > here. This is how things get heavy. Lets try to stick > to codifying those things that have a clear common > right way. Most of these don't meet that, in my > opinion. Pursuing many of these will lead to endless > arguments as they are just unproven which means > we should just let the diversity simmer for a while. > > Sessions *might* be doable. I absolutely agree. Web development (at least in Python) is moving *way* too fast for this to be a good time to introduce lots of specifications. Sessions have 1) been around forever, 2) have consistently been done similarly for each implementation, and 3) already require interoperation with third party code (database APIs), so introducing a common interface would not introduce any overhead. Therefore, session handling would be the only candidate I see for standardization. All the other proposals seem like ways to work around the weaknesses of WSGI by promoting lowest-common denominator agreements, and, since most consumers of WSGI have by now already worked around those weaknesses in ways that natively fit their code, any specification at this point would only introduce interface overhead into those codebases. The last thing I, as a framework dev, want to provide/require of my users is yet another point of composition. Let's standardize those points which already _require_ composition (like sessions), and worry about those points which merely _allow_ composition at a later date. Robert Brewer System Architect Amor Ministries fumanchu at amor.org -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/web-sig/attachments/20061112/8d717921/attachment.html From pje at telecommunity.com Mon Nov 13 05:03:08 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 12 Nov 2006 23:03:08 -0500 Subject: [Web-SIG] Possible specs In-Reply-To: <435DF58A933BA74397B42CDEB8145A86224C9E@ex9.hostedexchange. local> References: <4554C7B7.3070909@colorstudy.com> Message-ID: <5.1.1.6.0.20061112225422.01f3fa68@sparrow.telecommunity.com> At 07:45 PM 11/12/2006 -0800, Robert Brewer wrote: >Sessions have 1) been around forever, 2) have consistently been done >similarly for each implementation, and 3) already require interoperation >with third party code (database APIs), so introducing a common interface >would not introduce any overhead. Therefore, session handling would be the >only candidate I see for standardization. And even sessions are full of potential pitfalls for standardization, due to differences in things like authenticated vs. non-authenticated sessions, retention/expiration policies, and differing limitations on what can be stored. Of course, there are also those persons like myself, who believe that the very concept of sessions as something apart from an application's domain model is the devil's work, doing harm to both the users and the enterprise. ;-) (E.g. ask yourself if Amazon.com uses anything that would be traditionally called a "session" database - their sessions are permanent, identified, and even an anonymous person's shopping cart items are retained for *90 days*. And you can bet dollars to donuts that they can join "session" data to non-session data in their database queries.) So, from my POV, having session bags where you can just dump whatever old data you please, without regard to the application's DB design (not to mention backup/uptime guarantees of the DB service), are things of the vilest evil that should be cast back into the bowels of Microsoft from whence they came, or something like that. :) (I first encountered sessions in MS's early ASP stuff, but perhaps they got the idea from somewhere else.) Anyway, that's basically off-topic except to point out that even if you *are* a Satanis... er, sessionist, ;-), then you can still have wide-ranging disagreements over data retention policy, locking, etc., etc. From joe at bitworking.org Mon Nov 13 05:27:45 2006 From: joe at bitworking.org (Joe Gregorio) Date: Sun, 12 Nov 2006 23:27:45 -0500 Subject: [Web-SIG] wsgiref bug with HEAD request In-Reply-To: References: <45368B81.8090308@defuze.org> <3f1451f50611121828r201de504od8df368d3ef831db@mail.gmail.com> Message-ID: <3f1451f50611122027y1a1750c2p1d195653bdb1dff0@mail.gmail.com> On 11/12/06, Christian Wyglendowski wrote: > On 11/12/06, Joe Gregorio wrote: > > According to RFC 2616 the HEAD response MUST NOT > > return a message-body in the response: > > > > > http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.4 > > Right. I *think* the problem that Sylvain is bringing to light is that a > different headers are being returned for HEAD and GET requests. From the > section of RFC 2616 that you referenced above: > > "The metainformation contained in the HTTP headers in response to a HEAD > request SHOULD be identical to the information sent in response to a GET > request." Ah, I misunderstood the problem. You are correct: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.13 """ The Content-Length entity-header field indicates the size of the entity-body, in decimal number of OCTETs, sent to the recipient or, in the case of the HEAD method, the size of the entity-body that would have been sent had the request been a GET.""" -joe -- Joe Gregorio http://bitworking.org From joe at bitworking.org Mon Nov 13 20:32:31 2006 From: joe at bitworking.org (Joe Gregorio) Date: Mon, 13 Nov 2006 14:32:31 -0500 Subject: [Web-SIG] routing_args (was url_vars) In-Reply-To: References: Message-ID: <3f1451f50611131132p65259203vf4261639a224af1f@mail.gmail.com> I am positive on the spec, but I do have one question: What is the justification for altering the values of SCRIPT_NAME and PATH_INFO? At the very least this breaks conformance with RFC 3875[1], which PEP 333 references normatively. http://www.ietf.org/rfc/rfc3875 Thanks, -joe On 11/6/06, Luke Arno wrote: > So, it seemed to me like we had agreement on > this. Does anyone disagree? > > environ['routing_args'] == (list_or_tuple, a_dict) > > Should the spec get updated and marked as > accepted? > > Cheers, > - Luke > _______________________________________________ > Web-SIG mailing list > Web-SIG at python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: http://mail.python.org/mailman/options/web-sig/joe.gregorio%40gmail.com > -- Joe Gregorio http://bitworking.org From ianb at colorstudy.com Mon Nov 13 20:45:13 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 13 Nov 2006 13:45:13 -0600 Subject: [Web-SIG] routing_args (was url_vars) In-Reply-To: <3f1451f50611131132p65259203vf4261639a224af1f@mail.gmail.com> References: <3f1451f50611131132p65259203vf4261639a224af1f@mail.gmail.com> Message-ID: <4558CB49.2070502@colorstudy.com> Joe Gregorio wrote: > I am positive on the spec, but I do have one question: > > What is the justification for altering the values > of SCRIPT_NAME and PATH_INFO? > > At the very least this breaks conformance > with RFC 3875[1], which PEP 333 references > normatively. > > http://www.ietf.org/rfc/rfc3875 Why is this a problem? I put in the note about SCRIPT_NAME/PATH_INFO, because generally SCRIPT_NAME represents the consumed path. Dispatchers typically consume the path when they do their dispatching. If you leave SCRIPT_NAME, you are presuming that the application has knowledge of how the dispatcher works. The whole point of the spec is that consumers don't need to know how the variables got there. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From joe at bitworking.org Mon Nov 13 20:55:17 2006 From: joe at bitworking.org (Joe Gregorio) Date: Mon, 13 Nov 2006 14:55:17 -0500 Subject: [Web-SIG] routing_args (was url_vars) In-Reply-To: <4558CB49.2070502@colorstudy.com> References: <3f1451f50611131132p65259203vf4261639a224af1f@mail.gmail.com> <4558CB49.2070502@colorstudy.com> Message-ID: <3f1451f50611131155xb2c5395o8c168a82cab8fd8b@mail.gmail.com> On 11/13/06, Ian Bicking wrote: > Why is this a problem? > > I put in the note about SCRIPT_NAME/PATH_INFO, because generally > SCRIPT_NAME represents the consumed path. Dispatchers typically consume > the path when they do their dispatching. > > If you leave SCRIPT_NAME, you are presuming that the application has > knowledge of how the dispatcher works. The whole point of the spec is > that consumers don't need to know how the variables got there. So this lets me, for example, dispatch Selectors to other Selectors without them knowing about each other? That works for me. Thanks, -joe -- Joe Gregorio http://bitworking.org From ianb at colorstudy.com Mon Nov 13 21:31:16 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 13 Nov 2006 14:31:16 -0600 Subject: [Web-SIG] Possible specs In-Reply-To: <435DF58A933BA74397B42CDEB8145A86224C9E@ex9.hostedexchange.local> References: <4554C7B7.3070909@colorstudy.com> <435DF58A933BA74397B42CDEB8145A86224C9E@ex9.hostedexchange.local> Message-ID: <4558D614.8010700@colorstudy.com> Robert Brewer wrote: > Ian Bicking wrote: > > I brainstormed some ideas for wsgiorg specs and added them to the spec > > page, and also copied here. I offer them here to see if there's > > particular specifications that seem interesting, and might be worth > > pursuing sooner than other ones. > > ... > > and Luke Arno replied: > > > Could this actually decrease real interop? > > I am worried that things are getting a little inventive > > here. This is how things get heavy. Lets try to stick > > to codifying those things that have a clear common > > right way. Most of these don't meet that, in my > > opinion. Pursuing many of these will lead to endless > > arguments as they are just unproven which means > > we should just let the diversity simmer for a while. > > > > Sessions *might* be doable. > > I absolutely agree. Web development (at least in Python) is moving *way* > too fast for this to be a good time to introduce lots of specifications. Even if they are really really small? > Sessions have 1) been around forever, 2) have consistently been done > similarly for each implementation, and 3) already require interoperation > with third party code (database APIs), so introducing a common interface > would not introduce any overhead. Therefore, session handling would be > the only candidate I see for standardization. I don't have any problem with standardizing this, but I don't think it's particularly useful to share sessions across applications. It's useful to share configuration and backend setup, and probably good to share the session tracking policy. The actual contents of sessions are largely private to the application. But that can also be handled as an aspect of deployment, so whatever. Here's Ben's session ID proposal: http://mail.python.org/pipermail/web-sig/2006-January/001858.html Here's an interface I proposed last year for sessions: http://svn.colorstudy.com/home/ianb/proposed_session_interface.py It's not that simple of an interface :( > All the other proposals seem like ways to work around the weaknesses of > WSGI by promoting lowest-common denominator agreements, and, since most > consumers of WSGI have by now already worked around those weaknesses in > ways that natively fit their code, any specification at this point would > only introduce interface overhead into those codebases. The last thing > I, as a framework dev, want to provide/require of my users is yet > another point of composition. Let's standardize those points which > already _require_ composition (like sessions), and worry about those > points which merely _allow_ composition at a later date. While I don't have a problem with a session standard, I don't think sessions require or even benefit from composition. Authentication definitely does, but we have some light agreements on that. We could probably build a very conservative specification for authentication that makes explicit some things that are inherited from CGI. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ianb at colorstudy.com Mon Nov 13 21:51:07 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 13 Nov 2006 14:51:07 -0600 Subject: [Web-SIG] Possible specs In-Reply-To: References: <4554C7B7.3070909@colorstudy.com> <4554F2E9.2050107@colorstudy.com> <455542C3.2010705@colorstudy.com> Message-ID: <4558DABB.8080204@colorstudy.com> Luke Arno wrote: >> >> >> * Some systems prefer that unexpected exceptions bubble up, >> like >> >> >> test frameworks. A key could define this case (modelled on >> >> >> paste.throw_errors) and thus disable exception catchers. >> >> > >> >> > -1 I don't need it. Too abstract. >> >> > >> >> > Usually, you just have one error handling >> >> > middleware on the outside of the rest, no? >> >> >> >> Sometimes there's multiple pieces of middleware. For instance, I >> might >> >> have one on the outside, with another one in a subapplication >> (which is >> >> independently deployable, so needs some kind of wrapper). Also it is >> >> needed to avoid catching errors when running tests, where you want the >> >> exception to go up all the way to your test runner (which might have a >> >> debugger or special traceback formatter). >> > >> > This is probably just not relevant to my development >> > style. Or maybe I just don't get it. >> >> Well, the general pattern is: >> >> def error_catcher(app): >> def replacement_app(environ, start_response): >> if environ.get('paste.throw_errors'): >> return app(environ, start_response) >> try: >> return app(environ, start_response) >> except: >> # handle and report on error >> return replacement_app >> >> In a testing environment I always want errors to bubble up, I don't want >> a 500 response if it can be helped; so in paste.fixture.TestApp I always >> set this. >> >> It's not a big deal, but it's fairly simple to explain and handle. Low >> hanging fruit. > > I just put such things in a usually-outermost wrapping > context and test without it. I don't catch unexpected > exceptions at anyplace else in the stack so I guess > that is why this doesn't make sense to me. Sometimes it is hard to pull apart stacks, and sometimes you want to embed a stack that already includes an exception handler. For instance, Zope includes an option exactly like this. I think Zope 3 actually uses something like 'wsgi.throwErrors' for this same idea, which they came up with independently. Anyway, it's pretty obvious how it works and it's really easy to support. One of the other places where it is useful is to wrap an application with a different error catcher (like paste.evalexception) and not have to worry about unpacking the enclosed application to disable its exception handler. I've done this several times when debugging application deployment issues. [re: paste deploy] >> It gives you a consistent way to configure WSGI stacks, from a config >> file, database, or whatever. > > That much I do get. It seems clever but I have never > needed to do it. Outside of Paste and Paste users I always > see folks just building stacks in good old Python. How do non-developers ultimately deploy the application? How do they construct stacks of software to make a web site? I suspect (barring this sort of interface) that it's done in an ad hoc manner, which will seem fine so long as applications are one-off and/or deployed by the programmer. Note that I don't encourage people to use configuration to put together an application's stack; you should do that in Python code, and you may or may not use this interface to do so. But for putting together a single site, or for deploying an application, I don't think relying on Python code is a good idea. (Note that the interface doesn't *keep* you from using Python code, it just makes it possible to build other tools.) >> I'm not a big fan of forwarding and using exceptions to unwrap the >> middleware. But including content is much simpler. I think it can be >> as simple as: >> >> def middleware(app): >> def replacement(environ, start_response): >> anchors = environ.setdefault( >> 'x-wsgiorg.app_anchors', {}) >> anchors[environ['SCRIPT_NAME']] = (app, environ.copy()) >> return app(environ, start_response) >> return replacement >> >> This gives you the app and an indication of what the environ looks like >> when the app is typically reached. From here you can implement >> recursive calls fairly easily. >> >> Whether there should be support for multiple anchors, I'm not sure. I >> think it could be argued that the closest anchor is best to use, but the >> furthest one offers the most URI space (supposing there are multiple >> pieces of middleware like this in a stack). > > Fancy. Confusing. Only works under the right > conditions, no? I think this is cool experimentation > but I just don't think it is a good candidate for > standardization. Well, tools that could do recursive calls have to look for *something*. And something has to be put into the stack to make this possible. I'd like to write tools that do this that don't require a Pasty stack; without any particular standard there's no way to do that. In terms of its applicability and functionality, it works fine. It's a kind of functionality that is fairly common in other stacks, including Apache itself. Of course consuming this key is something a library routine should do, it's not easy to consume by hand. What I'm proposing is the minimum thing necessary to make the library routine feasible. Note if you do it statically (getting a hook to the original app without the environ, and without middleware) you will have to give information like hosts and paths that may or may not match up with the actual requests that come in. As a result I think it's much better to do this as middleware. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From luke.arno at gmail.com Mon Nov 13 21:57:30 2006 From: luke.arno at gmail.com (Luke Arno) Date: Mon, 13 Nov 2006 15:57:30 -0500 Subject: [Web-SIG] routing_args (was url_vars) In-Reply-To: <3f1451f50611131155xb2c5395o8c168a82cab8fd8b@mail.gmail.com> References: <3f1451f50611131132p65259203vf4261639a224af1f@mail.gmail.com> <4558CB49.2070502@colorstudy.com> <3f1451f50611131155xb2c5395o8c168a82cab8fd8b@mail.gmail.com> Message-ID: On 11/13/06, Joe Gregorio wrote: > On 11/13/06, Ian Bicking wrote: > > Why is this a problem? > > > > I put in the note about SCRIPT_NAME/PATH_INFO, because generally > > SCRIPT_NAME represents the consumed path. Dispatchers typically consume > > the path when they do their dispatching. > > > > If you leave SCRIPT_NAME, you are presuming that the application has > > knowledge of how the dispatcher works. The whole point of the spec is > > that consumers don't need to know how the variables got there. > > So this lets me, for example, dispatch Selectors to other > Selectors without them knowing about each other? That works for me. I didn't pipe up about this because, in general, you should consume unless you have a good reason not to. I will add explicit mention of all this to the Selector web page. I left this out of Selector because, given the typical usage, one would almost always end up consuming the whole remainder of the path and then I would just end up creating an environ key like 'selector.original_path' or some such. It does not seem readily generalizable to me in that form so I decided that if I were to chain selectors, I would just wrap them in small path-consuming middlewares as needed. Now that I think about this again, I will add a utility to Selector for creating path-consuming MWs. That way you can do the following to have '/foo' consumed before s2 gets called: s1 = Selector(prefix='/foo') s2 = Selector(prefix='/bar') con = consuming(s2, '/foo') s1.add('/bar', _ANY_=con) This is a lame patten unless you have a good reason for it, of course. I have not had occasion to do this in real life, as of yet. Cheers, - Luke From joe at bitworking.org Mon Nov 13 23:02:19 2006 From: joe at bitworking.org (Joe Gregorio) Date: Mon, 13 Nov 2006 17:02:19 -0500 Subject: [Web-SIG] Possible specs In-Reply-To: References: <4554C7B7.3070909@colorstudy.com> Message-ID: <3f1451f50611131402y46b1d6a0nc6b01fe0cbea569f@mail.gmail.com> On 11/10/06, Luke Arno wrote: > I am worried that things are getting a little inventive > here. This is how things get heavy. Lets try to stick > to codifying those things that have a clear common > right way. +1 Wsgiorg.routing_args came about from several independent WSGI middleware implemetations trying to interoperate, which is the right time to try to settle on a 'standard' way of doing things. -joe -- Joe Gregorio http://bitworking.org From lcrees at gmail.com Tue Nov 14 02:36:14 2006 From: lcrees at gmail.com (L. C. Rees) Date: Mon, 13 Nov 2006 18:36:14 -0700 Subject: [Web-SIG] (proto) request object spec Message-ID: <45591D8E.4020304@gmail.com> > That is my point. There are many smallish reasons to > prefer one request convenience interface to another. Why > try to sanctify some particular preferences? We don't > need it for interop because WSGI already gives us that. A not infrequent WSGI pattern is putting all of the various WSGI components in a request dictionary or even 'environ' itself and passing that to other WSGI applications. The dictionary, in effect, is used as a lightweight request object. In light of this pattern, an intermediate step between no request object standard and an overly standard might be agreement on standard environment keys common to most WSGI request objects. These keys could then be used in 'environ', a request dictionary, or by wrapping them in a request object. A few examples are: wsgiorg.parsed_cookie - The contents of 'HTTP_COOKIE' stored in a Cookie.SimpleCookie instance wsgiorg.start_request - The start_request function wsgiorg.headers - the headers list wsgiorg.status - the response status string wsgiorg.application_uri - the base request URI string, equivalent to the output of wsgiref.util.application_uri wsgiorg.request_uri - the full request URI including the query string, equivalent to the default output of wsgiref.util.request_uri Operations that produce entries such as these are frequently performed by middleware. Standard keys would provide a way for the operation to be performed once and the results of the operation to be stored and passed along to other WSGI software in a predictable way-lcr From pje at telecommunity.com Tue Nov 14 02:49:28 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 13 Nov 2006 20:49:28 -0500 Subject: [Web-SIG] (proto) request object spec In-Reply-To: <45591D8E.4020304@gmail.com> Message-ID: <5.1.1.6.0.20061113203928.01f41210@sparrow.telecommunity.com> At 06:36 PM 11/13/2006 -0700, L. C. Rees wrote: >A few examples are: > >wsgiorg.parsed_cookie - The contents of 'HTTP_COOKIE' stored in a >Cookie.SimpleCookie instance > >wsgiorg.start_request - The start_request function Do you mean start_response? If so, what do you need it in the environ for? >wsgiorg.headers - the headers list > >wsgiorg.status - the response status string Not too bad of an idea -- except that it's only useful within the scope of a framework that is *not* WSGI-based, unless you are also going to have a start_response that works by setting them in the environ. But even that isn't WSGI-compliant, because the spec explicitly allows the recipient of an environ to modify it in any way they like -- including clearing it altogether. So, I've already rejected this idea several times over, because I keep thinking it's a good idea myself until I think it through again. :) >wsgiorg.application_uri - the base request URI string, equivalent to >the output of wsgiref.util.application_uri > >wsgiorg.request_uri - the full request URI including the query string, >equivalent to the default output of wsgiref.util.request_uri These two cannot be used or implemented without breaking PEP 333 compatibility for middleware. Don't even think about doing this. >Operations that produce entries such as these are frequently performed by >middleware. This is not what middleware is for; please don't encourage people to do this. Library functions are the One Obvious way to perform operations on environ keys. Middleware is mainly useful for: * logging/debugging * response modification * dispatching * URL rewriting It isn't really intended as a mechanism to stick random features into the environ; such features can usually be implemented as libraries and gain the following benefits compared to middleware: * better overall server performance * cleaner APIs * easier to understand and use * no need to configure a middleware stack >Standard keys would provide a way for the operation to be >performed once and the results of the operation to be stored and passed >along to other WSGI software in a predictable way-lcr This kind of caching breaks middleware stacks. Remember, environ may be freely modified by middleware and applications, making the caches stale and invalid! (This is why wsgiref functions don't bother caching any of these values.) From ianb at colorstudy.com Tue Nov 14 04:26:43 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 13 Nov 2006 21:26:43 -0600 Subject: [Web-SIG] (proto) request object spec In-Reply-To: <45591D8E.4020304@gmail.com> References: <45591D8E.4020304@gmail.com> Message-ID: <45593773.1090408@colorstudy.com> L. C. Rees wrote: >> That is my point. There are many smallish reasons to >> prefer one request convenience interface to another. Why >> try to sanctify some particular preferences? We don't >> need it for interop because WSGI already gives us that. > > A not infrequent WSGI pattern is putting all of the various WSGI > components in a request dictionary or even 'environ' itself and > passing that to other WSGI applications. The dictionary, in effect, > is used as a lightweight request object. > > In light of this pattern, an intermediate step between no request > object standard and an overly standard might be agreement on > standard environment keys common to most WSGI request objects. These > keys could then be used in 'environ', a request dictionary, or by > wrapping them in a request object. Hrm... but this is neither that friendly to work with, nor very reliable (e.g., if the keys aren't there), nor particularly good WSGI practice when it includes the response. > A few examples are: > > wsgiorg.parsed_cookie - The contents of 'HTTP_COOKIE' stored in a > Cookie.SimpleCookie instance This would be possible. Phillip notes the problem of systems that rewrite the request; this can be alleviated by putting in (SimpleCookie, HTTP_COOKIE) so that you can check your cache. This is what I do in paste.request.get_cookies (storing the value in paste.cookies). Note, though, that you should never check this value directly, you should only use a function that will calculate the value when necessary. It almost makes me think something like this should be named paste._cookies, to discourage direct access. In this particular case, while a standard key for the cached value would be nice, the overhead of double-parsing the cookies is so low that it's not too big a deal. My (now withdrawn) suggestion of wsgiorg.form_vars was driven by this same need, but again with a function setting the value as needed. > wsgiorg.start_request - The start_request function > > wsgiorg.headers - the headers list > > wsgiorg.status - the response status string Erk. I think it's best to keep request and response separate. At least if you are using WSGI as the foundation for your system, WSGI does keep them separate. It is better for intermediaries; otherwise they'd have to rewrite the environment. Which isn't hard, but you already have to switch things around in the function call, having two switches is just asking for trouble. > wsgiorg.application_uri - the base request URI string, equivalent to > the output of wsgiref.util.application_uri This is puts redundant and potentially out of sync information in the environment. This is best done as a property on the request object that calculates application_uri from its base components. In this case the construction is fast enough that caching it isn't worth it. > wsgiorg.request_uri - the full request URI including the query string, > equivalent to the default output of wsgiref.util.request_uri Ditto this; great as a property on a request object, not so great as a key. > Operations that produce entries such as these are frequently performed by > middleware. Standard keys would provide a way for the operation to be > performed once and the results of the operation to be stored and passed > along to other WSGI software in a predictable way-lcr I know in my own code I avoid assuming any particular context for the code. I think this is generally best, and makes things easier to test and deploy in different layerings. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ianb at colorstudy.com Tue Nov 14 04:54:49 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 13 Nov 2006 21:54:49 -0600 Subject: [Web-SIG] New spec: simple authentication Message-ID: <45593E09.7010003@colorstudy.com> I added a new spec describing authentication middleware with WSGI. It doesn't describe anything new, really, it just describes what I think is the most basic best practice in doing WSGI-based authentication middleware: http://wsgi.org/wsgi/Specifications/simple_authentication I offer it more as a basis for other specifications to build upon. Also copied below: :Title: Simple Authentication :Author: Ian Bicking :Discussions-To: Python Web-SIG :Status: Proposed :Created: 13-Nov-2006 .. contents:: Abstract -------- This describes a simple pattern for implementing authentication in WSGI middleware. This does not propose any new features or environment keys; it only describes a baseline recommended practice. Rationale --------- Authentication is probably the most common detail that should be abstracted away from an application, as it is a concern most often bound to a *deployment*. Specification ------------- There are two components to authentication: 1. Indicating when a request is authenticated, and by who 2. Responding that authentication is necessary There are already two conventions for this: 1. Put the username in ``REMOTE_USER`` 2. Respond with ``401 Unauthorized`` .. note:: Please do not confused ``401 Unauthorized`` with "permission denied". Permission denied should be indicated with ``403 Forbidden``. ``REMOTE_USER``: This should be the string username of the user, nothing more. ``401 Unauthorized``: Because middleware is handling the authentication, additional information is not required. You do not (and should not) include a ``WWW-Authenticate`` header. The middleware may include that header, or may change the response in some other way to handle the login. Example -------- The first example implements simple HTTP Basic authentication:: class HTTPBasic(object): def __init__(self, app, user_database, realm='Website'): self.app = app self.user_database = user_database def __call__(self, environ, start_response): def repl_start_response(status, headers, exc_info=None): if status.startswith('401'): remove_header(headers, 'WWW-Authenticate') headers.append(('WWW-Authenticate', 'Basic realm="%s"' % self.realm)) return start_response(status, headers) auth = environ.get('HTTP_AUTHORIZATION') if auth: scheme, data = auth.split(None, 1) assert scheme.lower() == 'basic' username, password = data.decode('base64').split(':', 1) if self.user_database.get(username) != password: return self.bad_auth(environ, start_response) environ['REMOTE_USER'] = username del environ['HTTP_AUTHORIZATION'] return self.app(environ, repl_start_response) def bad_auth(self, environ, start_response): body = 'Please authenticate' headers = [ ('content-type', 'text/plain'), ('content-length', str(len(body))), ('WWW-Authenticate', 'Basic realm="%s"' % self.realm)] start_response('401 Unauthorized', headers) return [body] def remove_header(headers, name): for header in headers: if header[0].lower() == name.lower(): headers.remove(header) break Problems -------- * Strictly speaking, it is illegal to send a ``401 Unauthorized`` response without the WWW-Authenticate header. If no middleware is installed, most browsers will treat it like a ``200 OK``. There is also no way to detect if an appropriate middleware is installed. * This doesn't give any other information about the user. That information can go in other keys, but that is not addressed in this specification currently. * Some login methods will redirect the user, and any POST request data will possibly be lost. (Note that a specification like ["handling_post_forms"] helps address this problem.) Other Possibilities ------------------- * While you can add to this specification, I think it's the most logical and useful way to do authentication and better efforts can build on this base. Open Issues ----------- See Problems. From ianb at colorstudy.com Tue Nov 14 05:12:35 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 13 Nov 2006 22:12:35 -0600 Subject: [Web-SIG] New spec: throw_errors Message-ID: <45594233.3070205@colorstudy.com> I know other people weren't psyched about this one, but it's *so easy* to define. As evidence of its applicability to more than just Paste, I just checked the Zope 3 source, and it uses environ['wsgi.handleErrors'] for this same purpose. Also there are multiple WSGI testing frameworks that could use this key in their environments. The spec is at: http://wsgi.org/wsgi/Specifications/throw_errors Copied: :Title: x-wsgiorg.throw_errors :Author: Ian Bicking :Discussions-To: Python Web-SIG :Status: Proposed :Created: 13 Nov 2006 .. contents:: Abstract -------- WSGI applications are generally not supposed to raise exceptions, instead handling their own errors (possibly returning a ``500 Server Error`` response). But in some context it is *desired* that unexpected exceptions be allowed to bubble up. This specification defines a key to set in this circumstance. Rationale --------- When in a testing context it is undesirable for an application to handle its own errors. Typically the test framework is better at handling the errors, either through error formatting or by dropping into a debugger like `pdb `_. Additionally when an exception catcher is installed in a stack, ideally it will be used for all exceptions. This allows for centralized configuration (for example, when emails are sent when errors occur). Dynamically disabling any other exception catchers is often ideal in this situation. Specification ------------- An exception catcher should check for ``environ.get('x-wsgiorg.throw_errors')`` in the environment. If it is true, it should not try to catch exceptions. This need only be checked as the application is being entered, it should not be checked later. Applications should not try to set this to effect middleware that *wraps* them, only to effect applications they may call. Example -------- A simple exception catcher:: class ExceptionCatch(object): def __init__(self, app): self.app = app def __call__(self, environ, start_response): if environ.get('x-wsgiorg.throw_errors'): return self.app(environ, start_response) try: return self.app(environ, start_response) except: import sys, traceback, StringIO exc_info = sys.exc_info() start_response('500 Server Error', [('content-type', 'text/plain')], exc_info=exc_info) out = StringIO.StringIO() traceback.print_exc(file=out) return [out.getvalue()] Problems -------- * In theory an application may know better how to format an error response than the middleware exception catcher. Of course, an application can ignore ``x-wsgiorg.throw_errors`` if it thinks it is best (or if it has been explicitly configured to do so). Other Possibilities ------------------- * You can just get the unwrapped application object and test it. Open Issues ----------- * None I know of From sh at defuze.org Tue Nov 14 09:01:05 2006 From: sh at defuze.org (Sylvain Hellegouarch) Date: Tue, 14 Nov 2006 08:01:05 -0000 (GMT) Subject: [Web-SIG] New spec: simple authentication In-Reply-To: <45593E09.7010003@colorstudy.com> References: <45593E09.7010003@colorstudy.com> Message-ID: <33269.194.221.74.7.1163491265.squirrel@mail1.webfaction.com> Ian, Why disallowing the application to set the WWW-Authenticate header? If a middleware is present it will be overwritten anyway. If no middleware is there then at least you won't break the first MUST in section 10.4.2 of RFC 2616. What happens if part of my application is to be protected by OpenID, another by Digest and a third one by a in-house auth scheme? How does your spec. deal with this? - Sylvain > I added a new spec describing authentication middleware with WSGI. It > doesn't describe anything new, really, it just describes what I think is > the most basic best practice in doing WSGI-based authentication > middleware: > > http://wsgi.org/wsgi/Specifications/simple_authentication > > I offer it more as a basis for other specifications to build upon. > > Also copied below: > > :Title: Simple Authentication > :Author: Ian Bicking > :Discussions-To: Python Web-SIG > :Status: Proposed > :Created: 13-Nov-2006 > > .. contents:: > > Abstract > -------- > > This describes a simple pattern for implementing authentication in WSGI > middleware. This does not propose any new features or environment keys; > it only describes a baseline recommended practice. > > Rationale > --------- > > Authentication is probably the most common detail that should be > abstracted away from an application, as it is a concern most often bound > to a *deployment*. > > Specification > ------------- > > There are two components to authentication: > > 1. Indicating when a request is authenticated, and by who > 2. Responding that authentication is necessary > > There are already two conventions for this: > > 1. Put the username in ``REMOTE_USER`` > 2. Respond with ``401 Unauthorized`` > > .. note:: > Please do not confused ``401 Unauthorized`` with "permission > denied". Permission denied should be indicated with ``403 Forbidden``. > > ``REMOTE_USER``: > This should be the string username of the user, nothing more. > ``401 Unauthorized``: > Because middleware is handling the authentication, additional > information is not required. You do not (and should not) include a > ``WWW-Authenticate`` header. The middleware may include that header, or > may change the response in some other way to handle the login. > > Example > -------- > > The first example implements simple HTTP Basic authentication:: > > class HTTPBasic(object): > > def __init__(self, app, user_database, realm='Website'): > self.app = app > self.user_database = user_database > > def __call__(self, environ, start_response): > def repl_start_response(status, headers, exc_info=None): > if status.startswith('401'): > remove_header(headers, 'WWW-Authenticate') > headers.append(('WWW-Authenticate', 'Basic > realm="%s"' % self.realm)) > return start_response(status, headers) > auth = environ.get('HTTP_AUTHORIZATION') > if auth: > scheme, data = auth.split(None, 1) > assert scheme.lower() == 'basic' > username, password = data.decode('base64').split(':', 1) > if self.user_database.get(username) != password: > return self.bad_auth(environ, start_response) > environ['REMOTE_USER'] = username > del environ['HTTP_AUTHORIZATION'] > return self.app(environ, repl_start_response) > > def bad_auth(self, environ, start_response): > body = 'Please authenticate' > headers = [ > ('content-type', 'text/plain'), > ('content-length', str(len(body))), > ('WWW-Authenticate', 'Basic realm="%s"' % self.realm)] > start_response('401 Unauthorized', headers) > return [body] > > def remove_header(headers, name): > for header in headers: > if header[0].lower() == name.lower(): > headers.remove(header) > break > > Problems > -------- > > * Strictly speaking, it is illegal to send a ``401 Unauthorized`` > response without the WWW-Authenticate header. If no middleware is > installed, most browsers will treat it like a ``200 OK``. There is also > no way to detect if an appropriate middleware is installed. > > * This doesn't give any other information about the user. That > information can go in other keys, but that is not addressed in this > specification currently. > > * Some login methods will redirect the user, and any POST request data > will possibly be lost. (Note that a specification like > ["handling_post_forms"] helps address this problem.) > > Other Possibilities > ------------------- > > * While you can add to this specification, I think it's the most logical > and useful way to do authentication and better efforts can build on this > base. > > Open Issues > ----------- > > See Problems. > _______________________________________________ > Web-SIG mailing list > Web-SIG at python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: > http://mail.python.org/mailman/options/web-sig/sh%40defuze.org > From wilk at flibuste.net Tue Nov 14 09:27:38 2006 From: wilk at flibuste.net (William Dode) Date: Tue, 14 Nov 2006 08:27:38 +0000 (UTC) Subject: [Web-SIG] (proto) request object spec References: <45591D8E.4020304@gmail.com> <5.1.1.6.0.20061113203928.01f41210@sparrow.telecommunity.com> Message-ID: On 14-11-2006, Phillip J. Eby wrote: > At 06:36 PM 11/13/2006 -0700, L. C. Rees wrote: ... >>Operations that produce entries such as these are frequently performed >>by >>middleware. > > This is not what middleware is for; please don't encourage people to do > this. Library functions are the One Obvious way to perform operations on > environ keys. To prevent this what do you think about a wsgiorg "standard" library ? like parse_formvars of pythonpaste wich take environ as argument. -- William Dod? - http://flibuste.net From luke.arno at gmail.com Tue Nov 14 15:13:20 2006 From: luke.arno at gmail.com (Luke Arno) Date: Tue, 14 Nov 2006 09:13:20 -0500 Subject: [Web-SIG] (proto) request object spec In-Reply-To: References: <45591D8E.4020304@gmail.com> <5.1.1.6.0.20061113203928.01f41210@sparrow.telecommunity.com> Message-ID: On 11/14/06, William Dode wrote: > On 14-11-2006, Phillip J. Eby wrote: > > At 06:36 PM 11/13/2006 -0700, L. C. Rees wrote: > > ... > > >>Operations that produce entries such as these are frequently performed > >>by > >>middleware. > > > > This is not what middleware is for; please don't encourage people to do > > this. Library functions are the One Obvious way to perform operations on > > environ keys. > > To prevent this what do you think about a wsgiorg "standard" library ? > > like parse_formvars of pythonpaste wich take environ as argument. > I tend to think this is simply a case for education and continued exploration. It will take time for everyone to absorb the WSGI way. It will take time for us to evolve that way as a community. I would hate for us to create bloat or take wrong turns that we can't easily take back in our exuberant desire to rush that. There is a natural tendency to rush in to plant the colonial flag in a new territory. Let's avoid that too. Cheers, - Luke From luke.arno at gmail.com Tue Nov 14 15:22:43 2006 From: luke.arno at gmail.com (Luke Arno) Date: Tue, 14 Nov 2006 09:22:43 -0500 Subject: [Web-SIG] New spec: throw_errors In-Reply-To: <45594233.3070205@colorstudy.com> References: <45594233.3070205@colorstudy.com> Message-ID: -1 to the proposed spec On 11/13/06, Ian Bicking wrote: ... > Other Possibilities > ------------------- > > * You can just get the unwrapped application object and test it. +1, emphatically Let's encourage best practices, before we standardize specific workarounds. The unwrapped version of the object should also be made available for composition purposes, if it is intended for such. Cheers, - Luke From ianb at colorstudy.com Tue Nov 14 18:10:44 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 14 Nov 2006 11:10:44 -0600 Subject: [Web-SIG] New spec: throw_errors In-Reply-To: References: <45594233.3070205@colorstudy.com> Message-ID: <4559F894.7030905@colorstudy.com> Luke Arno wrote: > -1 to the proposed spec > > On 11/13/06, Ian Bicking wrote: > ... >> Other Possibilities >> ------------------- >> >> * You can just get the unwrapped application object and test it. > > +1, emphatically > > Let's encourage best practices, before we > standardize specific workarounds. > > The unwrapped version of the object should also be > made available for composition purposes, if it is > intended for such. OK, then, you're going to have to justify that, because I don't think it's best practice ;) In particular, in many frameworks the presence of an exception catcher is automatic and largely opaque to the user. Of course it doesn't *have* to be opaque, but because this is the only case where it really matters I don't see why it shouldn't be opaque -- it's not worth explaining. It's not opaque to me, of course, and yet I never feel a need to unpack the object this way, because the one reason I might have is satisfied transparently, automatically, and reliably by this environment convention. Exception handling is something that is generally handled by some particular pieces of software -- the exception catcher and test frameworks. If they can agree on this, I don't see a reason anyone else needs to think about it. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ianb at colorstudy.com Tue Nov 14 18:15:28 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 14 Nov 2006 11:15:28 -0600 Subject: [Web-SIG] New spec: simple authentication In-Reply-To: <33269.194.221.74.7.1163491265.squirrel@mail1.webfaction.com> References: <45593E09.7010003@colorstudy.com> <33269.194.221.74.7.1163491265.squirrel@mail1.webfaction.com> Message-ID: <4559F9B0.8050501@colorstudy.com> Sylvain Hellegouarch wrote: > Ian, > > > Why disallowing the application to set the WWW-Authenticate header? If a > middleware is present it will be overwritten anyway. If no middleware is > there then at least you won't break the first MUST in section 10.4.2 of > RFC 2616. I suppose there's no problem with it; though of course if you response with "WWW-Authenticate: Basic" and then don't have enough information to handle the response, you don't be doing anything useful. "WWW-Authenticate: x-internal-wsgi" would be... well, explicit at least ;) > What happens if part of my application is to be protected by OpenID, > another by Digest and a third one by a in-house auth scheme? How does your > spec. deal with this? You'd have to lay out and configure your middleware appropriately. Though I can imagine a problem where you have Digest in a specific area with OpenID everywhere else; the OpenID would catch the Digest response and rewrite it. I guess there might be some purpose to the WWW-Authenticate header there too. You could say that a 401 response with a x-internal-wsgi authentication type (or a missing WWW-Authenticate header) should be handled by middleware; other responses should be ignored. This is also useful for WebDAV areas, which probably live in a subdirectory somewhere and have more limited client authentication support. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From fumanchu at amor.org Tue Nov 14 19:03:11 2006 From: fumanchu at amor.org (Robert Brewer) Date: Tue, 14 Nov 2006 10:03:11 -0800 Subject: [Web-SIG] New spec: throw_errors Message-ID: <435DF58A933BA74397B42CDEB8145A8606AF1DFD@ex9.hostedexchange.local> Ian Bicking wrote: > http://wsgi.org/wsgi/Specifications/throw_errors > ... > Problems > -------- > > * In theory an application may know better how to format an error > response than the middleware exception catcher. Of course, an > application can ignore ``x-wsgiorg.throw_errors`` if it thinks it is > best (or if it has been explicitly configured to do so). I'm not sure I see the benefits of declaring this in the WSGI environ. If you're already conceding that each consumer of this may be configured to ignore it, that's duplicated effort. That is, instead of having a simple configuration item per app: throw_errors = True ...I now have to present the deployer with a more-confusing and more tightly-coupled option: ignore_wsgiorg_throw_errors = True If both the protocol and my app's API are now more complicated, what have we gained? Robert Brewer System Architect Amor Ministries fumanchu at amor.org From luke.arno at gmail.com Tue Nov 14 19:02:02 2006 From: luke.arno at gmail.com (Luke Arno) Date: Tue, 14 Nov 2006 13:02:02 -0500 Subject: [Web-SIG] New spec: throw_errors In-Reply-To: <4559F894.7030905@colorstudy.com> References: <45594233.3070205@colorstudy.com> <4559F894.7030905@colorstudy.com> Message-ID: On 11/14/06, Ian Bicking wrote: > Luke Arno wrote: > > -1 to the proposed spec > > > > On 11/13/06, Ian Bicking wrote: > > ... > >> Other Possibilities > >> ------------------- > >> > >> * You can just get the unwrapped application object and test it. > > > > +1, emphatically > > > > Let's encourage best practices, before we > > standardize specific workarounds. > > > > The unwrapped version of the object should also be > > made available for composition purposes, if it is > > intended for such. > > OK, then, you're going to have to justify that, because I don't think > it's best practice ;) I don't know what "justify" means in this context but I will try to explain. > In particular, in many frameworks the presence of an exception catcher > is automatic and largely opaque to the user. Of course it doesn't > *have* to be opaque, but because this is the only case where it really > matters I don't see why it shouldn't be opaque -- it's not worth explaining. I don't see why it should be opaque. It is just as easy to explain as your environ flag. Plus it is simpler and more explicit. This is a common pattern: application user interface and application programming interface. Here is the external, friendly, error-hiding app that you run by itself in production. Here is the raw app that you test and plug into the back of some larger app. And there is no additional config data to manage. Hooray! ;) > It's not opaque to me, of course, and yet I never feel a need to unpack > the object this way, because the one reason I might have is satisfied > transparently, automatically, and reliably by this environment convention. > > Exception handling is something that is generally handled by some > particular pieces of software -- the exception catcher and test > frameworks. If they can agree on this, I don't see a reason anyone else > needs to think about it. If the exception catcher isn't being put between the app and the test framework, there is no problem to solve or think about. I prefer non-problems to clever solutions. ;) Cheers, - Luke From ianb at colorstudy.com Tue Nov 14 19:38:20 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 14 Nov 2006 12:38:20 -0600 Subject: [Web-SIG] New spec: throw_errors In-Reply-To: <435DF58A933BA74397B42CDEB8145A8606AF1DFD@ex9.hostedexchange.local> References: <435DF58A933BA74397B42CDEB8145A8606AF1DFD@ex9.hostedexchange.local> Message-ID: <455A0D1C.2030905@colorstudy.com> Robert Brewer wrote: > Ian Bicking wrote: >> http://wsgi.org/wsgi/Specifications/throw_errors >> ... >> Problems >> -------- >> >> * In theory an application may know better how to format an error >> response than the middleware exception catcher. Of course, an >> application can ignore ``x-wsgiorg.throw_errors`` if it thinks it is >> best (or if it has been explicitly configured to do so). > > I'm not sure I see the benefits of declaring this in the WSGI environ. > If you're already conceding that each consumer of this may be configured > to ignore it, that's duplicated effort. That is, instead of having a > simple configuration item per app: > > throw_errors = True > > ...I now have to present the deployer with a more-confusing and more > tightly-coupled option: > > ignore_wsgiorg_throw_errors = True > > If both the protocol and my app's API are now more complicated, what > have we gained? I concede someone, somewhere might want to ignore the environ key. I've never wanted to do so myself. There's no good reason I can think of that any exception catcher I've written would ignore that environ key, because they don't add any particular value when tied more closely to the application. I can imagine someone, somewhere might write such an exception catcher that has some added value. And they might then add this configuration option (maybe phrased as "force_error_catching"). The specified (and presumably default) behavior applies to most cases, especially cases where the developer isn't entirely aware of the relationship of the application to its exception catcher, and of the test framework to the application. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ianb at colorstudy.com Tue Nov 14 19:38:49 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 14 Nov 2006 12:38:49 -0600 Subject: [Web-SIG] New spec: throw_errors In-Reply-To: References: <45594233.3070205@colorstudy.com> <4559F894.7030905@colorstudy.com> Message-ID: <455A0D39.1090507@colorstudy.com> Luke Arno wrote: > On 11/14/06, Ian Bicking wrote: >> Luke Arno wrote: >> > -1 to the proposed spec >> > >> > On 11/13/06, Ian Bicking wrote: >> > ... >> >> Other Possibilities >> >> ------------------- >> >> >> >> * You can just get the unwrapped application object and test it. >> > >> > +1, emphatically >> > >> > Let's encourage best practices, before we >> > standardize specific workarounds. >> > >> > The unwrapped version of the object should also be >> > made available for composition purposes, if it is >> > intended for such. >> >> OK, then, you're going to have to justify that, because I don't think >> it's best practice ;) > > I don't know what "justify" means in this context but > I will try to explain. > >> In particular, in many frameworks the presence of an exception catcher >> is automatic and largely opaque to the user. Of course it doesn't >> *have* to be opaque, but because this is the only case where it really >> matters I don't see why it shouldn't be opaque -- it's not worth >> explaining. > > I don't see why it should be opaque. It is just as easy > to explain as your environ flag. Plus it is simpler and > more explicit. The only people who have to understand the environ flag are testing system writers, and exception catching writers. For everyone else it Just Works. In your model you have to understand the layout of the middleware and where you can get the unwrapped application object. > This is a common pattern: application user interface > and application programming interface. Here is the > external, friendly, error-hiding app that you run by > itself in production. Here is the raw app that you test > and plug into the back of some larger app. And there > is no additional config data to manage. Hooray! ;) For the most common default case there is no additional config required; the default is for the key to be respected and then everything works appropriately. While you *can* ignore the key, I've never had a reason to do so. I can imagine such a case exists, which is why ignoring the key should be an option (and of course you can't *make* anyone pay attention to anything). >> It's not opaque to me, of course, and yet I never feel a need to unpack >> the object this way, because the one reason I might have is satisfied >> transparently, automatically, and reliably by this environment >> convention. >> >> Exception handling is something that is generally handled by some >> particular pieces of software -- the exception catcher and test >> frameworks. If they can agree on this, I don't see a reason anyone else >> needs to think about it. > > If the exception catcher isn't being put between the > app and the test framework, there is no problem to > solve or think about. > > I prefer non-problems to clever solutions. ;) You've pushed the problem out of the framework and into the user's understanding of and ability to decompose the framework. The problem still exists; if your exception catcher is in place when you run the tests you'll get really unhelpful output. And, as a final argument: this is a *really easy* spec to support. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From luke.arno at gmail.com Tue Nov 14 21:07:06 2006 From: luke.arno at gmail.com (Luke Arno) Date: Tue, 14 Nov 2006 15:07:06 -0500 Subject: [Web-SIG] New spec: throw_errors In-Reply-To: <455A0D39.1090507@colorstudy.com> References: <45594233.3070205@colorstudy.com> <4559F894.7030905@colorstudy.com> <455A0D39.1090507@colorstudy.com> Message-ID: On 11/14/06, Ian Bicking wrote: > Luke Arno wrote: > > On 11/14/06, Ian Bicking wrote: > >> Luke Arno wrote: > >> > -1 to the proposed spec > >> > > >> > On 11/13/06, Ian Bicking wrote: > >> > ... > >> >> Other Possibilities > >> >> ------------------- > >> >> > >> >> * You can just get the unwrapped application object and test it. > >> > > >> > +1, emphatically > >> > > >> > Let's encourage best practices, before we > >> > standardize specific workarounds. > >> > > >> > The unwrapped version of the object should also be > >> > made available for composition purposes, if it is > >> > intended for such. > >> > >> OK, then, you're going to have to justify that, because I don't think > >> it's best practice ;) > > > > I don't know what "justify" means in this context but > > I will try to explain. > > > >> In particular, in many frameworks the presence of an exception catcher > >> is automatic and largely opaque to the user. Of course it doesn't > >> *have* to be opaque, but because this is the only case where it really > >> matters I don't see why it shouldn't be opaque -- it's not worth > >> explaining. > > > > I don't see why it should be opaque. It is just as easy > > to explain as your environ flag. Plus it is simpler and > > more explicit. > > The only people who have to understand the environ flag are testing > system writers, and exception catching writers. For everyone else it > Just Works. In your model you have to understand the layout of the > middleware and where you can get the unwrapped application object. Who is this "middle class programmer" who conducts testing yet is so clueless of what he tests? Me thinks he just follows the instructions on his end to end framework and will not be affected either way. > > This is a common pattern: application user interface > > and application programming interface. Here is the > > external, friendly, error-hiding app that you run by > > itself in production. Here is the raw app that you test > > and plug into the back of some larger app. And there > > is no additional config data to manage. Hooray! ;) > > For the most common default case there is no additional config required; > the default is for the key to be respected and then everything works > appropriately. While you *can* ignore the key, I've never had a reason > to do so. I can imagine such a case exists, which is why ignoring the > key should be an option (and of course you can't *make* anyone pay > attention to anything). But what about composition? What about a reusable WebDAV component, for instance? You could run it as a stand-alone app and you would have the flag set to False. Then you plug it in as part of an app that just wants to include a little DAV. If the flag is set to False, the "master" application can't deal with errors intelligently or pass them along to it's error middleware, but if it is True, all the errors fly loose. I for one would like to see composability increase and avoid this sort of coupling. > >> It's not opaque to me, of course, and yet I never feel a need to unpack > >> the object this way, because the one reason I might have is satisfied > >> transparently, automatically, and reliably by this environment > >> convention. > >> > >> Exception handling is something that is generally handled by some > >> particular pieces of software -- the exception catcher and test > >> frameworks. If they can agree on this, I don't see a reason anyone else > >> needs to think about it. > > > > If the exception catcher isn't being put between the > > app and the test framework, there is no problem to > > solve or think about. > > > > I prefer non-problems to clever solutions. ;) > > You've pushed the problem out of the framework and into the user's > understanding of and ability to decompose the framework. I _want_ composability and decomposability. That is the exact approach I would like to see. Let's encourage this kind of understanding. > The problem > still exists; if your exception catcher is in place when you run the > tests you'll get really unhelpful output. So, don't do that. It is easy to explain why and how not to. If a programmer does not understand exception handling yet, he should be taught. Let's educate the programmer if needed rather than kludging around his ignorance. > And, as a final argument: this is a *really easy* spec to support. can != should :) Cheers, - Luke From hober0 at gmail.com Tue Nov 14 22:30:28 2006 From: hober0 at gmail.com (Edward O'Connor) Date: Tue, 14 Nov 2006 16:30:28 -0500 Subject: [Web-SIG] New spec: throw_errors References: <45594233.3070205@colorstudy.com> <4559F894.7030905@colorstudy.com> <455A0D39.1090507@colorstudy.com> Message-ID: <867ixx67ij.fsf@rakim.cfhp.org> > I _want_ composability and decomposability. > That is the exact approach I would like to see. > Let's encourage this kind of understanding. +1 To me, this is the biggest selling point for using WSGI at all. Ted -- Edward O'Connor hober0 at gmail.com Ense petit placidam sub libertate quietem. From chad at zetaweb.com Fri Nov 17 22:57:53 2006 From: chad at zetaweb.com (Chad Whitacre) Date: Fri, 17 Nov 2006 16:57:53 -0500 Subject: [Web-SIG] [ANN] Aspen 0.3 -- a WSGI-rific web server Message-ID: <455E3061.4090608@zetaweb.com> Greetings, program! I've just released what amounts to a dev snapshot of a project I'm calling Aspen. It's an attempt at a production-quality WSGI-centric web server, something to hang your WSGI on. Some of my goals with this are: - Don't write a framework. Aspen is a way to organize your existing WSGI apps, middleware, and frameworks into a website. - Support publication/hybrid websites as well as application websites. The frameworks that are out there generally treat application models as primary. Aspen gets back to the roots of the Web with a primarily hierarchical organization, into which applications (written with whatever WSGI-supporting framework) can be joined. Go filesystem! - Deeply drink the WSGI kool-aid. All direct extensions to Aspen are WSGI callables. (Actually, they're a slight super-set of WSGI [see also: httpy], but that's a bug I want to fix. I decided to release early though.) - Keep it Stupid, Stupid I like to use the same server for development and deployment. Aspen only exposes a few knobs on the command line, with further config and extension via simple plain-text config files in a UNIX-style userland hidden within your site hierarchy. The core extensibility code is pretty well tested. The plugin examples and the configuration/UI code are a bit rougher. Basically, anything documented is ready for scrutiny. I've got a couple sites using proto-versions of Aspen. This is a complete rewrite based around WSGI, but the basic publication-centric pattern hasn't changed. Paste is the closest thing I've found for wiring up WSGI. I think Aspen might be a stripped-down, publication-centric, production-intended Paste Script. Anyway, I would love any feedback you've got. To get started, just: $ easy_install aspen And then check out the tutorial: http://www.zetadev.com/software/aspen/0.3/doc/html/tutorial.html Thanks! chad From chad at zetaweb.com Fri Nov 17 23:01:02 2006 From: chad at zetaweb.com (Chad Whitacre) Date: Fri, 17 Nov 2006 17:01:02 -0500 Subject: [Web-SIG] Chad is a Dork (was: [ANN] Aspen 0.3 -- a WSGI-rific web server) In-Reply-To: <455E3061.4090608@zetaweb.com> References: <455E3061.4090608@zetaweb.com> Message-ID: <455E311E.2090100@zetaweb.com> > I've just released what amounts to a dev snapshot of a project I'm > calling Aspen. And the project homepage is here: http://www.zetadev.com/software/aspen/ chad From joe at bitworking.org Mon Nov 20 17:28:58 2006 From: joe at bitworking.org (Joe Gregorio) Date: Mon, 20 Nov 2006 11:28:58 -0500 Subject: [Web-SIG] New spec: throw_errors In-Reply-To: References: <45594233.3070205@colorstudy.com> <4559F894.7030905@colorstudy.com> <455A0D39.1090507@colorstudy.com> Message-ID: <3f1451f50611200828yd42edf1xce5315507933d55@mail.gmail.com> On 11/14/06, Luke Arno wrote: > I _want_ composability and decomposability. > That is the exact approach I would like to see. > Let's encourage this kind of understanding. +1 -joe -- Joe Gregorio http://bitworking.org From rayonny at netscape.net Mon Nov 20 21:55:22 2006 From: rayonny at netscape.net (rayonny at netscape.net) Date: Mon, 20 Nov 2006 15:55:22 -0500 Subject: [Web-SIG] Errors while viewing webpage Message-ID: <8C8DAFD543DAF99-860-C092@WEBMAIL-MB10.sysops.aol.com> I am not a Python programmer, but I was told this is a Python problem, and I am hoping that someone with Python experience can answer this. I am trying to use the New Jersey Jobs database on the internet, and I continually reach this page (I pasted it in below). There seems to be no one to contact there for help. Is there anything that I can do from my browser (Firefox) to get by this? Thank you. https://webos.dol.state.nj.us:444/seeker/job_search.asp ASP Error A system error has occurred. Click the Back button on your browser to continue. Program details: Traceback (most recent call last): File "/dat0/ajb/v4000/components/server_fw/handler_asp.py", line 62, in asp File "/dat0/ajb/v4000/components/server_fw/handler_asp.py", line 142, in asp_process File "/dat0/ajb/v4000/components/session/session.py", line 130, in get File "/dat0/ajb/v4000/components/dbora/dbi.py", line 184, in fetchone DatabaseError: Invalid handle! None Traceback (most recent call last): File "/dat0/ajb/v4000/components/server_fw/server_fw.py", line 336, in exec_child File "/dat0/ajb/v4000/components/server_fw/handler_asp.py", line 66, in asp File "/dat0/ajb/v4000/components/server_fw/asp_utils.py", line 150, in report_errors File "/dat0/ajb/v4000/components/server_fw/asp_utils.py", line 127, in report_errors_clean IndexError: list index out of range Traceback (most recent call last): File "/dat0/ajb/v4000/components/session/session.py", line 126, in get File "/dat0/ajb/v4000/components/dbora/dbi.py", line 164, in execute DatabaseError: ORA-03114: not connected to ORACLE ________________________________________________________________________ Check Out the new free AIM(R) Mail -- 2 GB of storage and industry-leading spam and email virus protection. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/web-sig/attachments/20061120/da185cfa/attachment.html From titus at caltech.edu Mon Nov 20 22:10:27 2006 From: titus at caltech.edu (Titus Brown) Date: Mon, 20 Nov 2006 13:10:27 -0800 Subject: [Web-SIG] Errors while viewing webpage In-Reply-To: <8C8DAFD543DAF99-860-C092@WEBMAIL-MB10.sysops.aol.com> References: <8C8DAFD543DAF99-860-C092@WEBMAIL-MB10.sysops.aol.com> Message-ID: <20061120211027.GB21761@caltech.edu> On Mon, Nov 20, 2006 at 03:55:22PM -0500, rayonny at netscape.net wrote: -> I am not a Python programmer, but I was told this is a Python problem, and I am hoping that someone with Python experience can answer this. I am trying to use the New Jersey Jobs database on the internet, and I continually reach this page (I pasted it in below). There seems to be no one to contact there for help. Is there anything that I can do from my browser (Firefox) to get by this? Thank you. -> -> https://webos.dol.state.nj.us:444/seeker/job_search.asp -> -> ASP Error A system error has occurred. Click the Back button on your browser to continue. Program details: Traceback (most recent call last): -> -> File "/dat0/ajb/v4000/components/server_fw/handler_asp.py", line 62, in asp -> -> File "/dat0/ajb/v4000/components/server_fw/handler_asp.py", line 142, in asp_process -> -> File "/dat0/ajb/v4000/components/session/session.py", line 130, in get -> -> File "/dat0/ajb/v4000/components/dbora/dbi.py", line 184, in fetchone -> -> DatabaseError: Invalid handle! Their Oracle database is not talking to their Python code properly. Nothing you can do ;(. A strategically timed power outage might help ;). And unfortunately this is really not the right place to post questions like this -- you need to talk directly to the New Jersey people. cheers, --titus From jfigueiras at sapo.pt Tue Nov 21 22:08:36 2006 From: jfigueiras at sapo.pt (jfigueiras at sapo.pt) Date: Tue, 21 Nov 2006 21:08:36 +0000 Subject: [Web-SIG] Capture file descriptors while running an external program Message-ID: <1164143316.15h60r71eqjk@w5.mail.sapo.pt> Hi all! I have a problem with the module subprocess! The problem is that the external software that I am running under subprocess.Popen opens stdin, stdout, stderr, and other file descriptors to write and read in the hard drive. I know how to capture stdin, stdout and stderr, what I cannot do is to capture the other file-descriptors. Is there any way to wrap those non-standard file descriptors and make them write and read from a specific object that I define? I know that I can use tmp files to do that, but i would like something running without tmp files. Regards... Joao ___________________________________________________________________ O SAPO j? est? livre de v?rus com a Panda Software, fique voc? tamb?m! Clique em: http://antivirus.sapo.pt From mikeal at osafoundation.org Tue Nov 21 22:50:43 2006 From: mikeal at osafoundation.org (Mikeal Rogers) Date: Tue, 21 Nov 2006 13:50:43 -0800 Subject: [Web-SIG] proxy-connection header in wsgiref.utils.is_hop_by_hop Message-ID: <8E587901-CA61-4E1A-BC78-7675332EEC62@osafoundation.org> I implemented a functional proxy last night using wsgiref and I noticed that 'proxy-connection' isn't in the _hoppish dict. Although proxy-connection isn't in the HTTP 1.1 spec as hop-by-hop it is an anomalous HTTP 1.0 header that is treated as a hop-by-hop header in most cases and is removed by the proxy when fulfilling requests. Should it be added to the _hoppish dict? -Mikeal From email at christoph-haas.de Fri Nov 24 15:43:23 2006 From: email at christoph-haas.de (Christoph Haas) Date: Fri, 24 Nov 2006 15:43:23 +0100 Subject: [Web-SIG] Naif wanting to improve CGI handling Message-ID: <200611241543.23545.email@christoph-haas.de> Dear SIG, I have just subscribed to this list and while I waded through a few postings in the list's archive I might not have a clear picture of what's currently really going on. So I may ask about things that you have decided on already. Frankly I'm unhappy with the 'cgi' module in the standard library. A posting on the python-user mailing list made me rethink why web programming is so painful for me in Python while it was easy in Perl (while everything else was hard in Perl which is easy in Python). IMHO Python deserves at least a good CGI implementation. And since I'm used to Perl's 'CGI' module I would at least expect the same functionality in Python. In my posting on python-user I wrote that I would want: - cookie handling - cookie-session handling (similar to PHP or Perl's CGI::Session; combined ? with database backends or simple text files) - handling of form fields (Perl's CGI class can easily redisplay what ? has been entered in the fields when they got submitted through a
) - accessing parameters (honestly I haven't yet understood why I need to use ? .value on FielStorage dictionaries - why isn't it just a plain ? dictionary?) - state keeping (storing the contents of all form fields on disk ? to retrieve it later) - creating form elements easily (option lists for example from ? dictionaries and lists like in Perl) - creating standard HTML elements (or do you remember how DOCTYPE ? looks without looking it up?) - handling of file uploads (which is just a recipe on ActivePython ? at the moment) - controlling HTTP headers (expires, redirections, charset) - convenience functions e.g. for getting the client IP address without ? needing to read ENV['REMOTE_ADDR'] So I started reading http://wiki.python.org/moin/WebStandardisation and found a pretty large list of frameworks, modules and other approaches to deal with the shortcomings of CGI handling in Python in real life. A user coming from Perl and being used to Perl's 'CGI' module will surely not know where to start. The standard library is not sufficient and makes him/her start to reinvent the wheel (that's what I did). Frameworks make him/her learn yet another language because frameworks often seem to transforming Python into some new language. While reading what is already done and learning about this SIG I found the PEP333 that really scared me. I'm personally not interested in Python acting as a web server. I'm used to run Apache and use CGIs that deal with stdin, stdout and environment variables. So my expectations are pretty close to Perl's 'CGI' and 'CGI::Session' modules. But it sounds like it's pretty mandatory to use WSGI if I want to add any layer of functionality. Currently my idea is to ignore PEP333 - mainly because I don't know whether I really need anything of that - and implement Perl's way of CGI handling in Python. I just fear that it will add to the pile of frameworks and get us even further away from providing *the* standard CGI library. It's nice to have the freedom to choose. But while everybody complains about Perl's TIMTOWTDI at least CGI programmers in Perl know what to use. While in Python everybody's going berserk on the large set of possible frameworks. IMHO whenever TIMTOWTDI in Python there is missing something basic. I'm dreaming of a generic way to handle CGIs in Python's common beautiful style while depending fully on the standard library. Perhaps something like... import newstuff import MySQLdb import datetime # Open database for storing sessions mydatabase = MySQLdb.connect(...) # Define a session handler session_handler = newstuff.session_cookies( database_type = 'mysql', database_handler = mydatabase ) # Create a CGI instance mycgi = newstuff.CGI( session_cookie_name = 'MYSESSION', session_cookie_handler = session_handler, session_max_age = datetime.timedelta(hours=1), # Cookie session based authentication authentication_needed = True, authentication_realm = 'Secure Area' ) # Print out the HTTP header print mycgi.http_header( type='text/html', expires='+3d' ) # Print out the HTML header print mycgi.start_html( title = 'Secrets of the Pyramids', target = '_blank', css_stylesheet = '/mystyle.css' ) # Provide a login form unless the user is authenticated through # a that is connected to the cookie session if not mycgi.authenticated: print mycgi.html.loginform( ... ) # Print a

HTML element print mycgi.html.h1( 'Hello world', style = 'funkytitle' ) # Access a form parameter print "In the last form the name you entered was:", \ mycgi.form('name') # Store a value into the cookie session mycgi.session_set('name', mycgi.form('name')) # Access a value from the cookie session print "Your cookie session contains this country:", \ mycgi.session_get('country') # Send mycgi.end_html() This is just a rough brainstormed draft and I haven't thought of a huge concept or plan so far. Should I just go ahead and create yet another framework and increase the chaos by joining the dark side? Or is there work going on where my ideas would fit in and where my revolutionary energy saves the world? I would love to see my Python skills - as average as they may be - create something useful not only for me. I hope someone of you has enough overview over the current activities to provide a little guidance. Thanks in advance. Kindly Christoph From mscott at springfieldtech.com Sat Nov 25 05:22:45 2006 From: mscott at springfieldtech.com (Matthew Scott) Date: Fri, 24 Nov 2006 22:22:45 -0600 Subject: [Web-SIG] Naif wanting to improve CGI handling In-Reply-To: <200611241543.23545.email@christoph-haas.de> References: <200611241543.23545.email@christoph-haas.de> Message-ID: On 2006 Nov 24, at 08:43, Christoph Haas wrote: > Frankly I'm unhappy with the 'cgi' module in the standard library. A > posting on the python-user mailing list made me rethink why web > programming is so painful for me in Python while it was easy in Perl > (while everything else was hard in Perl which is easy in Python). IMHO > Python deserves at least a good CGI implementation. And since I'm > used to > Perl's 'CGI' module I would at least expect the same functionality in > Python. Keep in mind that packages in the Python standard library tend to have a slower development pace than packages that are available elsewhere, e.g. the packages listed at cheeseshop.python.org Because of this tendency, there are packages in the stdlib that have not changed in terms of interface or implementation for quite a while, with the exception of bugfixes or to add support for new syntax (like the new 'with' keyword in Python 2.5) I have a feeling that even if the "cgi3k" package comes with the Python 3.0 standard library, there will still be a "cgi" package kept around to support the large amount of code out there that relies on that package. > While reading what is already done and learning about this SIG I > found the > PEP333 that really scared me. I'm personally not interested in Python > acting as a web server. I'm used to run Apache and use CGIs that > deal with > stdin, stdout and environment variables. So my expectations are pretty > close to Perl's 'CGI' and 'CGI::Session' modules. But it sounds > like it's > pretty mandatory to use WSGI if I want to add any layer of > functionality. Don't get too scared by PEP333 :) Phillip J. Eby, PEP 333's author, seems unabashedly detailed and verbose in his writing style, but I think that's a very good thing for a specification that is being read and implemented by a large audience. If you want a gentler introduction to WSGI, try reading some of the stuff at http://wsgi.org/ -- especially in the "Learn WSGI" section. The interesting thing about WSGI is that you don't have to use Python as a web server to use it. Once you wrap your head around what WSGI gives you and what it doesn't, it is not too much different from using stdin/stdout pipes and environment variables at a low level. Many people do end up using Python as a web server in some regard when using WSGI though, since doing so can provide increased performance and makes the handling of persistent objects (such as sessions, database connections, etc.) easier. It is possible to write a WSGI application that can run either as a CGI script, or as a long-running process, connected to Apache via proxying or FastCGI for example. As far as the list of features that you desire from CGI, there are a growing number of WSGI applications that act as filters. See for example http://pythonpaste.org/ -- there are more resources on the wsgi.org site, and also on cheeseshop if you search for WSGI. Basically, you write your application as a WSGI application, then use third-party WSGI applications to "wrap" your application inside session handlers, form handling helpers, and so forth. > Should I just go ahead and create yet another framework and > increase the > chaos by joining the dark side? Or is there work going on where my > ideas > would fit in and where my revolutionary energy saves the world? I > would > love to see my Python skills - as average as they may be - create > something useful not only for me. I hope someone of you has enough > overview over the current activities to provide a little guidance. > Thanks > in advance. The concept of programmers, seasoned or not, contributing to a larger pool of useful software and ideas is one area where WSGI is making some traction. Ian Bicking puts it well in this entry of his: http:// blog.ianbicking.org/wsgi-update.html """ WSGI development can seem kind of plodding at times, because we're building up slowly from a low-level foundation. But as the tools emerge they work together really well; I think we're starting to see real returns from that process. ... """ For instance, you may find that something like httpy -- see http:// www.zetadev.com/software/httpy/ -- gives you an interface that is easier to grok than "bare-bones" WSGI. By using such software though, you will find that it still fits right in with other WSGI applications that offer useful services like session management, even if those apps are written using different techniques. And again, if written in such a way as to provide for it, a WSGI app can be served up as a CGI script :-) -- Matthew Scott mscott at springfieldtech.com From paul at boddie.org.uk Sun Nov 26 00:34:12 2006 From: paul at boddie.org.uk (Paul Boddie) Date: Sun, 26 Nov 2006 00:34:12 +0100 Subject: [Web-SIG] Naif wanting to improve CGI handling Message-ID: <200611260034.12391.paul@boddie.org.uk> Christoph Haas wrote: > I have just subscribed to this list and while I waded through a few > postings in the list's archive I might not have a clear picture of what's > currently really going on. So I may ask about things that you have decided > on already. I've just resubscribed after a while, mainly because I think your proposal is interesting and provides a dimension that was originally considered in the establishment of this mailing list, but which has been lost over time. > Frankly I'm unhappy with the 'cgi' module in the standard library. A > posting on the python-user mailing list made me rethink why web > programming is so painful for me in Python while it was easy in Perl > (while everything else was hard in Perl which is easy in Python). IMHO > Python deserves at least a good CGI implementation. And since I'm used to > Perl's 'CGI' module I would at least expect the same functionality in > Python. While your focus is on the cgi module, and it may be the case that you're specifically interested in CGI as an execution model (with standard input, output, environment variables, and so on), I interpret your sentiments more broadly as a desire for a simple API which works with CGI and other technologies, and where the style of programming is as transparent as old-fashioned CGI scripts: you can clearly see how the program gets its inputs, makes its decisions, and then produces its output. Am I right in thinking this? If so, then you've found at least one other person who thinks there should be a fairly reasonable and obvious foundation for Web programming in the standard library, where "obvious" means an API which deals with high-level concepts familiar to Web programmers in other communities, and where "reasonable" means that you don't have to run some highly specific server solution to use it. Of course, there are arguments against advancing the standard library in this way, mostly expressing the sentiment that if you want things to be easy, you should be using a megaframework. But whilst some people have advocated the use of Django (for example), and whilst frameworks like Django provide a fairly reasonable API at some level, there are good reasons for not launching newcomers towards megaframeworks at the first opportunity. First of all, it makes Python itself difficult to advocate: "Use Python for Web programming? Batteries not included? You mean I have to choose from this list?" Secondly, the megaframeworks often have a confusing message: the Django tutorial goes off into creating database models at the very start, for example - something thankfully rectified in the draft Django book - and this doesn't appeal to people wanting to start simple and get "hello world" or "hello form data" up and running. Thirdly, it means that lots of different groups are doing things their own way when they don't have to; whilst WSGI reduces this, the phenomenon persists at the API level. And I think the API level is critical because that's what programmers relate to. As long as they can have some flexibility in the way they deploy their code, and as long as they don't have to spend a day just configuring servers before they can get started, that aspect isn't interesting at the earliest stage of exploration. Instead, people want to know what the code has to look like, whether they can stomach writing it that way, and whether there's anything out there to make it more pleasant to work with when doing common things like outputting forms. That web.py came along even after the most prominent megaframeworks made their entrance says a lot about the demand for simplicity and transparency in some quarters. [...] > Should I just go ahead and create yet another framework and increase the > chaos by joining the dark side? Or is there work going on where my ideas > would fit in and where my revolutionary energy saves the world? Probably against all other advice you're going to get, I'd recommend that you look at WebStack, Indra and httpy (all on the WebStandardisation page) and spend less time looking at the frameworks on the WebFrameworks page. What you'll get from WebStack is a sense of what it's like to make something like the Java Servlet API but applicable for anything from CGI to Zope; from Indra you'll see a fair amount of consideration given to the common interfaces needed in a Web framework (with the list of interfaces corresponding somewhat to your long list of desirable features); from httpy you'll see a wrapping around WSGI to make it more obvious to developers outside the WSGI community. Where your revolutionary energy saves the world is where you get to propose something that enters the standard library as a better API everyone can use to further whatever applications or megaframeworks they choose to develop and use. Good luck! Paul From chad at zetaweb.com Mon Nov 27 20:13:20 2006 From: chad at zetaweb.com (Chad Whitacre) Date: Mon, 27 Nov 2006 14:13:20 -0500 Subject: [Web-SIG] Naif wanting to improve CGI handling In-Reply-To: <200611241543.23545.email@christoph-haas.de> References: <200611241543.23545.email@christoph-haas.de> Message-ID: <456B38D0.4000900@zetaweb.com> Christoph, > While reading what is already done and learning about this SIG I found > the PEP333 that really scared me.... But it sounds like it's pretty > mandatory to use WSGI if I want to add any layer of functionality. WSGI is where the action is, yes. But don't let the PEP scare you: WSGI is just a Python-specific, stackable CGI. It's just starting to get past the early adopter phase, though, so if you're for something more mature, you might check out the following: neo_cgi -- I just found this today. The API is more basic than you describe, but similar in that you're dealing with a CGI object. Not sure if it's usable outside of Clearsilver. http://www.clearsilver.net/docs/python/module-neo_cgi.cst jonpy -- This library hasn't been marketed very well, but the code is good, it's lightweight, well-documented, and includes sessioning. http://jonpy.sourceforge.net/ chad From fumanchu at amor.org Tue Nov 28 23:09:40 2006 From: fumanchu at amor.org (Robert Brewer) Date: Tue, 28 Nov 2006 14:09:40 -0800 Subject: [Web-SIG] WSGI, cgi.FieldStorage incompatibility Message-ID: <435DF58A933BA74397B42CDEB8145A8607198936@ex9.hostedexchange.local> James Y Knight wrote: > On Sep 29, 2006, at 3:31 PM, Guido van Rossum wrote: > > > On 9/29/06, Michael Kerrin wrote: > >> But the current implementation of cgi.FieldStorage in the 2.4.4 > >> branch > >> and on Python 2.5 does call readline with the size argument. It has > >> started to do this in response to the Python bug #1112549 - > >> cgi.FieldStorage memory usage can spike in line-oriented ops. See > >> http://sourceforge.net/tracker/index.php? > >> func=detail&aid=1112549&group_id=5470&atid=105470 > >> > >> Since it is reasonable for a WSGI application to use > >> cgi.FieldStorage > >> I am wondering whether cgi.FieldStorage or the WSGI specification > >> needs > >> to changed in order to solve this incompatibility. > >> > >> Originally I thought it was cgi.FieldStorage that needs to be > >> changed, > >> and hence tried to fix it by wrapping the input stream so that the > >> readline method always uses the read method on the input stream. > >> While > >> this seems to work for me it introduces a level of > complexity in the > >> cgi.py file, and possible some other bugs, that makes me think that > >> adding the size argument for readline into the WSGI specification > >> isn't > >> such bad idea after all. > > > > Since that change to cgi.py was a security fix I would strongly > > recommend not to remove it and to change the WSGI spec instead. > > Given that this change is now part of python 2.4.4 and python > 2.5, it > seems to me it is now a defacto requirement that all WSGI server > implementations must support readline with a size argument in order > to run any interesting software, despite the spec explicitly saying > that you shouldn't. I suspect simply modifying the spec to > follow the > current reality would be the least bad option. > > But this kind of destabilizing breakage really shouldn't be allowed > to happen again. Once the error was discovered, the cgi.py change > should have been immediately reverted until either a decision was > made to change the WSGI spec, or else the change fixed to not break > WSGI compliant servers. This limbo situation is pretty bad. ...and it's still pretty bad. What can I do to speed up this process? Write a change proposal for the WSGI spec? Robert Brewer System Architect Amor Ministries fumanchu at amor.org From chad at zetaweb.com Thu Nov 30 17:11:19 2006 From: chad at zetaweb.com (Chad Whitacre) Date: Thu, 30 Nov 2006 11:11:19 -0500 Subject: [Web-SIG] Aspen screencast Message-ID: <456F02A7.1080804@zetaweb.com> Hey all, I've posted a screencast about Aspen, the Python web server that I first released a couple weeks ago. The 8-minute screencast demonstrates five development models that Aspen supports: 1. Static HTML 2. CGI-style 3. PHP-style 4. RoR-style 5. Zope-style You'll find the screencast plus other documentation and downloads here: http://www.zetadev.com/software/aspen/ Thanks. chad From rich at westpoint.ltd.uk Fri Nov 24 15:03:53 2006 From: rich at westpoint.ltd.uk (Richard Moore) Date: Fri, 24 Nov 2006 14:03:53 -0000 Subject: [Web-SIG] ANNOUNCE: WSGI XSS Prevention Middleware Message-ID: <4566F351.3010209@westpoint.ltd.uk> Hi, I've just written a python WSGI middleware class to mitigate XSS flaws, it's released under the python license. I've attached the docs below. Cheers Rich. WSGI Middleware class that prevents cross-site scripting flaws in WSGI applications being exploited. Potentially malicious GET and POST variables are checked for, and if found, a 403 Forbidden response is sent to the client. Note that this class can false positive on input such as XML or passwords containing the '<' character, so it is not useful in all contexts. In addition, you should note that this middleware is not a replacement for properly validating input and quoting output. This class can be downloaded from: http://www.westpoint.ltd.uk/dist/wsgisecurity.zip Author: Richard Moore, rich at westpoint.ltd.uk Copyright: (c) 2006 Westpoint Ltd License: Released under the Python License Version: 1.0 -- Richard Moore, Principal Software Engineer, Westpoint Ltd, Albion Wharf, 19 Albion Street, Manchester, M1 5LN, England Tel: +44 161 237 1028 Fax: +44 161 237 1031