From ianb at colorstudy.com Wed Feb 1 00:01:36 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 31 Jan 2006 17:01:36 -0600 Subject: [Web-SIG] Standardized template API In-Reply-To: <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> References: <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> Message-ID: <43DFEC50.9040908@colorstudy.com> Phillip J. Eby wrote: > At 02:03 PM 1/31/2006 -0500, Clark C. Evans wrote: > >> I'd stick with the notion of a "template_name" that is neither the >> template file nor the template body. Then you'd want a template factory >> method that takes the name and produces the template body (complied if >> necessary). This needs to be once indirect since a template may refer >> to other sub-templates. This way your template could be stored >> in-memory, on-disk, or in a database, or even remotely using an HTTP >> cashe. The actual storage mechanism for the template source code should >> not be part of this interface. > > > I'd go even further than this, to note that frameworks need to be able > to cache "compiled" versions of templates. The template "engine" should > be able to return a "compiled" template that the framework can use to do > rendering in future. The plugin instance itself can manage the caching. I believe current implementations do that (at least TurboCheetah does). Is there anything the framework can do for caching that the plugin instance can't? > Note too that frameworks such as Zope 3 and peak.web have concepts like > localization and "skinning" that require the ability to switch out the > actual provider of a named template or "view", and in at least the case > of peak.web this can be polymorphic. That is, one "skin" could > implement template "foo" using Kid and another one using ZPT. So the > name of a template needs to be independent of its implementation > language, or the ability to plug in different engines is moot. > (Currently, TurboGears embeds the chosen template language in code, > which means a deployer can't skin the application effectively.) I believe that custom find_template implementations could handle these cases. Maybe there also needs to be a relative_to_template_name argument to find_template, so that it can resolve template names relative to the calling template (so, for instance, you can use relative names to the template you inherit from). > Finally, I'd note that an increasingly common use case for template > storage is likely to be via pkg_resources lookup, so that applications > can be deployed as eggs. Ideally, future versions of Zope 3 and > peak.web would perhaps allow one egg to provide skins or "layers" for > views defined in other eggs, so that users can plug in third-party skins > for applications. If find_template returns a template object instead of a filename, this would be easier. As it is it could produce an actual filename (with pkg_resources.resource_filename), but that seems unnecessary as I think about it. > Actually, if we're going to come up with something really useful here, > it would probably be a good idea to expand the scope from just defining > templates, to defining a "resource access" protocol to cover both static > resource files and templates that may need to be localized or skinned. > That would be a much bigger win, IMO, since it would put more control in > the hands of deployers and customizers, instead of just making it > possible for developers to use whatever template language they fancy. :) That seems to be an extension of find_template, which isn't specified much in this proposal. But we could specify that, I suppose: def find_resource(resource_type, resource_name, relative_to=None): """ Return a resource_type kind of resource, with the given name. If relative_to (a resource name) is given and relative names are supported, then do so. """ class IResourceType: def load_resource(resource_name, finder, resource_filename=None, resource_string=None): """ Returns the resource. resource_name is an opaque identifier, but should be kept. finder is an implementation of find_resource. The resource content can be identified with a string or filename [other kinds?]. If both are given, filename can be used for debugging output, but string should be used as the actual content. If the resource refers to other resources then the finder should be used for retrieving those resources. """ class ITemplatePlugin(IResourceType): def __init__(extra_vars_func=None, options=None): """ Create a template plugin instance. extra_vars_func is a function that returns a dictionary of "standard" variables for the environment. options is a dictionary of options for the template (e.g., compiler settings) with no defined contents. """ def load_resource(...): """ Return an instance of the template. No interface for the return value is defined, but the return value can be used with .render() Note that some templating languages don't work well with render(), so the returned resource may be the more useful interface, and frameworks should allow users to get at this return value directly. """ def render(template_instance, vars, format="html", fragment=False): """ Render the template instance (as returned by load_resource) to a string [unicode?]. vars is typically a dictionary (-like object) of variables for use in substitution. format may be "xhtml", "plain", etc., but plugins may ignore that value. "fragment" indicates if an incomplete document should be created (e.g., with no DOCTYPE); again templates may ignore this. [instead of transform, can we have this not necessarily return strings, e.g., format="elementtree"?] """ So now you'd use it like: tmpl_plugin = CheetahPlugin() tmpl = my_finder(tmpl_plugin, 'foo.tmpl') print tmpl_plugin.render(tmpl, locals()) Where before it would be: tmpl_plugin = CheetahPlugin() print tmpl_plugin.render(locals(), template='foo.tmpl') -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From pje at telecommunity.com Wed Feb 1 01:43:25 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 31 Jan 2006 19:43:25 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <43DFEC50.9040908@colorstudy.com> References: <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> At 05:01 PM 1/31/2006 -0600, Ian Bicking wrote: >Phillip J. Eby wrote: >>At 02:03 PM 1/31/2006 -0500, Clark C. Evans wrote: >> >>>I'd stick with the notion of a "template_name" that is neither the >>>template file nor the template body. Then you'd want a template factory >>>method that takes the name and produces the template body (complied if >>>necessary). This needs to be once indirect since a template may refer >>>to other sub-templates. This way your template could be stored >>>in-memory, on-disk, or in a database, or even remotely using an HTTP >>>cashe. The actual storage mechanism for the template source code should >>>not be part of this interface. >> >>I'd go even further than this, to note that frameworks need to be able to >>cache "compiled" versions of templates. The template "engine" should be >>able to return a "compiled" template that the framework can use to do >>rendering in future. > >The plugin instance itself can manage the caching. I believe current >implementations do that (at least TurboCheetah does). Is there anything >the framework can do for caching that the plugin instance can't? The plugin doesn't know how long to retain a compiled template if it's referenced only via strings. If the plugin returns a compiled template object, then the framework can either hold onto it, or not. >>Note too that frameworks such as Zope 3 and peak.web have concepts like >>localization and "skinning" that require the ability to switch out the >>actual provider of a named template or "view", and in at least the case >>of peak.web this can be polymorphic. That is, one "skin" could implement >>template "foo" using Kid and another one using ZPT. So the name of a >>template needs to be independent of its implementation language, or the >>ability to plug in different engines is moot. >>(Currently, TurboGears embeds the chosen template language in code, which >>means a deployer can't skin the application effectively.) > >I believe that custom find_template implementations could handle these >cases. Maybe there also needs to be a relative_to_template_name argument >to find_template, so that it can resolve template names relative to the >calling template (so, for instance, you can use relative names to the >template you inherit from). I don't want plugins to affect *where* templates are retrieved from, or how their names are interpreted. A template engine's only job should be to turn a string or stream into a template object. What I was proposing we do was establish a common mechanism for finding templates and other resources, taking into account i18n, skinning, polymorphism, and eggs. That would have a much greater impact on the Python web world than simply formalizing the Buffet/TurboGears plugin mechanism. Indeed, I'd argue that it'd be better here to create a WSGI-based template interface, rather than a specialized template interface. That is, if the "compiled template" returned by a template engine is just a WSGI application object, then it provides a maximum of flexibility. If individual frameworks want to supply additional data or features, they can do so via additional environ keys. Combine that with a resource-locating API, and we'd really be getting somewhere. The way I see it, a framework would be responsible for creating locale and skin-layer search lists, and identifying a resource. The API would be responsible for locating the resource implementations and returning WSGI application objects. I don't know if it's politically or technically feasible to actually solve this, since some template engines bundle a lot of assumptions presently, like the idea that a template is a Python module or stuff like that. Some frameworks currently bundle assumptions of their own about such things. Getting to a workable standard might require some compromises. On the positive side, relatively few frameworks support the full range of i18n, skinning, polymorphism, and egg resource options, and so fewer conflicts are likely the more you get into the high end. For example, Zope 3 supports skins and i18n, but not really eggs and I'm not sure it does polymorphism. peak.web currently does polymorphism, skins, and eggs, but doesn't do much w/i18n as yet. Creating a standard would enhance the capabilities and deployability of *all* participating frameworks, however, so it seems to me to be worth a go. The downside, on the other hands, is that since so few frameworks try to solve the the high-end problems, there's likely to be less widespread understanding of what the problems are or why you'd want to solve them. This is the flip side of the issue Jim Fulton pointed out, which is that people who work at the high end of the problem scale don't see much benefit in the Turbo/Buffet templating API, because it bakes in way too many assumptions to allow you to solve any really interesting problems with it. From ianb at colorstudy.com Wed Feb 1 02:20:29 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 31 Jan 2006 19:20:29 -0600 Subject: [Web-SIG] Standardized template API In-Reply-To: <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> References: <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> Message-ID: <43E00CDC.4020904@colorstudy.com> Phillip J. Eby wrote: >> I believe that custom find_template implementations could handle these >> cases. Maybe there also needs to be a relative_to_template_name >> argument to find_template, so that it can resolve template names >> relative to the calling template (so, for instance, you can use >> relative names to the template you inherit from). > > > I don't want plugins to affect *where* templates are retrieved from, or > how their names are interpreted. A template engine's only job should be > to turn a string or stream into a template object. > > What I was proposing we do was establish a common mechanism for finding > templates and other resources, taking into account i18n, skinning, > polymorphism, and eggs. That would have a much greater impact on the > Python web world than simply formalizing the Buffet/TurboGears plugin > mechanism. find_template is provided *to* the plugin, *from* the framework, so I believe it fits what you describe. Well, the implementation of that finding algorithm is not in the spec, but making a good implementation can be tackled separately; it will be more interesting when we also have a specific resource to find (templates). > Indeed, I'd argue that it'd be better here to create a WSGI-based > template interface, rather than a specialized template interface. That > is, if the "compiled template" returned by a template engine is just a > WSGI application object, then it provides a maximum of flexibility. If > individual frameworks want to supply additional data or features, they > can do so via additional environ keys. I feel comfortable with ultimately doing a render(template, vars) call, which is widely supported in quite a few templating languages. Turning it into a WSGI app seems like a bigger stretch, and one that I'm not sure is necessary, and one I'm pretty sure will scare some people off ;) Separately from this, I have my own ideas as well about a WSGI middleware for skinning, but that's something else entirely. What we have here is very much within the reach of immediately implementing and using in several frameworks. > Combine that with a resource-locating API, and we'd really be getting > somewhere. The way I see it, a framework would be responsible for > creating locale and skin-layer search lists, and identifying a > resource. The API would be responsible for locating the resource > implementations and returning WSGI application objects. I proposed a resource-locating API as well in that email. Though I'm thinking that IResourceType needs just a little more stuff to be interesting, like maybe a string identifier (which would possible match a setuptools entry point group, e.g., 'python.templating.engine'). Anyway, it doesn't seem too complicated to extend that to other resources, and I can certainly see the utility. > I don't know if it's politically or technically feasible to actually > solve this, since some template engines bundle a lot of assumptions > presently, like the idea that a template is a Python module or stuff > like that. Some frameworks currently bundle assumptions of their own > about such things. Getting to a workable standard might require some > compromises. While a standard is useful, you won't be able to drop templates in willy-nilly into applications and have them work. This is abstraction at the framework level, where actual application writers are likely to write all sorts of things which depend on functionality in the template language they are using. Maybe later on template language mixing will happen, but there's real utility here for framework authors without that. > On the positive side, relatively few frameworks support the full range > of i18n, skinning, polymorphism, and egg resource options, and so fewer > conflicts are likely the more you get into the high end. For example, > Zope 3 supports skins and i18n, but not really eggs and I'm not sure it > does polymorphism. peak.web currently does polymorphism, skins, and > eggs, but doesn't do much w/i18n as yet. Creating a standard would > enhance the capabilities and deployability of *all* participating > frameworks, however, so it seems to me to be worth a go. > > The downside, on the other hands, is that since so few frameworks try to > solve the the high-end problems, there's likely to be less widespread > understanding of what the problems are or why you'd want to solve them. > This is the flip side of the issue Jim Fulton pointed out, which is that > people who work at the high end of the problem scale don't see much > benefit in the Turbo/Buffet templating API, because it bakes in way too > many assumptions to allow you to solve any really interesting problems > with it. Then Jim and others should give more pushback about the issues they see. I, for one, am listening. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From pje at telecommunity.com Wed Feb 1 02:55:22 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 31 Jan 2006 20:55:22 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <43E00CDC.4020904@colorstudy.com> References: <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> At 07:20 PM 1/31/2006 -0600, Ian Bicking wrote: >Phillip J. Eby wrote: >>>I believe that custom find_template implementations could handle these >>>cases. Maybe there also needs to be a relative_to_template_name >>>argument to find_template, so that it can resolve template names >>>relative to the calling template (so, for instance, you can use relative >>>names to the template you inherit from). >> >>I don't want plugins to affect *where* templates are retrieved from, or >>how their names are interpreted. A template engine's only job should be >>to turn a string or stream into a template object. >> >>What I was proposing we do was establish a common mechanism for finding >>templates and other resources, taking into account i18n, skinning, >>polymorphism, and eggs. That would have a much greater impact on the >>Python web world than simply formalizing the Buffet/TurboGears plugin >>mechanism. > >find_template is provided *to* the plugin, *from* the framework, so I >believe it fits what you describe. Well, the implementation of that >finding algorithm is not in the spec, but making a good implementation can >be tackled separately; it will be more interesting when we also have a >specific resource to find (templates). What I'm getting at is that a standardized way for a deployer to provide templates to "skin" or localize an application has vastly wider applicability than the template plugin mechanism being discussed, and is much more relevant to standardization. >>Indeed, I'd argue that it'd be better here to create a WSGI-based >>template interface, rather than a specialized template interface. That >>is, if the "compiled template" returned by a template engine is just a >>WSGI application object, then it provides a maximum of flexibility. If >>individual frameworks want to supply additional data or features, they >>can do so via additional environ keys. > >I feel comfortable with ultimately doing a render(template, vars) call, >which is widely supported in quite a few templating languages. Turning it >into a WSGI app seems like a bigger stretch, and one that I'm not sure is >necessary, and one I'm pretty sure will scare some people off ;) But the upsides are: 1. Less to specify 2. More flexibility 3. Fewer protocols for a framework developer to learn 4. Complete access to the request 5. Complete control over the response >Separately from this, I have my own ideas as well about a WSGI middleware >for skinning, but that's something else entirely. What we have here is >very much within the reach of immediately implementing and using in >several frameworks. And like Jim, I have little interest in it, as it doesn't solve any of the interesting problems. Unlike Jim, I'm also actively *against* having such a spec because it creates the illusion that a useful problem has been solved. I don't have anything against the Turbo/Buffet API, mind you, I just don't want it anywhere near a PEP. It's a niche solution to a niche problem, which is allowing web frameworks to offer an illusion of choice to developers. However, the existence of a "blessed" spec along those lines would send a message that this is a solved problem, just use that spec and everything is fine, when a whole area of solutions is completely closed off. I can just see people complaining about Zope not supporting the spec, no matter that the spec doesn't come anywhere close to addressing the problems Zope is solving. The problem I'd like to see a template/resource spec solve is this: give developers and deployers a way to package templates and other static or dynamic resources for an application. All frameworks can benefit from this, especially if there need only be a single implementation of the API. Solving this problem *also* solves the problems solved by the Turbo/Buffet plugin API as a side effect, because you have to be able to specify somehow what code gets used to turn a resource specification into something executable. >>I don't know if it's politically or technically feasible to actually >>solve this, since some template engines bundle a lot of assumptions >>presently, like the idea that a template is a Python module or stuff like >>that. Some frameworks currently bundle assumptions of their own about >>such things. Getting to a workable standard might require some compromises. > >While a standard is useful, you won't be able to drop templates in >willy-nilly into applications and have them work. That's not really the goal of what I'm proposing, and it's also why I think focusing on the interface between the framework and templating tool is way too narrow a niche. >>On the positive side, relatively few frameworks support the full range of >>i18n, skinning, polymorphism, and egg resource options, and so fewer >>conflicts are likely the more you get into the high end. For example, >>Zope 3 supports skins and i18n, but not really eggs and I'm not sure it >>does polymorphism. peak.web currently does polymorphism, skins, and >>eggs, but doesn't do much w/i18n as yet. Creating a standard would >>enhance the capabilities and deployability of *all* participating >>frameworks, however, so it seems to me to be worth a go. >>The downside, on the other hands, is that since so few frameworks try to >>solve the the high-end problems, there's likely to be less widespread >>understanding of what the problems are or why you'd want to solve them. >>This is the flip side of the issue Jim Fulton pointed out, which is that >>people who work at the high end of the problem scale don't see much >>benefit in the Turbo/Buffet templating API, because it bakes in way too >>many assumptions to allow you to solve any really interesting problems with it. > >Then Jim and others should give more pushback about the issues they see. That's what I'm doing. :) From ianb at colorstudy.com Wed Feb 1 03:11:12 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 31 Jan 2006 20:11:12 -0600 Subject: [Web-SIG] Standardized template API In-Reply-To: <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> References: <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> Message-ID: <43E018C0.7050905@colorstudy.com> Phillip J. Eby wrote: > At 07:20 PM 1/31/2006 -0600, Ian Bicking wrote: > >> Phillip J. Eby wrote: >> >>>> I believe that custom find_template implementations could handle >>>> these cases. Maybe there also needs to be a >>>> relative_to_template_name argument to find_template, so that it can >>>> resolve template names relative to the calling template (so, for >>>> instance, you can use relative names to the template you inherit from). >>> >>> >>> I don't want plugins to affect *where* templates are retrieved from, >>> or how their names are interpreted. A template engine's only job >>> should be to turn a string or stream into a template object. >>> >>> What I was proposing we do was establish a common mechanism for >>> finding templates and other resources, taking into account i18n, >>> skinning, polymorphism, and eggs. That would have a much greater >>> impact on the Python web world than simply formalizing the >>> Buffet/TurboGears plugin mechanism. >> >> >> find_template is provided *to* the plugin, *from* the framework, so I >> believe it fits what you describe. Well, the implementation of that >> finding algorithm is not in the spec, but making a good implementation >> can be tackled separately; it will be more interesting when we also >> have a specific resource to find (templates). > > > What I'm getting at is that a standardized way for a deployer to provide > templates to "skin" or localize an application has vastly wider > applicability than the template plugin mechanism being discussed, and is > much more relevant to standardization. I don't see why that needs to be related to this spec, except to allow for it. find_template/find_resource allows for arbitrarily complex rules to identify resources (at least seem so to me), but does not provide or specify any standard way to do this. >>> Indeed, I'd argue that it'd be better here to create a WSGI-based >>> template interface, rather than a specialized template interface. >>> That is, if the "compiled template" returned by a template engine is >>> just a WSGI application object, then it provides a maximum of >>> flexibility. If individual frameworks want to supply additional data >>> or features, they can do so via additional environ keys. >> >> >> I feel comfortable with ultimately doing a render(template, vars) >> call, which is widely supported in quite a few templating languages. >> Turning it into a WSGI app seems like a bigger stretch, and one that >> I'm not sure is necessary, and one I'm pretty sure will scare some >> people off ;) > > > But the upsides are: > > 1. Less to specify > 2. More flexibility > 3. Fewer protocols for a framework developer to learn > 4. Complete access to the request > 5. Complete control over the response The spec we have has few or no bindings to HTTP, and is usable (and useful) outside of HTTP. You could build a single WSGI implementation on top of this spec, and use the spec to make that WSGI implementation support any template language that uses this spec. >> Separately from this, I have my own ideas as well about a WSGI >> middleware for skinning, but that's something else entirely. What we >> have here is very much within the reach of immediately implementing >> and using in several frameworks. > > > And like Jim, I have little interest in it, as it doesn't solve any of > the interesting problems. > > Unlike Jim, I'm also actively *against* having such a spec because it > creates the illusion that a useful problem has been solved. I don't > have anything against the Turbo/Buffet API, mind you, I just don't want > it anywhere near a PEP. It's a niche solution to a niche problem, which > is allowing web frameworks to offer an illusion of choice to developers. I don't buy that. I think if frameworks expose not just the result of render, but also the result of load_template, then this can to some degree support all templating languages. I think find_template/find_resource can support a wide variety of techniques for finding resources, and if a compelling spec or implementation of that function comes about, then that's great, but that doesn't invalidate the spec, nor is it required to use the spec. > However, the existence of a "blessed" spec along those lines would send > a message that this is a solved problem, just use that spec and > everything is fine, when a whole area of solutions is completely closed > off. I can just see people complaining about Zope not supporting the > spec, no matter that the spec doesn't come anywhere close to addressing > the problems Zope is solving. OK, then you'll have to go into more depth why the API provided is wrong. Not why it isn't enough -- it's not the end-all and be-all of templating or anything -- but why what it *does* provide is not sufficient or correct. > The problem I'd like to see a template/resource spec solve is this: give > developers and deployers a way to package templates and other static or > dynamic resources for an application. All frameworks can benefit from > this, especially if there need only be a single implementation of the > API. Solving this problem *also* solves the problems solved by the > Turbo/Buffet plugin API as a side effect, because you have to be able to > specify somehow what code gets used to turn a resource specification > into something executable. That's a separate issue. If you -- or someone else at some future time -- can provide an implementation of find_template/find_resource that does these things, then I think you'll have what you want. >>> I don't know if it's politically or technically feasible to actually >>> solve this, since some template engines bundle a lot of assumptions >>> presently, like the idea that a template is a Python module or stuff >>> like that. Some frameworks currently bundle assumptions of their own >>> about such things. Getting to a workable standard might require some >>> compromises. >> >> >> While a standard is useful, you won't be able to drop templates in >> willy-nilly into applications and have them work. > > > That's not really the goal of what I'm proposing, and it's also why I > think focusing on the interface between the framework and templating > tool is way too narrow a niche. I, and several other people who have already started usefully using this spec as is, don't feel that way. We feel it's useful enough right now, even with notable problems around its current (as described on the TG page) module-oriented perspective. Please, say what is wrong about the spec itself, not what is wrong about the scope of the spec. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From pje at telecommunity.com Wed Feb 1 03:46:22 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 31 Jan 2006 21:46:22 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <43E018C0.7050905@colorstudy.com> References: <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060131211609.04086258@mail.telecommunity.com> At 08:11 PM 1/31/2006 -0600, Ian Bicking wrote: >Please, say what is wrong about the spec itself, not what is wrong about >the scope of the spec. The first things that come to mind are: 1. It disadvantages better solutions (whether frameworks or templates) by reducing everything to the least common denominator 2. It doesn't give templates access to the request or response without creating new specifications for how they're to be encoded in vars -- which would end up recapitulating the pre-WSGI "common request/response" arguments all over again 3. Doesn't allow the framework to control the lifetime of compiled resources 4. Lack of relationship to HTTP makes it irrelevant/out-of-scope for Web-SIG; it should go to a string or text processing SIG. 5. Makes Web-SIG do-over a problem that TurboGears and Buffet have already solved. (That is, there's already a usable solution for what this does; it doesn't need *another* specification; let's use the current focus of attention to go after an actual opportunity to *improve*.) Problems 1-4 are simple to solve by having resources be WSGI application objects. The rest of the standardization could then be reduced to what additional environ keys should be supplied by frameworks. And then we could get to work on solving the resource deployment problem instead. Look-and-feel customizability of reusable web application components is an incredibly important problem, on a scale comparable to the problems that setuptools and eggs solve, and it's a much bigger issue for man-in-the-street use of Python as a web platform than WSGI itself is. My wife isn't a programmer, but she's hacked PHP applications to customize their look and feel. (Of course, these apps then aren't really upgradeable any more, because their code has effectively been branched.) A Python standard for resource location would allow us to have tools that run against a project and spit out a directory tree with just resources for you to edit, and to build eggs of your newly-customized UIs. If this is a *standard* for Python web apps, then it means you only learn it once. Sure, you'll have to learn the template syntax used by different apps or components, but that's not a big deal if you're mainly looking to change the HTML anyway. Also, it enables a market for third-party customizations: i.e., people creating skins for popular components/apps. If we can solve the resource deployment problem in a way that allows creating tools that people like my wife can use, we will have succeeded in moving the Python web platform forward significantly, in terms of what will be available to users, not just developers. From ianb at colorstudy.com Wed Feb 1 04:04:43 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 31 Jan 2006 21:04:43 -0600 Subject: [Web-SIG] Standardized template API In-Reply-To: <5.1.1.6.0.20060131211609.04086258@mail.telecommunity.com> References: <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131211609.04086258@mail.telecommunity.com> Message-ID: <43E0254B.4060203@colorstudy.com> Phillip J. Eby wrote: > At 08:11 PM 1/31/2006 -0600, Ian Bicking wrote: > >> Please, say what is wrong about the spec itself, not what is wrong >> about the scope of the spec. > > > The first things that come to mind are: > > 1. It disadvantages better solutions (whether frameworks or templates) > by reducing everything to the least common denominator I think exposing load_template is an escape for template languages that don't fit into the standard dictionary-in-string-out approach. Besides that, I don't know what Better Solution we're talking about. > 2. It doesn't give templates access to the request or response without > creating new specifications for how they're to be encoded in vars -- > which would end up recapitulating the pre-WSGI "common request/response" > arguments all over again The spec doesn't say anything about the variables passed in. I guess there's an implication that it is a dictionary, though that might not be necessary. The request and response are variables. Why would they be something other than variables? Even in Zope they are variables, right? The actual templates written are implicitly bound to the application and the framework, and the application is bound to the template language, and maybe we can solve that problem later. This just handles the template/framework binding. > 3. Doesn't allow the framework to control the lifetime of compiled > resources Ok... so what should we do? We can say that .load_template is not allowed to cache the thing it returns, and require find_resource to do the caching. > 4. Lack of relationship to HTTP makes it irrelevant/out-of-scope for > Web-SIG; it should go to a string or text processing SIG. There isn't such a sig. And it's all just mailing lists anyway. The people who care about this now are doing web work. > 5. Makes Web-SIG do-over a problem that TurboGears and Buffet have > already solved. (That is, there's already a usable solution for what > this does; it doesn't need *another* specification; let's use the > current focus of attention to go after an actual opportunity to *improve*.) We have discussed some improvements on the TG list, and I've tried to bring those up here as well. And the interface is like a couple weeks old, so it's not like this particular solution is As Old As The Hills and shouldn't be considered by a wider audience before moving forward. > Problems 1-4 are simple to solve by having resources be WSGI application > objects. The rest of the standardization could then be reduced to what > additional environ keys should be supplied by frameworks. And then we > could get to work on solving the resource deployment problem instead. > > Look-and-feel customizability of reusable web application components is > an incredibly important problem, on a scale comparable to the problems > that setuptools and eggs solve, and it's a much bigger issue for > man-in-the-street use of Python as a web platform than WSGI itself is. > My wife isn't a programmer, but she's hacked PHP applications to > customize their look and feel. (Of course, these apps then aren't > really upgradeable any more, because their code has effectively been > branched.) These are all important issues -- I entirely agree. When I wrote a list of features I'd want in an ideal web development environment (http://blog.ianbicking.org/ideal-webdev-environment.html), these kind of features take up a sizable portion of the list. But I don't see how this spec -- or particularly the spec I proposed in response to you (http://mail.python.org/pipermail/web-sig/2006-February/001926.html) -- disallows these kinds of features being developed in the future. In fact quite the contrary! By giving that resource finding process a name and purpose -- even if not an implementation -- I think this spec strongly encourages such development (or extraction) in the future. I just can't see why one needs to be a prerequesite for the other. But people want the template plugin *now*, and working on what people want is generally a good strategy. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From pje at telecommunity.com Wed Feb 1 04:07:33 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 31 Jan 2006 22:07:33 -0500 Subject: [Web-SIG] A trivial template API counter-proposal Message-ID: <5.1.1.6.0.20060131215710.04141ac0@mail.telecommunity.com> API to be provided by a "template engine": compile_string(text), compile_stream(input_stream) - return a WSGI application object for the template described by the text or file-like object write_compiled(app, output_stream) - save the compiled form of the template to output_stream read_compiled(input_stream) - read a template from input_stream and return a WSGI application object I believe that this is all that can reasonably be standardized by the Web-SIG at this time for a template API, while still treating its constituents fairly. (i.e., not disadvantaging more advanced frameworks by being a least-common-denominator approach). The only other thing that might be suitable for standardization would be any common extra 'environ' keys that might be considered desirable, like a "published object" key to hold the "vars" (for "controller"-style frameworks that postprocess a function return value) or to hold the "self" object (for Zope-style object publishing frameworks that apply a template as a view of the object addressed in the URL). From millerdev at gmail.com Wed Feb 1 04:14:56 2006 From: millerdev at gmail.com (Daniel Miller) Date: Tue, 31 Jan 2006 22:14:56 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <5.1.1.6.0.20060131211609.04086258@mail.telecommunity.com> References: <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131211609.04086258@mail.telecommunity.com> Message-ID: <43E027B0.4090401@gmail.com> OK Phillip. Time for you to put your money on the line. Give us an interface so WE can shoot YOUR great ideas down for a change :) ~ Daniel Phillip J. Eby wrote: > At 08:11 PM 1/31/2006 -0600, Ian Bicking wrote: >> Please, say what is wrong about the spec itself, not what is wrong about >> the scope of the spec. > > The first things that come to mind are: > > 1. It disadvantages better solutions (whether frameworks or templates) by > reducing everything to the least common denominator > > 2. It doesn't give templates access to the request or response without > creating new specifications for how they're to be encoded in vars -- which > would end up recapitulating the pre-WSGI "common request/response" > arguments all over again > > 3. Doesn't allow the framework to control the lifetime of compiled resources > > 4. Lack of relationship to HTTP makes it irrelevant/out-of-scope for > Web-SIG; it should go to a string or text processing SIG. > > 5. Makes Web-SIG do-over a problem that TurboGears and Buffet have already > solved. (That is, there's already a usable solution for what this does; it > doesn't need *another* specification; let's use the current focus of > attention to go after an actual opportunity to *improve*.) > > Problems 1-4 are simple to solve by having resources be WSGI application > objects. The rest of the standardization could then be reduced to what > additional environ keys should be supplied by frameworks. And then we > could get to work on solving the resource deployment problem instead. > > Look-and-feel customizability of reusable web application components is an > incredibly important problem, on a scale comparable to the problems that > setuptools and eggs solve, and it's a much bigger issue for > man-in-the-street use of Python as a web platform than WSGI itself is. My > wife isn't a programmer, but she's hacked PHP applications to customize > their look and feel. (Of course, these apps then aren't really upgradeable > any more, because their code has effectively been branched.) > > A Python standard for resource location would allow us to have tools that > run against a project and spit out a directory tree with just resources for > you to edit, and to build eggs of your newly-customized UIs. If this is a > *standard* for Python web apps, then it means you only learn it > once. Sure, you'll have to learn the template syntax used by different > apps or components, but that's not a big deal if you're mainly looking to > change the HTML anyway. Also, it enables a market for third-party > customizations: i.e., people creating skins for popular components/apps. > > If we can solve the resource deployment problem in a way that allows > creating tools that people like my wife can use, we will have succeeded in > moving the Python web platform forward significantly, in terms of what will > be available to users, not just developers. > From foom at fuhm.net Wed Feb 1 04:28:57 2006 From: foom at fuhm.net (James Y Knight) Date: Tue, 31 Jan 2006 22:28:57 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <43E027B0.4090401@gmail.com> References: <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131211609.04086258@mail.telecommunity.com> <43E027B0.4090401@gmail.com> Message-ID: On Jan 31, 2006, at 10:07 PM, Phillip J. Eby wrote: > Subject: [Web-SIG] A trivial template API counter-proposal > API to be provided by a "template engine": > [...] On Jan 31, 2006, at 10:14 PM, Daniel Miller wrote: > OK Phillip. Time for you to put your money on the line. Give us an > interface so WE can shoot YOUR great ideas down for a change :) Awesome timing. :) James From pje at telecommunity.com Wed Feb 1 05:06:38 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 31 Jan 2006 23:06:38 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: References: <43E027B0.4090401@gmail.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131211609.04086258@mail.telecommunity.com> <43E027B0.4090401@gmail.com> Message-ID: <5.1.1.6.0.20060131230608.02390e58@mail.telecommunity.com> At 10:28 PM 1/31/2006 -0500, James Y Knight wrote: >On Jan 31, 2006, at 10:07 PM, Phillip J. Eby wrote: >> Subject: [Web-SIG] A trivial template API counter-proposal >>API to be provided by a "template engine": >>[...] > >On Jan 31, 2006, at 10:14 PM, Daniel Miller wrote: >>OK Phillip. Time for you to put your money on the line. Give us an >>interface so WE can shoot YOUR great ideas down for a change :) > >Awesome timing. :) Let the shooting begin. :) From pje at telecommunity.com Wed Feb 1 05:19:50 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 31 Jan 2006 23:19:50 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <43E02D68.40709@colorstudy.com> References: <5.1.1.6.0.20060131214816.04136ab0@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131214816.04136ab0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060131230703.04104bf8@mail.telecommunity.com> [back to the Web-SIG] At 09:39 PM 1/31/2006 -0600, Ian Bicking wrote: >How do you pass in variables? environ, or a nested variable therein > How do you get non-string output? What kind of non-string output? For doing what? >But there's a kind of templating that occurs in a typical application >that is much more intimately tied to the calling code, and shouldn't be >pushed through an HTTP-like interface. But that HTTP-like interface can be concealed in a library wrapper for systems that don't orient that way, and if you *don't* supply such an interface, then template systems that orient that way are put at a disadvantage that *can't* be worked around by library wrapping. Granted, using WSGI means that frameworks with deeper WSGI integration are rewarded by this approach (because they can just expose their existing WSGI context to the template), but that doesn't seem like a *bad* thing to me. :) (By the way, there are a few WSGI features that could be dropped for the resource interface in any case, lessening the overhead of those frameworks that have to recreate a WSGI context. We could remove the requirements for the caller to supply a write() function, for example, along with the wsgi.input and wsgi.errors streams, which have negligible utility for templates.) From ianb at colorstudy.com Wed Feb 1 05:32:01 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 31 Jan 2006 22:32:01 -0600 Subject: [Web-SIG] Standardized template API In-Reply-To: <5.1.1.6.0.20060131230703.04104bf8@mail.telecommunity.com> References: <5.1.1.6.0.20060131214816.04136ab0@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131214816.04136ab0@mail.telecommunity.com> <5.1.1.6.0.20060131230703.04104bf8@mail.telecommunity.com> Message-ID: <43E039C1.8070306@colorstudy.com> Phillip J. Eby wrote: > [back to the Web-SIG] > > At 09:39 PM 1/31/2006 -0600, Ian Bicking wrote: > >> How do you pass in variables? > > > environ, or a nested variable therein OK, if you invert that (put the environ in the variables) then you get... variables, like in the original spec. >> How do you get non-string output? > > > What kind of non-string output? For doing what? Like ElementTree output, that you might pipe through other transformations. > >But there's a kind of templating that occurs in a typical application > >that is much more intimately tied to the calling code, and shouldn't be > >pushed through an HTTP-like interface. > > But that HTTP-like interface can be concealed in a library wrapper for > systems that don't orient that way, and if you *don't* supply such an > interface, then template systems that orient that way are put at a > disadvantage that *can't* be worked around by library wrapping. I just don't get it... why do I care about the status or headers from the template? I don't want the template returning those things. I probably want the template returning a unicode string. Providing a content-type would be fine, but hardly important. So, I want to send variables in, and I want to get a string out... and that's what the original spec does. Why phrase it in terms of WSGI? -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From pje at telecommunity.com Wed Feb 1 06:22:23 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 01 Feb 2006 00:22:23 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <43E039C1.8070306@colorstudy.com> References: <5.1.1.6.0.20060131230703.04104bf8@mail.telecommunity.com> <5.1.1.6.0.20060131214816.04136ab0@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131214816.04136ab0@mail.telecommunity.com> <5.1.1.6.0.20060131230703.04104bf8@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060131233646.02388550@mail.telecommunity.com> At 10:32 PM 1/31/2006 -0600, Ian Bicking wrote: >Phillip J. Eby wrote: >>[back to the Web-SIG] >>At 09:39 PM 1/31/2006 -0600, Ian Bicking wrote: >> >>>How do you pass in variables? >> >>environ, or a nested variable therein > >OK, if you invert that (put the environ in the variables) then you get... >variables, like in the original spec. Well, if you're going to include the environ, you're halfway to WSGI, so why bother making a new spec? :) >>> How do you get non-string output? >> >>What kind of non-string output? For doing what? > >Like ElementTree output, that you might pipe through other transformations. But how's that a *template* any more, then? >> >But there's a kind of templating that occurs in a typical application >> >that is much more intimately tied to the calling code, and shouldn't be >> >pushed through an HTTP-like interface. >>But that HTTP-like interface can be concealed in a library wrapper for >>systems that don't orient that way, and if you *don't* supply such an >>interface, then template systems that orient that way are put at a >>disadvantage that *can't* be worked around by library wrapping. > >I just don't get it... why do I care about the status or headers from the >template? Because in some frameworks, the template is the resource and it's what controls these things. > I don't want the template returning those things. And other people do. Zope 2 certainly wants its templates to be able to control such things. peak.web wants it. ASP and PHP-style templating systems want it. I'm sure there are others. >I probably want the template returning a unicode string. Providing a >content-type would be fine, but hardly important. You asked what was wrong with the spec; this is a good example. It's conceived in the context of a very narrow set of frameworks whose paradigms differ so little that they might as well be one framework to begin with, at least with respect to how they treat templates. That's not to say that there's anything wrong with them, just that there is a lot of useful diversity in Python web frameworks and I'd prefer not to rule out entire *classes* of web frameworks from consideration, just because some other ideas are more familiar to some people. We should use the wider interface in this case, because it means everybody can play. If you make a variables-to-strings interface, only variables-to-strings frameworks and templates can play. It's straightforward to wrap variables-to-strings templates in a trivial WSGI interface and to pull string output from a WSGI app (you can, after all, just use the body and ignore the headers), but it's not possible to support WSGI functionality inside a variables-to-strings paradigm. Notice, by the way, that a resources-as-WSGI-apps approach also enables approaches like WSGI servers that run templates straight from a directory, ASP/PHP-style. This is just *one* kind of paradigm that's enabled by my counter-proposal but isn't practical with the variables-to-string approaches. Another paradigm that's enabled: imagine having a "template" whose text is actually a Paste Deploy specification. Now imagine that Zope implemented the WSGI-templating proposal using "WSGI Methods" that allowed you to select whatever "template engine" you wanted, and allowed editing the text through the web. Now, you could deploy a Paste-based application inside a folder in a Zope site! So, those are just two ideas I pulled out of my hat for what a WSGI-templating spec could permit, that variables-and-strings can't match. It's not so much that the vars-and-strings proposal is *wrong* per se, it's just that compared to my counterproposal, it just ain't right. A "good is the enemy of the best" scenario is what I'm worried about here, which is why even though the TG/B interface is a *good* thing in and of itself, I don't want it to get promoted outside its area of expertise. The Web-SIG should put forth a WSGI-based templating API, because it's better for the Python web platform as a whole. The only cost that I see is disadvantaging frameworks that don't make good use of WSGI, and that was always going to be a natural consequence of having a standard in the first place. (i.e., some will always make better use of it than others, and the better ones will have a better time of it.) From mike_mp at zzzcomputing.com Wed Feb 1 08:03:12 2006 From: mike_mp at zzzcomputing.com (Michael Bayer) Date: Wed, 1 Feb 2006 02:03:12 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <5.1.1.6.0.20060131233646.02388550@mail.telecommunity.com> References: <5.1.1.6.0.20060131230703.04104bf8@mail.telecommunity.com> <5.1.1.6.0.20060131214816.04136ab0@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131214816.04136ab0@mail.telecommunity.com> <5.1.1.6.0.20060131230703.04104bf8@mail.telecommunity.com> <5.1.1.6.0.20060131233646.02388550@mail.telecommunity.com> Message-ID: do either of these specs address the concept of componentized templates ? that is, when you are referencing a template, compiling its output, and delivering its textual output, in reality that template would consist of possibly dozens of smaller sub-templates, to form different parts of the page, and also to form various nesting schemes. In myghty, the resolver stack, i.e. the thing that takes some kind of string, runs it through some rules (which can be a straight down list or a whole structure of conditionals and groups), and produces a template object (actually called a component), is not used only for handling the initial URL from an HTTP request. It is used entirely within the request for every sub-component, nested- template, subrequest, template inheritance scheme, what-have-you, so that a single request actually calls a lot of templates which call each other, all using the same resolver stack to find things. Extra contextual information can also be passed to the resolver to give it more things with which to make decisions; such as the "context", which can say "im the top-level request calling you", "im a sub- component calling you", "im your super-(inherited) parent calling you", etc. Information about the resolved component is available as well, such as getting back the re.Match object that resulted from a regexp call, or other parameters that indicate things about the resolver rule that actually matched. All of that stuff is important. A cross-template resolution/execution API introduces the concept of one kind of template including snips from other kinds of templates, or someone using a template-inheritance-enabled system to enable that capability for a system that doesnt have that feature, etc. So it would be nice if this system could support that use case, and not just be request-oriented. When I first wrote WSGI support for Myghty and saw what Ian was doing with Paste, I thought, wow, why dont I take my component model and make them all into little WSGI application() objects...but in reality, for those sub-templates, the typical HTTP request/response as well as the start_response doesnt really seem appropriate, especially for the kinds of interactions those components have with each other, such as intricate wrapping schemes, iterative patterns. Also i dont believe any templating system should ever be hardcoded to an "HTTP request" concept....too many places have we had JSP or PHP sites that hit a brick wall when we want to use the same templates to pre-generate files, or emails, etc. because those systems have no notion of non-HTTP usage. Forgive my rambling, I havent gotten to dig deeply into Ian's spec and I'm still trying to get an idea of what the more encompassing spec is about (like what is template polymorphism exactly ?) - mike On Feb 1, 2006, at 12:22 AM, Phillip J. Eby wrote: > At 10:32 PM 1/31/2006 -0600, Ian Bicking wrote: >> Phillip J. Eby wrote: >>> [back to the Web-SIG] >>> At 09:39 PM 1/31/2006 -0600, Ian Bicking wrote: >>> >>>> How do you pass in variables? >>> >>> environ, or a nested variable therein >> >> OK, if you invert that (put the environ in the variables) then you >> get... >> variables, like in the original spec. > > Well, if you're going to include the environ, you're halfway to > WSGI, so > why bother making a new spec? :) > > >>>> How do you get non-string output? >>> >>> What kind of non-string output? For doing what? >> >> Like ElementTree output, that you might pipe through other >> transformations. > > But how's that a *template* any more, then? > > >>>> But there's a kind of templating that occurs in a typical >>>> application >>>> that is much more intimately tied to the calling code, and >>>> shouldn't be >>>> pushed through an HTTP-like interface. >>> But that HTTP-like interface can be concealed in a library >>> wrapper for >>> systems that don't orient that way, and if you *don't* supply >>> such an >>> interface, then template systems that orient that way are put at a >>> disadvantage that *can't* be worked around by library wrapping. >> >> I just don't get it... why do I care about the status or headers >> from the >> template? > > Because in some frameworks, the template is the resource and it's what > controls these things. > > >> I don't want the template returning those things. > > And other people do. Zope 2 certainly wants its templates to be > able to > control such things. peak.web wants it. ASP and PHP-style templating > systems want it. I'm sure there are others. > > >> I probably want the template returning a unicode string. Providing a >> content-type would be fine, but hardly important. > > You asked what was wrong with the spec; this is a good example. It's > conceived in the context of a very narrow set of frameworks whose > paradigms > differ so little that they might as well be one framework to begin > with, at > least with respect to how they treat templates. > > That's not to say that there's anything wrong with them, just that > there is > a lot of useful diversity in Python web frameworks and I'd prefer > not to > rule out entire *classes* of web frameworks from consideration, just > because some other ideas are more familiar to some people. > > We should use the wider interface in this case, because it means > everybody > can play. If you make a variables-to-strings interface, only > variables-to-strings frameworks and templates can play. It's > straightforward to wrap variables-to-strings templates in a trivial > WSGI > interface and to pull string output from a WSGI app (you can, after > all, > just use the body and ignore the headers), but it's not possible to > support > WSGI functionality inside a variables-to-strings paradigm. > > Notice, by the way, that a resources-as-WSGI-apps approach also > enables > approaches like WSGI servers that run templates straight from a > directory, > ASP/PHP-style. This is just *one* kind of paradigm that's enabled > by my > counter-proposal but isn't practical with the variables-to-string > approaches. > > Another paradigm that's enabled: imagine having a "template" whose > text is > actually a Paste Deploy specification. Now imagine that Zope > implemented > the WSGI-templating proposal using "WSGI Methods" that allowed you to > select whatever "template engine" you wanted, and allowed editing > the text > through the web. Now, you could deploy a Paste-based application > inside a > folder in a Zope site! > > So, those are just two ideas I pulled out of my hat for what a > WSGI-templating spec could permit, that variables-and-strings can't > match. > > It's not so much that the vars-and-strings proposal is *wrong* per > se, it's > just that compared to my counterproposal, it just ain't right. A > "good is > the enemy of the best" scenario is what I'm worried about here, > which is > why even though the TG/B interface is a *good* thing in and of > itself, I > don't want it to get promoted outside its area of expertise. The > Web-SIG > should put forth a WSGI-based templating API, because it's better > for the > Python web platform as a whole. > > The only cost that I see is disadvantaging frameworks that don't > make good > use of WSGI, and that was always going to be a natural consequence of > having a standard in the first place. (i.e., some will always make > better > use of it than others, and the better ones will have a better time > of it.) > > _______________________________________________ > Web-SIG mailing list > Web-SIG at python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: http://mail.python.org/mailman/options/web-sig/mike_mp% > 40zzzcomputing.com From ianb at colorstudy.com Wed Feb 1 08:27:21 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 01 Feb 2006 01:27:21 -0600 Subject: [Web-SIG] Standardized template API In-Reply-To: References: <5.1.1.6.0.20060131230703.04104bf8@mail.telecommunity.com> <5.1.1.6.0.20060131214816.04136ab0@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131214816.04136ab0@mail.telecommunity.com> <5.1.1.6.0.20060131230703.04104bf8@mail.telecommunity.com> <5.1.1.6.0.20060131233646.02388550@mail.telecommunity.com> Message-ID: <43E062D9.4000304@colorstudy.com> Michael Bayer wrote: > do either of these specs address the concept of componentized templates > ? that is, when you are referencing a template, compiling its output, > and delivering its textual output, in reality that template would > consist of possibly dozens of smaller sub-templates, to form different > parts of the page, and also to form various nesting schemes. In > myghty, the resolver stack, i.e. the thing that takes some kind of > string, runs it through some rules (which can be a straight down list > or a whole structure of conditionals and groups), and produces a > template object (actually called a component), is not used only for > handling the initial URL from an HTTP request. I think you can do this with my second proposal, which uses find_resource. In that model I imagine something like: find_resource = MyghtyResourceFinder(request info...) plugin = MyghtyPlugin() tmpl = find_resource(plugin, '/index.myt') plugin.render(tmpl, {}) The tmpl object knows that it is named '/index.myt', and it has a reference to find_resource, so it can use that to find sub-templates. The only context that is passed to find_resource in that model is the (optional) template that is getting the new template. There's no subrequests, and maybe not some of the other things you are considering... but that might not be a problem, I'm not sure. Because when you get a subtemplate with find_resource you are just *getting* it, you aren't filling it quite yet. So if you phrase that inclusion as a subrequest, maybe that just means that you fill that subtemplate with different arguments that indicate that. Unless you need to know that it is a subrequest (beyond just checking for the presence of the relative_to argument), or need to know the chain of calls or something. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From dangoor at gmail.com Wed Feb 1 14:28:15 2006 From: dangoor at gmail.com (Kevin Dangoor) Date: Wed, 1 Feb 2006 08:28:15 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> References: <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <43E00CDC.4020904@colorstudy.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> Message-ID: <3f085ecd0602010528r40a70e39g62fa3d8793fa96e9@mail.gmail.com> On 1/31/06, Phillip J. Eby wrote: > Unlike Jim, I'm also actively *against* having such a spec because it > creates the illusion that a useful problem has been solved. I don't have > anything against the Turbo/Buffet API, mind you, I just don't want it > anywhere near a PEP. It's a niche solution to a niche problem, which is > allowing web frameworks to offer an illusion of choice to developers. There may need to be two discussions here. There are some minor tweaks to the current TurboGears template plugin spec that people want. I don't know how many people are using those plugins, but I do know that there are at least three. I'm fine with taking a first step of making our changes to the simple variable-to-string interface and having that be a de facto standard among those of us using these plugins. If we can devise a standard that builds on WSGI in some useful way and allows for more uses and wider adoption, as Phillip suggests, that does seem like a fine goal for the web-sig. That effort is not going to stop or hinder those of us who are already using these template engine plugins happily, so I don't think we need to look at this as an either-or proposition. The PEP would only cover the larger standard, but we can still make good use of the tools we have today. Kevin From pje at telecommunity.com Wed Feb 1 16:38:20 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 01 Feb 2006 10:38:20 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <3f085ecd0602010528r40a70e39g62fa3d8793fa96e9@mail.gmail.co m> References: <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <43E00CDC.4020904@colorstudy.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060201102611.0417d408@mail.telecommunity.com> At 08:28 AM 2/1/2006 -0500, Kevin Dangoor wrote: >On 1/31/06, Phillip J. Eby wrote: > > Unlike Jim, I'm also actively *against* having such a spec because it > > creates the illusion that a useful problem has been solved. I don't have > > anything against the Turbo/Buffet API, mind you, I just don't want it > > anywhere near a PEP. It's a niche solution to a niche problem, which is > > allowing web frameworks to offer an illusion of choice to developers. > >There may need to be two discussions here. There are some minor tweaks >to the current TurboGears template plugin spec that people want. I >don't know how many people are using those plugins, but I do know that >there are at least three. I'm fine with taking a first step of making >our changes to the simple variable-to-string interface and having that >be a de facto standard among those of us using these plugins. > >If we can devise a standard that builds on WSGI in some useful way and >allows for more uses and wider adoption, as Phillip suggests, that >does seem like a fine goal for the web-sig. That effort is not going >to stop or hinder those of us who are already using these template >engine plugins happily, so I don't think we need to look at this as an >either-or proposition. The PEP would only cover the larger standard, >but we can still make good use of the tools we have today. +1. I just want to make it so that frameworks get a bigger win by supporting WSGI templating than by only implementing the vars-to-strings convention, in order to encourage more frameworks to let WSGI pass all the way through their stack. This gets us a lot closer to the place where anything works from inside of anything. From pje at telecommunity.com Wed Feb 1 17:07:30 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 01 Feb 2006 11:07:30 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: References: <5.1.1.6.0.20060131233646.02388550@mail.telecommunity.com> <5.1.1.6.0.20060131230703.04104bf8@mail.telecommunity.com> <5.1.1.6.0.20060131214816.04136ab0@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131214816.04136ab0@mail.telecommunity.com> <5.1.1.6.0.20060131230703.04104bf8@mail.telecommunity.com> <5.1.1.6.0.20060131233646.02388550@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060201103909.02440a88@mail.telecommunity.com> At 02:03 AM 2/1/2006 -0500, Michael Bayer wrote: >do either of these specs address the concept of componentized >templates ? that is, when you are referencing a template, compiling >its output, and delivering its textual output, in reality that >template would consist of possibly dozens of smaller sub-templates, >to form different parts of the page, and also to form various nesting >schemes. In myghty, the resolver stack, i.e. the thing that takes >some kind of string, runs it through some rules (which can be a >straight down list or a whole structure of conditionals and groups), >and produces a template object (actually called a component), is not >used only for handling the initial URL from an HTTP request. It is >used entirely within the request for every sub-component, nested- >template, subrequest, template inheritance scheme, what-have-you, so >that a single request actually calls a lot of templates which call >each other, all using the same resolver stack to find things. Extra >contextual information can also be passed to the resolver to give it >more things with which to make decisions; such as the "context", >which can say "im the top-level request calling you", "im a sub- component >calling you", "im your super-(inherited) parent calling >you", etc. Information about the resolved component is available as >well, such as getting back the re.Match object that resulted from a >regexp call, or other parameters that indicate things about the >resolver rule that actually matched. All of that stuff is important. My assumption is that framework-specific functionality of this nature would be made available via standard WSGI "server extension APIs", which is to say objects or callbacks inside of the 'environ' dict. So, you might have a 'myghty.resolver' key in environ to provide access to that API. >A cross-template resolution/execution API introduces the concept of >one kind of template including snips from other kinds of templates, >or someone using a template-inheritance-enabled system to enable that >capability for a system that doesnt have that feature, etc. So it >would be nice if this system could support that use case, and not >just be request-oriented. > >When I first wrote WSGI support for Myghty and saw what Ian was doing >with Paste, I thought, wow, why dont I take my component model and >make them all into little WSGI application() objects...but in >reality, for those sub-templates, the typical HTTP request/response >as well as the start_response doesnt really seem appropriate, >especially for the kinds of interactions those components have with >each other, such as intricate wrapping schemes, iterative patterns. Note that different templating engines have different notions of what a sub-template would consist of; some think of them as functions, others as DOM trees, and still others as strings or objects implementing some kind of rendering interface. Some need subtemplates that are effectively bound as methods of some object being rendered, so there's a need for a "self" parameter to the subtemplate. Thus, it seems to me that all this stuff needs to remain on the template side of the API to start with, as it isn't currently generalizable to some sort of generic API. (Which is why my proposal doesn't currently include a standard way to find other templates.) >Also i dont believe any templating system should ever be hardcoded to >an "HTTP request" concept....too many places have we had JSP or PHP >sites that hit a brick wall when we want to use the same templates to >pre-generate files, or emails, etc. because those systems have no >notion of non-HTTP usage. Sorry, you lost me there. I see no reason why you can't use a WSGI response to write to a file or send an email. >Forgive my rambling, I havent gotten to dig deeply into Ian's spec >and I'm still trying to get an idea of what the more encompassing >spec is about (like what is template polymorphism exactly ?) Specifying only a template's *name* in code, not its type. In peak.web, if I refer to a resource or template named "foo", the system looks in the resource directory and determines the template type based on extensions. So if there's a foo.pwt or foo.html or foo.gif or foo.jpeg, it determines the resource type from that. When you combine this feature with skinning, it means that deployers are free to change the technology used to deliver a particular template or resource. For example, replacing a .gif or .jpeg with a .png or vice versa, or using a different template engine for a template. I don't know if any other frameworks support it; IIRC TurboGears currently embeds the template engine name inside the code that refers to the template. Anyway, resource polymorphism is a deep assumption in peak.web, and it obviously affects how it does subtemplate lookups -- another reason why I think we should start with a spec that doesn't get into subtemplating to start with. There's nothing stopping the a WSGI-based templating proposal from providing some kind of subtemplate support, though; it's just an extension API after all. What I'm opposed to is giving official status to half-solutions to the subtemplating problem that only work with a few string-oriented templates and frameworks. However, I'm thinking maybe we could take a step in the direction of making it easier for templating systems to play without needing full WSGI support. We could require a WSGI templating environ to include a 'wti.return_string' key, defined such that you could do something like this: def template(environ, start_response): return environ['wti.return_string']( start_response, 'text/plain', "rendered text" ) This would make it trivial to wrap string-based template engines in WTI (the WSGI Template Interface). Heck, it might not be a bad idea to have a key like that in WSGI 1.1! But I digress. What I think we should do as we figure out other models to return for things besides strings, we can add other 'wti.return_foo' keys to allow templates to return non-string things that can nonetheless be reduced to strings if middleware is in play and needs them. From christian at dowski.com Wed Feb 1 17:02:13 2006 From: christian at dowski.com (Christian Wyglendowski) Date: Wed, 01 Feb 2006 10:02:13 -0600 Subject: [Web-SIG] Standardized template API In-Reply-To: <3f085ecd0602010528r40a70e39g62fa3d8793fa96e9@mail.gmail.com> References: <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <43E00CDC.4020904@colorstudy.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <3f085ecd0602010528r40a70e39g62fa3d8793fa96e9@mail.gmail.com> Message-ID: <43E0DB85.1090606@dowski.com> Kevin Dangoor wrote: > On 1/31/06, Phillip J. Eby wrote: >> Unlike Jim, I'm also actively *against* having such a spec because it >> creates the illusion that a useful problem has been solved. I don't have >> anything against the Turbo/Buffet API, mind you, I just don't want it >> anywhere near a PEP. It's a niche solution to a niche problem, which is >> allowing web frameworks to offer an illusion of choice to developers. > > There may need to be two discussions here. There are some minor tweaks > to the current TurboGears template plugin spec that people want. I > don't know how many people are using those plugins, but I do know that > there are at least three. I'm fine with taking a first step of making > our changes to the simple variable-to-string interface and having that > be a de facto standard among those of us using these plugins. I am fine with that. While the current vars-to-strings template plugins might not solve any "interesting" problems, they can (and do) make life easier for folks writing web apps in frameworks that use them. > If we can devise a standard that builds on WSGI in some useful way and > allows for more uses and wider adoption, as Phillip suggests, that > does seem like a fine goal for the web-sig. That effort is not going > to stop or hinder those of us who are already using these template > engine plugins happily, so I don't think we need to look at this as an > either-or proposition. The PEP would only cover the larger standard, > but we can still make good use of the tools we have today. Right. Christian http://www.dowski.com ps - my first post to the WEB-SIG :-) From cce at clarkevans.com Wed Feb 1 17:43:10 2006 From: cce at clarkevans.com (Clark C. Evans) Date: Wed, 1 Feb 2006 11:43:10 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> References: <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> Message-ID: <20060201164310.GA13344@prometheusresearch.com> On Tue, Jan 31, 2006 at 08:55:22PM -0500, Phillip J. Eby wrote: | >>Indeed, I'd argue that it'd be better here to create a WSGI-based | >>template interface, rather than a specialized template interface. | >>That is, if the "compiled template" returned by a template engine is | >>just a WSGI application object, then it provides a maximum of | >>flexibility. If individual frameworks want to supply additional data | >>or features, they can do so via additional environ keys. Let's follow this idea through. Would the ``environ`` essentially act like the dict that most template engines already use for input? Or would it require something like environ['wsgi.result'] be a dict that acts as this primary input? How does this application get its input? As I've been converting my application to WSGI, I've put my "templating" stuff as a *response* filter. It gets JSON input (as a string) from the output stream of the previous stage... yea, I encode and then decode so it isn't fast. But it works and is very general. Perhaps the WSGI application specification should be modified to allow the response to be an interation over any sort of object that can be converted to a str? This would allow filters to be written w/o emitting and re-parsing content in an intermediate format. | >I feel comfortable with ultimately doing a render(template, vars) call, | >which is widely supported in quite a few templating languages. Turning | >it into a WSGI app seems like a bigger stretch, and one that I'm not | >sure is necessary, and one I'm pretty sure will scare some people off ;) | | But the upsides are: | | 1. Less to specify | 2. More flexibility | 3. Fewer protocols for a framework developer to learn | 4. Complete access to the request | 5. Complete control over the response I can see reasons why you'd want this. A templating engine could want to use the ``HTTP_ACCEPT`` environment variable to determine what sort content to return (or raise an error, 404 if it doesn't support any of the content type suggested). Similarly, I can see reasons why a templating engine might want to set specific response headers. ... On Tue, Jan 31, 2006 at 09:04:43PM -0600, Ian Bicking wrote: | Phillip J. Eby wrote: | >1. It disadvantages better solutions (whether frameworks or templates) | >by reducing everything to the least common denominator | | I think exposing load_template is an escape for template languages that | don't fit into the standard dictionary-in-string-out approach. Besides | that, I don't know what Better Solution we're talking about. I don't see how something like this disadvantages better solutions. If anything, I can see how it would enable quicker adoption of better solutions: an adapter from this "simple" interface to the "better" interface would be easy to construct and thus give out of the gate any better interface a leg-up. | >2. It doesn't give templates access to the request or response without | >creating new specifications for how they're to be encoded in vars -- | >which would end up recapitulating the pre-WSGI "common | >request/response" arguments all over again | | The spec doesn't say anything about the variables passed in. I guess | there's an implication that it is a dictionary, though that might not be | necessary. The request and response are variables. Why would they be | something other than variables? Even in Zope they are variables, right? Ian, my "templates" use the Accept header to figure out what sort of content variant to return; plus a quick peek at the User-Agent for last-minute compatibility tweaks. How does the current/proposed templating system support this? These things arn't exactly "inputs" that get renderd into the output string. | >3. Doesn't allow the framework to control the lifetime of compiled | >resources | | Ok... so what should we do? We can say that .load_template is not | allowed to cache the thing it returns, and require find_resource to do | the caching. Phillip, I fail to see how this is a problem. Doesn't Python have garbage collecting and weak references? Best, Clark From mike_mp at zzzcomputing.com Wed Feb 1 18:12:49 2006 From: mike_mp at zzzcomputing.com (Michael Bayer) Date: Wed, 1 Feb 2006 12:12:49 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <5.1.1.6.0.20060201103909.02440a88@mail.telecommunity.com> References: <5.1.1.6.0.20060131233646.02388550@mail.telecommunity.com> <5.1.1.6.0.20060131230703.04104bf8@mail.telecommunity.com> <5.1.1.6.0.20060131214816.04136ab0@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131214816.04136ab0@mail.telecommunity.com> <5.1.1.6.0.20060131230703.04104bf8@mail.telecommunity.com> <5.1.1.6.0.20060131233646.02388550@mail.telecommunity.com> <5.1.1.6.0.20060201103909.02440a88@mail.telecommunity.com> Message-ID: On Feb 1, 2006, at 11:07 AM, Phillip J. Eby wrote: >> spec is about (like what is template polymorphism exactly ?) > > Specifying only a template's *name* in code, not its type. In > peak.web, if I refer to a resource or template named "foo", the > system looks in the resource directory and determines the template > type based on extensions. So if there's a foo.pwt or foo.html or > foo.gif or foo.jpeg, it determines the resource type from that. > When you combine this feature with skinning, it means that > deployers are free to change the technology used to deliver a > particular template or resource. For example, replacing a .gif > or .jpeg with a .png or vice versa, or using a different template > engine for a template. > It seems like myghty resolver does this as well...you send in a URI, and as it walks down the chain, it can hit FileResolver (returns file- based templates), ModuleResolver (returns python callables), each of which are free to return whatever Component object they want if they have something to offer for that URI; each Component just has a run() method. you can inject whatever customized rule you want to return something else at some point. rules could be constructed to return an adapted version of any other template's template object too. From pje at telecommunity.com Wed Feb 1 18:59:09 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 01 Feb 2006 12:59:09 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: References: <5.1.1.6.0.20060201103909.02440a88@mail.telecommunity.com> <5.1.1.6.0.20060131233646.02388550@mail.telecommunity.com> <5.1.1.6.0.20060131230703.04104bf8@mail.telecommunity.com> <5.1.1.6.0.20060131214816.04136ab0@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131214816.04136ab0@mail.telecommunity.com> <5.1.1.6.0.20060131230703.04104bf8@mail.telecommunity.com> <5.1.1.6.0.20060131233646.02388550@mail.telecommunity.com> <5.1.1.6.0.20060201103909.02440a88@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060201125808.03de1280@mail.telecommunity.com> At 12:12 PM 2/1/2006 -0500, Michael Bayer wrote: >On Feb 1, 2006, at 11:07 AM, Phillip J. Eby wrote: >>>spec is about (like what is template polymorphism exactly ?) >> >>Specifying only a template's *name* in code, not its type. In >>peak.web, if I refer to a resource or template named "foo", the >>system looks in the resource directory and determines the template >>type based on extensions. So if there's a foo.pwt or foo.html or >>foo.gif or foo.jpeg, it determines the resource type from that. >>When you combine this feature with skinning, it means that >>deployers are free to change the technology used to deliver a >>particular template or resource. For example, replacing a .gif >>or .jpeg with a .png or vice versa, or using a different template >>engine for a template. > >It seems like myghty resolver does this as well...you send in a URI, >and as it walks down the chain, it can hit FileResolver (returns file- >based templates), ModuleResolver (returns python callables), each of >which are free to return whatever Component object they want if they >have something to offer for that URI; each Component just has a run() >method. you can inject whatever customized rule you want to return >something else at some point. rules could be constructed to return >an adapted version of any other template's template object too. So asking for "foo" can resolve to "foo.gif" or "foo.jpeg" in the file system? If you have to say in the code which it is, that's not what I mean. From pje at telecommunity.com Wed Feb 1 19:02:36 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 01 Feb 2006 13:02:36 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <20060201164310.GA13344@prometheusresearch.com> References: <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060201115123.038607e0@mail.telecommunity.com> At 11:43 AM 2/1/2006 -0500, Clark C. Evans wrote: >On Tue, Jan 31, 2006 at 08:55:22PM -0500, Phillip J. Eby wrote: >| >>Indeed, I'd argue that it'd be better here to create a WSGI-based >| >>template interface, rather than a specialized template interface. >| >>That is, if the "compiled template" returned by a template engine is >| >>just a WSGI application object, then it provides a maximum of >| >>flexibility. If individual frameworks want to supply additional data >| >>or features, they can do so via additional environ keys. > >Let's follow this idea through. Would the ``environ`` essentially act >like the dict that most template engines already use for input? Or >would it require something like environ['wsgi.result'] be a dict that >acts as this primary input? How does this application get its input? As I mentioned in my counter-proposal, there should probably be a key like 'wti.source' to contain either the object to be published (for publisher-oriented frameworks) or a dictionary of variables (for controller-oriented frameworks). I originally called it "published object", but that's biased towards publisher frameworks so perhaps a more neutral name like 'source' or 'data' would be more appropriate. Ideally, I'd like for us to move forward with hammering out the details of this and any other APIs that should be present in a WSGI Template or WSGI Resource interface. >Perhaps the WSGI application specification should be modified to allow >the response to be an interation over any sort of object that can be >converted to a str? This would allow filters to be written w/o emitting >and re-parsing content in an intermediate format. Actually, my 'wti.return_string' idea in a previous post does this in a slightly different way. Having return_foo APIs allows you to return other types of objects that can be converted to normal WSGI or left alone, according to whether the caller understands them. Think of it as a Python form of HTTP_ACCEPT. :) For an existing example of this idea, consider WSGI's file_wrapper feature, that wraps a file as an iterable, but can bypass it and use the file directly if no intervening middleware uses the iterable. This idea could be extended to DOMs or other kinds of objects. >On Tue, Jan 31, 2006 at 09:04:43PM -0600, Ian Bicking wrote: >| Phillip J. Eby wrote: >| >1. It disadvantages better solutions (whether frameworks or templates) >| >by reducing everything to the least common denominator >| >| I think exposing load_template is an escape for template languages that >| don't fit into the standard dictionary-in-string-out approach. Besides >| that, I don't know what Better Solution we're talking about. > >I don't see how something like this disadvantages better solutions. If >anything, I can see how it would enable quicker adoption of better >solutions: an adapter from this "simple" interface to the "better" >interface would be easy to construct and thus give out of the gate >any better interface a leg-up. No, it wouldn't. The WSGI templating proposal effectively requires the framework to allow resources full control of a WSGI response, whereas a framework that just strips off the WSGI bits and only keeps a string completely disempowers the template. That's what I mean by disadvantaging better solutions. We can and should turn that around: disadvantage the frameworks that *don't* let the resource have control, instead of disadvantaging the resources that want to get control. In fact, "template" is probably the wrong word for us to be using here; the issue is more about embedding rich dynamic resources, only some of which happen to be simple templates. If we get that, we get templates as a side effect. But if we only do templates, we make it harder for resource embedding to get a foothold. (This reminds me a bit of the complaints when the Microsoft CLR came out, that it basically makes every language into syntax sugar for C#, while disadvantaging more dynamic languages by making them relatively harder to implement and/or slower. Only in this case, we're not talking about something that can be worked around with partial interpretation and fancier compiler techniques.) >| >3. Doesn't allow the framework to control the lifetime of compiled >| >resources >| >| Ok... so what should we do? We can say that .load_template is not >| allowed to cache the thing it returns, and require find_resource to do >| the caching. > >Phillip, I fail to see how this is a problem. Doesn't Python have >garbage collecting and weak references? The earlier proposal, if I understood it correctly, never exposed a compiled template object to the framework, so GC and weak references wouldn't have helped. Any caching would've had to be done by the engine without any input from the framework as to how long something was needed. Arranging the API to expose compiled resource objects would fix that, since the framework could hold on to them as long as it liked, and normal refcounting would take care of resource management. And if the engine wants to cache also, it can use a weak reference. From ianb at colorstudy.com Wed Feb 1 19:29:35 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 01 Feb 2006 12:29:35 -0600 Subject: [Web-SIG] Standardized template API In-Reply-To: <20060201164310.GA13344@prometheusresearch.com> References: <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> Message-ID: <43E0FE0F.2010805@colorstudy.com> Clark C. Evans wrote: > On Tue, Jan 31, 2006 at 09:04:43PM -0600, Ian Bicking wrote: > | Phillip J. Eby wrote: > | >1. It disadvantages better solutions (whether frameworks or templates) > | >by reducing everything to the least common denominator > | > | I think exposing load_template is an escape for template languages that > | don't fit into the standard dictionary-in-string-out approach. Besides > | that, I don't know what Better Solution we're talking about. > > I don't see how something like this disadvantages better solutions. If > anything, I can see how it would enable quicker adoption of better > solutions: an adapter from this "simple" interface to the "better" > interface would be easy to construct and thus give out of the gate > any better interface a leg-up. My criteria is that it would be easy to implement Phillip's proposal in terms of mine (and I could provide a sample implementation of that if it would help), but it doesn't seem so easy to do the opposite. Maybe not possible, because templates often need to get to raw (not plugin or WSGI wrapped) version of subtemplates (using communication APIs that are not standard and are not part of any of these specs). For instance, there's no way for a Cheetah template to extend a WSGI application. It can extend other Cheetah templates, and even other Python classes, but not a WSGI app. > | >2. It doesn't give templates access to the request or response without > | >creating new specifications for how they're to be encoded in vars -- > | >which would end up recapitulating the pre-WSGI "common > | >request/response" arguments all over again > | > | The spec doesn't say anything about the variables passed in. I guess > | there's an implication that it is a dictionary, though that might not be > | necessary. The request and response are variables. Why would they be > | something other than variables? Even in Zope they are variables, right? > > Ian, my "templates" use the Accept header to figure out what sort of > content variant to return; plus a quick peek at the User-Agent for > last-minute compatibility tweaks. How does the current/proposed > templating system support this? These things arn't exactly "inputs" > that get renderd into the output string. It's up to you to handle that in your application. So you might pass in the WSGI environment as some variable, or the specific header, or you might select a template based on that header. For instance, you might have a convention that templates are named like foo.text.plain.tmpl for a template that produces text/plain output. Then you make a find_resource implementation that is bound to the current request, and looks at the accept header to select the appropriate template from the intersection of Accept and the available templates. Or, you could simply pass in the environment to the template and assume that the template will produce the correct template. For instance, maybe you make your own template plugin that wraps other template plugins, and does automatic conversion from text/html to text/plain if necessary. There isn't any very good way to send metadata about the result of the template rendering, and I can see there being some utility to allowing that. For instance, indicating what mime type was actually generated, instead of trusting that the mime type given was used. Or indicating what encoding the output is in (if you don't return unicode). While these aren't strictly necessary -- you can just trust your template to do what you tell it to do -- it would be nice. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From mike_mp at zzzcomputing.com Wed Feb 1 19:50:15 2006 From: mike_mp at zzzcomputing.com (Michael Bayer) Date: Wed, 1 Feb 2006 13:50:15 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <5.1.1.6.0.20060201125808.03de1280@mail.telecommunity.com> References: <5.1.1.6.0.20060201103909.02440a88@mail.telecommunity.com> <5.1.1.6.0.20060131233646.02388550@mail.telecommunity.com> <5.1.1.6.0.20060131230703.04104bf8@mail.telecommunity.com> <5.1.1.6.0.20060131214816.04136ab0@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131214816.04136ab0@mail.telecommunity.com> <5.1.1.6.0.20060131230703.04104bf8@mail.telecommunity.com> <5.1.1.6.0.20060131233646.02388550@mail.telecommunity.com> <5.1.1.6.0.20060201103909.02440a88@mail.telecommunity.com> <5.1.1.6.0.20060201125808.03de1280@mail.telecommunity.com> Message-ID: On Feb 1, 2006, at 12:59 PM, Phillip J. Eby wrote: > At 1 > > So asking for "foo" can resolve to "foo.gif" or "foo.jpeg" in the > file system? If you have to say in the code which it is, that's > not what I mean. you can write a resolver that searches the filesystem for "foo.*" and does some type calculation based on the extension it located, sure. or you get a similar effect with two resolvers in a chain, one which is configured to look for ".jpeg" and return a jpeg-template, and one configured to look for ".gif" to return a gif- template. you can switch the order of those resolvers to specify one extension taking precendence over the other. the common use case in myghty is looking for ".py", which gives you a ModuleComponent, or ".myt" or another file-based extension which gives you a FileComponent. From chrism at plope.com Wed Feb 1 19:53:59 2006 From: chrism at plope.com (Chris McDonough) Date: Wed, 1 Feb 2006 13:53:59 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <5.1.1.6.0.20060201115123.038607e0@mail.telecommunity.com> References: <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060201115123.038607e0@mail.telecommunity.com> Message-ID: <10EC77CF-0218-4397-982E-5BB243762E1C@plope.com> One specific concern about the "returning the published object" for publisher-based frameworks is that often the published object has references to other objects that might not make sense in the context of the thread handling the rendering of the template. For example, if you're using a thread pool behind a Twisted server, and the thing doing the rendering is in "the main thread", methods hanging off of the "published object" might try to make use of thread-local storage, which would fail. Zope 3 uses thread-local storage for request objects, IIRC. This might be a nonissue, because I'm a little fuzzy on which component(s) actually do(es) the rendering of the template in the models being proposed. But the amount of fuzziness I have about what's trying to be specified here makes me wonder if there aren't better things to go specify. > > As I mentioned in my counter-proposal, there should probably be a > key like > 'wti.source' to contain either the object to be published (for > publisher-oriented frameworks) or a dictionary of variables (for > controller-oriented frameworks). I originally called it "published > object", but that's biased towards publisher frameworks so perhaps > a more > neutral name like 'source' or 'data' would be more appropriate. From pje at telecommunity.com Wed Feb 1 20:49:08 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 01 Feb 2006 14:49:08 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <43E0FE0F.2010805@colorstudy.com> References: <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> Message-ID: <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> At 12:29 PM 2/1/2006 -0600, Ian Bicking wrote: >Clark C. Evans wrote: >>On Tue, Jan 31, 2006 at 09:04:43PM -0600, Ian Bicking wrote: >>| Phillip J. Eby wrote: >>| >1. It disadvantages better solutions (whether frameworks or templates) >>| >by reducing everything to the least common denominator >>| | I think exposing load_template is an escape for template languages >>that | don't fit into the standard dictionary-in-string-out >>approach. Besides | that, I don't know what Better Solution we're >>talking about. >>I don't see how something like this disadvantages better solutions. If >>anything, I can see how it would enable quicker adoption of better >>solutions: an adapter from this "simple" interface to the "better" >>interface would be easy to construct and thus give out of the gate >>any better interface a leg-up. > >My criteria is that it would be easy to implement Phillip's proposal in >terms of mine (and I could provide a sample implementation of that if it >would help), but it doesn't seem so easy to do the opposite. Actually, I believe you've got that backwards. Many things the WSGI embedding proposal can do, yours can't do *at all*, no matter how much programming you do. vars'n'strings isn't a substitute for WSGI, but it's easy to piggyback other payloads *out* of WSGI using file_wrapper-like APIs or dual-use objects with "__call__" methods. > Maybe not possible, because templates often need to get to raw (not > plugin or WSGI wrapped) version of subtemplates (using communication APIs > that are not standard and are not part of any of these specs). For one thing, you can use extension APIs, like wsgi.file_wrapper. For another, note that requiring a compiled template to be a WSGI app doesn't require a calling or extending template to restrict itself to interacting via WSGI. If you have some kind of "find_template" extension API that's called, the template is free to sniff the returned resource for something "better" than WSGI for its purposes, like a Cheetah base template or a peak.web IDOMlet, or a PSO tag, or any of a dozen other specialized templating technologies. They just have to *also* be WSGI apps if there's any chance they might be used to respond to a page request. (Example: peak.web IDOMlets have separate methods to behave as a WSGI-like response handler and as a fragment generator for insertion into another DOMlet's page, which also allows them to be smart about including surrounding content; thus they can dynamically "inherit" (to use the common slang for applying a layout) depending on whether they are being used as a fragment or a full page, and optionally applying content negotiation as well. This is another example of a kind of feature where the vars'n'strings proposal seems to fall short of enabling.) >For instance, there's no way for a Cheetah template to extend a WSGI >application. It can extend other Cheetah templates, and even other Python >classes, but not a WSGI app. Provide extension APIs to allow finding other templates. Putting them in this context also helps make it clear that such APIs aren't -- and *can't* be -- universal, because the templates themselves aren't heterogeneous. You can't give a Cheetah template a Kid subtemplate. I'd prefer to take up template finding in the context of a resource deployment proposal, where I think we can get a lot more leverage. But if it enables needed use cases today, I could get behind providing a simple -- and *optional* -- template finding API as part of the WSGI template/resource proposal, especially if discussing it helps lay the groundwork for the resource deployment proposal. >It's up to you to handle that in your application. So you might pass in >the WSGI environment as some variable, or the specific header, or you >might select a template based on that header. You're presuming a controller-based architecture here, not an object-publishing one or a PHP/ASP style "active pages" one. That's ignoring what, maybe half the web frameworks? From pje at telecommunity.com Wed Feb 1 20:59:22 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 01 Feb 2006 14:59:22 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <10EC77CF-0218-4397-982E-5BB243762E1C@plope.com> References: <5.1.1.6.0.20060201115123.038607e0@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <5.1.1.6.0.20060201115123.038607e0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060201144913.0239e3e0@mail.telecommunity.com> At 01:53 PM 2/1/2006 -0500, Chris McDonough wrote: >One specific concern about the "returning the published object" for >publisher-based frameworks is that often the published object has >references to other objects that might not make sense in the context >of the thread handling the rendering of the template. For example, >if you're using a thread pool behind a Twisted server, and the thing >doing the rendering is in "the main thread", methods hanging off of >the "published object" might try to make use of thread-local storage, >which would fail. Zope 3 uses thread-local storage for request >objects, IIRC. > >This might be a nonissue, because I'm a little fuzzy on which >component(s) actually do(es) the rendering of the template in the >models being proposed. But the amount of fuzziness I have about >what's trying to be specified here makes me wonder if there aren't >better things to go specify. At least WSGI already has something approaching a defined threading model. :) Granted, recent discussion has shown that it's lacking in a few areas, but we could kill two birds with one stone here by fixing those bits. I would also note, however, that your comments actually suggest that the use of thread-locals for context is in fact a bad thing, and in fact I have written something better that allows contexts to be handed off to other threads or used in massively parallel pseudothreads. I.e., they are "context locals" rather than "thread locals", and you can instantaneously switch an unbounded number of them in or out in an O(1) operation, without even having to know which ones may be in use. You simply get *all* of them, so it's quite scalable. If you needed to hand off tasks between threads, you could hand off their complete execution context that way. Of course, this is off-topic for Web-SIG, so anyone can email me privately if they're curious. However, if the library ever does find its way into the stdlib, then it might be reasonable to reference it in later WSGI versions. (E.g. as "if you run iterations of the same app return value from multiple threads, a server MUST save and restore the active context locals across thread switches".) I think this would actually fix most of the thread-related issues that have popped up in recent discussion. From ianb at colorstudy.com Wed Feb 1 21:48:09 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 01 Feb 2006 14:48:09 -0600 Subject: [Web-SIG] Standardized template API In-Reply-To: <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> References: <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> Message-ID: <43E11E89.7020308@colorstudy.com> Phillip J. Eby wrote: >> My criteria is that it would be easy to implement Phillip's proposal >> in terms of mine (and I could provide a sample implementation of that >> if it would help), but it doesn't seem so easy to do the opposite. > > > Actually, I believe you've got that backwards. Many things the WSGI > embedding proposal can do, yours can't do *at all*, no matter how much > programming you do. vars'n'strings isn't a substitute for WSGI, but > it's easy to piggyback other payloads *out* of WSGI using > file_wrapper-like APIs or dual-use objects with "__call__" methods. What exactly can't I do with a WSGI application (a specific implementation) that uses the non-WSGI plugin API? >> For instance, there's no way for a Cheetah template to extend a WSGI >> application. It can extend other Cheetah templates, and even other >> Python classes, but not a WSGI app. > > > Provide extension APIs to allow finding other templates. Putting them > in this context also helps make it clear that such APIs aren't -- and > *can't* be -- universal, because the templates themselves aren't > heterogeneous. You can't give a Cheetah template a Kid subtemplate. My API always indicates the type of object you are looking for, though that may be too strict, as you don't *always* want to have a single type. Though if there is a defined exception when a resource can't be found, maybe that could be handled at a higher level than find_resource. > I'd prefer to take up template finding in the context of a resource > deployment proposal, where I think we can get a lot more leverage. But > if it enables needed use cases today, I could get behind providing a > simple -- and *optional* -- template finding API as part of the WSGI > template/resource proposal, especially if discussing it helps lay the > groundwork for the resource deployment proposal. > > >> It's up to you to handle that in your application. So you might pass >> in the WSGI environment as some variable, or the specific header, or >> you might select a template based on that header. > > > You're presuming a controller-based architecture here, not an > object-publishing one or a PHP/ASP style "active pages" one. That's > ignoring what, maybe half the web frameworks? object-publishing is a subset of controller-based frameworks, where there is a single controller known as the "object publisher". In an active page model the controller is the framework (insofar as you want to project MVC terminology onto such systems). Here's a rough WSGI implementation of active pages using the resource spec: class TemplateApp(object): def __init__(self, find_resource, ext_to_plugin, index_name, default_ext): self.find_resource = find_resource # a dictionary of extensions to instantiated plugins: self.ext_to_plugin = ext_to_plugin self.index_name = index_name self.default_ext = default_ext def __call__(self, environ, start_response): # actually empty path_info should cause a redirect... path_info = environ.get('PATH_INFO', '/') if path_info.endswith('/'): path_info += self.index_name parts = path_info.rsplit('.', 1) if len(parts) == 1: ext = self.default_ext resource_name = path_info + ext else: ext = '.'+parts[1] resource_name = path_info plugin = self.ext_to_plugin[ext] environ['response.headers'] = {'Content-type': 'text/html'} resource = self.find_resource(plugin, resource_name) body = response.plugin.render(resource, {'environ': environ}) start_response('200 OK', environ['response.headers'].items()) return [body] There's a bunch of sloppy WSGI constructs, and the environment isn't a great request object, but that's not really the point. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From pje at telecommunity.com Wed Feb 1 23:15:26 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 01 Feb 2006 17:15:26 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <43E11E89.7020308@colorstudy.com> References: <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <43DF94F5.7080108@colorstudy.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> At 02:48 PM 2/1/2006 -0600, Ian Bicking wrote: >Phillip J. Eby wrote: >>>My criteria is that it would be easy to implement Phillip's proposal in >>>terms of mine (and I could provide a sample implementation of that if it >>>would help), but it doesn't seem so easy to do the opposite. >> >>Actually, I believe you've got that backwards. Many things the WSGI >>embedding proposal can do, yours can't do *at all*, no matter how much >>programming you do. vars'n'strings isn't a substitute for WSGI, but it's >>easy to piggyback other payloads *out* of WSGI using file_wrapper-like >>APIs or dual-use objects with "__call__" methods. > >What exactly can't I do with a WSGI application (a specific >implementation) that uses the non-WSGI plugin API? The "template" doesn't get to control the status or headers if all it can do is return is a string. >>>For instance, there's no way for a Cheetah template to extend a WSGI >>>application. It can extend other Cheetah templates, and even other >>>Python classes, but not a WSGI app. >> >>Provide extension APIs to allow finding other templates. Putting them in >>this context also helps make it clear that such APIs aren't -- and >>*can't* be -- universal, because the templates themselves aren't >>heterogeneous. You can't give a Cheetah template a Kid subtemplate. > >My API always indicates the type of object you are looking for, though >that may be too strict, as you don't *always* want to have a single >type. Though if there is a defined exception when a resource can't be >found, maybe that could be handled at a higher level than find_resource. I don't see what this gains over having WSGI as a universal fallback protocol in this context. You always get WSGI, and maybe it also speaks something "native" to you. >Here's a rough WSGI implementation of active pages using the resource spec: You've just presented a *convention*, not a standard. There's a big difference. Standardizing on WSGI gives a level playing field for all paradigms. Standardizing on strings clips other frameworks' wings, as your example clearly demonstrates, and forces them to reinvent wheels that we've already spent plenty of sweat and tears inventing. Why are you against this, anyway, btw? There are plenty of obvious benefits to extending the existing standard, and I'm not the only person who's pointed out that a string-oriented approach doesn't allow them to do things they want to do. There's also been scarcely any argument that you can't trivially wrap string-oriented templates as WSGI resources. So the question then boils down to this: who should be inconvenienced and what capabilities should be crippled, and by how much? Your position calls for inconveniencing nobody, but crippling any frameworks or templates that are more powerful or flexible than the ones you favor. Mine calls for a mild inconvenience to any framework that doesn't doesn't currently support embedded WSGI, but it cripples *nobody*, and encourages frameworks to *add* entirely new capabilities while enabling new kinds of WSGI-based tools. From cce at clarkevans.com Wed Feb 1 23:42:26 2006 From: cce at clarkevans.com (Clark C. Evans) Date: Wed, 1 Feb 2006 17:42:26 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> References: <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> Message-ID: <20060201224226.GA16631@prometheusresearch.com> On Wed, Feb 01, 2006 at 05:15:26PM -0500, Phillip J. Eby wrote: | The "template" doesn't get to control the status or headers if all it | can do is return is a string. WSGI the hammer! Must we necessarly have a nail here? I think what's missing in this discussion is that templates are often used to build but a small part of an result; a left toolbar, a menu, perhaps even a row in a table -- but no more. I think that the requirements and design constraints involved make them so different from full-fledged WSGI applications/filters that even thinking about them in WSGI terms isn't helpful. That said, I do like the idea of passing templates a WSGI ``environ`` in addition to the input data that they should render; however, everything else in WSGI doesn't seem particularly applicable. The "composition" patterns are completely different. Best, Clark From pje at telecommunity.com Thu Feb 2 00:52:24 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 01 Feb 2006 18:52:24 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <20060201224226.GA16631@prometheusresearch.com> References: <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> At 05:42 PM 2/1/2006 -0500, Clark C. Evans wrote: >| The "template" doesn't get to control the status or headers if all it >| can do is return is a string. > >WSGI the hammer! Must we necessarly have a nail here? Who's "we"? Strings don't meet *my* use cases, or those of other template system and application developers who are not necessarily here to defend themselves against the promulgation of a standard that will exclude or marginalize them, or who have been here but haven't figured out yet how the exclusion will affect them, or what opportunities will be lost by going down this road. (Or who have already given up and left, as it appears Jim may have done, though it's also likely he's just busy at the moment.) Anyway, for the "we" that I'm a member of, yes, there is definitely a nail. >I think what's missing in this discussion is that templates are often >used to build but a small part of an result; a left toolbar, a menu, >perhaps even a row in a table -- but no more. Actually, I've explicitly covered this in my most recent messages, even giving an example of a templating system that already supports dual usage of "page" and "fragment" modes, where the page mode is more like WSGI and the fragment mode is more like strings for the *same* template, so it can be used as a page *or* in embedded form. > I think that the >requirements and design constraints involved make them so different from >full-fledged WSGI applications/filters that even thinking about them in >WSGI terms isn't helpful. Neither is it harmful. I'm not proposing that subtemplates be required to use WSGI, simply because most templating systems don't offer much heterogeneity in what their subtemplates can consist of, especially for layouts ("macros" in ZPT terms, or "template inheritance" in many other systems' terms). But, going the opposite way, to string-centrism, *will* be harmful. Even the example of PHP or ASP should be sufficient to show that some templates *need* to have the option of controlling their response. A standard that ignored this would clearly be broken. I'm beginning to think that I'm going to have to come up with a better metaphor to explain what I'm seeing here. Focusing on plugging string templates and aggregation is like trying to design a standard for USB-based power when we don't have *wall current yet*. Sure, it's nice for powering all your little mice and keyboards and such, but you're never going to be able to plug a dishwasher into it. You can always transform *down* from wall current to run a peripheral, or to power a USB hub, but if all you have is USB power, everybody who needs household appliances is out of luck. Ergo, it makes more sense to standardize wall current (WSGI embedding) *first*, and then move down to more fine-grained concerns like powering USB hubs (sub-template linkage in homogeneous or heterogeneous template systems). So far, the whole of the arguments presented *against* the WSGI embedding approach seem to be that I've proposed something that has *too much* ability. What a lousy no-good person I am, proposing things that accomplish more and allow more templating systems to participate in the standard. Shame on me! ;) Seriously, though, does anybody have any arguments against WSGI embedding *besides* it requiring a little extra wrapping in certain subtemplate scenarios? Anything that stacks up against the huge list of benefits it offers? Does anybody believe they have a way to support response control from templates without a kludged-on convention that's not part of the standard, and isn't just another WSGI in sheep's clothing? C'mon guys, where's the shooting down of my ideas that y'all promised? ;) From renesd at gmail.com Thu Feb 2 01:32:07 2006 From: renesd at gmail.com (Rene Dudfield) Date: Thu, 2 Feb 2006 11:32:07 +1100 Subject: [Web-SIG] Standardized template API In-Reply-To: <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> References: <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <20060201224226.GA16631@prometheusresearch.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> Message-ID: <64ddb72c0602011632y154d63c2n83cb13039f82ee28@mail.gmail.com> On 2/2/06, Phillip J. Eby wrote: > > C'mon guys, where's the shooting down of my ideas that y'all promised? ;) > Bang. From mike_mp at zzzcomputing.com Thu Feb 2 19:48:24 2006 From: mike_mp at zzzcomputing.com (Michael Bayer) Date: Thu, 2 Feb 2006 13:48:24 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> References: <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> Message-ID: <251B5A06-5026-48CE-A128-91C4777C41AE@zzzcomputing.com> is it possibly of value to just put some of these ideas into practice first without shooting any of them down ? I find it very hard to come up with broad-ranging specs, and then declare it as "thats the way it will be", without having a decent community of folks trying out the ideas and seeing how far they fly, having a spec be more of a "living document" that can iteratively settle down to something known to be solid, rather than a hypothetical notion that becomes a rule without being put through real-world paces. Can those of us who have worked less with "the high end" see some real examples of what they need, and how they will be marginalized by a template adapter API? Should those of us concerned about WSGI maybe try out a WSGI- centric idea and see how that goes ? Even though I am skeptical of it, I've hardly any concept of what it would even look like. The way it seems, there is a whole bunch of people playing with template languages and connecting them to view/controller frameworks, who want interoperability, and something has grown out of that which is definitely useful. Then there is a whole host of other issues being laid out which to me at least, arent so tangible, that such a spec is too limiting and shuts out a lot of important use cases...can we get tangible examples down of this ? can we get a clearer view, for those of us who cant necessarily work the entire problem space out in the hypothetical, of some real-world examples ? Discussions like these, which go on and on without too many tangibles, I think can suffer due to an excessive amount of abstraction which shuts out a lot of potentially valuable participants...there needs to be more code and less hand-waving. On Feb 1, 2006, at 6:52 PM, Phillip J. Eby wrote: > At 05:42 PM 2/1/2006 -0500, Clark C. Evans wrote: >> | The "template" doesn't get to control the status or headers if >> all it >> | can do is return is a string. >> >> WSGI the hammer! Must we necessarly have a nail here? > > Who's "we"? Strings don't meet *my* use cases, or those of other > template > system and application developers who are not necessarily here to > defend > themselves against the promulgation of a standard that will exclude or > marginalize them, or who have been here but haven't figured out yet > how the > exclusion will affect them, or what opportunities will be lost by > going > down this road. (Or who have already given up and left, as it > appears Jim > may have done, though it's also likely he's just busy at the moment.) > > Anyway, for the "we" that I'm a member of, yes, there is definitely > a nail. > > >> I think what's missing in this discussion is that templates are often >> used to build but a small part of an result; a left toolbar, a menu, >> perhaps even a row in a table -- but no more. > > Actually, I've explicitly covered this in my most recent messages, > even > giving an example of a templating system that already supports dual > usage > of "page" and "fragment" modes, where the page mode is more like > WSGI and > the fragment mode is more like strings for the *same* template, so > it can > be used as a page *or* in embedded form. > > >> I think that the >> requirements and design constraints involved make them so >> different from >> full-fledged WSGI applications/filters that even thinking about >> them in >> WSGI terms isn't helpful. > > Neither is it harmful. I'm not proposing that subtemplates be > required to > use WSGI, simply because most templating systems don't offer much > heterogeneity in what their subtemplates can consist of, especially > for > layouts ("macros" in ZPT terms, or "template inheritance" in many > other > systems' terms). > > But, going the opposite way, to string-centrism, *will* be > harmful. Even > the example of PHP or ASP should be sufficient to show that some > templates > *need* to have the option of controlling their response. A > standard that > ignored this would clearly be broken. > > I'm beginning to think that I'm going to have to come up with a better > metaphor to explain what I'm seeing here. Focusing on plugging string > templates and aggregation is like trying to design a standard for > USB-based > power when we don't have *wall current yet*. Sure, it's nice for > powering > all your little mice and keyboards and such, but you're never going > to be > able to plug a dishwasher into it. You can always transform *down* > from > wall current to run a peripheral, or to power a USB hub, but if all > you > have is USB power, everybody who needs household appliances is out > of luck. > > Ergo, it makes more sense to standardize wall current (WSGI embedding) > *first*, and then move down to more fine-grained concerns like > powering USB > hubs (sub-template linkage in homogeneous or heterogeneous template > systems). > > So far, the whole of the arguments presented *against* the WSGI > embedding > approach seem to be that I've proposed something that has *too much* > ability. What a lousy no-good person I am, proposing things that > accomplish more and allow more templating systems to participate in > the > standard. Shame on me! ;) > > Seriously, though, does anybody have any arguments against WSGI > embedding > *besides* it requiring a little extra wrapping in certain subtemplate > scenarios? Anything that stacks up against the huge list of > benefits it > offers? Does anybody believe they have a way to support response > control > from templates without a kludged-on convention that's not part of the > standard, and isn't just another WSGI in sheep's clothing? > > C'mon guys, where's the shooting down of my ideas that y'all > promised? ;) > > _______________________________________________ > Web-SIG mailing list > Web-SIG at python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: http://mail.python.org/mailman/options/web-sig/mike_mp% > 40zzzcomputing.com From pje at telecommunity.com Thu Feb 2 20:49:24 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 02 Feb 2006 14:49:24 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <251B5A06-5026-48CE-A128-91C4777C41AE@zzzcomputing.com> References: <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> At 01:48 PM 2/2/2006 -0500, Michael Bayer wrote: >Can those of us who >have worked less with "the high end" see some real examples of what >they need, and how they will be marginalized by a template adapter >API? Have you ever used PHP? ASP? DTML? Specifically, have you ever used them to generate any content-type other than HTML? This is a trivial example of a use case that a vars'n'strings approach doesn't support, without basically reinventing a new protocol that duplicates WSGI. I totally understand that for the frameworks where the current vars'n'strings proposal originated, this is a non-issue because they always have Python code between the framework and the template. But there are plenty of frameworks (and templating engines) where what you want is to *just have a template* in order to implement a URL, either as a method of a published object, or just as an "active page". The vars'n'strings approach can't do active pages, the WSGI approach can. >Should those of us concerned about WSGI maybe try out a WSGI- centric idea >and see how that goes ? Even though I am skeptical of >it, I've hardly any concept of what it would even look like. For frameworks like TurboGears that treat a template as a formatter of the return value of some Python code, the difference wouldn't be user-visible. And for frameworks, servers, or tools that use WSGI internally now (e.g. Paste, Routes, maybe Pylons?) there might not be any implementation to do at all, except to add support for loading/compiling resources. These tools can already run WSGI apps, they just need support to allow mapping URLs to templates. Tools that don't do WSGI yet internally would have to add a WSGI "server" component that creates an environ dict and add a start_response/write/iter facility. These aren't that complex, and if the framework is itself a WSGI application, it can simply pass through a modified version of the 'environ' it received from the original WSGI server, along with the 'start_response' it received. It then has to simply return the template's return value to the WSGI server. In short, the framework just has to act like WSGI middleware. A trivial example: def framework_app(environ, start_response): # framework-specific code to parse environ and figure out # where the URL points to, updating SCRIPT_NAME and # PATH_INFO as it goes, so SCRIPT_NAME points to the template # when done, while adding any framework-specific variables to # environ. if found_a_template: # framework-specific code to figure out what template # engine the template uses # ask the engine to load a WSGI app from a file # (or wherever) app = engine.compile_stream(open(filename,'r')) # Run the app against the original environ/start response # and return its return value, unless the framework needs to # alter the output some way (in which case it should do normal # WSGI middleware stuff). return app(environ, start_response) Now, if you compare this to what you have in the case of vars'n'strings, you'll notice that one thing this code is *missing* is the code to set a default content type, status, etc., while dumping out the "string". In the WSGI approach, that code moves to the *template* side of the fence. If the framework needs to add or modify headers, it can do so by passing in an altered start_response, e.g.: def my_start(status,headers): # framework-specific code here to munge headers return start_response(munged_status, munged_headers) return app(environ,my_start) Does this put more burden on framework developers? Probably so. It will be the most burdensome for frameworks that don't carry WSGI deep inside themselves, or whose APIs don't provide any way to do the things WSGI allows. They may have to write code that recreates WSGI from an internal request or response object, possibly containing already-parsed form data and cookies and such. Or, they may simply need to move their traversal or routing logic nearer to their WSGI input, so that any framework-specific request/response objects are created "just in time". Does this put any significant burden on template developers? Probably not. The most trivial template wrapper would consist of something like: def trivial_template_app(environ, start_response): output = ["some %(someVar)s text" % environ['wti.source']] start_response("200 OK", [('Content-type','text/html')]) return output Templating tools are then in a position to offer response-management API directly. For example, a ZPT wrapper could offer Zope-style REQUEST/RESPONSE objects for use in the template, because it would be dealing with WSGI. The single strongest argument that I think can be made *against* the WSGI approach is quite simply that I'm asking framework developers to help make their frameworks obsolete, by moving their request and response objects to the periphery of their frameworks, so that developers have not only a choice of template *syntax*, but also a choice of request/response APIs associated with those templates. Perhaps that's an unrealistic idea; maybe it's too soon to push for WSGI adoption phase 2 (the de-frameworkification of the frameworks). Maybe the only realistic consensus at this point in time is a niche API for vars'n'strings. Maybe the only way to showcase the power of what I'm proposing is to make WSGI wrappers for some template engines and implement the load/compile/save API I proposed on some server-side stuff. I just don't want to end up in the position of having spent time to write Yet Another Framework, as opposed to contributing code to a common standard. From ianb at colorstudy.com Fri Feb 3 02:08:46 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 02 Feb 2006 19:08:46 -0600 Subject: [Web-SIG] Standardized template API In-Reply-To: <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> References: <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> Message-ID: <43E2AD1E.30605@colorstudy.com> Phillip J. Eby wrote: > At 01:48 PM 2/2/2006 -0500, Michael Bayer wrote: > >>Can those of us who >>have worked less with "the high end" see some real examples of what >>they need, and how they will be marginalized by a template adapter >>API? > > > Have you ever used PHP? ASP? DTML? Specifically, have you ever used them > to generate any content-type other than HTML? This is a trivial example of > a use case that a vars'n'strings approach doesn't support, without > basically reinventing a new protocol that duplicates WSGI. Yes, I've used them, and they work pretty much like the example WSGI app I gave. There are functions -- callbacks from our perspective -- that let you set the headers. All the output is sent on to the browser, or perhaps returned as a string. Though usually there is no "return" value, it is all sent to the browser -- for instance, if you wanted to capture the output of a PHP template included from another template, I believe you have to essentially redirect output, run the other PHP script, then restore output and get the text that was captured. There's no particular support for capturing other information, like headers that were set. This sounds like a dumb design, you say? Sure, but we can be dumb-design-compatible! DTML allows capturing, but no special support for headers or metadata. And I guess it has the concept of a return value, but I suppose I've never understood or used that, since Python Scripts existed by the time I started using DTML, and they handled those kinds of cases better. I dunno, I just don't get it when it comes to template-as-WSGI-apps. And if it's gettable, I kinda feel like I should get it. So I'm not sure it's going to make sense to people. I still feel quite comfortable with the API I posted earlier, though I'm sure some additions would be good. It is close to the TG/Buffet API, but with a little inversion that I think allows (but does not specify) for intelligent resource resolution. I'm all for decomposing things into WSGI when it is appropriate. But I also think there's lots of complimentary APIs that aren't directly associated to WSGI. My motto: small standards, loosely joined. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From pje at telecommunity.com Fri Feb 3 04:07:58 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 02 Feb 2006 22:07:58 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <43E2ACD4.1060501@imagescape.com> References: <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060202220152.02221ba8@mail.telecommunity.com> At 07:07 PM 2/2/2006 -0600, Ian Bicking wrote: >Phillip J. Eby wrote: >>At 01:48 PM 2/2/2006 -0500, Michael Bayer wrote: >> >>>Can those of us who >>>have worked less with "the high end" see some real examples of what >>>they need, and how they will be marginalized by a template adapter >>>API? >> >>Have you ever used PHP? ASP? DTML? Specifically, have you ever used >>them to generate any content-type other than HTML? This is a trivial >>example of a use case that a vars'n'strings approach doesn't support, >>without basically reinventing a new protocol that duplicates WSGI. > >Yes, I've used them, and they work pretty much like the example WSGI app >I gave. There are functions -- callbacks from our perspective -- that >let you set the headers. All the output is sent on to the browser, or >perhaps returned as a string. Though usually there is no "return" >value, it is all sent to the browser -- for instance, if you wanted to >capture the output of a PHP template included from another template, I >believe you have to essentially redirect output, run the other PHP >script, then restore output and get the text that was captured. There's >no particular support for capturing other information, like headers that >were set. This sounds like a dumb design, you say? Sure, but we can be >dumb-design-compatible! > >DTML allows capturing, but no special support for headers or metadata. >And I guess it has the concept of a return value, but I suppose I've >never understood or used that, since Python Scripts existed by the time >I started using DTML, and they handled those kinds of cases better. I don't understand what you're trying to say here at all. What's all this stuff about return values? My only reference to return values was WSGI application return iterables, which are an implementation detail and nothing to do with some sort of template return value. >I dunno, I just don't get it when it comes to template-as-WSGI-apps. >And if it's gettable, I kinda feel like I should get it. So I'm not sure >it's going to make sense to people. Trying to keep it as simple as possible: Step 1. Pretend that Apache could run DTML pages from .dtml files, using the Zope request/response API to look at request contents, or a PHP/ASP API for manipulating headers or request data. Step 2. Take away Apache and replace it with WSGI. Take away DTML and the Zope API, and replace it with your choice of template language and associated request/response API. Voila. You can now run arbitrary active resources in any framework that supports WSGI embedding. And that would include frameworks like TurboGears, they'd just be putting additional data in the WSGI request fed to the template engine. From ben at groovie.org Fri Feb 3 20:17:59 2006 From: ben at groovie.org (Ben Bangert) Date: Fri, 3 Feb 2006 11:17:59 -0800 Subject: [Web-SIG] Standardized template API In-Reply-To: <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> References: <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> Message-ID: <8944CE8B-B4D5-477A-A5C9-B9263E871BBE@groovie.org> On Feb 2, 2006, at 11:49 AM, Phillip J. Eby wrote: > For frameworks like TurboGears that treat a template as a formatter > of the > return value of some Python code, the difference wouldn't be > user-visible. And for frameworks, servers, or tools that use WSGI > internally now (e.g. Paste, Routes, maybe Pylons?) there might not > be any > implementation to do at all, except to add support for loading/ > compiling > resources. These tools can already run WSGI apps, they just need > support > to allow mapping URLs to templates. Since Pylons is just using Myghty, it still has full access to the WSGI call, so does RhubarbTart and Paste, so we'd all be set for this style. Recent discussion/patches to CherryPy indicate that they're putting in support to retain the full WSGI environ/start_response stuff so they'd be able to use it as well. > # framework-specific code to parse environ and figure out > # where the URL points to, updating SCRIPT_NAME and > # PATH_INFO as it goes, so SCRIPT_NAME points to the template > # when done, while adding any framework-specific variables to > # environ My main concern here, though its likely just a detail I'm not seeing. When you call the template engine with the WSGI environ, I'm a bit worried about URL generation. For example, Routes is WSGI aware, and if it sees SCRIPT_NAME, it'll prepend it to generated URL's. When you render templates, the template name in many cases has nothing to do with anything present in the URL. The URL might be / comment/view/4, and the template rendered relative to the template root is '/viewers/comment.myt'. Myghty already has a WSGI handler, and has its own internal response/request objects, so its already suited quite well for the interface it sounds like you're proposing. However, to tell Myghty I'd want that path rendered, I'd have to set PATH_INFO to '/viewers/comment.myt'. In this case, its possible Routes could still generate proper URL's as long as the SCRIPT_NAME only contained whatever was necessary to reach the WSGI app that ran the Route match. If SCRIPT_NAME is further messed with on the way to the template engine, the generated URL would be broken. For this reason, I think sending the request to a template engine via WSGI is fine, except SCRIPT_NAME should not be touched, since thats still where the "application" responding is located. Also, since PATH_INFO is now referring only to the template to be rendered, its un-reliable for purposes of determining the full URL. Actually, now that I think about it, if we want to use WSGI for the template engine, I think it'd better to put additional keys in the environ for what template path to render, etc. This way vital information for URL generation isn't altered in ways that might result in broken URL's. > def trivial_template_app(environ, start_response): > output = ["some %(someVar)s text" % environ['wti.source']] > start_response("200 OK", [('Content-type','text/html')]) > return output Looks fine to me, except for the concerns I cited above for how the template engine should determine the template to render. > Templating tools are then in a position to offer response- > management API > directly. For example, a ZPT wrapper could offer Zope-style > REQUEST/RESPONSE objects for use in the template, because it would be > dealing with WSGI. Yep, makes sense to me, especially since Myghty already does this. > The single strongest argument that I think can be made *against* > the WSGI > approach is quite simply that I'm asking framework developers to > help make > their frameworks obsolete, by moving their request and response > objects to > the periphery of their frameworks, so that developers have not only a > choice of template *syntax*, but also a choice of request/response > APIs > associated with those templates. Excellent. :) Cheers, Ben From pje at telecommunity.com Fri Feb 3 20:57:44 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 03 Feb 2006 14:57:44 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <8944CE8B-B4D5-477A-A5C9-B9263E871BBE@groovie.org> References: <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> At 11:17 AM 2/3/2006 -0800, Ben Bangert wrote: >On Feb 2, 2006, at 11:49 AM, Phillip J. Eby wrote: > >>For frameworks like TurboGears that treat a template as a formatter >>of the >>return value of some Python code, the difference wouldn't be >>user-visible. And for frameworks, servers, or tools that use WSGI >>internally now (e.g. Paste, Routes, maybe Pylons?) there might not >>be any >>implementation to do at all, except to add support for loading/ compiling >>resources. These tools can already run WSGI apps, they just need >>support >>to allow mapping URLs to templates. > >Since Pylons is just using Myghty, it still has full access to the >WSGI call, so does RhubarbTart and Paste, so we'd all be set for this >style. Recent discussion/patches to CherryPy indicate that they're >putting in support to retain the full WSGI environ/start_response >stuff so they'd be able to use it as well. Great. >> # framework-specific code to parse environ and figure out >> # where the URL points to, updating SCRIPT_NAME and >> # PATH_INFO as it goes, so SCRIPT_NAME points to the template >> # when done, while adding any framework-specific variables to >> # environ > >My main concern here, though its likely just a detail I'm not seeing. >When you call the template engine with the WSGI environ, I'm a bit >worried about URL generation. For example, Routes is WSGI aware, and >if it sees SCRIPT_NAME, it'll prepend it to generated URL's. > >When you render templates, the template name in many cases has >nothing to do with anything present in the URL. The URL might be / >comment/view/4, and the template rendered relative to the template >root is '/viewers/comment.myt'. Myghty already has a WSGI handler, >and has its own internal response/request objects, so its already >suited quite well for the interface it sounds like you're proposing. >However, to tell Myghty I'd want that path rendered, I'd have to set >PATH_INFO to '/viewers/comment.myt'. I think maybe there's some confusion here as to what I'm suggesting SCRIPT_NAME be set to. I probably should've said that SCRIPT_NAME should represent the "consumed" part of the URL, and PATH_INFO should be empty unless there's some unconsumed path portion. So if the URL path is /comment/view/4, then SCRIPT_NAME should be /comment/view/4 appended to whatever the original SCRIPT_NAME was, and PATH_INFO would be empty. The purpose of this is just that some frameworks can potentially have unprocessed path segments following the URL that led to rendering of a template, and SCRIPT_NAME and PATH_INFO should reflect this, just as though the template were itself a WSGI application. (Because it is one, and some "templates" may actually be whole applications "mounted" at the template location.) >In this case, its possible Routes could still generate proper URL's >as long as the SCRIPT_NAME only contained whatever was necessary to >reach the WSGI app that ran the Route match. If SCRIPT_NAME is >further messed with on the way to the template engine, the generated >URL would be broken. > >For this reason, I think sending the request to a template engine via >WSGI is fine, except SCRIPT_NAME should not be touched, since thats >still where the "application" responding is located. Also, since >PATH_INFO is now referring only to the template to be rendered, its >un-reliable for purposes of determining the full URL. My suggestion would be to add an extra WSGI key, maybe 'wsgi.application_root' to represent the "original application SCRIPT_NAME" for frameworks that have such a concept. Templates using Routes could then use that variable in place of SCRIPT_NAME. It seems to me that Zope request/response objects also need this information, in order to generate the magic URL0-URL9 and other such variables. The application root should of course be set at the entry point of the framework, so in the case of Routes, Routes could simply copy SCRIPT_NAME to application_root in environ if there isn't one already set. It could then simply use application_root first when generating URLs, and for that matter it could add extension APIs to the environ to allow accessing Routes APIs from embedded apps. >Actually, now that I think about it, if we want to use WSGI for the >template engine, I think it'd better to put additional keys in the >environ for what template path to render, etc. This way vital >information for URL generation isn't altered in ways that might >result in broken URL's. I'd like to preserve the normal WSGI invariants instead, because this allows entire mini-applications with their own URL space to be embedded as "templates" within a containing application. Such apps can then rely on SCRIPT_NAME as being their root, even if they weren't originally written for embedding. I don't see a use for including a template path anywhere, and that really wasn't what I was suggesting. Locating templates' implementation data is a job for the enclosing framework, and it's not really the template's business to know about that. >> def trivial_template_app(environ, start_response): >> output = ["some %(someVar)s text" % environ['wti.source']] >> start_response("200 OK", [('Content-type','text/html')]) >> return output > >Looks fine to me, except for the concerns I cited above for how the >template engine should determine the template to render. The template engine wouldn't; the framework simply hands the engine the "template" source (as a string or a stream) and gets back an app object like the above. See my "trivial counterproposal API" for a summary of the engine interface. As we flesh out these details, I'm realizing that what I'm actually proposing might be best described as a "WSGI Embedding" standard, that describes how a framework can use an "application factory" to obtain a WSGI app from a string or stream. Wrapping a template engine as an application factory makes it possible to embed active pages in any framework that supports WSGI embedding, and supporting WSGI embedding in a framework lets you run any WSGI application inside it that has an application factory implementation. So, we're basically solving the old "How do you deploy/configure a WSGI app?" question, not by having a single file format for deployment, but by allowing pluggable file formats, some of which just happen to be "templates". Meanwhile, view-oriented frameworks like Zope 3 and peak.web can basically embed entire applications or sub-applications wrapping published objects, and even simple servers like the mod_python WSGI server could serve templates and applications straight from the file system. And even controller-style frameworks that use templates strictly for postprocessing will end up being able to embed fairly sophisticated things. >>Templating tools are then in a position to offer response- management API >>directly. For example, a ZPT wrapper could offer Zope-style >>REQUEST/RESPONSE objects for use in the template, because it would be >>dealing with WSGI. > >Yep, makes sense to me, especially since Myghty already does this. > >>The single strongest argument that I think can be made *against* >>the WSGI >>approach is quite simply that I'm asking framework developers to >>help make >>their frameworks obsolete, by moving their request and response >>objects to >>the periphery of their frameworks, so that developers have not only a >>choice of template *syntax*, but also a choice of request/response >>APIs >>associated with those templates. > >Excellent. :) When WSGI was created, I had no idea that people would come up with cool stuff like Ian's debugging and lint middleware components, so I'm really excited thinking about what kinds of things people will be able to do with WSGI embedding. From ben at groovie.org Fri Feb 3 22:34:59 2006 From: ben at groovie.org (Ben Bangert) Date: Fri, 3 Feb 2006 13:34:59 -0800 Subject: [Web-SIG] Standardized template API In-Reply-To: <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> References: <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> Message-ID: <3330749F-9EC1-4101-8477-0E0B9512AEA0@groovie.org> On Feb 3, 2006, at 11:57 AM, Phillip J. Eby wrote: > I think maybe there's some confusion here as to what I'm suggesting > SCRIPT_NAME be set to. I probably should've said that SCRIPT_NAME > should represent the "consumed" part of the URL, and PATH_INFO > should be empty unless there's some unconsumed path portion. So if > the URL path is /comment/view/4, then SCRIPT_NAME should be / > comment/view/4 appended to whatever the original SCRIPT_NAME was, > and PATH_INFO would be empty. What would be an example where PATH_INFO would not be empty? > My suggestion would be to add an extra WSGI key, maybe > 'wsgi.application_root' to represent the "original application > SCRIPT_NAME" for frameworks that have such a concept. Templates > using Routes could then use that variable in place of SCRIPT_NAME. > It seems to me that Zope request/response objects also need this > information, in order to generate the magic URL0-URL9 and other > such variables. > > The application root should of course be set at the entry point of > the framework, so in the case of Routes, Routes could simply copy > SCRIPT_NAME to application_root in environ if there isn't one > already set. It could then simply use application_root first when > generating URLs, and for that matter it could add extension APIs to > the environ to allow accessing Routes APIs from embedded apps. That would work fine for me, and is a rather trivial Routes tweak as well. > I'd like to preserve the normal WSGI invariants instead, because > this allows entire mini-applications with their own URL space to be > embedded as "templates" within a containing application. Such apps > can then rely on SCRIPT_NAME as being their root, even if they > weren't originally written for embedding. Sure, again, I see no issue with that, though I'm sure many might wonder why you'd want to. :) > I don't see a use for including a template path anywhere, and that > really wasn't what I was suggesting. Locating templates' > implementation data is a job for the enclosing framework, and it's > not really the template's business to know about that. Wait, then how does the template engine know what template to render? > The template engine wouldn't; the framework simply hands the engine > the "template" source (as a string or a stream) and gets back an > app object like the above. See my "trivial counterproposal API" > for a summary of the engine interface. Ok, so in my case, I'd pass the Myghty template engine the string indicating the template I want rendered. Mainly because Myghty has its own powerful compiling/caching system for templates, I wouldn't want to try loading them myself and passing them as streams. But I see how this affords that flexibility should you want this. > As we flesh out these details, I'm realizing that what I'm actually > proposing might be best described as a "WSGI Embedding" standard, > that describes how a framework can use an "application factory" to > obtain a WSGI app from a string or stream. Wrapping a template > engine as an application factory makes it possible to embed active > pages in any framework that supports WSGI embedding, and supporting > WSGI embedding in a framework lets you run any WSGI application > inside it that has an application factory implementation. Sweet, I read that twice and got my eyes to bug out. The interface is flexible with WSGI, leaves lots of options open, I like... :) > So, we're basically solving the old "How do you deploy/configure a > WSGI app?" question, not by having a single file format for > deployment, but by allowing pluggable file formats, some of which > just happen to be "templates". > > Meanwhile, view-oriented frameworks like Zope 3 and peak.web can > basically embed entire applications or sub-applications wrapping > published objects, and even simple servers like the mod_python WSGI > server could serve templates and applications straight from the > file system. And even controller-style frameworks that use > templates strictly for postprocessing will end up being able to > embed fairly sophisticated things. Ok, so.... I'm going to try and summarize to ensure that I have a firm grasp on the proposal. 1) The template spec will use a WSGI call to ask the template engine to return a WSGI application (iterable) of the rendered template (resource). 2) The spec shall declare 2 environ keys to be used: 2a) wti.source - either a stream or string which is the template source (source as in code, or as in the name of the template is up to the engine) 2b) wsgi.application_root - shall be used to access the original SCRIPT_NAME if desired, ie, the SCRIPT_NAME of the application calling the template engine That's it. For a template language like Myghty, it already has a WSGI handler that parses the template to render given the PATH_INFO, so I'd merely sub-class its WSGIHandler to have it use wti.source as the name of the template to render. Now, there's a few things this doesn't touch at all, that the TurboGears plug-in does address. Because wti.source is left open as either a string, or a stream, and the meaning of said string could be that its a string to be rendered as if it was a template, or it could be as I'm used it here, as the string representing what template Myghty should search for and render. Though a small layer on top of the spec could handle it in whatever way it preferred to, definitely a lot of flexibility involved. A common question given the minimality of the spec as I'm summarizing would be, "How do I tell my framework to pass a dict of variables into the template so it can use them?" This would probably be best done as adding a key to the environ. So I propose in addition to the 2 keys: 2c) wti.vars - (optional) A dict representing variables you'd like access to in the template should it provide ways to make them available To configure said template language implementing this spec would differ slightly depending on what options the template language needs. So each template language should include a small bit on what the minimum configuration required to use it looks like. For Myghty it might look like: # setup the myghty engine from myghty.wsgi import engine myghtyengine = engine(template_root='/some/path/to/templates') # render a template def render_myghty(template): environ['wti-source'] = template environ['wti-vars'] = myvars environ['wsgi.application_root'] = environ['SCRIPT_NAME'] environ['SCRIPT_NAME'] += environ['PATH_INFO'] environ['PATH_INFO'] = '' return myghtyengine(environ, start_response) So, the scope of this proposal is to cover the basics that the template engine has access to, so that it can do what it will with them. For those looking like a 'higher-level' interface like the TurboGears template plug-in, it'd be very trivial to make a higher abstraction based on this that can sort out differences in what template languages "want" when it comes to configuration. Anyways, hopefully this helps. Assuming I'm interpreting the WSGI as template spec thing properly, I think it'll work great, and provide a solid base for powerful template languages, as well as more minimal ones (they can use a small WSGI wrapper). Cheers, Ben From ben at groovie.org Fri Feb 3 22:45:37 2006 From: ben at groovie.org (Ben Bangert) Date: Fri, 3 Feb 2006 13:45:37 -0800 Subject: [Web-SIG] Standardized template API In-Reply-To: <3330749F-9EC1-4101-8477-0E0B9512AEA0@groovie.org> References: <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> <3330749F-9EC1-4101-8477-0E0B9512AEA0@groovie.org> Message-ID: On Feb 3, 2006, at 1:34 PM, Ben Bangert wrote: > # setup the myghty engine > from myghty.wsgi import engine > myghtyengine = engine(template_root='/some/path/to/templates') > > # render a template > def render_myghty(template): > environ['wti-source'] = template > environ['wti-vars'] = myvars > environ['wsgi.application_root'] = environ['SCRIPT_NAME'] > environ['SCRIPT_NAME'] += environ['PATH_INFO'] > environ['PATH_INFO'] = '' > return myghtyengine(environ, start_response) Thought I'd ammend this. Since the engine would implement the Template Spec, it'd do what it needs to inside, so a realistic example would be: # render a template environ['wti-source'] = template environ['wti-vars'] = myvars return myghtyengine(environ, start_response) And of course the implementation details of how it plays with environ are left up to it. Someone who wanted something even easier could just have a wrapper on that call in their framework, etc. Cheers, Ben From pje at telecommunity.com Fri Feb 3 23:24:23 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 03 Feb 2006 17:24:23 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <3330749F-9EC1-4101-8477-0E0B9512AEA0@groovie.org> References: <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060203164932.02156ce0@mail.telecommunity.com> At 01:34 PM 2/3/2006 -0800, Ben Bangert wrote: >On Feb 3, 2006, at 11:57 AM, Phillip J. Eby wrote: > >>I think maybe there's some confusion here as to what I'm suggesting >>SCRIPT_NAME be set to. I probably should've said that SCRIPT_NAME >>should represent the "consumed" part of the URL, and PATH_INFO >>should be empty unless there's some unconsumed path portion. So if >>the URL path is /comment/view/4, then SCRIPT_NAME should be / >>comment/view/4 appended to whatever the original SCRIPT_NAME was, >>and PATH_INFO would be empty. > >What would be an example where PATH_INFO would not be empty? When the "template" is responsible for the URL space underneath it. This is more of a use case for "active page" frameworks, and for application embedding scenarios. >>I'd like to preserve the normal WSGI invariants instead, because >>this allows entire mini-applications with their own URL space to be >>embedded as "templates" within a containing application. Such apps >>can then rely on SCRIPT_NAME as being their root, even if they >>weren't originally written for embedding. > >Sure, again, I see no issue with that, though I'm sure many might >wonder why you'd want to. :) It lets you create mount points from one WSGI app/server into another app. Like having a configuration file that Apache "serves" by treating the app's config file as a "template". >>I don't see a use for including a template path anywhere, and that >>really wasn't what I was suggesting. Locating templates' >>implementation data is a job for the enclosing framework, and it's >>not really the template's business to know about that. > >Wait, then how does the template engine know what template to render? The framework gives the engine the source of the template, and asks for a WSGI app object in return. The framework is allowed to hold on to this application object and reuse it as often as desired. >>The template engine wouldn't; the framework simply hands the engine >>the "template" source (as a string or a stream) and gets back an >>app object like the above. See my "trivial counterproposal API" >>for a summary of the engine interface. > >Ok, so in my case, I'd pass the Myghty template engine the string >indicating the template I want rendered. Mainly because Myghty has >its own powerful compiling/caching system for templates, I wouldn't >want to try loading them myself and passing them as streams. But I >see how this affords that flexibility should you want this. Well, the idea I was trying to do here was to allow the embedder to control how the data is stored. I think of this as a bit like OLE embedding, where the enclosing application just hands off a stream to the class that implements the embedded object or document. What I'd suggest is that this is actually an opportunity for Myghty to put its lookup and caching on one side of the interface, and an actual template rendering facility on the other side. That is, treat Myghty as an embedd*er* rather than the embedd*ee*, so that the engine can be used with arbitrary apps or templates on the embedded side. I'm not really familiar with Myghty, though, so I might not be saying something sensible here. I'm just saying that if you have a mechanism for locating, compiling, and caching templates, then it should live on the framework side of the "engine" interface. An embedding engine offers the "compile_string, compile_stream, write_compiled, read_compiled" methods, and your locator/compiler/cache system would just call down to those instead of hardwiring the implementation of compiling and serializing. You would of course have to add in a way to tell what embedding engine to select in that case, but now Myghty itself (if I understand you correctly) becomes a WSGI embedding host that can run any kind of template using its lookup/cache/etc. facilities. Does that make sense? >>So, we're basically solving the old "How do you deploy/configure a >>WSGI app?" question, not by having a single file format for >>deployment, but by allowing pluggable file formats, some of which >>just happen to be "templates". >> >>Meanwhile, view-oriented frameworks like Zope 3 and peak.web can >>basically embed entire applications or sub-applications wrapping >>published objects, and even simple servers like the mod_python WSGI >>server could serve templates and applications straight from the >>file system. And even controller-style frameworks that use >>templates strictly for postprocessing will end up being able to >>embed fairly sophisticated things. > >Ok, so.... I'm going to try and summarize to ensure that I have a >firm grasp on the proposal. > >1) The template spec will use a WSGI call to ask the template engine >to return a WSGI application (iterable) of the rendered template >(resource). No; what happens is that the framework locates the serialized form of the template it wants to use, and requests that a specific engine deserialize it to get a WSGI app (callable, not iterable). The framework is allowed to cache the returned app object and reuse it for multiple requests. The engine doesn't get any data about the current request. The app object is used to "render" the template. It's just called like a normal WSGI app object. >2) The spec shall declare 2 environ keys to be used: > 2a) wti.source - either a stream or string which is the >template source (source as in code, or as in the name of the template >is up to the engine) The idea of wti.source was that it would be either the "published object" or the "vars", depending on the kind of framework that was involved. It's whatever is the subject of the request. For "active page" style frameworks, there would be no object. I don't think that source is the best name for this, though; it should probably be "target" or "param" or something like that. Hell, maybe it shouldn't be just one key; maybe there should be a "params" key and a "self" key, such that a given framework can define one or both. That way, the template implementations will know what they're dealing with. > 2b) wsgi.application_root - shall be used to access the original >SCRIPT_NAME if desired, ie, the SCRIPT_NAME of the application >calling the template engine Yeah, this needs to be nailed down a bit more clearly, because some frameworks don't have such a concept. For example, in a pure "active page" model running under mod_python there is really no "calling application", so there would be no application_root set. >That's it. For a template language like Myghty, it already has a WSGI >handler that parses the template to render given the PATH_INFO, so >I'd merely sub-class its WSGIHandler to have it use wti.source as the >name of the template to render. Hrm. Well, it's a bit different than that, as you can see. I guess we'll revisit this after you've read the above notes. >Now, there's a few things this doesn't touch at all, that the >TurboGears plug-in does address. Because wti.source is left open as >either a string, or a stream, and the meaning of said string could be >that its a string to be rendered as if it was a template, or it could >be as I'm used it here, as the string representing what template >Myghty should search for and render. This actually isn't an issue; the engine interface has 'compile_string' and 'compile_stream' methods, and wti.source has nothing to do with it. See my earlier post at: http://mail.python.org/pipermail/web-sig/2006-February/001933.html >Though a small layer on top of the spec could handle it in whatever >way it preferred to, definitely a lot of flexibility involved. A >common question given the minimality of the spec as I'm summarizing >would be, "How do I tell my framework to pass a dict of variables >into the template so it can use them?" > >This would probably be best done as adding a key to the environ. So I >propose in addition to the 2 keys: > 2c) wti.vars - (optional) A dict representing variables you'd >like access to in the template should it provide ways to make them >available Yeah, this is actually what the wti.source was supposed to be, as you see above. >Anyways, hopefully this helps. Assuming I'm interpreting the WSGI as >template spec thing properly, I think it'll work great, and provide a >solid base for powerful template languages, as well as more minimal >ones (they can use a small WSGI wrapper). I think you'll find that the actual design is even better than the one you interpreted it as, so I think we're doing good. :) I also think I'm really going to need to carve out some time soon to write up a proper spec, or at least something a lot closer to one than the current tangled threads of explanation. It's all pretty clear in my mind, but that's mainly because I have only a *vague* understanding of most frameworks, so I know how little I can rely on things working a particular way. Thus, my design is such that it doesn't rely on much of anything about either the frameworks or templates; it just draws a sharp dividing line between three distinct sets of responsibilities: the embedding host, the embedded app, and the application factory. These divisions may cut things at slightly different points than what people were originally thinking. For example, it sounds from your description like Myghty might have parts in all three areas. So, if it were to implement WSGI embedding, it would become componentized along those lines, making each part potentially interchangeable or reusable. I appreciate your feedback and questions a lot, as they're helping me improve how I communicate the ideas. Hopefully, I'll soon be able to explain it with examples and metaphors so that it doesn't take this much back-and-forth. Despite Ian's fears, I'm not really worried that it's too complex; it's hugely simpler than WSGI itself, and is probably less complex than even the idea of WSGI middleware. I am just not that good at *communicating* it simply yet. From ben at groovie.org Sat Feb 4 00:01:43 2006 From: ben at groovie.org (Ben Bangert) Date: Fri, 3 Feb 2006 15:01:43 -0800 Subject: [Web-SIG] Standardized template API In-Reply-To: <5.1.1.6.0.20060203164932.02156ce0@mail.telecommunity.com> References: <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> <5.1.1.6.0.20060203164932.02156ce0@mail.telecommunity.com> Message-ID: <99FE035C-0288-4697-9562-75E1F6767D8B@groovie.org> On Feb 3, 2006, at 2:24 PM, Phillip J. Eby wrote: >> Wait, then how does the template engine know what template to render? > > The framework gives the engine the source of the template, and asks > for a WSGI app object in return. The framework is allowed to hold > on to this application object and reuse it as often as desired. I'm really not sure why the framework developer should be further burdened with trying to cache templates. Putting additional demands on the framework developer will create such a burden that I'd highly prefer to use TurboGears plug-in system and ignore any spec requiring such additional effort. > Well, the idea I was trying to do here was to allow the embedder to > control how the data is stored. I think of this as a bit like OLE > embedding, where the enclosing application just hands off a stream > to the class that implements the embedded object or document. I don't understand that at all, but probably because I don't do OLE embedding and all that jazz. > What I'd suggest is that this is actually an opportunity for Myghty > to put its lookup and caching on one side of the interface, and an > actual template rendering facility on the other side. That is, > treat Myghty as an embedd*er* rather than the embedd*ee*, so that > the engine can be used with arbitrary apps or templates on the > embedded side. Myghty's lookup and caching is rather complex and requires significant synchronization efforts to ensure that under heavy load, if the template has changed, its only re-compiled once while the rest of the requests continue to use the prior compiled version until the updated one is ready. It's complex, and putting it on completely separate sides is beyond a nightmare of effort. The lookup mechanism it uses is also very complex, and it doesn't belong outside of Myghty. > I'm not really familiar with Myghty, though, so I might not be > saying something sensible here. I'm just saying that if you have a > mechanism for locating, compiling, and caching templates, then it > should live on the framework side of the "engine" interface. An > embedding engine offers the "compile_string, compile_stream, > write_compiled, read_compiled" methods, and your locator/compiler/ > cache system would just call down to those instead of hardwiring > the implementation of compiling and serializing. You would of > course have to add in a way to tell what embedding engine to select > in that case, but now Myghty itself (if I understand you correctly) > becomes a WSGI embedding host that can run any kind of template > using its lookup/cache/etc. facilities. Does that make sense? Yes, but its 100% incompatible with Myghty. For the flexibility and reloading functionality that Myghty provides, splitting such functionality up will only cripple features of Myghty that I really enjoy. It also seems to go far beyond the scope of merely having a standardized way of calling a template, as its now somehow going to handle caching it, and knowing when its ok to use a cached copy, vs a new one, etc. This might work for other template languages that merely render a single template and give you its output, but Myghty has things like automatic inheritance with its own internal resolver for assembling the fully rendered template in the context of its location. Moving this stuff onto the framework side reduces the power of the template language. I'm not sure about the other template systems, some of them do some inheritance stuff that might be affected as well. Also, Myghty has per-component (template) caching systems built-in which allow individual parts to be cached on separate criteria (maybe your side-bar refreshes every 10 seconds, and the page header every 2 minutes). Caching from outside would remove this ability since these fine-grained caching is only available when Myghty is handling its caching. While the thought that you can split off the lookup/cache functionality is nice in theory, the reality is that imposing such a system also reduces the flexibility and capabilities of the template language. For example, in Myghty (and I believe Kid, Cheetah, etc.), you can include and call other components (templates) from within one template. How does the template lookup the other template you're including? It has its own resolver. When it loads that other template inside a template, what if the other template should be cached? This kind of stuff won't work with what you're proposing as far as I can tell. If it can, the amount of work it will push onto every framework developer will likely doom such a proposal for the start, especially with something much simpler available (TG's template-plugin approach) > No; what happens is that the framework locates the serialized form > of the template it wants to use, and requests that a specific > engine deserialize it to get a WSGI app (callable, not iterable). > The framework is allowed to cache the returned app object and reuse > it for multiple requests. The engine doesn't get any data about > the current request. This kind of work sounds best suited for the template language to deal with. Especially since template languages differ on how they do it substantially, and some of them can't decouple the process from the actual render without reducing powerful functionality it affords. > The app object is used to "render" the template. It's just called > like a normal WSGI app object. Since Myghty allows template's and template calls within templates to be cached on varying keys and expiration times, this cannot be decoupled from the compile step. > I think you'll find that the actual design is even better than the > one you interpreted it as, so I think we're doing good. :) Not really, the one I interpreted kept vital steps together. Without those steps coupled closely, Myghty cannot be compatible without very significant changes to underlying structures. I'm not sure what work will be necessary for other template languages that compile and manage their own caches, but since it can be a very integrated process I'd imagine it'd be equally horrible. > These divisions may cut things at slightly different points than > what people were originally thinking. For example, it sounds from > your description like Myghty might have parts in all three areas. > So, if it were to implement WSGI embedding, it would become > componentized along those lines, making each part potentially > interchangeable or reusable. I thought Cheetah also compiled its templates and would reload them if it detected changes. If a template changes on disk, thats the great thing about template systems that handle stuff like this. You ask for the rendered template, and if it changes, it'll recompile/re- cache it, and render it for you. Decoupling this process removes a very powerful and convenient feature. One I'm not ready to part with. > I appreciate your feedback and questions a lot, as they're helping > me improve how I communicate the ideas. Hopefully, I'll soon be > able to explain it with examples and metaphors so that it doesn't > take this much back-and-forth. Despite Ian's fears, I'm not really > worried that it's too complex; it's hugely simpler than WSGI > itself, and is probably less complex than even the idea of WSGI > middleware. I am just not that good at *communicating* it simply yet. Perhaps, but if this spec is extremely difficult to implement (as is Myghty's case, and maybe Kid/Cheetah as well), or removes functionality from existing solutions, I don't see it going anywhere. It might not be complex for whatever template system you use, but its a drastically different situation for others. A pure WSGI call to a template engine like I thought you were referring to was more agnostic on how the template language deals with it, the key appeal and flexibility to me. Cheers, Ben From mike_mp at zzzcomputing.com Sat Feb 4 00:08:39 2006 From: mike_mp at zzzcomputing.com (Michael Bayer) Date: Fri, 3 Feb 2006 18:08:39 -0500 (EST) Subject: [Web-SIG] Standardized template API In-Reply-To: <5.1.1.6.0.20060203164932.02156ce0@mail.telecommunity.com> References: <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> <5.1.1.6.0.20060203164932.02156ce0@mail.telecommunity.com> Message-ID: <19836.66.192.34.8.1139008119.squirrel@www.geekisp.com> Phillip J. Eby wrote: > What I'd suggest is that this is actually an opportunity for Myghty to put > its lookup and caching on one side of the interface, and an actual > template > rendering facility on the other side. That is, treat Myghty as an > embedd*er* rather than the embedd*ee*, so that the engine can be used with > arbitrary apps or templates on the embedded side. > > I'm not really familiar with Myghty, though, so I might not be saying > something sensible here. I'm just saying that if you have a mechanism for > locating, compiling, and caching templates, then it should live on the > framework side of the "engine" interface. An embedding engine offers the > "compile_string, compile_stream, write_compiled, read_compiled" methods, > and your locator/compiler/cache system would just call down to those > instead of hardwiring the implementation of compiling and serializing. of course, this is sensible. i am not aware immediately of any major technical issues to componentizing the internals of myghty, there might be some; i doubt they are fundamnetally insurmoutnable. it would also make myghty a more solid system, allow its reloading functionality to be useable by other systems that can benefit from it, etc. but, to do this would be an intense royal pain in the ass and take several weeks/months, render new releases of myghty as less stable for awhile, etc. which leads me to believe that if this spec also came along with a salaried position +401K benefits for template system developers to spend full time re-architecting their systems, id say just great ! otherwise, might be a little ambitious for the near-to-medium term. From mike_mp at zzzcomputing.com Sat Feb 4 00:12:45 2006 From: mike_mp at zzzcomputing.com (Michael Bayer) Date: Fri, 3 Feb 2006 18:12:45 -0500 (EST) Subject: [Web-SIG] Standardized template API In-Reply-To: <19836.66.192.34.8.1139008119.squirrel@www.geekisp.com> References: <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> <5.1.1.6.0.20060203164932.02156ce0@mail.telecommunity.com> <19836.66.192.34.8.1139008119.squirrel@www.geekisp.com> Message-ID: <23477.66.192.34.8.1139008365.squirrel@www.geekisp.com> I would add that the locating/compiling system, like Ben is mentioning, would also need to be accessible from within the template engine; i.e. these components need to be linkable together in recursive chains. that is because most template engines have the ability to call sub-templates, and this is a core functionality of myghty...very nested and recursive, also keeps an internal stack frame going to track the whole thing. Michael Bayer wrote: > Phillip J. Eby wrote: >> What I'd suggest is that this is actually an opportunity for Myghty to >> put >> its lookup and caching on one side of the interface, and an actual >> template >> rendering facility on the other side. That is, treat Myghty as an >> embedd*er* rather than the embedd*ee*, so that the engine can be used >> with >> arbitrary apps or templates on the embedded side. >> >> I'm not really familiar with Myghty, though, so I might not be saying >> something sensible here. I'm just saying that if you have a mechanism >> for >> locating, compiling, and caching templates, then it should live on the >> framework side of the "engine" interface. An embedding engine offers >> the >> "compile_string, compile_stream, write_compiled, read_compiled" methods, >> and your locator/compiler/cache system would just call down to those >> instead of hardwiring the implementation of compiling and serializing. > > of course, this is sensible. i am not aware immediately of any major > technical issues to componentizing the internals of myghty, there might be > some; i doubt they are fundamnetally insurmoutnable. it would also make > myghty a more solid system, allow its reloading functionality to be > useable by other systems that can benefit from it, etc. > > but, to do this would be an intense royal pain in the ass and take several > weeks/months, render new releases of myghty as less stable for awhile, > etc. which leads me to believe that if this spec also came along with a > salaried position +401K benefits for template system developers to spend > full time re-architecting their systems, id say just great ! otherwise, > might be a little ambitious for the near-to-medium term. > > > _______________________________________________ > Web-SIG mailing list > Web-SIG at python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: > http://mail.python.org/mailman/options/web-sig/mike_mp%40zzzcomputing.com > From pje at telecommunity.com Sat Feb 4 00:49:53 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 03 Feb 2006 18:49:53 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <99FE035C-0288-4697-9562-75E1F6767D8B@groovie.org> References: <5.1.1.6.0.20060203164932.02156ce0@mail.telecommunity.com> <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> <5.1.1.6.0.20060203164932.02156ce0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060203182248.02154a60@mail.telecommunity.com> At 03:01 PM 2/3/2006 -0800, Ben Bangert wrote: >On Feb 3, 2006, at 2:24 PM, Phillip J. Eby wrote: > >>>Wait, then how does the template engine know what template to render? >> >>The framework gives the engine the source of the template, and asks >>for a WSGI app object in return. The framework is allowed to hold >>on to this application object and reuse it as often as desired. > >I'm really not sure why the framework developer should be further >burdened with trying to cache templates. Well, somebody's got to do it. :) Not every template facility out there currently has any kind of location ability or caching. And different frameworks have their own mechanisms for locating, caching, etc. However, it may just be that there are more pieces here than I thought, and I've been lumping some of them together under the name "framework". >>What I'd suggest is that this is actually an opportunity for Myghty >>to put its lookup and caching on one side of the interface, and an >>actual template rendering facility on the other side. That is, >>treat Myghty as an embedd*er* rather than the embedd*ee*, so that >>the engine can be used with arbitrary apps or templates on the >>embedded side. > >Myghty's lookup and caching is rather complex and requires >significant synchronization efforts to ensure that under heavy load, >if the template has changed, its only re-compiled once while the rest >of the requests continue to use the prior compiled version until the >updated one is ready. It's complex, and putting it on completely >separate sides is beyond a nightmare of effort. I don't understand. Are you saying there's nothing that corresponds to reading source, compiling, and writing out a compiled version, that would just change to simply call those methods on a different object than the ones they're called on now? >The lookup mechanism it uses is also very complex, and it doesn't >belong outside of Myghty. I don't understand what you mean by "outside of Myghty" in this context. My suggestion was simply that would gain pluggability in one main area: the implementation of compiling and serialization of templates. (i.e., the syntax/semantics of the templates used) >Yes, but its 100% incompatible with Myghty. For the flexibility and >reloading functionality that Myghty provides, splitting such >functionality up will only cripple features of Myghty that I really >enjoy. I'm still not understanding. Since Myghty will be in control of all the streams, it will have full ability to monitor changes etc. The only thing that's different under the embedding scheme is that 1: it would support multiple formats for the actual templates, and 2: it would return WSGI apps that frontend access to an individual template. >Not really, the one I interpreted kept vital steps together. Without >those steps coupled closely, Myghty cannot be compatible without very >significant changes to underlying structures. It's not as different as you think. In your idea, Myghty took each WSGI call. In the actual idea, Myghty would take a call that returns a callable. That callable gets called for each WSGI call. You still have total control over what happens in each WSGI call; it's just that *which* template is to be used gets factored out by the first call. It doesn't decouple any of the things you're talking about, as best I understand it. >I thought Cheetah also compiled its templates and would reload them >if it detected changes. If a template changes on disk, thats the >great thing about template systems that handle stuff like this. You >ask for the rendered template, and if it changes, it'll recompile/re- >cache it, and render it for you. Ah. I didn't realize this was so common. I'm accustomed to template systems that don't control so much stuff, but which are more at the compile/save level. That may be more of a mismatch, making my idea much less useful for the template use cases that people currently have in mind. :( >It might not be complex for whatever template system you use, but its >a drastically different situation for others. A pure WSGI call to a >template engine like I thought you were referring to was more >agnostic on how the template language deals with it, the key appeal >and flexibility to me. I'm seeing now that the key problem with my view of the situation is that I've been assuming *frameworks* have control over where and how templates are stored, and considering the template systems to be just that - implementation of the template itself. One reason why I also wanted us to focus on template/resource deployment, was because that would help unify caching/compilation concerns and reach a common way to address and access such resources across e.g. different eggs. I think the best thing to do at this point is for me to try to actually write up a spec that's clear enough to be understood. I think that probably the issue here is that there are actually *four* roles, not three or two, and I've been lumping some of them together to make three, and most everybody else has been lumping them into two, and so none of our divisions match each other or reality. :) Restating it in four roles, I'd say there's a publisher, a manager, a compiler, and a resource. The publisher is that part of a framework that determines which template is needed - a name or symbol identifying it. The manager is the thing that translates a template's identity to some form of storage, and manages compilation/caching/reloading/etc. The compiler is the thing that actually does the compilation and knows how to write out a compiled template. Finally, the resource is a WSGI wrapper around a compiled template. So, on a given request, the publisher asks the manager for a resource by saying what template it wants. The manager does whatever it needs to do, possibly invoking the compiler if it needs to. The publisher then makes a WSGI call on the resource to render it. Now, all four of these parts are there in the vars'n'strings proposal too, more or less. It's just that it seems to bundle up everything other than the publisher into one unit, and instead of getting back a resource, the publisher receives the result of the rendering. So we get "framework" (==publisher) and "engine" (==manager+compiler+resources). I was at first assuming that "framework" meant the publisher and manager, and "engine" was the compiler and resource. Then to explain better I split compiler and resource, but now I see that I should've split (conceptually) the publisher and manager. Now, if I'm understanding correctly, Myghty is manager+compiler+resources, so from the publisher point of view there's no real change from your interpretation; a name goes in and a WSGI callable comes out. And, from a WSGI embedding point of view, that's really enough. The only change from your previous idea is that you return some object that the publisher makes a WSGI call on, instead of doing it as a single WSGI call. However, Myghty is still integrated in its control over the handling. Does that make sense now, or am I missing something again? From pje at telecommunity.com Sat Feb 4 00:56:36 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 03 Feb 2006 18:56:36 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <19836.66.192.34.8.1139008119.squirrel@www.geekisp.com> References: <5.1.1.6.0.20060203164932.02156ce0@mail.telecommunity.com> <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> <5.1.1.6.0.20060203164932.02156ce0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060203185002.042f28f8@mail.telecommunity.com> At 06:08 PM 2/3/2006 -0500, Michael Bayer wrote: >Phillip J. Eby wrote: > > What I'd suggest is that this is actually an opportunity for Myghty to put > > its lookup and caching on one side of the interface, and an actual > > template > > rendering facility on the other side. That is, treat Myghty as an > > embedd*er* rather than the embedd*ee*, so that the engine can be used with > > arbitrary apps or templates on the embedded side. > > > > I'm not really familiar with Myghty, though, so I might not be saying > > something sensible here. I'm just saying that if you have a mechanism for > > locating, compiling, and caching templates, then it should live on the > > framework side of the "engine" interface. An embedding engine offers the > > "compile_string, compile_stream, write_compiled, read_compiled" methods, > > and your locator/compiler/cache system would just call down to those > > instead of hardwiring the implementation of compiling and serializing. > >of course, this is sensible. i am not aware immediately of any major >technical issues to componentizing the internals of myghty, there might be >some; i doubt they are fundamnetally insurmoutnable. it would also make >myghty a more solid system, allow its reloading functionality to be >useable by other systems that can benefit from it, etc. > >but, to do this would be an intense royal pain in the ass and take several >weeks/months, render new releases of myghty as less stable for awhile, >etc. which leads me to believe that if this spec also came along with a >salaried position +401K benefits for template system developers to spend >full time re-architecting their systems, id say just great ! otherwise, >might be a little ambitious for the near-to-medium term. Heh. My hope would be that this discussion might lead to a situation where a bunch of framework authors could agree to "outsource" much of this handling to a single library, which is why I originally wanted to focus on template deployment issues. If we were able to end up with a common way to find and load templates (factoring out their "language" to plugins), then there would be a need for only a limited number of implementations of that mechanism, maybe only one in a future stdlib. (Granted, specialized needs like loading templates from a database would want distinct implementations of the "manager" portion, but be able to reuse the "compiler" plugins.) You may be right about it being too ambitious; I suspect I will need to go look at the code and design of at least Myghty and Cheetah in order to better understand the issues here. I must confess I have been thinking about template engines strictly in language terms, probably because of my own bias towards putting the "manager" part on the "framework" side of the line rather than on the "template engine" side of the line. And I thought the Python web framework world was pretty complex already! :) From pje at telecommunity.com Sat Feb 4 01:14:26 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 03 Feb 2006 19:14:26 -0500 Subject: [Web-SIG] Terminology and a question Message-ID: <5.1.1.6.0.20060203190539.0217dc18@mail.telecommunity.com> Okay, so now it's clear to me that some of the crosstalk on my proposal was due to terminology issues. I'd like to propose a revised terminology for discussing the various attempts at a templating standard, specifically: publisher - the part of a system that determines what template is to be used, usually in the form of an ID, path, or other symbolic name of a template manager - the part of a system that manages the conversion from a template identifier to something executable, by finding the source code and compiling it. (or retrieving it from a cache, etc.) compiler - the part of a system that knows how to convert the source of a template into something executable resource - the "something executable" created by the compiler and returned or cached by the manager, for use by the publisher. Does this make sense to everybody? I think this will help us figure out what will and won't work for systems that put different things on different sides of the "framework/template" line, since it seems there are some template systems that include a "manager" and some that do not, and there are some frameworks that include a "manager" and some that do not. My proposal was based on an assumption that framework usually means a publisher+manager, and a template engine means a compiler+resources. I'm not sure if it is as useful when the template engine means a manager+compiler+resources, but think we should explore that a bit more. It may also be that there are useful opportunities to standardize or library-ize other parts of this stack than have been discussed so far. From ben at groovie.org Sat Feb 4 02:07:56 2006 From: ben at groovie.org (Ben Bangert) Date: Fri, 3 Feb 2006 17:07:56 -0800 Subject: [Web-SIG] Standardized template API In-Reply-To: <5.1.1.6.0.20060203182248.02154a60@mail.telecommunity.com> References: <5.1.1.6.0.20060203164932.02156ce0@mail.telecommunity.com> <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> <5.1.1.6.0.20060203164932.02156ce0@mail.telecommunity.com> <5.1.1.6.0.20060203182248.02154a60@mail.telecommunity.com> Message-ID: On Feb 3, 2006, at 3:49 PM, Phillip J. Eby wrote: > Well, somebody's got to do it. :) Not every template facility out > there currently has any kind of location ability or caching. And > different frameworks have their own mechanisms for locating, > caching, etc. True, and caching can be a tough problem, so it'd be nice if caching code could be re-used at least. > I don't understand. Are you saying there's nothing that > corresponds to reading source, compiling, and writing out a > compiled version, that would just change to simply call those > methods on a different object than the ones they're called on now? No, you're right, there is mechanisms that do exactly what you describe. > I don't understand what you mean by "outside of Myghty" in this > context. My suggestion was simply that would gain pluggability in > one main area: the implementation of compiling and serialization of > templates. (i.e., the syntax/semantics of the templates used) I meant decoupled from the request to render the template. Since rendering the template may or may not require reloading/re-caching it, it seems like a process that the framework shouldn't have to consider. > I'm still not understanding. Since Myghty will be in control of > all the streams, it will have full ability to monitor changes etc. > The only thing that's different under the embedding scheme is that > 1: it would support multiple formats for the actual templates, and > 2: it would return WSGI apps that frontend access to an individual > template. Yes, that is useful functionality. Would the call to the WSGI app that renders the template also trigger something that ensures its re- compiled if it needs to be? > It's not as different as you think. In your idea, Myghty took each > WSGI call. In the actual idea, Myghty would take a call that > returns a callable. That callable gets called for each WSGI call. > You still have total control over what happens in each WSGI call; > it's just that *which* template is to be used gets factored out by > the first call. It doesn't decouple any of the things you're > talking about, as best I understand it. Got it, that makes sense. > Ah. I didn't realize this was so common. I'm accustomed to > template systems that don't control so much stuff, but which are > more at the compile/save level. That may be more of a mismatch, > making my idea much less useful for the template use cases that > people currently have in mind. :( It usually is, which is part of the reason it might be nice to have it decoupled so the most robust compiler/loader can be used, if thats possible. Such template agnosticism might come at a very hefty workload cost, as Mike mentioned about how much time it'd take to splice this functionality into separate parts. > I'm seeing now that the key problem with my view of the situation > is that I've been assuming *frameworks* have control over where and > how templates are stored, and considering the template systems to > be just that - implementation of the template itself. Ah, yes. That explains a lot. In my experience with various template systems, the directory structure of the template root can also influence the rendering of the template. In Myghty this structure is used for automatic inheritance, Cheetah, Kid, Django templates, and others all include the ability to "extend" existing templates within a template. This means they all need to resolve the included template, and cache it. That reason is why all of them already have their own caching/compiling stuff integrated. > One reason why I also wanted us to focus on template/resource > deployment, was because that would help unify caching/compilation > concerns and reach a common way to address and access such > resources across e.g. different eggs. Indeed, since the resolver scheme is likely different for different template systems, it'd be more likely that purely caching can be done separately. Some templates aren't compiled at all for example (I think Django's are always rendered on the fly, not 100% sure though). There's variations in how the template is handled though. > I think the best thing to do at this point is for me to try to > actually write up a spec that's clear enough to be understood. I > think that probably the issue here is that there are actually > *four* roles, not three or two, and I've been lumping some of them > together to make three, and most everybody else has been lumping > them into two, and so none of our divisions match each other or > reality. :) Quite likely. > So, on a given request, the publisher asks the manager for a > resource by saying what template it wants. The manager does > whatever it needs to do, possibly invoking the compiler if it needs > to. The publisher then makes a WSGI call on the resource to render > it. Where along these steps, are the variables passed into the template? Once the template is compiled/cached, in addition to calling it, I need to pass it variables that it will be using. Are all templates going to be expected to pull variables they should see from the environ? > I was at first assuming that "framework" meant the publisher and > manager, and "engine" was the compiler and resource. Then to > explain better I split compiler and resource, but now I see that I > should've split (conceptually) the publisher and manager. Ok, so if I'm using this in my web application. Somehow, I'm going to need to check to see if the template has been compiled/cached, then I have to go and call the WSGI app? With all these compiled/cached templates that've been used, what's holding onto all those many, many WSGI apps that've been created? Some of my web applications have upwards of thousands of templates, is there going to be thousands of WSGI apps in memory waiting to be called? I'm guessing/hoping I missed something and this isn't the case. > Now, if I'm understanding correctly, Myghty is manager+compiler > +resources, so from the publisher point of view there's no real > change from your interpretation; a name goes in and a WSGI callable > comes out. And, from a WSGI embedding point of view, that's really > enough. The only change from your previous idea is that you return > some object that the publisher makes a WSGI call on, instead of > doing it as a single WSGI call. However, Myghty is still > integrated in its control over the handling. That's correct. Typically, in a lot of the MVC frameworks, like Aquarium, CherryPy, Pylons, Django, etc. the web developer will be doing some logic in the controller. They will then decide to render a template. The controller is the 'publisher', and the template to be rendered is always provided by name, perhaps based on what variables the controller sees. I can see from a Zope oriented view that this is rather different from where you'd have a Publisher that determines the template to be used for an object. In the frameworks I mentioned, it works a bit differently so all thats of most concern to us is that we can render a template by name, against a configured template directory root path. > Does that make sense now, or am I missing something again? I think it makes sense, we're definitely coming at this from different perspectives on what's going on, and where. Since templates in a bunch of these template languages will frequently include other templates, all the 'resource' instances will need access to call back to the 'publisher' so that they can do what they need to for the template to be rendered and included. Some template systems go farther still than basic including of another template and start extending other templates. Kid will have one template override parts of templates it "inherits" from. Myghty will have parent templates that will insert their "child" templates in various fashions. Cheetah and Django also do things like this. All these systems will need ways to tie back into the manager so that they're all loaded, compiled, cached when needed, and that individual parts of the whole get reloaded when necessary without reloading everything. If there's a way to separate out this very convoluted process (that isn't necessarily the same for each system), it'd be very compelling I think. Especially since solid code to handle many of the problems that can occur under load when doing all this reloading/caching/etc is not easy stuff. I think many of us are actually talking about two different proposals. The proposal for standardized templating you're talking to sounds like WSGI for template languages. That is, it pushes common parts many template languages have, into a re-usable layer. I'm not sure such a thing is possible, its definitely quite ambitious, and I look forward to the more clearly defined spec you put out. The proposal many of us were looking for (I think), was merely a more unified way of calling a template. That is, we want a standard 'publisher' interface, so that we can call someengine.render('some/ template.myt', myvars) and its rendered. The TurboGears template plug- in thing describes such a thing, but it could use some more work. The way I interpreted your proposal originally also set things up in such a way that there was a base system for configuring a template engine, then you just call the app and put your vars in environ. As the more ambitious one is likely to take significantly more effort from everyone involved (assuming it'll work), can we unify a setup/ call structure first? I see no reason it won't be compatible in the end with standardizing the "internals" of the template languages as well. Cheers, Ben From pje at telecommunity.com Sat Feb 4 04:40:27 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 03 Feb 2006 22:40:27 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: References: <5.1.1.6.0.20060203182248.02154a60@mail.telecommunity.com> <5.1.1.6.0.20060203164932.02156ce0@mail.telecommunity.com> <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> <5.1..1.6.0.20060203164932.02156ce0@mail.telecommunity.com> <5.1.1.6.0.20060203182248.02154a60@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060203215009.02157120@mail.telecommunity.com> At 05:07 PM 2/3/2006 -0800, Ben Bangert wrote: >On Feb 3, 2006, at 3:49 PM, Phillip J. Eby wrote: >>I don't understand what you mean by "outside of Myghty" in this >>context. My suggestion was simply that would gain pluggability in >>one main area: the implementation of compiling and serialization of >>templates. (i.e., the syntax/semantics of the templates used) > >I meant decoupled from the request to render the template. Since >rendering the template may or may not require reloading/re-caching >it, it seems like a process that the framework shouldn't have to >consider. My thoughts on caching were in the context of assuming the "manager" was part of the framework. Frameworks that implement skins/layers and localizable resources have to do this, because template systems generally don't have any concept of skins or layers. But it's seeming like we'll have to punt on this to some extent for now. >>I'm still not understanding. Since Myghty will be in control of >>all the streams, it will have full ability to monitor changes etc. >>The only thing that's different under the embedding scheme is that >>1: it would support multiple formats for the actual templates, and >>2: it would return WSGI apps that frontend access to an individual >>template. > >Yes, that is useful functionality. Would the call to the WSGI app >that renders the template also trigger something that ensures its re- >compiled if it needs to be? Sure. You can just do something like: def resource_returned_by_manager(environ, start_response): if needs_recompiling(actual_resource): recompile(actual_resource) return actual_resource(environ, start_response) That is, the manager doesn't have to return the resource returned by the compiler; it can wrap a trivial reloader middleware around it and return that. Of course, the more I look at this, the more I realize that your idea of just putting the entire "manager" interface in one WSGI call might make good sense too. Because unless the "publisher" is going to hang on to the returned app, you might as well just integrate this stuff at the manager level. The problem for me is, I was hoping we would end up defining a manager->compiler interface and a publisher->resource interface rather than a publisher->manager one. Then again, maybe we should define both, and let people plug things in where they can. It would provide a path for gradually evolving to the place where everything's pluggable. >It usually is, which is part of the reason it might be nice to have >it decoupled so the most robust compiler/loader can be used, if thats >possible. Such template agnosticism might come at a very hefty >workload cost, as Mike mentioned about how much time it'd take to >splice this functionality into separate parts. I'm leaning more now towards defining a publisher->manager and publisher->resource interface pair based on WSGI as making the most sense for the present. A manager->compiler interface would be nice too, but perhaps not strictly necessary for the original use cases that prompted this discussion. >>I'm seeing now that the key problem with my view of the situation >>is that I've been assuming *frameworks* have control over where and >>how templates are stored, and considering the template systems to >>be just that - implementation of the template itself. > >Ah, yes. That explains a lot. In my experience with various template >systems, the directory structure of the template root can also >influence the rendering of the template. In Myghty this structure is >used for automatic inheritance, Cheetah, Kid, Django templates, and >others all include the ability to "extend" existing templates within >a template. This means they all need to resolve the included >template, and cache it. That reason is why all of them already have >their own caching/compiling stuff integrated. In my view, this is a bit suboptimal in the long haul, because there's all this duplicated code for managing files, and probably little -- if any -- of it works with eggs, or templates in a database (e.g. ZODB), etc. This is part of why my mental model didn't really consider much about template systems having their own "manager" parts - the idea seems manifestly unworkable if you need to do things like what Zope 3 supports. Of course, my long-term ambition here would be that we get to a place where you can do skinning and localization with templates and other resources, all using a Python-wide standard for how you identify and deploy these resources. But that's out of scope for the current interface discussions, alas. It might be nice, though, to put together a session at PyCon to do some brainstorming on templates, embedding, and deployment to see if there are currently any easily-reached opportunities. >>So, on a given request, the publisher asks the manager for a >>resource by saying what template it wants. The manager does >>whatever it needs to do, possibly invoking the compiler if it needs >>to. The publisher then makes a WSGI call on the resource to render >>it. > >Where along these steps, are the variables passed into the template? When the publisher makes the WSGI call, it would add a 'params' key and/or a 'published_object' key, depending on whether the publisher is an object publisher or does parameters or both. I can imagine that Zope, for example, might do both, drawing on its request parsing to do parameters. A TurboGears or other controller-type framework that uses templates for postprocessing would put the variables in 'params'. These names are hypothetical and subject to getting a firm spec, of course. >Once the template is compiled/cached, in addition to calling it, I >need to pass it variables that it will be using. Are all templates >going to be expected to pull variables they should see from the environ? Yeah, from the 'params' key or 'published_object' or both. >>I was at first assuming that "framework" meant the publisher and >>manager, and "engine" was the compiler and resource. Then to >>explain better I split compiler and resource, but now I see that I >>should've split (conceptually) the publisher and manager. > >Ok, so if I'm using this in my web application. Somehow, I'm going to >need to check to see if the template has been compiled/cached, then I >have to go and call the WSGI app? With all these compiled/cached >templates that've been used, what's holding onto all those many, many >WSGI apps that've been created? Some of my web applications have >upwards of thousands of templates, is there going to be thousands of >WSGI apps in memory waiting to be called? I'm guessing/hoping I >missed something and this isn't the case. Nah, the issue here was that I envisioned the division differently than Ian; my concern about being able to cache was because I envisioned the "manager" being on the framework side of the line; ergo, the framework needed caching. If the interface is strictly between the publisher and the manager, then you could potentially just do it all with a single WSGI call, in which the manager pulls the template identifier from the environ. But it seems a little ugly to me to cram another key into environ to tell the manager what template to use, when you could just have a 'get(template_id)' call that gets you the WSGI app to use. It also makes the objects' responsibilities a little more clearly defined, since a WSGI app in the general case doesn't take a template_id parameter, if you see what I mean. >Since templates in a bunch of these template languages will >frequently include other templates, all the 'resource' instances will >need access to call back to the 'publisher' so that they can do what To the manager actually, since the manager is responsible for id->template lookups. >they need to for the template to be rendered and included. Some >template systems go farther still than basic including of another >template and start extending other templates. Kid will have one >template override parts of templates it "inherits" from. Myghty will >have parent templates that will insert their "child" templates in >various fashions. Cheetah and Django also do things like this. ZPT has macros, peak.web has DOMlets and layouts, etc., etc. Lots of variety to choose from. :) >All these systems will need ways to tie back into the manager so that >they're all loaded, compiled, cached when needed, and that individual >parts of the whole get reloaded when necessary without reloading >everything. > >If there's a way to separate out this very convoluted process (that >isn't necessarily the same for each system), it'd be very compelling >I think. Especially since solid code to handle many of the problems >that can occur under load when doing all this reloading/caching/etc >is not easy stuff. I'm beginning to think that this actually ties in pretty closely with the deployment issues, in the sense that you may need basically multiple "sources" of template/resource implementations, especially if you're supporting skins or localization. It's almost like you need a registry of template ids to ways to get them, that have been registered via plugins. Of course, in the use cases you guys are talking about, you presume that there's just a "template root", which is way too simple for the kinds of stuff Zope 3 and peak.web want to do. Both have the notion of "skins", which are composed of "layers". A layer can provide alternate versions of any given template, and layers can be combined to form skins. There's no particular requirement that these layers have any filesystem relationships, and in the Zope case they may be in ZODB in any case, not the filesystem. In PEAK's case, the layers can live in different eggs, and I suspect Zope will want to allow the same as they increase their support for eggs. Anyway, that's why I think that if we can come up with a "bottom up" way to provide template sources, then we can have a manager that's just a thin layer over these providers. Stuff that's supplied by eggs, for example, will probably be precompiled and won't need to be refreshed. Stuff in ZODB has its own caching. So, to a certain extent, I think the ultimate future will be that our "manager" will just be something that figures out what backend is supplying the stuff and hand off the handling to it. We'll have some "file" managers, "egg" managers, "ZODB" managers, etc. plugged in via entry points or some such. Of course, this is all "future" stuff. I had hoped we were closer to being able to do it, though, as I'd been meaning to build some of it for peak.web, and didn't want to end up in the position of appearing to say, "hey, everybody should do this stuff I did for peak.web". :) Instead, I'd be saying, "peak.web now supports the new WSGI embedding and resource deployment specs". :) >I think many of us are actually talking about two different >proposals. The proposal for standardized templating you're talking to >sounds like WSGI for template languages. That is, it pushes common >parts many template languages have, into a re-usable layer. I'm not >sure such a thing is possible, its definitely quite ambitious, and I >look forward to the more clearly defined spec you put out. Well, I'm realizing now that the more ambitious bits are harder because there's more that's actually been done out there in the template engines than I thought. If there were less, more standardization would've been an obvious move forward and more doable now. >The proposal many of us were looking for (I think), was merely a more >unified way of calling a template. That is, we want a standard >'publisher' interface, so that we can call someengine.render('some/ >template.myt', myvars) and its rendered. The TurboGears template plug- in >thing describes such a thing, but it could use some more work. The >way I interpreted your proposal originally also set things up in such >a way that there was a base system for configuring a template engine, >then you just call the app and put your vars in environ. > >As the more ambitious one is likely to take significantly more effort >from everyone involved (assuming it'll work), can we unify a setup/ call >structure first? I see no reason it won't be compatible in the >end with standardizing the "internals" of the template languages as >well. I concur. What I propose right now is that we have a "manager" interface that gets asked for a template by a name or ID, and returns a WSGI app object. Calls to that object can then include 'params' and/or 'published_object'. (Exact spelling TBD.) I don't think this interface should suggest/recommend an addressing scheme, nor a plugin scheme. That is, I don't think it should include specifiers for entry points or anything like that. Instead, to do the TG/B style of plugins, you should just have a manager that figures out what template engine/plugin to use. The identifier can and should be opaque. By that I mean, you could have a manager object that looks for a prefix like "kid:" at the beginning of the ID and then looks up an entry point to forward to another manager. But that would just be one way of implementing the spec, not something required by the spec. The reason is that this then leaves open both the possibility of developing generic and/or polymorphic "manager" objects. The big open question is how to create and configure a manager object, since different managers will have different needs. For example, peak.web uses module-relative paths in the default layer of the default skin; it has no "root" directory. Each layer can also have its own directory, and I believe Zope 3 does the same. From guido at python.org Sat Feb 4 17:17:54 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 4 Feb 2006 08:17:54 -0800 Subject: [Web-SIG] A trivial template API counter-proposal Message-ID: [Phillip] > API to be provided by a "template engine": > > compile_string(text), compile_stream(input_stream) - return a WSGI > application object for the template described by the text or file-like object > > write_compiled(app, output_stream) - save the compiled form of the template > to output_stream > > read_compiled(input_stream) - read a template from input_stream and return > a WSGI application object I am probably missing something, but shouldn't there also be an API to render the template to a string or stream given some context of variable names? I'm looking at this from a very different perspective, namely *using* various templating engines from an app that otherwise doesn't use a framework but still needs templating. (PS having tried WSGI a bit now I'm fine with it. Perhaps wsgiref should go into the Python 2.5 standard library?) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ben at groovie.org Sat Feb 4 18:17:15 2006 From: ben at groovie.org (Ben Bangert) Date: Sat, 4 Feb 2006 09:17:15 -0800 Subject: [Web-SIG] A trivial template API counter-proposal In-Reply-To: References: Message-ID: <6ACB2EC3-CA90-4D41-976C-9B46F7B8CA86@groovie.org> On Feb 4, 2006, at 8:17 AM, Guido van Rossum wrote: > I am probably missing something, but shouldn't there also be an API to > render the template to a string or stream given some context of > variable names? I'm looking at this from a very different perspective, > namely *using* various templating engines from an app that otherwise > doesn't use a framework but still needs templating. Yes, there was a lot of convoluted thought in the larger thread of Standardized template API which got intermingled with a related proposal to standardize common parts used within template languages themselves. We're currently steering back to just standardizing *using* the various templates, and saving the much more complex issue of their internals for a later point. Cheers, Ben From ben at groovie.org Sat Feb 4 18:31:26 2006 From: ben at groovie.org (Ben Bangert) Date: Sat, 4 Feb 2006 09:31:26 -0800 Subject: [Web-SIG] Standardized template API In-Reply-To: <5.1.1.6.0.20060203215009.02157120@mail.telecommunity.com> References: <5.1.1.6.0.20060203182248.02154a60@mail.telecommunity.com> <5.1.1.6.0.20060203164932.02156ce0@mail.telecommunity.com> <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> <5.1..1.6.0.20060203164932.02156ce0@mail.telecommunity.com> <5.1.1.6.0.20060203182248.02154a60@mail.telecommunity.com> <5.1.1.6.0.20060203215009.02157120@mail.telecommunity.com> Message-ID: <5E9F03A7-670E-45F5-B9F3-5D0E3B7FD275@groovie.org> On Feb 3, 2006, at 7:40 PM, Phillip J. Eby wrote: > The problem for me is, I was hoping we would end up defining a > manager->compiler interface and a publisher->resource interface > rather than a publisher->manager one. Then again, maybe we should > define both, and let people plug things in where they can. It > would provide a path for gradually evolving to the place where > everything's pluggable. Sure, as this thread started with standardizing purely the usage of various template languages, I'd suggest we split the thread so that any talk of standardizing internals regarding those relations be separate from the issue of just using them. > Yeah, from the 'params' key or 'published_object' or both. Ok, that sounds fine. > But it seems a little ugly to me to cram another key into environ > to tell the manager what template to use, when you could just have > a 'get(template_id)' call that gets you the WSGI app to use. It > also makes the objects' responsibilities a little more clearly > defined, since a WSGI app in the general case doesn't take a > template_id parameter, if you see what I mean. Sure, though I'm thinking back to Ian's original proposed addition to TG's spec regarding being able to pass in a find_template callable. > I concur. What I propose right now is that we have a "manager" > interface that gets asked for a template by a name or ID, and > returns a WSGI app object. Calls to that object can then include > 'params' and/or 'published_object'. (Exact spelling TBD.) We still need a setup/prepare step for the template engines involved because they have different requirements. Some of them need an initial root directory to be declared, etc. The TurboGears spec covered this nicely, can we use it for a starting point at least since it exists right now, and it works? > I don't think this interface should suggest/recommend an addressing > scheme, nor a plugin scheme. That is, I don't think it should > include specifiers for entry points or anything like that. > Instead, to do the TG/B style of plugins, you should just have a > manager that figures out what template engine/plugin to use. The > identifier can and should be opaque. Sure, that sounds appealing as well, what would using said interface look like, and how would you configure the various template engines it supports? (ie, if one of them needs a template root, or one of them needs a cache directory, etc.) > By that I mean, you could have a manager object that looks for a > prefix like "kid:" at the beginning of the ID and then looks up an > entry point to forward to another manager. But that would just be > one way of implementing the spec, not something required by the > spec. The reason is that this then leaves open both the > possibility of developing generic and/or polymorphic "manager" > objects. > > The big open question is how to create and configure a manager > object, since different managers will have different needs. For > example, peak.web uses module-relative paths in the default layer > of the default skin; it has no "root" directory. Each layer can > also have its own directory, and I believe Zope 3 does the same. I think at this point, Ian's suggestion regarding a find_template callable solves several of the issues you noted in how differently Zope looks up its layers/templates. If you create a callable that can do your lookup's, the existing TG spec for using templates would need little modification to gain the majority of the flexibility we're looking at. So... if the existing TG spec can't/shouldn't be extended to support this, what's the alternative easy-to-use method for *using* various template languages to look like? - Ben From pje at telecommunity.com Sat Feb 4 19:36:55 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 04 Feb 2006 13:36:55 -0500 Subject: [Web-SIG] A trivial template API counter-proposal In-Reply-To: Message-ID: <5.1.1.6.0.20060204131722.02179008@mail.telecommunity.com> At 08:17 AM 2/4/2006 -0800, Guido van Rossum wrote: >[Phillip] > > API to be provided by a "template engine": > > > > compile_string(text), compile_stream(input_stream) - return a WSGI > > application object for the template described by the text or file-like > object > > > > write_compiled(app, output_stream) - save the compiled form of the template > > to output_stream > > > > read_compiled(input_stream) - read a template from input_stream and return > > a WSGI application object > >I am probably missing something, but shouldn't there also be an API to >render the template to a string or stream given some context of >variable names? I'm looking at this from a very different perspective, >namely *using* various templating engines from an app that otherwise >doesn't use a framework but still needs templating. The API I proposed assumes that the template engine gives you WSGI application objects, and that you simply pipe them through to your WSGI stack. That is, in the simplest case: def some_wsgi_app(environ, start_response): template = some_engine.compile_stream(open("mytemplate.foo")) environ['wsgi.params'] = {'some_var':'bar'} return template(environ, start_response) The assumption here is that a "template" is just a WSGI application object, that maybe has a few extra keys added to the environ, like a params dict and/or a "published object". From current discussion, however, it seems like the above API will be deferred a while to focus on a higher-level API in which you simply ask for a template by an opaque ID, e.g.: def some_wsgi_app(environ, start_response): template = some_engine.find_template("mytemplate") environ['wsgi.params'] = {'some_var':'bar'} return template(environ, start_response) So, it's the same except an even simpler, higher-level API. The API I proposed is more useful in the context of a template manager (such as Zope 3 skins/layers) that might need to support pluggable template compilers. In such circumstances, the compile/write APIs I proposed would be useful, but a different API layer than the "get me a template to run" API. Anyway, part of the reasoning for using WSGI itself as a template API is that it allows for template languages that may need to manipulate response headers and status, such as the ones that embed Python in HTML or HTML in Python. Or more typically, templates being used to generate non-HTML text, such as XML, tab-delimited files, or other kinds of downloads. It also makes it possible for the template language to offer its own native API for getting request data. For example, a ZPT template engine would be free to rewrap the WSGI data to appear as standard Zope request/response objects from the template's point of view. >(PS having tried WSGI a bit now I'm fine with it. WSGI - it's no worse than a trip to East Berlin. :) > Perhaps wsgiref >should go into the Python 2.5 standard library?) Sure. I'd suggest that we might want to expand it a little to include some of the fine contributions of other authors, such as Ian's "lint" middleware (which sits between an app and a server and checks both for compliance) and maybe even his "Python debugger in the browser" middleware. There have also been people who've written a few utilities for CGI-in-WSGI, WSGI in mod_python, etc. And it'd be nice to squeeze in a FastCGI implementation if somebody's got a suitable one. Docs are also nonexistent at the moment. I'd be happy to co-ordinate and consolidate the additions, and of course I'll write documentation, though any assistance people can offer would be greatly appreciated. Some people have written nice WSGI tutorials or explanations, so if anybody wants to offer those up for consolidation into docs, that would be good too. From pje at telecommunity.com Sat Feb 4 19:55:19 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 04 Feb 2006 13:55:19 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <5E9F03A7-670E-45F5-B9F3-5D0E3B7FD275@groovie.org> References: <5.1.1.6.0.20060203215009.02157120@mail.telecommunity.com> <5.1.1.6.0.20060203182248.02154a60@mail.telecommunity.com> <5.1.1.6.0.20060203164932.02156ce0@mail.telecommunity.com> <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1..1.6.0.20060203143323.0215b908@mail.telecommunity.com> <5.1..1.6.0.20060203164932.02156ce0@mail.telecommunity.com> <5.1.1.6.0.20060203182248.02154a60@mail.telecommunity.com> <5.1.1.6.0.20060203215009.02157120@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060204133704.021562a8@mail.telecommunity.com> At 09:31 AM 2/4/2006 -0800, Ben Bangert wrote: >On Feb 3, 2006, at 7:40 PM, Phillip J. Eby wrote: >>But it seems a little ugly to me to cram another key into environ >>to tell the manager what template to use, when you could just have >>a 'get(template_id)' call that gets you the WSGI app to use. It >>also makes the objects' responsibilities a little more clearly >>defined, since a WSGI app in the general case doesn't take a >>template_id parameter, if you see what I mean. > >Sure, though I'm thinking back to Ian's original proposed addition to >TG's spec regarding being able to pass in a find_template callable. But that's the case that threw this whole thing off track to begin with, because you can only pass in a find_template if *you're the template manager*. In the interface we're currently discussing, finding templates is internal to the engine, so that bit doesn't come up. That is, if you have a Myghty "get(template_id)", then clearly it's Myghty that does the finding, so how would you *give* Myghty a finder? You've already said that Myghty has to use its own finder, didn't you? >We still need a setup/prepare step for the template engines involved >because they have different requirements. Some of them need an >initial root directory to be declared, etc. The TurboGears spec >covered this nicely, can we use it for a starting point at least >since it exists right now, and it works? If you mean the __init__() signature, I'd agree, but IIUC the extra_vars_func isn't needed since the caller's responsible for adding those to the WSGI environ at call time. Unless I'm misunderstanding its purpose? >>I don't think this interface should suggest/recommend an addressing >>scheme, nor a plugin scheme. That is, I don't think it should >>include specifiers for entry points or anything like that. >>Instead, to do the TG/B style of plugins, you should just have a >>manager that figures out what template engine/plugin to use. The >>identifier can and should be opaque. > >Sure, that sounds appealing as well, what would using said interface >look like, and how would you configure the various template engines >it supports? (ie, if one of them needs a template root, or one of >them needs a cache directory, etc.) I was assuming that the selection or implementation of such "uber-managers" would be hardcoded by the framework, not a point of pluggability. These "uber-managers" effectively correspond to the TurboGears loader or Buffet loader that exist today. That is, they're the implementation of a thing that finds other managers and calls them. It's just that in this spec, they become "manager middleware" because they both implement the manager "get" interface, and they invoke it. E.g.: class TGManager: def get_template(self, id): if ':' in id: engine_name, id = id.split(':',1) else: engine_name = self.default_engine return self.get_engine(engine_name).get_template(id) So, the framework can expose a TGManager that offers the standard get_template() and follows the framework's policy for determining what engine to invoke; the above is just implementing what I believe is Turbogears' current policy for such. So, initialization and policy of one of these animals is framework-specific as I understand it. But the framework should expose the manager, and it can be recommended that it be included as an environ item, so that an application could use something like 'environ["wsgi.template_engine"].get_template("some_id")' to obtain a template. (Of course, that only matters for frameworks that are exposing WSGI to application code, or if you want to give a template access to the template engine, even though it should already have a direct line to the engine internally.) >I think at this point, Ian's suggestion regarding a find_template >callable solves several of the issues you noted in how differently >Zope looks up its layers/templates. If you create a callable that can >do your lookup's, the existing TG spec for using templates would need >little modification to gain the majority of the flexibility we're >looking at. Unfortunately, it doesn't. Didn't you just point out that Myghty isn't going to actually *use* that find_template callable? If not, then what good does it do to pass it in? >So... if the existing TG spec can't/shouldn't be extended to support >this, what's the alternative easy-to-use method for *using* various >template languages to look like? I think the TG API is pretty close to what we're going to end up with, except that rendering takes place via WSGI. I don't see a problem with treating the other existing TG methods (render and transform) as optional extension APIs; I just don't want those to be "the standard" for how you plug templates into a framework, since it makes object publisher and active page systems into second-class citizens grubbing for scraps of text, so to speak. :) From ben at groovie.org Sat Feb 4 20:37:01 2006 From: ben at groovie.org (Ben Bangert) Date: Sat, 4 Feb 2006 11:37:01 -0800 Subject: [Web-SIG] Standardized template API In-Reply-To: <5.1.1.6.0.20060204133704.021562a8@mail.telecommunity.com> References: <5.1.1.6.0.20060203215009.02157120@mail.telecommunity.com> <5.1.1.6.0.20060203182248.02154a60@mail.telecommunity.com> <5.1.1.6.0.20060203164932.02156ce0@mail.telecommunity.com> <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1..1.6.0.20060203143323.0215b908@mail.telecommunity.com> <5.1..1.6.0.20060203164932.02156ce0@mail.telecommunity.com> <5.1.1.6.0.20060203182248.02154a60@mail.telecommunity.com> <5.1.1.6.0.20060203215009.02157120@mail.telecommunity.com> <5.1.1.6.0.20060204133704.021562a8@mail.telecommunity.com> Message-ID: <8E439B9E-BF16-4A8A-9E87-BA3E5FE4385E@groovie.org> On Feb 4, 2006, at 10:55 AM, Phillip J. Eby wrote: > But that's the case that threw this whole thing off track to begin > with, because you can only pass in a find_template if *you're the > template manager*. In the interface we're currently discussing, > finding templates is internal to the engine, so that bit doesn't > come up. Yes, I noticed this, because the next step it led to was having a load_template, then we're down the slippery slope to standardizing internals. > That is, if you have a Myghty "get(template_id)", then clearly it's > Myghty that does the finding, so how would you *give* Myghty a > finder? You've already said that Myghty has to use its own finder, > didn't you? Agreed, it does. find_template should be optional, because its of no use to Myghty. Myghty needs merely the template name and the template root. Of course, merely having a find_template does imply that if a template include or uses another template, the template language itself must be aware it should be using find_template.... So yes, find_template quickly leads to standardizing internals of the template language, I agree that it should either not be present, or be optional. > If you mean the __init__() signature, I'd agree, but IIUC the > extra_vars_func isn't needed since the caller's responsible for > adding those to the WSGI environ at call time. Unless I'm > misunderstanding its purpose? No, I think you got it, I'm referring to the __init__ for each individual template engine. > So, the framework can expose a TGManager that offers the standard > get_template() and follows the framework's policy for determining > what engine to invoke; the above is just implementing what I > believe is Turbogears' current policy for such. Makes sense. > So, initialization and policy of one of these animals is framework- > specific as I understand it. But the framework should expose the > manager, and it can be recommended that it be included as an > environ item, so that an application could use something like > 'environ["wsgi.template_engine"].get_template("some_id")' to obtain > a template. (Of course, that only matters for frameworks that are > exposing WSGI to application code, or if you want to give a > template access to the template engine, even though it should > already have a direct line to the engine internally.) I think that'll be fine, we probably need an updated bit of sample code to look at now. > Unfortunately, it doesn't. Didn't you just point out that Myghty > isn't going to actually *use* that find_template callable? If not, > then what good does it do to pass it in? I think the main reason Ian suggested it was because he wanted to implement his own lookup scheme to find templates. As I mentioned above, this begins to take us to the internals.... so I'm inclined to agree that its not going to help here. > I think the TG API is pretty close to what we're going to end up > with, except that rendering takes place via WSGI. I don't see a > problem with treating the other existing TG methods (render and > transform) as optional extension APIs; I just don't want those to > be "the standard" for how you plug templates into a framework, > since it makes object publisher and active page systems into second- > class citizens grubbing for scraps of text, so to speak. :) Ok, sounds great. Anyone up for throwing some pseudo-code? Cheers, Ben From pje at telecommunity.com Sat Feb 4 21:59:29 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 04 Feb 2006 15:59:29 -0500 Subject: [Web-SIG] Standardized template API In-Reply-To: <8E439B9E-BF16-4A8A-9E87-BA3E5FE4385E@groovie.org> References: <5.1.1.6.0.20060204133704.021562a8@mail.telecommunity.com> <5.1.1.6.0.20060203215009.02157120@mail.telecommunity.com> <5.1.1.6.0.20060203182248.02154a60@mail.telecommunity.com> <5.1.1.6.0.20060203164932.02156ce0@mail.telecommunity.com> <5.1.1.6.0.20060203143323.0215b908@mail.telecommunity.com> <5.1.1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <6654eac40601301920n2448c068q83b33ee4c0d4fa37@mail.gmail.com> <43DEDD22.9010208@colorstudy.com> <3f085ecd0601310753o19edc503we008f2c4c6b80ebf@mail.gmail.com> <43DF94F5.7080108@colorstudy.com> <5.1.1.6.0.20060131144811.03ff2ff0@mail.telecommunity.com> <5.1.1.6.0.20060131191743.02384868@mail.telecommunity.com> <5.1.1.6.0.20060131203203.040f3d10@mail.telecommunity.com> <20060201164310.GA13344@prometheusresearch.com> <5.1.1.6.0.20060201142436.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201164355.0239dc00@mail.telecommunity.com> <5.1.1.6.0.20060201180703.023a1e78@mail.telecommunity.com> <5.1..1.6.0.20060202140047.023b4a50@mail.telecommunity.com> <5.1..1.6.0.20060203143323.0215b908@mail.telecommunity.com> <5.1..1.6.0.20060203164932.02156ce0@mail.telecommunity.com> <5.1.1.6.0.20060203182248.02154a60@mail.telecommunity.com> <5.1.1.6.0.20060203215009.02157120@mail.telecommunity.com> <5.1.1.6.0.20060204133704.021562a8@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060204154122.02173bc8@mail.telecommunity.com> At 11:37 AM 2/4/2006 -0800, Ben Bangert wrote: >I think the main reason Ian suggested it was because he wanted to >implement his own lookup scheme to find templates. As I mentioned >above, this begins to take us to the internals.... so I'm inclined to >agree that its not going to help here. Actually, I'm thinking that the whole discussion is definitely resembling OLE, at least as a conceptual model. We have "linking" (symbolic identifier passed off to an engine that finds and returns the desired object) and we have "embedding", where the engine is just handed raw data or access to a place to store the object. These are *both* useful things to do, even if some are what would currently be considered "internals" in some frameworks and template systems right now. But they probably *both* deserve to be standardized, and then a given template or framework can support one or the other or both according to its choice and ability. Thus, Myghty would be an example of a tool supporting linking (as a link resolver), but not embedding, so it would describe itself as supplying a WSGI link resolver. My hypothetical templates-from-the-filesystem server would be an embedder, but wouldn't support linking (at least not directly), so it would describe itself as providing a WSGI resource container. Anyway, we could then call the full spec "WELT" - WSGI Embedding, Linking, and Templates. Our slogan could then be, "Got WELT?" :) From ianb at colorstudy.com Sun Feb 5 00:07:21 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Sat, 04 Feb 2006 17:07:21 -0600 Subject: [Web-SIG] WSGI in standard library (was: A trivial template API counter-proposal) In-Reply-To: <5.1.1.6.0.20060204131722.02179008@mail.telecommunity.com> References: <5.1.1.6.0.20060204131722.02179008@mail.telecommunity.com> Message-ID: <43E533A9.6080504@colorstudy.com> Phillip J. Eby wrote: >> Perhaps wsgiref >>should go into the Python 2.5 standard library?) > > > Sure. I'd suggest that we might want to expand it a little to include some > of the fine contributions of other authors, such as Ian's "lint" middleware > (which sits between an app and a server and checks both for compliance) and > maybe even his "Python debugger in the browser" middleware. There have > also been people who've written a few utilities for CGI-in-WSGI, WSGI in > mod_python, etc. And it'd be nice to squeeze in a FastCGI implementation > if somebody's got a suitable one. Docs are also nonexistent at the > moment. I'd be happy to co-ordinate and consolidate the additions, and of > course I'll write documentation, though any assistance people can offer > would be greatly appreciated. Some people have written nice WSGI tutorials > or explanations, so if anybody wants to offer those up for consolidation > into docs, that would be good too. There's some pieces that I think are not yet ready for standardization. For instance, the error catching middleware would deserve a much larger discussion, I think, as it also involves the question of how to extend tracebacks (it uses Zope's code that uses magic local variables). Also, the interactive implementation is complex enough (including Javascript) that I can't envision it stabablizing sufficiently. (Paste includes a cgitb-based catching middleware as well) I think a WSGI-based HTTP server is fairly easy to stabalize. Clark Evans contributed a server based on SimpleHTTPServer, which I think is both pretty complete and doesn't do anything besides HTTP serving (e.g., no static file serving): http://svn.pythonpaste.org/Paste/trunk/paste/httpserver.py The CGI server in wsgiref is also very stable -- there's not much for it to do. Flup includes FastCGI, SCGI, and AJK, both forking and threaded. I think all these pieces have relatively few dependencies, which could be extracted. No forking WSGI HTTP server exists that I know of. paste.lint is pretty stable and complete, I think. With a little adaptation, paste.urlparser.StaticURLParser and paste.urlparser.PkgResourceParser also could be useful -- these are static file serving apps (they should be renamed, of course). Serving with pkg_resources doesn't make sense unless pkg_resources is in the standard library. But both those are based on paste.fileapp.FileApp -- which serves a single file -- and while Clark has also done some work on that and it is improved, I feel less confident that it is complete (but it could be completed -- its scope is well-understood). Finding a file is easy (that's what StaticURLParser does), but actually serving a file with full HTTP support is a little more complex. Also support for encoding and expiration times is missing from those -- mostly a matter of passing related configuration. I think a good static file serving implementation is really valuable. paste.cgiapp is relatively simple (runs a CGI script). I think paste.proxy (just added) would be a really nice contribution to the standard library -- an HTTP proxy to another server, appearing as a WSGI application -- but the implementation is rough and only a couple of days old. There's also a bunch of the routines in paste.request, paste.response, and paste.wsgilib could be good for the standard library. There's also some code in paste.httpheaders, which perhaps could be made simpler and more primitive-feeling -- for things like parsing header values according to their meaning in HTTP. There's several other bits of Paste that are pretty straight-forward, but not necessarily important. E.g., a middleware that profiles the internal request and includes that in the output. And some parts that I think are really useful, but abstract in a way that might not make everyone comfortable -- like paste.urlmap for mounting applications, or paste.cascade for trying multiple apps in a row until one "succedes" (where success usually means doesn't-return-404). If stuff was put into the standard library, I think it would be important to consider up-front how the code would be backported. I'm not really happy with the ad hoc backporting of libraries that currently occurs. And I don't want to maintain the same code in Paste and a standard library module, or use cascading imports in my code. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ianb at colorstudy.com Sun Feb 5 00:21:45 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Sat, 04 Feb 2006 17:21:45 -0600 Subject: [Web-SIG] My original template API proposal Message-ID: <43E53709.4040007@colorstudy.com> Well, here's the proposal I had before, tweaked slightly. This is mostly like the TG/Buffet API, but with find_resource extracted. I realize this is the basis of the whole WSGI offshoot before, and maybe an API without that would be useful or easier to implement. But it would also be way more useful to me with a resource finding callback, and I know Kevin has also indicated he wants that feature in TG. It *is* hard to implement find_resource and will likely require updates or monkeypatching of existing template systems, because they don't always expose the internal process of fetching some other template. The API is here: http://svn.pythonpaste.org/home/ianb/templateapi/interface.py and copied below: """ These interfaces represent two sides of a resource-fetching implementation. ``IFindResource`` is meant to be implemented by the framework, and can implement things like search paths. ``IResourceType`` produces resources. Major open issues: * Where does caching occur? What are the promises we can make about caching? How can we know about the dependencies between templates, so we know when a template has to be recompiled because something it depends upon has been recompiled? * How will this really be used for non-template resources? Most immediately important is media files like CSS files, which are often changed in concert with template changes, but are usually just static. * Should the expected output type be separate from the expected input type? Right now IResourceType distinguishes template engines, but doesn't necessarily distinguish the expected mime type. Other questions are in [] in the docstrings below... """ class IResourceFinder: def __call__(resource_name, resource_type=None, relative_to_name=None, relative_to_object=None): """ Finds a resource by the given name. If this is being called *by* a resource, then both ``relative_to_name`` and ``relative_to_object`` should be given. The implementation of this callable is *not* part of a template plugin, but is something provided by the framework. This is only a callable; it could be created in such a way that it is bound to the current request in a web environment, or with any other implementation that seems appropriate. ``resource_name`` and ``relative_to_name`` are strings or unicode objects. ``relative_to_object`` is something that ``find_resource()`` itself returned. ``resource_type`` is the type of object we are looking for, an instance of IResourceType. If not given (or None) then this object should try to determine the type itself, possibly changing the name and calling itself recursively. [I think there should be some convention on what names should look like -- like filenames (with / separators), or like module names...? Should they include extensions?] """ class IResourceType: """ A kind of resource, like 'cheetah templates' or 'CSS files'. [Should this include some attribute with the mime type, extensions, name of the type, ...?] """ def load_resource(resource_name, find_resource, resource_stream, resource_location=None): """ Returns the resource. ``resource_name`` is an opaque identifier, but should be kept. ``find_resource`` is the implementation of ``IResourceFinder`` that is creating the resource. This same implementation should be used for any recursive resource fetching. The resource content is given in ``resource_stream``, a file-like object. [Does the caller close the file?] ``resource_location`` is a string identifier for the actual location of the resource. This might be a filename, or some other unambiguous representation of where the resource came from. This is meant for use in tracebacks or debugging, and is only meant to mean something to a human and is not used by the resource itself. """ class ITemplatePlugin(IResourceType): def __init__(extra_vars_func=None, options=None): """ Create a template plugin instance. ``extra_vars_func`` is a function that returns a dictionary of 'standard' variables for the environment. ``options`` is a dictionary of options for the template (e.g., compiler settings) with no defined contents. [Is ``extra_vars_func`` necessary? Couldn't it be handled by a wrapper around ``render``?] """ def load_resource(...): """ Return an instance of the template. No interface for the return value is defined, but the return value can be used with ``render``. Note that some templating languages don't work well with ``render``, so the returned resource may be the more useful interface, and frameworks should allow users to get at this return value directly. This value should typically be the 'native' template object for the template language. """ def render(template_instance, vars, format="html", fragment=False): """ Render the template instance (as returned by ``load_resource``) to a string or unicode object. [Should there be some way to return metadata about what was rendered, like the type of object being returned -- e.g., text/html or text/plain] ``vars`` is typically a dictionary (-like object) of variables for use in substitution. format may be 'xhtml', 'plain', 'elementtree' etc., but plugins may ignore that value. Depending on the value, the return value may not be a string. However, the caller should try to handle a string return value. For instance, if you ask for 'elementtree' and get a string response, you should be ready to call ``XML()`` on that string. [Should this optionally be a list of formats, in order of preference?] 'fragment' indicates if an incomplete document should be created (e.g., with no DOCTYPE); again templates may ignore this. """ From foom at fuhm.net Sun Feb 5 00:21:24 2006 From: foom at fuhm.net (James Y Knight) Date: Sat, 4 Feb 2006 18:21:24 -0500 Subject: [Web-SIG] WSGI in standard library (was: A trivial template API counter-proposal) In-Reply-To: <43E533A9.6080504@colorstudy.com> References: <5.1.1.6.0.20060204131722.02179008@mail.telecommunity.com> <43E533A9.6080504@colorstudy.com> Message-ID: On Feb 4, 2006, at 6:07 PM, Ian Bicking wrote: > I thought the entire point of the development of a better packaging system (eggs) is so that not everything needs to go into the standard library. James From guido at python.org Sun Feb 5 01:20:54 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 4 Feb 2006 16:20:54 -0800 Subject: [Web-SIG] WSGI in standard library (was: A trivial template API counter-proposal) In-Reply-To: <43E533A9.6080504@colorstudy.com> References: <5.1.1.6.0.20060204131722.02179008@mail.telecommunity.com> <43E533A9.6080504@colorstudy.com> Message-ID: On 2/4/06, Ian Bicking wrote: > Phillip J. Eby wrote: > >> Perhaps wsgiref > >>should go into the Python 2.5 standard library?) > > > > Sure. I'd suggest that we might want to expand it a little to include some > > of the fine contributions of other authors, such as Ian's "lint" middleware > > (which sits between an app and a server and checks both for compliance) and > > maybe even his "Python debugger in the browser" middleware. There have > > also been people who've written a few utilities for CGI-in-WSGI, WSGI in > > mod_python, etc. And it'd be nice to squeeze in a FastCGI implementation > > if somebody's got a suitable one. Docs are also nonexistent at the > > moment. I'd be happy to co-ordinate and consolidate the additions, and of > > course I'll write documentation, though any assistance people can offer > > would be greatly appreciated. Some people have written nice WSGI tutorials > > or explanations, so if anybody wants to offer those up for consolidation > > into docs, that would be good too. > > There's some pieces that I think are not yet ready for standardization. > For instance, the error catching middleware would deserve a much > larger discussion, I think, as it also involves the question of how to > extend tracebacks (it uses Zope's code that uses magic local variables). > Also, the interactive implementation is complex enough (including > Javascript) that I can't envision it stabablizing sufficiently. (Paste > includes a cgitb-based catching middleware as well) > > I think a WSGI-based HTTP server is fairly easy to stabalize. Clark > Evans contributed a server based on SimpleHTTPServer, which I think is > both pretty complete and doesn't do anything besides HTTP serving (e.g., > no static file serving): > http://svn.pythonpaste.org/Paste/trunk/paste/httpserver.py I'm confused. I downloaded wsgiref and it seemed to be implementing an HTTP server based on BaseHTTPServer. > The CGI server in wsgiref is also very stable -- there's not much for it > to do. Flup includes FastCGI, SCGI, and AJK, both forking and threaded. > I think all these pieces have relatively few dependencies, which could > be extracted. No forking WSGI HTTP server exists that I know of. > > paste.lint is pretty stable and complete, I think. > > With a little adaptation, paste.urlparser.StaticURLParser and > paste.urlparser.PkgResourceParser also could be useful -- these are > static file serving apps (they should be renamed, of course). Serving > with pkg_resources doesn't make sense unless pkg_resources is in the > standard library. But both those are based on paste.fileapp.FileApp -- > which serves a single file -- and while Clark has also done some work on > that and it is improved, I feel less confident that it is complete (but > it could be completed -- its scope is well-understood). Finding a file > is easy (that's what StaticURLParser does), but actually serving a file > with full HTTP support is a little more complex. Also support for > encoding and expiration times is missing from those -- mostly a matter > of passing related configuration. I think a good static file serving > implementation is really valuable. > > paste.cgiapp is relatively simple (runs a CGI script). I think > paste.proxy (just added) would be a really nice contribution to the > standard library -- an HTTP proxy to another server, appearing as a WSGI > application -- but the implementation is rough and only a couple of days > old. > > There's also a bunch of the routines in paste.request, paste.response, > and paste.wsgilib could be good for the standard library. There's also > some code in paste.httpheaders, which perhaps could be made simpler and > more primitive-feeling -- for things like parsing header values > according to their meaning in HTTP. > > There's several other bits of Paste that are pretty straight-forward, > but not necessarily important. E.g., a middleware that profiles the > internal request and includes that in the output. And some parts that I > think are really useful, but abstract in a way that might not make > everyone comfortable -- like paste.urlmap for mounting applications, or > paste.cascade for trying multiple apps in a row until one "succedes" > (where success usually means doesn't-return-404). Can I recommend that you be *extremely* conservative in suggesting stuff to go into the stdlib? Once it's in there, you have no way of upgrading it until the next major release (i.e. 2.6) which is typically an 18-24 months timeframe -- minor releases (e.g. 2.5.1) aren't allowed to add new features, only to fix bugs. EGGs and other 3rds party install methids (by design, actually) can't easily be used to upgrade standard library modules/packages. ISTM that wsgiref itself is stable enough, and having it in the stdlib would lower the bar for anybody to develop WSGI-compliant apps even if they refuse to use a toolkit. A lot of the other things you're mentioning above seem to be of a different kind -- nice-to-haves, for sure, but either they're not 100% stable yet, or they enter the slippery slope of appearing to be a specific framework choice (cgitb already has that feel to me). > If stuff was put into the standard library, I think it would be > important to consider up-front how the code would be backported. I'm > not really happy with the ad hoc backporting of libraries that currently > occurs. And I don't want to maintain the same code in Paste and a > standard library module, or use cascading imports in my code. I'm not sure what you mean by "backporting". Can you clarify? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sun Feb 5 02:01:03 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 4 Feb 2006 17:01:03 -0800 Subject: [Web-SIG] A trivial template API counter-proposal In-Reply-To: <5.1.1.6.0.20060204131722.02179008@mail.telecommunity.com> References: <5.1.1.6.0.20060204131722.02179008@mail.telecommunity.com> Message-ID: > >I am probably missing something, but shouldn't there also be an API to > >render the template to a string or stream given some context of > >variable names? I'm looking at this from a very different perspective, > >namely *using* various templating engines from an app that otherwise > >doesn't use a framework but still needs templating. > > The API I proposed assumes that the template engine gives you WSGI > application objects, and that you simply pipe them through to your WSGI > stack. That is, in the simplest case: > > def some_wsgi_app(environ, start_response): > template = some_engine.compile_stream(open("mytemplate.foo")) > environ['wsgi.params'] = {'some_var':'bar'} > return template(environ, start_response) > > The assumption here is that a "template" is just a WSGI application object, > that maybe has a few extra keys added to the environ, like a params dict > and/or a "published object". > > From current discussion, however, it seems like the above API will be > deferred a while to focus on a higher-level API in which you simply ask for > a template by an opaque ID, e.g.: > > def some_wsgi_app(environ, start_response): > template = some_engine.find_template("mytemplate") > environ['wsgi.params'] = {'some_var':'bar'} > return template(environ, start_response) > > So, it's the same except an even simpler, higher-level API. The API I > proposed is more useful in the context of a template manager (such as Zope > 3 skins/layers) that might need to support pluggable template > compilers. In such circumstances, the compile/write APIs I proposed would > be useful, but a different API layer than the "get me a template to run" API. > > Anyway, part of the reasoning for using WSGI itself as a template API is > that it allows for template languages that may need to manipulate response > headers and status, such as the ones that embed Python in HTML or HTML in > Python. Or more typically, templates being used to generate non-HTML text, > such as XML, tab-delimited files, or other kinds of downloads. It also > makes it possible for the template language to offer its own native API for > getting request data. For example, a ZPT template engine would be free to > rewrap the WSGI data to appear as standard Zope request/response objects > from the template's point of view. I see. But doesn't this still tie the templates API rather strongly to a web API? What if I want to use a template engine to generate spam^H^H^H^Hpersonalized emails? Or static HTML pages that are written to the filesystem and then served by a classic Apache setup? Or source code (program generators are fun toys!)? ISTM that a template that does anything besides producing the *content* of a download would be a rather extreme exception -- e.g. the decision to generate a redirect feels like *application* logic, while templates ought to limit themselves to *presentation* logic, and for that, an API that produces a string or writes to a stream seems to make more sense. What am I missing? Are you proposing that I should fake out the wsgi API and capture the strings passed to write() and the sequence returned? > >(PS having tried WSGI a bit now I'm fine with it. > > WSGI - it's no worse than a trip to East Berlin. :) Well, it *is* quite a trip to memory lane... Intellectually I understand that the CGI feel is simply practical; but emotionally, using environ["PATH_INFO"] feels like a big step back compared to request.path. > > Perhaps wsgiref > >should go into the Python 2.5 standard library?) > > Sure. I'd suggest that we might want to expand it a little to include some > of the fine contributions of other authors, such as Ian's "lint" middleware > (which sits between an app and a server and checks both for compliance) and > maybe even his "Python debugger in the browser" middleware. There have > also been people who've written a few utilities for CGI-in-WSGI, WSGI in > mod_python, etc. And it'd be nice to squeeze in a FastCGI implementation > if somebody's got a suitable one. Docs are also nonexistent at the > moment. I'd be happy to co-ordinate and consolidate the additions, and of > course I'll write documentation, though any assistance people can offer > would be greatly appreciated. Some people have written nice WSGI tutorials > or explanations, so if anybody wants to offer those up for consolidation > into docs, that would be good too. Adding support for mod_python in the stdlib seems excessive -- shouldn't that be included with mod_python instead? FastCGI OTOH seems handy to have. See my response to Ian recommending a conservative approach here. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ianb at colorstudy.com Sun Feb 5 02:12:55 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Sat, 04 Feb 2006 19:12:55 -0600 Subject: [Web-SIG] WSGI in standard library In-Reply-To: References: <5.1.1.6.0.20060204131722.02179008@mail.telecommunity.com> <43E533A9.6080504@colorstudy.com> Message-ID: <43E55117.9040007@colorstudy.com> Guido van Rossum wrote: >>I think a WSGI-based HTTP server is fairly easy to stabalize. Clark >>Evans contributed a server based on SimpleHTTPServer, which I think is >>both pretty complete and doesn't do anything besides HTTP serving (e.g., >>no static file serving): >>http://svn.pythonpaste.org/Paste/trunk/paste/httpserver.py > > > I'm confused. I downloaded wsgiref and it seemed to be implementing an > HTTP server based on BaseHTTPServer. Yes, I had forgotten about that. I myself haven't used wsgiref that much. It would be useful to analyse the differences between these two implementations. Paste's HTTP server has some more HTTP 1.1 features, but that might actually be problematic for the standard library. > Can I recommend that you be *extremely* conservative in suggesting > stuff to go into the stdlib? Once it's in there, you have no way of > upgrading it until the next major release (i.e. 2.6) which is > typically an 18-24 months timeframe -- minor releases (e.g. 2.5.1) > aren't allowed to add new features, only to fix bugs. EGGs and other > 3rds party install methids (by design, actually) can't easily be used > to upgrade standard library modules/packages. ISTM that wsgiref itself > is stable enough, and having it in the stdlib would lower the bar for > anybody to develop WSGI-compliant apps even if they refuse to use a > toolkit. A lot of the other things you're mentioning above seem to be > of a different kind -- nice-to-haves, for sure, but either they're not > 100% stable yet, or they enter the slippery slope of appearing to be a > specific framework choice (cgitb already has that feel to me). Yes, I'm not really seriously proposing anything now, just brainstorming things. Phillip brought up the interactive debugger, and thought there were *lots* of things far more appropriate than that; that doesn't necessarily mean those parts are really appropriate. Also, I suppose there's no reason that the standard library has to be the landing place for "standard" code. Stuff in the standard library needs to have a PEP, but a PEP can describe something that never intends to go in the standard library. >>If stuff was put into the standard library, I think it would be >>important to consider up-front how the code would be backported. I'm >>not really happy with the ad hoc backporting of libraries that currently >>occurs. And I don't want to maintain the same code in Paste and a >>standard library module, or use cascading imports in my code. > > > I'm not sure what you mean by "backporting". Can you clarify? Well, for instance optik and logging are provided for installation on older versions of Python. And PyXML. But these aren't consistently backported, enhancements are often not backported, and what backports that exist aren't listed anywhere. And users have to do various kinds of cascading imports if they want compatibility. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From pje at telecommunity.com Sun Feb 5 02:41:50 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 04 Feb 2006 20:41:50 -0500 Subject: [Web-SIG] WSGI in standard library (was: A trivial template API counter-proposal) In-Reply-To: References: <43E533A9.6080504@colorstudy.com> <5.1.1.6.0.20060204131722.02179008@mail.telecommunity.com> <43E533A9.6080504@colorstudy.com> Message-ID: <5.1.1.6.0.20060204203451.021532a0@mail.telecommunity.com> At 04:20 PM 2/4/2006 -0800, Guido van Rossum wrote: >ISTM that wsgiref itself >is stable enough, and having it in the stdlib would lower the bar for >anybody to develop WSGI-compliant apps even if they refuse to use a >toolkit. A lot of the other things you're mentioning above seem to be >of a different kind -- nice-to-haves, for sure, but either they're not >100% stable yet, or they enter the slippery slope of appearing to be a >specific framework choice (cgitb already has that feel to me). Good point. I'd still like to throw in Ian's "lint" tool or something like it, though, since wsgiref only implements app compliancy tests, not server compliance. But most of the other stuff is still evolving too much, I agree. wsgiref, OTOH, doesn't change much because the spec hasn't changed, and it's not particularly ambitious anyway. I wrote it with stdlib-ish conservatism in mind, i.e. no fancy stuff. :) FastCGI would also be nice, but it's easy to install fancy things with eggs now; it makes more sense to put the core and slowly-changing functionality in the stdlib. From floydophone at gmail.com Sun Feb 5 03:35:56 2006 From: floydophone at gmail.com (Peter Hunt) Date: Sat, 4 Feb 2006 21:35:56 -0500 Subject: [Web-SIG] WSGI in standard library Message-ID: <6654eac40602041835u22e10470ia6dc1990d2d8dcdb@mail.gmail.com> I think CherryPy's WSGI server should go in: it's stable, and the best-performing WSGI HTTP server out there. Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/web-sig/attachments/20060204/4e03ab63/attachment.html From srichter at cosmos.phy.tufts.edu Sun Feb 5 12:17:38 2006 From: srichter at cosmos.phy.tufts.edu (Stephan Richter) Date: Sun, 5 Feb 2006 06:17:38 -0500 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <6654eac40602041835u22e10470ia6dc1990d2d8dcdb@mail.gmail.com> References: <6654eac40602041835u22e10470ia6dc1990d2d8dcdb@mail.gmail.com> Message-ID: <200602050617.38943.srichter@cosmos.phy.tufts.edu> On Saturday 04 February 2006 21:35, Peter Hunt wrote: > I think CherryPy's WSGI server should go in: it's stable, and the > best-performing WSGI HTTP server out there. Are you sure? Is it really scalable? Can you prove it? Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training From pywebsig at xhaus.com Sun Feb 5 14:12:42 2006 From: pywebsig at xhaus.com (Alan Kennedy) Date: Sun, 05 Feb 2006 13:12:42 +0000 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <6654eac40602041835u22e10470ia6dc1990d2d8dcdb@mail.gmail.com> References: <6654eac40602041835u22e10470ia6dc1990d2d8dcdb@mail.gmail.com> Message-ID: <43E5F9CA.9010001@xhaus.com> [Peter Hunt] > I think CherryPy's WSGI server should go in: it's stable, and the > best-performing WSGI HTTP server out there. I disagree. I think that if a WSGI server is to go into the standard library, it should be the most basic one possible, e.g. one that builds on the *HttpServer.py hierarchy already there, and one that makes it as easy as possible for coders to understand how WSGI works. HTTP servers can be complex beasts. Security is a major consideration, robustness and stability being next. Performance is also a major concern, with flexibility and ease-of-use being important as well. That's too many concerns to balance against each other for a python library module. Instead, I think the right approach is to continue with the existing approach: put the most basic possible WSGI server in the standard library, for educational purposes only, and a warning that it shouldn't really be used for production purposes. The following quote is from the docstring of the CGIHTTPServer module """ In all cases, the implementation is intentionally naive -- all requests are executed sychronously. SECURITY WARNING: DON'T USE THIS CODE UNLESS YOU ARE INSIDE A FIREWALL -- it may execute arbitrary Python code or external programs. """ And that's a good thing. If I really want to use python CGI, then I should find a robust HTTP server which supports it, e.g. Apache. The same reasoning should apply to WSGI, IMHO. Just another ?0,02. Regards, Alan. From pywebsig at xhaus.com Sun Feb 5 14:28:39 2006 From: pywebsig at xhaus.com (Alan Kennedy) Date: Sun, 05 Feb 2006 13:28:39 +0000 Subject: [Web-SIG] A trivial template API counter-proposal In-Reply-To: References: <5.1.1.6.0.20060204131722.02179008@mail.telecommunity.com> Message-ID: <43E5FD87.10106@xhaus.com> [Guido van Rossum] > I see. But doesn't this still tie the templates API rather strongly to > a web API? What if I want to use a template engine to generate > spam^H^H^H^Hpersonalized emails? Or static HTML pages that are written > to the filesystem and then served by a classic Apache setup? Or source > code (program generators are fun toys!)? > > ISTM that a template that does anything besides producing the > *content* of a download would be a rather extreme exception -- e.g. > the decision to generate a redirect feels like *application* logic, > while templates ought to limit themselves to *presentation* logic, and > for that, an API that produces a string or writes to a stream seems to > make more sense. What am I missing? Are you proposing that I should > fake out the wsgi API and capture the strings passed to write() and > the sequence returned? At last! A voice of sanity! I've been dismayed over the last few days trying to follow the direction of this thread: it appears to me that something very simple has now become very complex. Templating is about taking a pattern of bytes (the template), somehow mixing it with user data (the user context), and generating a series of bytes (the output). Full stop. In relation to text, this means taking a textual template, with embedded code/processing-instructions/whatever, "cooking" it in a user namespace, delivering a final piece of output. With text, the only major concern is with character encoding. And if I were designing a templating API, I'd make everything unicode-only, leaving the user responsible for transcoding to their desired encoding at serialisation time. I can understand why the web-sig has fallen into the trap of tying a tmeplating API to its nice web standard, WSGI: all web applications must generate output. But web apps need to generate a wide range of media types, e.g. image/*, application/msword, etc, etc, etc. This topic started with Buffet, the de-facto standard templating API for CherryPy. Buffet is just about textual templating, which is a good thing. That's why it's very simple, and is thus actually being used. Perhaps web-sig is the wrong place to discuss a textual templating API. Maybe xml-sig would be a better place, or a new text-sig should be formed? In relation to Guido's point above about usage scenarios for this API: I'm quite interested because I have a jython implementation of ZPT/TAL that I'll be open-sourcing in the coming weeks, and which I intend to make compatible with whatever API is produced by this current discussion. I used that TAL implementation to generate the documentation for various things, usually just a flat set of HTML files in a directory: not a HTTP request in sight. Theoretically, I can envision a situation where I might want to swap TAL implementations for that offline generation process, meaning that it would be helpful to have a standardised API for controlling template cooking. Why should I have to use WSGI in that scenario? Just my ?0,02. Alan. From pje at telecommunity.com Sun Feb 5 18:40:12 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 05 Feb 2006 12:40:12 -0500 Subject: [Web-SIG] A trivial template API counter-proposal Message-ID: <5.1.1.6.0.20060205121213.021562b8@mail.telecommunity.com> >I see. But doesn't this still tie the templates API rather strongly to >a web API? What if I want to use a template engine to generate >spam^H^H^H^Hpersonalized emails? Then set the content-type to message/rfc822. ;) > Or static HTML pages that are written >to the filesystem and then served by a classic Apache setup? Or source >code (program generators are fun toys!)? Use the output file's write() method as the "write" callback returned by start_response(), or use one of the wsgiref handlers that you give arbitrary file-like objects to. >ISTM that a template that does anything besides producing the >*content* of a download would be a rather extreme exception -- e.g. >the decision to generate a redirect feels like *application* logic, >while templates ought to limit themselves to *presentation* logic, Not every framework developer agrees with you on this point; some explicitly eschew the separation of presentation and application, although I personally tend to agree with you. In the absence of BDFL pronouncement on which frameworks are "right" or "wrong", my policy has been to advocate framework neutrality - to the extent I'm able to identify and get past my own biases, of course. For ASP/PHP-style frameworks, it's routine for the "template" to make these kinds of response-control decisions, and it's certainly seen a lot in Zope 2 applications using DTML as well. Even if only for backward compatibility, it seems wrong to me to leave all these applications unable to operate under the new standard. (Also, even in frameworks where logic and presentation are kept separate, it doesn't necessarily mean that a template doesn't still need to control its content-type, or that code invoked from the template in order to access data may not result in data being added to the response headers. In Zope 3, a "view" can do such things, as might components rendered in the view.) >and >for that, an API that produces a string or writes to a stream seems to >make more sense. What am I missing? Are you proposing that I should >fake out the wsgi API and capture the strings passed to write() and >the sequence returned? In that case, why don't you just use the template engine directly? But I don't see how such an API is a *web* templating API in that case. Shouldn't such things just follow the stdlib string template API? Why do we need a *web* standard for string templating? As for capturing the output, it's pretty trivial to do if for some reason you have to: handler = BaseCGIHandler(StringIO(""),StringIO(),StringIO(),environ) handler.run(app) output = handler.stdout.getvalue().split('\r\n\r\n',1)[1] This could probably be added as a utility function to wsgiref, or perhaps a simpler and more efficient form thereof. By contrast, to allow response-manipulating templates with a non-web API, there would have to be some way to provide WSGI or a WSGI-like API thereto. So far, nobody has proposed a way of doing that to be included in the templating standard. Ian has suggested that one could stick some API functions in the variables passed to the template, but hasn't proposed a standard for doing so. My counterproposal is that since WSGI already exists, it isn't necessary to create an entirely new standard for same. Currently, the progress of the discussion I've been having with Ben Bangert is that we would split the templating API into two or three layers. The layer I'm describing now would be part of "WSGI Linking" and the kind of API you're describing would be an optional extension to "WSGI Embedding". Since not all template systems can reasonably be used without access to a request/response, it seems to me that the string-only facility should be considered an optional extension. While it means that there will be at least three different parts to the spec, it will hopefully be clearer what different frameworks and templates do and don't do with regard to using each other's templates or vice versa. > > >(PS having tried WSGI a bit now I'm fine with it. > > > > WSGI - it's no worse than a trip to East Berlin. :) > >Well, it *is* quite a trip to memory lane... Intellectually I >understand that the CGI feel is simply practical; but emotionally, >using environ["PATH_INFO"] feels like a big step back compared to >request.path. The idea of course was to allow existing framework APIs to wrap WSGI, so that you can have request.path or REQUEST.path_info or whatever suits your fancy. It was never intended that anybody but framework developers write "bare metal" WSGI code, and such developers have to touch either CGI or HTTP in their code anyway. From guido at python.org Sun Feb 5 18:48:28 2006 From: guido at python.org (Guido van Rossum) Date: Sun, 5 Feb 2006 09:48:28 -0800 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <43E5F9CA.9010001@xhaus.com> References: <6654eac40602041835u22e10470ia6dc1990d2d8dcdb@mail.gmail.com> <43E5F9CA.9010001@xhaus.com> Message-ID: > [Peter Hunt] > > I think CherryPy's WSGI server should go in: it's stable, and the > > best-performing WSGI HTTP server out there. [Alan Kennedy] > I disagree. > > I think that if a WSGI server is to go into the standard library, it > should be the most basic one possible, e.g. one that builds on the > *HttpServer.py hierarchy already there, and one that makes it as easy as > possible for coders to understand how WSGI works. Right. > HTTP servers can be complex beasts. Security is a major consideration, > robustness and stability being next. Performance is also a major > concern, with flexibility and ease-of-use being important as well. > > That's too many concerns to balance against each other for a python > library module. Exactly. > Instead, I think the right approach is to continue with the existing > approach: put the most basic possible WSGI server in the standard > library, for educational purposes only, and a warning that it shouldn't > really be used for production purposes. > > The following quote is from the docstring of the CGIHTTPServer module > > """ > In all cases, the implementation is intentionally naive -- all > requests are executed sychronously. > > SECURITY WARNING: DON'T USE THIS CODE UNLESS YOU ARE INSIDE A FIREWALL > -- it may execute arbitrary Python code or external programs. > """ > > And that's a good thing. If I really want to use python CGI, then I > should find a robust HTTP server which supports it, e.g. Apache. > > The same reasoning should apply to WSGI, IMHO. Except for the security warning, probably -- that only applies to CGIHttpServer.py, not to BasicHTTPServer.py. (It does in a sense apply to SimpleHTTPServer.py since that serves up the current directory.) Thanks for correctly channeling the sentiment behind the standard library server hierarchy! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Sun Feb 5 19:40:41 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 05 Feb 2006 13:40:41 -0500 Subject: [Web-SIG] A trivial template API counter-proposal In-Reply-To: <43E5FD87.10106@xhaus.com> References: <5.1.1.6.0.20060204131722.02179008@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060205124025.021885b0@mail.telecommunity.com> At 01:28 PM 2/5/2006 +0000, Alan Kennedy wrote: >I've been dismayed over the last few days trying to follow the direction >of this thread: it appears to me that something very simple has now >become very complex. On the contrary, the situation is and always has been very complex. It only appears simple if you restrict your view to a particular narrow set of use cases and ignore everyone else's. Python web frameworks are extremely diverse in audience, needs, philosophies, and implementation. A "standard" that doesn't take this into consideration is divisive, when what we need to be doing is unifying. Having a standard that takes diversity into consideration may be slightly more complex, but it allows *consolidation* to occur over time. The *real* additional complexity here is communication. Since each framework has its own conceptual model of how things work, we've been spending a lot of time just learning to understand each other. While time consuming, it is absolutely a worthwhile effort if your goal is to get everybody closer together. If we only discuss what we already have in common, we are simply dividing the world into "us" and "them", preaching to the choir and getting nowhere new. That is why I've been fighting to hold off a mere rubber-stamping of a non-web standard that happens to work for one family of frameworks; it would be better for all of us if those use cases are handled as part of a spec that brings everybody closer together. That's not going to be as easy, true. It certainly hasn't been easy for the people discussing it so far (myself absolutely included), and it certainly won't be for the people reading the spec or implementing it. However, not everything that is worthwhile is easy. Developing WSGI was not easy, either, as I'm sure you recall. You and I certainly argued a bit about iterators and file-like objects and such, and it took a while before I understood all of your use cases and we were able to resolve them in the spec. If you had given up on convincing me then, or if I had given up on your use cases as "too complex", the spec would have suffered for it. (And you wouldn't have gotten that prominent credit in the PEP 333 "Acknowledgments" section. ;) ) >I can understand why the web-sig has fallen into the trap of tying a >tmeplating API to its nice web standard, WSGI: all web applications must >generate output. But web apps need to generate a wide range of media >types, e.g. image/*, application/msword, etc, etc, etc. And in many frameworks, it is the *template* that decides what media type it is generating - and it may not even be outputting text or unicode. Again, this is something that would be neglected by a text-only spec. >This topic started with Buffet, the de-facto standard templating API for >CherryPy. Buffet is just about textual templating, which is a good >thing. That's why it's very simple, and is thus actually being used. Which is precisely why we don't need to rush into blessing it as some kind of web standard, when it doesn't have anything to do with the "web", as far as I can tell. If I needed a plugin for pure text or XML templating (as opposed to *web application* templates), or I were creating a pure text or XML output generator, I would absolutely use the TG/Buffet API, no question. It is 100% the right tool for the job, and the de facto standard as well. But I'm also 100% positive that it is *not* suitable to be declared a Python *web* standard by itself: it's not about the web at all. The web just happens to be where it's being used at the moment, and only in frameworks where the use cases slot neatly together with it. Heck, just look at the API documentation and see if you can find the word "web" or even "request" mentioned on the page: http://www.turbogears.org/docs/plugins/template.html >I used that TAL implementation to generate the documentation for various >things, usually just a flat set of HTML files in a directory: not a HTTP >request in sight. Theoretically, I can envision a situation where I >might want to swap TAL implementations for that offline generation >process, meaning that it would be helpful to have a standardised API for >controlling template cooking. > >Why should I have to use WSGI in that scenario? You don't have to. As I mentioned many times in the discussion on subtemplating, nothing stops template objects from offering *additional* APIs besides WSGI. It's just that if we're making a *web* standard here, then it should be possible for templates to do *web* things, not just play with strings. Ergo, the string part of the API should be an optional extension, or a standard part of the API but allowed to throw exceptions if the template tries to do something that requires WSGI. In contrast, I've listed numerous scenarios and use cases where providing WSGI or something like it is absolutely essential. Nobody is proposing any alternatives that would be part of the standard, as opposed to non-standard conventions. I want to make sure that *your* use cases are supported by the spec. In the absolute worst case scenario, you might have to do a little wrapping to meet them. But I also want to support *my* use cases, and those of other frameworks with similar ones. Those cases can't be supported without a *web* API, which TG/Buffet does not provide. We're all having this conversation because we want to achieve consensus so we can achieve something in common, and use each other's tools. But we're not going to have consensus while people's use cases are being left out altogether. Nor will we have the greatest interoperability of tools if we exclude entire classes of frameworks and templates from playing. I believe that we can have a spec that allows for all the use cases being discussed. Certainly, it will not be as simple as the TG/Buffet API. However, if what you want is only the TG/Buffet API with some tweaks, then you are saying you do not want co-operation with at least Zope and Python server pages (which latter group includes at least users of Spyce, Webware, Snakecharmer, and mod_python, as far as I can tell). And that would seem to me to be a big enough subset of Python web users/developers to say you don't want a Python web standard at all. Why you'd want that, I'm not sure. Note that making it possible for people to run their existing code in *your* frameworks means that they have the option of migrating to your framework. But if you don't support that, they can't. It seems to me that it is in framework authors' best interest, then, to acknowledge the reality of these existing codebases and what will be required to let them migrate to another system. From guido at python.org Sun Feb 5 19:40:38 2006 From: guido at python.org (Guido van Rossum) Date: Sun, 5 Feb 2006 10:40:38 -0800 Subject: [Web-SIG] A trivial template API counter-proposal In-Reply-To: <5.1.1.6.0.20060205121213.021562b8@mail.telecommunity.com> References: <5.1.1.6.0.20060205121213.021562b8@mail.telecommunity.com> Message-ID: [Me] > > Or static HTML pages that are written > >to the filesystem and then served by a classic Apache setup? Or source > >code (program generators are fun toys!)? [Phillip] > Use the output file's write() method as the "write" callback returned by > start_response(), or use one of the wsgiref handlers that you give > arbitrary file-like objects to. I didn't see an output file in the proposed standard. All I saw was a WSGI interface. The output file is a figment of your implementation. > >ISTM that a template that does anything besides producing the > >*content* of a download would be a rather extreme exception -- e.g. > >the decision to generate a redirect feels like *application* logic, > >while templates ought to limit themselves to *presentation* logic, > > Not every framework developer agrees with you on this point; some > explicitly eschew the separation of presentation and application, although > I personally tend to agree with you. In the absence of BDFL pronouncement > on which frameworks are "right" or "wrong", my policy has been to advocate > framework neutrality - to the extent I'm able to identify and get past my > own biases, of course. It is my strong preference that the standard API we are attempting to design here does *not* tie you strongly to WSGI or the generation of dynamic web content. > For ASP/PHP-style frameworks, it's routine for the "template" to make these > kinds of response-control decisions, and it's certainly seen a lot in Zope > 2 applications using DTML as well. Even if only for backward > compatibility, it seems wrong to me to leave all these applications unable > to operate under the new standard. There's nothing wrong with having some things that can invoked from the template that set or add headers behind your back, if the template engine supports such a thing. But I think the API to let it do so might not be part of the standard, or it might be an optional part of the standard; I want to be able to use a templating engine in a non-web context and still use it (of course I won't be able to set headers then, but I'd assume this functionality is optional and I can just elect not to use it). > (Also, even in frameworks where logic and presentation are kept separate, > it doesn't necessarily mean that a template doesn't still need to control > its content-type, or that code invoked from the template in order to access > data may not result in data being added to the response headers. In Zope > 3, a "view" can do such things, as might components rendered in the view.) I'm okay if the proposed standard doesn't provide access to every aspect of Zope 3 templates. If I were intereste in using the web-specific features of Zope3 templates, I should probably be using Zope 3. > >and > >for that, an API that produces a string or writes to a stream seems to > >make more sense. What am I missing? Are you proposing that I should > >fake out the wsgi API and capture the strings passed to write() and > >the sequence returned? > > In that case, why don't you just use the template engine directly? Because every template engine has a @#$%! different API!!!! The whole *point* of asking for a standard template API was to reduce the effort needed to switch template engines. I realize that I still need to rewrite my templates to conform to the new template syntax, and I may need to provide my context in a different form because not all templating engines have the same power of reaching into objects to extract information. But I still see the different template APIs as an unnecessary barrier to switching. Even if I'm not switching within one project, I may have to maintain two projects that use different templating engines. Like the DB-API standard, having analogous APIs helps. > But I > don't see how such an API is a *web* templating API in that > case. Shouldn't such things just follow the stdlib string template > API? Why do we need a *web* standard for string templating? I didn't ask for a *web* templating API; I suspect you've been reading too much into my request. I specifically asked for a *standard templating API*. Note that both Cheetah and Django templates *have* such an API, and provide examples of how to use them -- the grating part is that they are gratuitously *different*. > As for capturing the output, it's pretty trivial to do if for some reason > you have to: > > handler = BaseCGIHandler(StringIO(""),StringIO(),StringIO(),environ) > handler.run(app) > output = handler.stdout.getvalue().split('\r\n\r\n',1)[1] This assumes wegiref though. I'd rather not make wsgiref a required part of the standard. > This could probably be added as a utility function to wsgiref, or perhaps a > simpler and more efficient form thereof. No, I'd like to see the templating API and WSGI completely decoupled, except for an optional part where in the evaluation context passed into a template at render time, you pass in an object that lets the template manipulate headers etc. It would be fine if the latter used WSGI somehow. But there *must* be a standard way yo invoke the template engine (context in, string or stream out) without assuming it's an HTTP response. > By contrast, to allow response-manipulating templates with a non-web API, > there would have to be some way to provide WSGI or a WSGI-like API > thereto. So far, nobody has proposed a way of doing that to be included in > the templating standard. Ian has suggested that one could stick some API > functions in the variables passed to the template, but hasn't proposed a > standard for doing so. My counterproposal is that since WSGI already > exists, it isn't necessary to create an entirely new standard for same. Well, I don't like it. :-( > Currently, the progress of the discussion I've been having with Ben Bangert > is that we would split the templating API into two or three layers. The > layer I'm describing now would be part of "WSGI Linking" and the kind of > API you're describing would be an optional extension to "WSGI > Embedding". Since not all template systems can reasonably be used without > access to a request/response, it seems to me that the string-only facility > should be considered an optional extension. How can it be considered a templating system if it doesn't have access to a request/response? Surely somewhere in the bowels is an engine that renders a template in a context to a text stream. The rest ought to be considered part of the web framework that hots the templating engine. ISTM we may be confused (or simply disagreeing) about the boundary between the templating engine and the framework. > While it means that there will be at least three different parts to the > spec, it will hopefully be clearer what different frameworks and templates > do and don't do with regard to using each other's templates or vice versa. Please keep coming up with proposals and rationales for them. I will certainly admit that I haven't used a Python web framework in a while (not since I left Zope :-). But when I see Cheetah and Django and some competitors that were mentioned in responses to my blogs, a certain "Platonic ideal" of a templating engine emerges in my brain, and I'd like at least the API for that standardardized, if at all possible, so people writing frameworks can choose from the many templating engines available, and people with templating needs *outside* a web context can have the same selection. > > > >(PS having tried WSGI a bit now I'm fine with it. > > > > > > WSGI - it's no worse than a trip to East Berlin. :) > > > >Well, it *is* quite a trip to memory lane... Intellectually I > >understand that the CGI feel is simply practical; but emotionally, > >using environ["PATH_INFO"] feels like a big step back compared to > >request.path. > > The idea of course was to allow existing framework APIs to wrap WSGI, so > that you can have request.path or REQUEST.path_info or whatever suits your > fancy. It was never intended that anybody but framework developers write > "bare metal" WSGI code, and such developers have to touch either CGI or > HTTP in their code anyway. Yeah, but potentially this causes several translations of the data (especially since WSGI requires a *real* dict, not something with mapping behavior) if the server's native API doesn't come in he form of CGI-compatible environment variables. This seems counter to the desire for it to be fast. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Sun Feb 5 20:34:36 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 05 Feb 2006 14:34:36 -0500 Subject: [Web-SIG] A trivial template API counter-proposal In-Reply-To: References: <5.1.1.6.0.20060205121213.021562b8@mail.telecommunity.com> <5.1.1.6.0.20060205121213.021562b8@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060205135745.0371d938@mail.telecommunity.com> At 10:40 AM 2/5/2006 -0800, Guido van Rossum wrote: >[Me] > > > Or static HTML pages that are written > > >to the filesystem and then served by a classic Apache setup? Or source > > >code (program generators are fun toys!)? > >[Phillip] > > Use the output file's write() method as the "write" callback returned by > > start_response(), or use one of the wsgiref handlers that you give > > arbitrary file-like objects to. > >I didn't see an output file in the proposed standard. All I saw was a >WSGI interface. The output file is a figment of your implementation. You said you wanted to generate static HTML files in the file system; that's the "output file" I meant. >It is my strong preference that the standard API we are attempting to >design here does *not* tie you strongly to WSGI or the generation of >dynamic web content. That seems to be a popular position, to be sure. However, since there is already a de facto standard support for that simpler use case, that is already supported by two frameworks for at least 6 templating systems(!) it seems that it's already both a solved problem and outside the scope of the Web-SIG. You already *have* this standard API; why shouldn't the web-sig then focus on something that supports *web* templating, not just strings? > > For ASP/PHP-style frameworks, it's routine for the "template" to make these > > kinds of response-control decisions, and it's certainly seen a lot in Zope > > 2 applications using DTML as well. Even if only for backward > > compatibility, it seems wrong to me to leave all these applications unable > > to operate under the new standard. > >There's nothing wrong with having some things that can invoked from >the template that set or add headers behind your back, if the template >engine supports such a thing. But I think the API to let it do so >might not be part of the standard, or it might be an optional part of >the standard; I want to be able to use a templating engine in a >non-web context and still use it (of course I won't be able to set >headers then, but I'd assume this functionality is optional and I can >just elect not to use it). My goal here is to make it so that the standard encourages web frameworks to offer the WSGI capabilities to the template. If this is not part of the standard, there is no encouragement, there are simply scattered non-standard APIs, if anything at all. An example of why I'd want this: legacy code written in DTML and Python ASP. If I'm migrating a bunch of it, I'd like the option to stick it in a new framework en masse, but actually port/convert one template at a time to newer approaches. Any framework that uses only a string-based approach, isn't going to allow porting this code. A WSGI interface makes it possible to have the template use its native web APIs. One could create Zope request/response objects wrapping WSGI for use in the template, thus allowing the template to initially remain unchanged. >I'm okay if the proposed standard doesn't provide access to every >aspect of Zope 3 templates. If I were intereste in using the >web-specific features of Zope3 templates, I should probably be using >Zope 3. And what if you're trying to port something away from Zope 3? or Zope 2? What if you are porting Python Server Pages *to* Zope 3? DTML to TurboGears? I'm trying to make it so that these kinds of embeddings are possible, so that we can start consolidating frameworks. This is a Prisoner's Dilemna situation to some extent. Nobody wants to do extra work, even though we all want the framework situation to improve. Few people want *their* framework to be the one that goes away, either. > > >and > > >for that, an API that produces a string or writes to a stream seems to > > >make more sense. What am I missing? Are you proposing that I should > > >fake out the wsgi API and capture the strings passed to write() and > > >the sequence returned? > > > > In that case, why don't you just use the template engine directly? > >Because every template engine has a @#$%! different API!!!! There's a bunch now that use the TG/Buffet plugin API. Why invent a *new* standard if all you want is strings? > > But I > > don't see how such an API is a *web* templating API in that > > case. Shouldn't such things just follow the stdlib string template > > API? Why do we need a *web* standard for string templating? > >I didn't ask for a *web* templating API; I suspect you've been reading >too much into my request. Well, it's being discussed on the Web-SIG. Note that this discussion started before your web framework posts, as did the TG/Buffet template API, so it's not really "your" request that we're talking about here. >But there *must* be a standard way yo invoke the >template engine (context in, string or stream out) without assuming >it's an HTTP response. See http://www.turbogears.org/docs/plugins/template.html for the documentation on how to do this now. Not to mention: http://projects.dowski.com/projects/buffet , which lists the currently-available plugins for Stan, Cheetah, Kid, string.Template, Myghty, and XSLT. > > Currently, the progress of the discussion I've been having with Ben Bangert > > is that we would split the templating API into two or three layers. The > > layer I'm describing now would be part of "WSGI Linking" and the kind of > > API you're describing would be an optional extension to "WSGI > > Embedding". Since not all template systems can reasonably be used without > > access to a request/response, it seems to me that the string-only facility > > should be considered an optional extension. > >How can it be considered a templating system if it doesn't have access >to a request/response? But that's what you're proposing. If there are only variables in and a string out, there is no request/response, and no ability to use the request/response API that would normally be used with that templating language. With my approach, if you were using a DTML template plugin, you would get to use the REQUEST/RESPONSE objects you normally use, since the template engine would be wrapping the WSGI call with them. This would make porting templates between frameworks *much* easier. > Surely somewhere in the bowels is an engine >that renders a template in a context to a text stream. The rest ought >to be considered part of the web framework that hots the templating >engine. ISTM we may be confused (or simply disagreeing) about the >boundary between the templating engine and the framework. In my discussion with Ben Bangert and others to date, it's very clear that there is no common terminology or conceptual model as soon as you get outside of a relatively narrow range of frameworks. The frameworks involved in the TG/Buffet standard have a model that looks like this: * framework calls some Python code you wrote * your code returns a dictionary of values you want rendered * framework passes your returned values to the template, gets back a string * framework sends the string back to the browser There is *no* request/response API available from the template in this model, which is why the folks who use this model don't see a reason why anybody would want them. Meanwhile, it seems like you're implicitly assuming that *of course* the template would have some kind of request/response, or maybe I'm misunderstanding you now. The point is, we've barely explored this enough to understand what models are in use. I think I have enough info now to draft a proposal that covers all the bases that have been brought up so far, but of course there's no guarantee that that's all the bases that exist. > > While it means that there will be at least three different parts to the > > spec, it will hopefully be clearer what different frameworks and templates > > do and don't do with regard to using each other's templates or vice versa. > >Please keep coming up with proposals and rationales for them. I will >certainly admit that I haven't used a Python web framework in a while >(not since I left Zope :-). But when I see Cheetah and Django and some >competitors that were mentioned in responses to my blogs, a certain >"Platonic ideal" of a templating engine emerges in my brain, and I'd >like at least the API for that standardardized, if at all possible, so >people writing frameworks can choose from the many templating engines >available, and people with templating needs *outside* a web context >can have the same selection. A non-web de facto standard already exists, and I would certainly encourage people to support it. I think the *web* standard should include it or something very close to it as an optional extension. I just view that part of the problem as mostly solved, but not very germane to actual template portability. The problem that I see is that everybody seems to me to be saying, "real template portability is impossible", and when I say, "but just support WSGI and everything is there", they go, "but that's too hard". It seems to me that this is a reflection of the a priori belief that it's impossible, rather than an actual reflection of the difficulty. Most frameworks now already have code to turn WSGI into whatever their existing request/response API is. And some frameworks already have WSGI passthrough capability to bring WSGI down to where they would need it to call a template using it. > > The idea of course was to allow existing framework APIs to wrap WSGI, so > > that you can have request.path or REQUEST.path_info or whatever suits your > > fancy. It was never intended that anybody but framework developers write > > "bare metal" WSGI code, and such developers have to touch either CGI or > > HTTP in their code anyway. > >Yeah, but potentially this causes several translations of the data >(especially since WSGI requires a *real* dict, not something with >mapping behavior) if the server's native API doesn't come in he form >of CGI-compatible environment variables. This seems counter to the >desire for it to be fast. Note that using "WSGI all the way down" to the template level means you have at most two translations: HTTP->WSGI->API, and the API can be a lazy wrapper over WSGI. However, being *fast* was never a WSGI goal; interoperability and portability are more important if we're ever to reduce the chaos around web frameworks. Even if you were to now Pronounce a One True Framework, there are still millions of lines of code already out there, and migration paths are essential. That has always been my vision for WSGI, and I always thought it was going to be a *lot* easier to agree on and write than it actually was. Even on this second go-round, I'm being surprised by lots of interesting little twists that I hadn't thought of. From ben at groovie.org Sun Feb 5 20:47:09 2006 From: ben at groovie.org (Ben Bangert) Date: Sun, 5 Feb 2006 11:47:09 -0800 Subject: [Web-SIG] A trivial template API counter-proposal In-Reply-To: References: <5.1.1.6.0.20060205121213.021562b8@mail.telecommunity.com> Message-ID: <62388AF2-7B46-42AB-ADC2-A008BD87BE91@groovie.org> On Feb 5, 2006, at 10:40 AM, Guido van Rossum wrote: > I didn't see an output file in the proposed standard. All I saw was a > WSGI interface. The output file is a figment of your implementation. The WSGI interface indicates that the WSGI app will return its output as an iterable. While WSGI was intended to unify the calling of these frameworks, if we go with a WSGI style template call, it'd be trivial to wrap it in something letting you do: mytemplate = render_template('django', '/the/template.html', wsgi=False) And in such a case, render_template will call the Django WSGI interface with a basic WSGI environ, capture the headers, and return a string of the rendered content. You'd be able to use the same command with any of the template languages, and for the template languages that do want to create their own response/request API's out of the WSGI request, they still can. > It is my strong preference that the standard API we are attempting to > design here does *not* tie you strongly to WSGI or the generation of > dynamic web content. The fact is, that many template languages do provide abstractions of objects present only in a web context though. Having an API that fulfills this, like WSGI, lets you easily put a little function on top like the render_template I showed that still does what you want. > There's nothing wrong with having some things that can invoked from > the template that set or add headers behind your back, if the template > engine supports such a thing. But I think the API to let it do so > might not be part of the standard, or it might be an optional part of > the standard; I want to be able to use a templating engine in a > non-web context and still use it (of course I won't be able to set > headers then, but I'd assume this functionality is optional and I can > just elect not to use it). Of course, as I showed, its pretty trivial to use WSGI apps in such a context. I can put together a more full fledged example as well, but having WSGI under the hood avoids crippling the more full featured template languages that do provide additional web context objects. > This assumes wegiref though. I'd rather not make wsgiref a required > part of the standard. No need to, environ is basic dict, start_response is just a callable (in non-wsgi, just supply a callable that eats the headers since you won't use them), the response is an iterable, feel free to iterate over it writing the content back out (imagine the template's rendered response is huge, so you don't want it all in memory). Or maybe you do want the full rendered output, just list() it. The freedom is left up to you, and the template language still has the ability to do its thing regardless of if you want to use it. > No, I'd like to see the templating API and WSGI completely decoupled, > except for an optional part where in the evaluation context passed > into a template at render time, you pass in an object that lets the > template manipulate headers etc. It would be fine if the latter used > WSGI somehow. But there *must* be a standard way yo invoke the > template engine (context in, string or stream out) without assuming > it's an HTTP response. WSGI at its heart is a very basic API. To come to an API for template languages, that provides flexibility for the language, and options for its output, we'll end up with an extremely similar API to WSGI I think. > Please keep coming up with proposals and rationales for them. I will > certainly admit that I haven't used a Python web framework in a while > (not since I left Zope :-). But when I see Cheetah and Django and some > competitors that were mentioned in responses to my blogs, a certain > "Platonic ideal" of a templating engine emerges in my brain, and I'd > like at least the API for that standardardized, if at all possible, so > people writing frameworks can choose from the many templating engines > available, and people with templating needs *outside* a web context > can have the same selection. I see no reason the template spec can't also provide for a unified render call on top of the WSGI interface (for use in non-web context). This would give you, django = DjangoTemplate(**django_setup_args) # in case it needs a template_root path, etc. content, headers = django('/some/template.html', wsgi=False) # content is an iterable # web context return django(environ, start_response) # Or use Kid: kidengine = KidTemplate(**kid_args) content, headers = kidengine('some.template.kid', wsgi=False) # web return kidengine(environ, start_response) Or if people don't like relying on the object to determine if there's a keyword with wsgi = False, content, headers = kidengine.render('some.template.kid', wsgi=False) return kidengine.wsgi(environ, start_response) Regardless, everyone now has a uniform way to make a template renderer, the template renderer can designate the options it should take, and its output can be returned as an iterable for you, or it can directly handle the WSGI call. The polymorphic calling nature Phillip Eby describes could easily be built in his framework by wrapping this process, but anyone will have a rather uniform way to make a template engine instance, and use it. Automated code could also be easily made to setup a few of these and handle them accordingly (no need for it in the API). How's that look? Cheers, Ben From pywebsig at xhaus.com Sun Feb 5 21:02:03 2006 From: pywebsig at xhaus.com (Alan Kennedy) Date: Sun, 05 Feb 2006 20:02:03 +0000 Subject: [Web-SIG] A trivial template API counter-proposal In-Reply-To: <5.1.1.6.0.20060205124025.021885b0@mail.telecommunity.com> References: <5.1.1.6.0.20060204131722.02179008@mail.telecommunity.com> <5.1.1.6.0.20060205124025.021885b0@mail.telecommunity.com> Message-ID: <43E659BB.9000301@xhaus.com> [Phillip J. Eby] > Developing WSGI was not easy, either, as I'm sure you recall.You and I > certainly argued a bit about iterators and file-like objects and such, > and it took a while before I understood all of your use cases and we > were able to resolve them in the spec. If you had given up on > convincing me then, or if I had given up on your use cases as "too > complex", the spec would have suffered for it. And I am indeed most grateful that you took the time to understand my tiresome ramblings on the subject: WSGI is indeed a most excellent spec: well done! :-) [Alan Kennedy] >> I can understand why the web-sig has fallen into the trap of tying a >> tmeplating API to its nice web standard, WSGI: all web applications >> must generate output. But web apps need to generate a wide range of >> media types, e.g. image/*, application/msword, etc, etc, etc. [Phillip J. Eby] > And in many frameworks, it is the *template* that decides what media > type it is generating - and it may not even be outputting text or > unicode. Again, this is something that would be neglected by a > text-only spec. Ah, now there I have a problem! IMHO, templates should generate only a single media type. Whatever code is managing resource-delivery to the browser should decide which template to use, and set the media type accordingly, outside of the template. Let me explain in terms of an actual use case. I used to work for an e-learning company (widelearning.com), which delivered multimedia financial training materials. As much as possible, the content was delivered as video and Macromedia Flash, with fallback to simple image/* and text/html if the multimedia plugins were not available. This was done through two primary mechanisms: 1. Through plugin detection, i.e. running script in the browser to detect certain plugins, e.g. Flash. 2. Through user profiles, i.e. where the user selected their media preference, which was stored in a database. In both scenarios, entirely different templating engines were used. For text/*html, we used XSLT and JSP (ugh ;-) For Flash, we used a bespoke templating system, akin to Macromedia Generator (something like jgenerator: http://www.jzox.com). This was a templating engine that took a binary template as input, "cooked" the template with reference to a user data namespace, and generated a binary output stream representing a personalised Flash "movie". Neither rendering engine had any knowledge that other media types could potentially be returned to the user. Before any templates were rendered, a decision was made as to what media type was suitable to service the request, maximising the capabilities of the users browser, and the relevant rendering engine invoked, with the relevant template. This "separation of concerns" greatly simplified our development and QA process. IMO, permitting templates to select the media type is akin to the old problem of dealing with exceptions in various templating languages which intermingle code and presentation, e.g. JSP, ASP, PHP, etc. If a JSP caused an exception halfway through page-rendering, it was too late to do anything meaningful about it: the first half of the rendered page had already been transmitted to the user. What should really have happened is that the page should not have been transmitted to the user until the template was completely successfully rendered. That way, if an exception occurred, a suitable error page could be returned to the user, and the half-cooked template response discarded. Similarly, if a template is permitted to set HTTP headers, then it might discover too late that it is generating a media type that is unsuitable for the client. IMHO, some functionality in the HTTP application should decide the media type to be returned, call the relevant templating engine and set the relevant HTTP headers. Looking at this in an MVC context, the application is responsible for populating the Model (user namespace), and selecting which View (template<->media-type) is suitable for return to the user. Templates should not vary media types. HTTP headers do need to be set for different templates/media-types. But that should be the responsibility of the HTTP application, not the template, which should be unaware of the application contect in which it is running, except for the contents of the Model/user-namespace. Regards, Alan. From ben at groovie.org Sun Feb 5 21:10:24 2006 From: ben at groovie.org (Ben Bangert) Date: Sun, 5 Feb 2006 12:10:24 -0800 Subject: [Web-SIG] A trivial template API counter-proposal In-Reply-To: <62388AF2-7B46-42AB-ADC2-A008BD87BE91@groovie.org> References: <5.1.1.6.0.20060205121213.021562b8@mail.telecommunity.com> <62388AF2-7B46-42AB-ADC2-A008BD87BE91@groovie.org> Message-ID: > # Or use Kid: > kidengine = KidTemplate(**kid_args) > content, headers = kidengine('some.template.kid', wsgi=False) > > # web > return kidengine(environ, start_response) > > Or if people don't like relying on the object to determine if there's > a keyword with wsgi = False, > > content, headers = kidengine.render('some.template.kid', wsgi=False) > return kidengine.wsgi(environ, start_response) Actually, to clarify, it'd be: content, headers = kidengine.render('some.template', vars, **extra_args) content, headers = kidengine.render(template_string, vars, is_string=True) environ['wti.source'] = 'some.template' environ['wti.vars'] = vars environ['wti.extra'] = dict(is_string=True) return kidengine.wsgi(environ, start_response) Here's my thinking on this: First, I don't believe it's terribly realistic to try and unify how the template resource appears. Different template languages use different approaches, Myghty uses a normal path string (relative to a template root dir), Kid uses a module path, etc. You will *always* need to know a certain amount about the template language you're using, whether it needs to have a template root configured, do you give it a resource name as a module path, or filesystem path, etc. Trying to bash them all together will only cripple many or leave them working in bizarre ways. Second, different template languages can take additional options, we need to retain the ability to pass these in, thus **extra_args. Kid can take the fragment option, Myghty can toggle inheritance, etc. Third, afaik, every template language lets you give it variables, these will be in the vars dict, how the template language gets them to the template is up to the template language. So, in this proposal, it is assumed that underneath, there is a WSGI interface that can be called. Whether the render function wants to emulate a WSGI call to the underlying WSGI ability, or just render is up to the template spec implementor. What you get out of this: - Ability to call as WSGI - Ability to send it the template data, or the name of the template - Template engine is configured individually depending on what it needs for setup - Ability to have the output returned to you as the iterable There's lots of room for individual template languages to supply the necessary things they need, as if we try and restrict too much, we quickly end up crippling or severely limiting the usefullness. You do get a consistent way to make a template engine object, and ask for output. How the template language does such things is left for another discussion, this is about using it. I'm sure some of the environ keys I proposed here could be refined as well. - Ben From ianb at colorstudy.com Sun Feb 5 21:36:40 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 05 Feb 2006 14:36:40 -0600 Subject: [Web-SIG] A trivial template API counter-proposal In-Reply-To: <5.1.1.6.0.20060205124025.021885b0@mail.telecommunity.com> References: <5.1.1.6.0.20060204131722.02179008@mail.telecommunity.com> <5.1.1.6.0.20060205124025.021885b0@mail.telecommunity.com> Message-ID: <43E661D8.9090000@colorstudy.com> Phillip J. Eby wrote: >>This topic started with Buffet, the de-facto standard templating API for >>CherryPy. Buffet is just about textual templating, which is a good >>thing. That's why it's very simple, and is thus actually being used. > > > Which is precisely why we don't need to rush into blessing it as some kind > of web standard, when it doesn't have anything to do with the "web", as far > as I can tell. If I needed a plugin for pure text or XML templating (as > opposed to *web application* templates), or I were creating a pure text or > XML output generator, I would absolutely use the TG/Buffet API, no > question. It is 100% the right tool for the job, and the de facto standard > as well. > > But I'm also 100% positive that it is *not* suitable to be declared a > Python *web* standard by itself: it's not about the web at all. The web > just happens to be where it's being used at the moment, and only in > frameworks where the use cases slot neatly together with it. Heck, just > look at the API documentation and see if you can find the word "web" or > even "request" mentioned on the page: > > http://www.turbogears.org/docs/plugins/template.html It is not a complete spec that fits many use cases at this time. It won't be sufficient for TG either before long -- Kevin has already said that he'd like a find_template function somewhere in there. Because the spec suggests that templates are given module-like names coinciding with the package of the consuming software, it is hard to use in any way where the templates themselves are taken from arbitrary locations. It doesn't actually specify how the names are interpreted or how the templates are found, but all the implementations so far use pkg_resources and load templates in a way that makes it difficult to load arbitrary files. This probably seemed sensible since the first couple languages -- Kid and Cheetah -- compile to Python modules. But while it seems sensible, Python modules are far more difficult to work with than simpler metaphors for a compiled template -- you get import hooks, rather complicated recompilation steps, and add one more thing to the already complex task of managing sys.path. So no, this isn't sufficient now. I think very minor fixes could improve it -- for instance, allow template_filename as an argument instead of only template_name. Then at least simple template filling would be handled well (e.g., enough for Paste Script), though it is unlikely (with only those specs) that templates will handle including other templates very well, as there's not even sufficient information given for the template language to figure out the programmer's intent. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ianb at colorstudy.com Sun Feb 5 21:46:35 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 05 Feb 2006 14:46:35 -0600 Subject: [Web-SIG] My original template API proposal In-Reply-To: <43E53709.4040007@colorstudy.com> References: <43E53709.4040007@colorstudy.com> Message-ID: <43E6642B.50006@colorstudy.com> Ian Bicking wrote: > def render(template_instance, vars, format="html", fragment=False): Here I can magically turn this into a WEB templating spec: def render(template_instance, vars, format="html", fragment=False, wsgi_environ=None, set_header_callback=None) wsgi_environ is the environ dictionary if this is being called in a WSGI context. set_header_callback can be called like set_header_callback(header_name, header_value) to write such a header to the response. Frameworks may or may not allow for setting headers. If they don't allow for it, they shouldn't provide that callback (thus headers will not be mysteriously thrown away -- instead they will be rejected immediately). [Should set_header_callback('Status', '404 Not Found') be used, or a separate callback, or...?] This follows what all "server pages" templates I know of do. That is, they do not have special syntax related to any metadata (i.e., headers) or even any special syntax related to web requests. Instead the web request is represented through some set of variables available in the template. By putting these variables directly in the call to render(), we give a standard location for template languages to look for these values, and they can represent them internally however they choose. This does not preclude sending request- or response-related values in through vars directly. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ianb at colorstudy.com Sun Feb 5 22:00:19 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 05 Feb 2006 15:00:19 -0600 Subject: [Web-SIG] A trivial template API counter-proposal In-Reply-To: <5.1.1.6.0.20060205135745.0371d938@mail.telecommunity.com> References: <5.1.1.6.0.20060205121213.021562b8@mail.telecommunity.com> <5.1.1.6.0.20060205121213.021562b8@mail.telecommunity.com> <5.1.1.6.0.20060205135745.0371d938@mail.telecommunity.com> Message-ID: <43E66763.5010902@colorstudy.com> Phillip J. Eby wrote: >>It is my strong preference that the standard API we are attempting to >>design here does *not* tie you strongly to WSGI or the generation of >>dynamic web content. > > > That seems to be a popular position, to be sure. However, since there is > already a de facto standard support for that simpler use case, that is > already supported by two frameworks for at least 6 templating systems(!) it > seems that it's already both a solved problem and outside the scope of the > Web-SIG. You already *have* this standard API; why shouldn't the web-sig > then focus on something that supports *web* templating, not just strings? Here is exactly why: * There was an ad hoc standardization happening with TG and Buffet. * Other people were becoming interested. * Discussion was happening on the TG list, where most people on the TG list were not particularly interested about this sort of thing. * There are still several outstanding issues with the spec. * People came upon this ad hoc standard rather randomly through word of mouth, and Peter Hunt thought that maybe the spec should be promoted more. * Almost everyone doing stuff on the web cares about templates. * Web applications and frameworks are the primary consumers of templating languages, and the vast majority of templating languages were written with web use in mind. Except maybe string.Template, I can't think of ANY templating language not written with the web in mind. * Most templating languages are interfaced in very similar ways -- dict of variables in, string out. Despite the fact that these languages were written for the web, people still haven't felt a need to go beyond this. * But the exact details of invoking the template vary widely, in ways that are very annoying for people who want to support multiple languages. It is even very annoying for people who only want to support one language, because the interfaces often have several options and there's never been a particular target API for template language authors to shoot for. There's no "best practice" for this side of the API design. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ianb at colorstudy.com Sun Feb 5 22:26:13 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 05 Feb 2006 15:26:13 -0600 Subject: [Web-SIG] A trivial template API counter-proposal In-Reply-To: References: <5.1.1.6.0.20060205121213.021562b8@mail.telecommunity.com> <62388AF2-7B46-42AB-ADC2-A008BD87BE91@groovie.org> Message-ID: <43E66D75.9010803@colorstudy.com> Ben Bangert wrote: >># Or use Kid: >>kidengine = KidTemplate(**kid_args) >>content, headers = kidengine('some.template.kid', wsgi=False) >> >># web >>return kidengine(environ, start_response) >> >>Or if people don't like relying on the object to determine if there's >>a keyword with wsgi = False, >> >>content, headers = kidengine.render('some.template.kid', wsgi=False) >>return kidengine.wsgi(environ, start_response) > > > Actually, to clarify, it'd be: > content, headers = kidengine.render('some.template', vars, **extra_args) > > content, headers = kidengine.render(template_string, vars, > is_string=True) > > environ['wti.source'] = 'some.template' > environ['wti.vars'] = vars > environ['wti.extra'] = dict(is_string=True) > return kidengine.wsgi(environ, start_response) This involves a lot of background from the discussion to understand. Can this be summarized in a way that is complete? > Here's my thinking on this: > > First, I don't believe it's terribly realistic to try and unify how > the template resource appears. Different template languages use > different approaches, Myghty uses a normal path string (relative to a > template root dir), Kid uses a module path, etc. You will *always* > need to know a certain amount about the template language you're > using, whether it needs to have a template root configured, do you > give it a resource name as a module path, or filesystem path, etc. > Trying to bash them all together will only cripple many or leave them > working in bizarre ways. Many templates provide multiple ways of indicating the template. So they'll have to choose one. And a good chance that it won't be the one that works for me :( Even the most trivial of web applications needs templates to include other templates, so the fact that this doesn't do anything to aid or specify that makes the spec feel leaky. I can indicate where the template comes from initially, but all bets are off after that. > Second, different template languages can take additional options, we > need to retain the ability to pass these in, thus **extra_args. Kid > can take the fragment option, Myghty can toggle inheritance, etc. That seems fine. Most of the options to render() in the TG spec only really applied to Kid. TG also had the options during instantiation, which were intended to be more like compiler options. I assume that templates should ignore options they don't expect. > Third, afaik, every template language lets you give it variables, > these will be in the vars dict, how the template language gets them > to the template is up to the template language. > > So, in this proposal, it is assumed that underneath, there is a WSGI > interface that can be called. Whether the render function wants to > emulate a WSGI call to the underlying WSGI ability, or just render is > up to the template spec implementor. > > What you get out of this: > - Ability to call as WSGI Of course, you can't *actually* call it as a WSGI app without some container that properly instantiates the engine after it finds the template. So you can't drop templates into an application anymore than you can with a non-WSGI spec. All this does is put an incomplete WSGI wrapper into template plugin -- a wrapper which would only have to be written once against the TG-style spec anyway. Except for Phillip's insistence on this, I feel thoroughly unconvinced of what a WSGI interface accomplishes. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From pje at telecommunity.com Sun Feb 5 23:19:36 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 05 Feb 2006 17:19:36 -0500 Subject: [Web-SIG] My original template API proposal In-Reply-To: <43E6642B.50006@colorstudy.com> References: <43E53709.4040007@colorstudy.com> <43E53709.4040007@colorstudy.com> Message-ID: <5.1.1.6.0.20060205170548.021522d8@mail.telecommunity.com> At 02:46 PM 2/5/2006 -0600, Ian Bicking wrote: >Ian Bicking wrote: > > def render(template_instance, vars, format="html", fragment=False): > >Here I can magically turn this into a WEB templating spec: > >def render(template_instance, vars, format="html", fragment=False, >wsgi_environ=None, set_header_callback=None) > >wsgi_environ is the environ dictionary if this is being called in a WSGI >context. set_header_callback can be called like >set_header_callback(header_name, header_value) to write such a header to >the response. Frameworks may or may not allow for setting headers. If >they don't allow for it, they shouldn't provide that callback (thus >headers will not be mysteriously thrown away -- instead they will be >rejected immediately). [Should set_header_callback('Status', '404 Not >Found') be used, or a separate callback, or...?] > >This follows what all "server pages" templates I know of do. That is, >they do not have special syntax related to any metadata (i.e., headers) >or even any special syntax related to web requests. Instead the web >request is represented through some set of variables available in the >template. Yes, but different template systems offer different APIs based on it; the idea of using WSGI here was to make it possible for them to offer their *own*, native APIs under this spec, not to force the use of the host framework's API. The only thing that's missing from your proposal is streaming control or large file support. I'll agree that it's an edge use case, but it seems to me just as easy to just offer a plain WSGI interface and not have to document a bunch of differences and limitations. OTOH, if this is what it takes to get consensus, so be it. The additional advantage to using plain ol' WSGI as the calling interface, however, is that it also lets you embed *anything* as a template, including whole applications if they provide a "template engine" whose syntax is actually the application's configuration. Anyway, the only differences I'm aware of between what you're proposing and what I'm proposing are: 1. Syntax sugar (each proposal sweetens different use cases) 2. Feature restrictions (yours takes away streaming) 3. What's optional (you consider WSGI optional, I want strings to be optional) It would be better, I think, to address further discussion to addressing the actual points of difference. Regarding #2, I'm willing to compromise to get consensus. Regarding #3, I'd be willing to compromise by making *both* optional, with clearly defined variations of the spec so that plugins and frameworks that support each are clearly distinguishable. This would also mean that we'd both be able to get the syntaxes we want under #1. From pje at telecommunity.com Sun Feb 5 23:33:59 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 05 Feb 2006 17:33:59 -0500 Subject: [Web-SIG] A trivial template API counter-proposal In-Reply-To: <43E66D75.9010803@colorstudy.com> References: <5.1.1.6.0.20060205121213.021562b8@mail.telecommunity.com> <62388AF2-7B46-42AB-ADC2-A008BD87BE91@groovie.org> Message-ID: <5.1.1.6.0.20060205173119.0385b718@mail.telecommunity.com> At 03:26 PM 2/5/2006 -0600, Ian Bicking wrote: >Even the most trivial of web applications needs templates to include >other templates, so the fact that this doesn't do anything to aid or >specify that makes the spec feel leaky. I can indicate where the >template comes from initially, but all bets are off after that. As Ben has previously pointed out, systems like Myghty are going to ignore your 'find_template()' because they do their own finding. So the spec will leak no matter what, until we get to the level of specification called for by the "embedding" side of my proposal. (The compile/write stuff.) And Ben and Michael have both pointed out that trying to meet a spec that calls for them to change how their inner find_template works would be costly. Note, however, that these constraints don't have anything to do with whether WSGI is used or not; the essential complexity of the problem is still the same. From pje at telecommunity.com Sun Feb 5 23:31:55 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 05 Feb 2006 17:31:55 -0500 Subject: [Web-SIG] A trivial template API counter-proposal In-Reply-To: <43E661D8.9090000@colorstudy.com> References: <5.1.1.6.0.20060205124025.021885b0@mail.telecommunity.com> <5.1.1.6.0.20060204131722.02179008@mail.telecommunity.com> <5.1.1.6.0.20060205124025.021885b0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060205172056.03fced80@mail.telecommunity.com> At 02:36 PM 2/5/2006 -0600, Ian Bicking wrote: >I think very minor fixes could improve it -- for instance, allow >template_filename as an argument instead of only template_name. Then at >least simple template filling would be handled well (e.g., enough for >Paste Script), though it is unlikely (with only those specs) that >templates will handle including other templates very well, as there's not >even sufficient information given for the template language to figure out >the programmer's intent. But this is precisely why I think a spec to the level you're describing is premature. You're basically saying that the template deployment situation is all fouled up, so we need to allow for more balkanization. My original statement on this topic was that what we need to do is get a standard for template/resource *deployment* so that there's a path for cleaning up the balkanization. Sure, no framework that exists is going to currently match exactly whatever spec we come up with for deployment, but I think we could come up with a layout that would support every framework's use cases and that everybody could agree it made sense to add support for. From ianb at colorstudy.com Mon Feb 6 00:15:22 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 05 Feb 2006 17:15:22 -0600 Subject: [Web-SIG] My original template API proposal In-Reply-To: <5.1.1.6.0.20060205170548.021522d8@mail.telecommunity.com> References: <43E53709.4040007@colorstudy.com> <43E53709.4040007@colorstudy.com> <5.1.1.6.0.20060205170548.021522d8@mail.telecommunity.com> Message-ID: <43E6870A.8080908@colorstudy.com> Phillip J. Eby wrote: > At 02:46 PM 2/5/2006 -0600, Ian Bicking wrote: > >> Ian Bicking wrote: >> > def render(template_instance, vars, format="html", >> fragment=False): >> >> Here I can magically turn this into a WEB templating spec: >> >> def render(template_instance, vars, format="html", fragment=False, >> wsgi_environ=None, set_header_callback=None) >> >> wsgi_environ is the environ dictionary if this is being called in a WSGI >> context. set_header_callback can be called like >> set_header_callback(header_name, header_value) to write such a header to >> the response. Frameworks may or may not allow for setting headers. If >> they don't allow for it, they shouldn't provide that callback (thus >> headers will not be mysteriously thrown away -- instead they will be >> rejected immediately). [Should set_header_callback('Status', '404 Not >> Found') be used, or a separate callback, or...?] >> >> This follows what all "server pages" templates I know of do. That is, >> they do not have special syntax related to any metadata (i.e., headers) >> or even any special syntax related to web requests. Instead the web >> request is represented through some set of variables available in the >> template. > > > Yes, but different template systems offer different APIs based on it; > the idea of using WSGI here was to make it possible for them to offer > their *own*, native APIs under this spec, not to force the use of the > host framework's API. There's nothing that specifies what the template does with vars. It might be reasonable, for instance, in ZPT for the global ("context") variables to be build from wsgi_environ, and the vars dict to be put in as "options". > The only thing that's missing from your proposal is streaming control or > large file support. I'll agree that it's an edge use case, but it seems > to me just as easy to just offer a plain WSGI interface and not have to > document a bunch of differences and limitations. OTOH, if this is what > it takes to get consensus, so be it. We could change the return value of render to be an iterator. I don't see any problem with this, as it's so easy to do ''.join(return_value), or return [content]. I do *strongly* feel that templates should be allowed (and encouraged) to return unicode strings. There's also the possibility of returning values other than strings, though, like an ElementTree object. I think this is interesting, and the fallback if strings are returned is pretty easy -- the template consumer should just make sure they handle strings in addition to whatever they asked for (which just means parsing the output into an ElementTree object in this case). Returning an iterator makes this harder. There's probably more likely performance benefits in returning ElementTree objects (or similar) for use when chaining transformations, than there is in returning iterators for large content bodies. Maybe this could be handled with two methods instead of the one render(). Which actually is what the original TG spec has (though I'd like the spec to indicate the string fallback implementation). Well... another option is just to expect consumers to try to get a method (e.g., render_elementtree), catch AttributeError, and fall back that way. That seems the most straight-forward. > The additional advantage to using plain ol' WSGI as the calling > interface, however, is that it also lets you embed *anything* as a > template, including whole applications if they provide a "template > engine" whose syntax is actually the application's configuration. The template plugin can do whatever you want, if it takes string content and returns string output. With the WSGI-based spec, for some cases (e.g., not in a web environment) you have wrappers that create a pretend WSGI environment and wrappers that cover up the WSGI response. In the TG model, you have a wrapper that makes a WSGI application from the plugin, or presents a template-like interface to a WSGI application. Both are fairly simple wrappers. Some aspects of the WSGI spec can remain as suggestions -- for instance, if you are wrapping a WSGI application, we can suggest that you put vars into 'wti.vars'. This doesn't have to be built on WSGI in order to suggest in the spec how it would interact with WSGI. Just like we can suggest a setuptools entry point group, without building this on setuptools. > Anyway, the only differences I'm aware of between what you're proposing > and what I'm proposing are: > > 1. Syntax sugar (each proposal sweetens different use cases)` In addition to sweetness, there's the question of comfort. Will you have to understand WSGI to understand the spec? How comfortable will people feel implementing the spec, or using the plugins? > 2. Feature restrictions (yours takes away streaming) Streaming is fairly trivial; it could be added easily enough. The only really big one is find_resource, which I'll concede is quite hard. If there was at least a way to suggest such a hook while also providing a sane degredation, that would be ideal. Then at least plugin/template developers would have something to shoot for. > 3. What's optional (you consider WSGI optional, I want strings to be > optional) This is kind of 1, since both interfaces can be used to implement the other. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ianb at colorstudy.com Mon Feb 6 00:17:58 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 05 Feb 2006 17:17:58 -0600 Subject: [Web-SIG] My original template API proposal In-Reply-To: <43E6642B.50006@colorstudy.com> References: <43E53709.4040007@colorstudy.com> <43E6642B.50006@colorstudy.com> Message-ID: <43E687A6.3070900@colorstudy.com> Ian Bicking wrote: > Ian Bicking wrote: > >> def render(template_instance, vars, format="html", fragment=False): > > > Here I can magically turn this into a WEB templating spec: > > def render(template_instance, vars, format="html", fragment=False, > wsgi_environ=None, set_header_callback=None) I've updated this spec (http://svn.pythonpaste.org/home/ianb/templateapi/interface.py) with a new version of the render method: def render(template_instance, vars, wsgi_environ=None, set_header_callback=None, **kwargs): """ Render the template instance (as returned by ``load_resource``). [Should there be some way to return metadata about what was rendered, like the type of object being returned -- e.g., text/html or text/plain?] ``vars`` is typically a dictionary (-like object) of variables for use in substitution. ``wsgi_environ``, if given, is the WSGI environment dictionary. Templates are free to create their own native wrappers of this environment. [Is there anything specifying where *here* is? Is SCRIPT_NAME authoritative in this case? Any standard for the application root? Maybe a special key, e.g., wsgi.application_root] ``set_header_callback`` is a function that can be called like ``set_header_callback(header_name, header_value)``. Arguments can only be strings (not unicode) [encode unicode with ASCII?]. The header named ``'Status'`` can be used to set the status. Even if this template is rendered in a web environment, frameworks may not provide this callback if they do not expect or wish the template to effect the headers. Other keyword arguments can be provided, with meaning specific to the templating language. Plugins should ignore keyword arguments they do not understand. [Is this a good idea?] This function returns an iterator which produces strings or unicode strings. (It should produce one or the other, not both.) Unicode is preferred. If you want a template to render to something else -- for example, ElementTree objects -- you should provide another method with the same signature as this. Consumers can fall back to getting the string with this method, like:: template = plugin.load_template(...) try: meth = plugin.render_elementtree except AttributeError: result = ''.join(plugin.render(template, ...)) result = ElementTree.XML(result) else: result = meth(template, ...) """ -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ianb at colorstudy.com Mon Feb 6 00:25:29 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 05 Feb 2006 17:25:29 -0600 Subject: [Web-SIG] A trivial template API counter-proposal In-Reply-To: <5.1.1.6.0.20060205173119.0385b718@mail.telecommunity.com> References: <5.1.1.6.0.20060205121213.021562b8@mail.telecommunity.com> <62388AF2-7B46-42AB-ADC2-A008BD87BE91@groovie.org> <5.1.1.6.0.20060205173119.0385b718@mail.telecommunity.com> Message-ID: <43E68969.6080205@colorstudy.com> Phillip J. Eby wrote: > At 03:26 PM 2/5/2006 -0600, Ian Bicking wrote: > >> Even the most trivial of web applications needs templates to include >> other templates, so the fact that this doesn't do anything to aid or >> specify that makes the spec feel leaky. I can indicate where the >> template comes from initially, but all bets are off after that. > > > As Ben has previously pointed out, systems like Myghty are going to > ignore your 'find_template()' because they do their own finding. So the > spec will leak no matter what, until we get to the level of > specification called for by the "embedding" side of my proposal. (The > compile/write stuff.) And Ben and Michael have both pointed out that > trying to meet a spec that calls for them to change how their inner > find_template works would be costly. Yes, I understand it is difficult. I think it is possible to refactor Myghty in terms of find_template without loss of functionality (and perhaps allowing Myghty's resolution to be used with other templating languages) -- but I'm sure it would not be easy. If there was some way to support find_template/find_resource for languages that supported it, but still make other languages useful, that would be fine with me. I personally would focus on adding that support where I needed it, and using those templating languages that supported it. Support for that functionality is more important to me than any one templating language. However, I acknowledge that people will also want a spec they can use without changing the underlying template implementation. We could remove find_template/find_resource, and use a load_template method with a stream input (and a object name for debugging output), or with purely a template name and not handle actual loading at all, or some combination of the two. Though I have a hard time envisioning how that would all work together to be usable. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From foom at fuhm.net Mon Feb 6 00:37:01 2006 From: foom at fuhm.net (James Y Knight) Date: Sun, 5 Feb 2006 18:37:01 -0500 Subject: [Web-SIG] My original template API proposal In-Reply-To: <43E687A6.3070900@colorstudy.com> References: <43E53709.4040007@colorstudy.com> <43E6642B.50006@colorstudy.com> <43E687A6.3070900@colorstudy.com> Message-ID: On Feb 5, 2006, at 6:17 PM, Ian Bicking wrote: > ``set_header_callback`` is a function that can be called > like > ``set_header_callback(header_name, header_value)``. > Arguments can only be strings (not unicode) [encode unicode > with ASCII?]. The header named ``'Status'`` can be used to > set the status. Even if this template is rendered in a web > environment, frameworks may not provide this callback if > they > do not expect or wish the template to effect the headers. > > This function returns an iterator which produces strings or > unicode strings. (It should produce one or the other, not > both.) Unicode is preferred. Now that you're re-specifying something similar to WSGI, you need to consider issues like what should happen if you call the set_header_callback after having yielded one string out of the iterator. Is that an error, or is it silently ignored, or can the container decide? Also, is the caller required to send the strings returned by the iterator to the browser immediately, or can it buffer it all up to send when the template iterator finishes? Or doesn't it matter? James From ianb at colorstudy.com Mon Feb 6 00:44:48 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 05 Feb 2006 17:44:48 -0600 Subject: [Web-SIG] My original template API proposal In-Reply-To: References: <43E53709.4040007@colorstudy.com> <43E6642B.50006@colorstudy.com> <43E687A6.3070900@colorstudy.com> Message-ID: <43E68DF0.9030906@colorstudy.com> James Y Knight wrote: > On Feb 5, 2006, at 6:17 PM, Ian Bicking wrote: > >> ``set_header_callback`` is a function that can be called like >> ``set_header_callback(header_name, header_value)``. >> Arguments can only be strings (not unicode) [encode unicode >> with ASCII?]. The header named ``'Status'`` can be used to >> set the status. Even if this template is rendered in a web >> environment, frameworks may not provide this callback if they >> do not expect or wish the template to effect the headers. >> >> This function returns an iterator which produces strings or >> unicode strings. (It should produce one or the other, not >> both.) Unicode is preferred. > > > Now that you're re-specifying something similar to WSGI, you need to > consider issues like what should happen if you call the > set_header_callback after having yielded one string out of the > iterator. Is that an error, or is it silently ignored, or can the > container decide? Also, is the caller required to send the strings > returned by the iterator to the browser immediately, or can it buffer > it all up to send when the template iterator finishes? Or doesn't it > matter? I'm not sure about order of execution -- the same as WSGI, I guess (what does PHP do?) There's no promises on when the yielded strings get sent, or if they get sent at all. In most cases I expect the results will be fully buffered. I suspect most templates will buffer their output internally, unless somehow configured or dynamically set not to do so. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From guido at python.org Mon Feb 6 05:09:55 2006 From: guido at python.org (Guido van Rossum) Date: Sun, 5 Feb 2006 20:09:55 -0800 Subject: [Web-SIG] My original template API proposal In-Reply-To: <43E68DF0.9030906@colorstudy.com> References: <43E53709.4040007@colorstudy.com> <43E6642B.50006@colorstudy.com> <43E687A6.3070900@colorstudy.com> <43E68DF0.9030906@colorstudy.com> Message-ID: On 2/5/06, Ian Bicking wrote: > I suspect most templates will buffer their output internally, unless > somehow configured or dynamically set not to do so. Why would they? Isn't that a function that the web server typically does? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Feb 6 05:32:29 2006 From: guido at python.org (Guido van Rossum) Date: Sun, 5 Feb 2006 20:32:29 -0800 Subject: [Web-SIG] A trivial template API counter-proposal In-Reply-To: <43E66763.5010902@colorstudy.com> References: <5.1.1.6.0.20060205121213.021562b8@mail.telecommunity.com> <5.1.1.6.0.20060205135745.0371d938@mail.telecommunity.com> <43E66763.5010902@colorstudy.com> Message-ID: On 2/5/06, Ian Bicking wrote: > * There was an ad hoc standardization happening with TG and Buffet. > > * Other people were becoming interested. > > * Discussion was happening on the TG list, where most people on the TG > list were not particularly interested about this sort of thing. > > * There are still several outstanding issues with the spec. > > * People came upon this ad hoc standard rather randomly through word of > mouth, and Peter Hunt thought that maybe the spec should be promoted more. > > * Almost everyone doing stuff on the web cares about templates. > > * Web applications and frameworks are the primary consumers of > templating languages, and the vast majority of templating languages were > written with web use in mind. Except maybe string.Template, I can't > think of ANY templating language not written with the web in mind. > > * Most templating languages are interfaced in very similar ways -- dict > of variables in, string out. Despite the fact that these languages were > written for the web, people still haven't felt a need to go beyond this. > > * But the exact details of invoking the template vary widely, in ways > that are very annoying for people who want to support multiple > languages. It is even very annoying for people who only want to support > one language, because the interfaces often have several options and > there's never been a particular target API for template language authors > to shoot for. There's no "best practice" for this side of the API design. Thanks for confirming that I'm not entirely out of my mind. There's too much written here for me to respond on directly, but I like the way things are going -- genuine consideration of the complexity of the issues, and the desire to have an inclusive standard. I seem to accidentally have caused some confusion by writing "how can it be considered a templating system if it has no access to request/response?" -- Reading back I can't quite reconstruct what I *meant* to say, but it was more like the opposite -- e.g. "how can you exclude templating systems that don't need access to request/response?" My apologies. Phillip described the workflow for Django/Cheetah style templates as follows: > * framework calls some Python code you wrote > * your code returns a dictionary of values you want rendered > * framework passes your returned values to the template, gets back a string > * framework sends the string back to the browser But there's an even lower-level variant, where the Python code referenced invokes the template (with variables) and receives a string back, which it possibly munges further and then passes to the framework. (For example, my first attempt at using Django templates had one piece of Python code that concatenated the results from several templates.) I'd like someone to write up a similar list explaining how e.g. Zope3 differs -- I suppose at one point in the past I used to know, but that knowledge is on a back-up tape that was lost after I moved to California. :-( Phillip also wrote this: > As Ben has previously pointed out, systems like Myghty are going to ignore > your 'find_template()' because they do their own finding. So the spec will > leak no matter what, until we get to the level of specification called for > by the "embedding" side of my proposal. (The compile/write stuff.) And > Ben and Michael have both pointed out that trying to meet a spec that calls > for them to change how their inner find_template works would be costly. I can't parse this because I don't know what relationship Ben and Michael have with the systems mentioned. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From mike_mp at zzzcomputing.com Mon Feb 6 06:14:55 2006 From: mike_mp at zzzcomputing.com (Michael Bayer) Date: Mon, 6 Feb 2006 00:14:55 -0500 Subject: [Web-SIG] A trivial template API counter-proposal In-Reply-To: <5.1.1.6.0.20060205173119.0385b718@mail.telecommunity.com> References: <5.1.1.6.0.20060205121213.021562b8@mail.telecommunity.com> <62388AF2-7B46-42AB-ADC2-A008BD87BE91@groovie.org> <5.1.1.6.0.20060205173119.0385b718@mail.telecommunity.com> Message-ID: <42884277-B14A-43B4-9ED0-3D2935A04A4A@zzzcomputing.com> On Feb 5, 2006, at 5:33 PM, Phillip J. Eby wrote: > As Ben has previously pointed out, systems like Myghty are going to > ignore > your 'find_template()' because they do their own finding. So the > spec will > leak no matter what, until we get to the level of specification > called for > by the "embedding" side of my proposal. (The compile/write > stuff.) And > Ben and Michael have both pointed out that trying to meet a spec > that calls > for them to change how their inner find_template works would be > costly. > Im not totally following every detail of this discussion....but correct me if I'm wrong, if we are just talking about a specification of how the various components of a template engine are to look and interact with each other, there is no reason Myghty's resolver couldnt implement a find_template() method...since thats what it is ! I would say that the argument or arguments passed to find_template() need to be pretty open ended, since you cant predict what kinds of things might be meaningful to a template finder, such as in myghty's case, the resolver wants to know if you are asking for this template to be used in a request, or an "embedded" template call, since there may be rules set up to return different components based on that context. The only thing about myghty's "find_template()" that is special and possibly complicating to this whole idea, is that its used not just at the request, but within the execution of the template itself to find other templates to include within or inherit from. Thats the gist of where myghty is coming from....you have this system of finding, compiling, caching the compiled version of, executing, and (optionally) caching the generated output of templates...but its totally recursive. one template call can result in a hundred more before the request is complete, and each step in that chain of template operations can occur at the componentized level as well as the overall request....there is also a request context active the whole time which maintains a stack of frame objects, etc. So if we come up with this standardized system to denote all those steps, for Myghty they might be built all the way through internally....perhaps at the internal level they will communicate more natively to reduce overhead, but in general we will be looking to organize its pieces in a way that corresponds closely to this "generalized template scheme". From ianb at colorstudy.com Mon Feb 6 07:17:08 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 06 Feb 2006 00:17:08 -0600 Subject: [Web-SIG] A trivial template API counter-proposal In-Reply-To: <42884277-B14A-43B4-9ED0-3D2935A04A4A@zzzcomputing.com> References: <5.1.1.6.0.20060205121213.021562b8@mail.telecommunity.com> <62388AF2-7B46-42AB-ADC2-A008BD87BE91@groovie.org> <5.1.1.6.0.20060205173119.0385b718@mail.telecommunity.com> <42884277-B14A-43B4-9ED0-3D2935A04A4A@zzzcomputing.com> Message-ID: <43E6E9E4.3030805@colorstudy.com> Michael Bayer wrote: > Im not totally following every detail of this discussion....but correct > me if I'm wrong, if we are just talking about a specification of how > the various components of a template engine are to look and interact > with each other, there is no reason Myghty's resolver couldnt implement > a find_template() method...since thats what it is ! Yes, this is always how I've seen it. > I would say that > the argument or arguments passed to find_template() need to be pretty > open ended, since you cant predict what kinds of things might be > meaningful to a template finder, such as in myghty's case, the resolver > wants to know if you are asking for this template to be used in a > request, or an "embedded" template call, since there may be rules set > up to return different components based on that context. Do these need to be arguments, or can the context be instance variables of a find_template callable? That is, you might do: def myghty_entry_point(environ, start_response): find_template = MyghtyTemplateFinder(environ) ... At least in my spec, find_template is not meant to be a global value. (However, I believe this complicates caching substantially, unless we leave it up to the find_template callable to handle that). Another option for more complicated situations would be looking for methods on find_template, and using those, or falling back to the calling interface. One substantial one that I think comes up in Myghty is that you get a template, and then you get all its implicit parents (I'm not sure what the term is in Myghty). I don't even know what to call that, and certainly not how to deal with it in find_template. One possible workaround for that would be for the plugin's load_template to return an object that encompasses all the implicit parents. But that takes some functionality from find_template that seems like it should be in find_template. Unless there was some really simple rule, like: load_template(find_template('/foo/bar/baz.myt'))... tries to call: load_template(find_template('/foo/template.myt')) also tries to call: load_template(find_template('/template.myt')) My spec allows for this, I believe. But it depends on the template plugin invoking these rules. > The only thing about myghty's "find_template()" that is special and > possibly complicating to this whole idea, is that its used not just at > the request, but within the execution of the template itself to find > other templates to include within or inherit from. Thats the gist of > where myghty is coming from....you have this system of finding, > compiling, caching the compiled version of, executing, and (optionally) > caching the generated output of templates...but its totally recursive. > one template call can result in a hundred more before the request is > complete, and each step in that chain of template operations can occur > at the componentized level as well as the overall request....there is > also a request context active the whole time which maintains a stack of > frame objects, etc. So if we come up with this standardized system > to denote all those steps, for Myghty they might be built all the way > through internally....perhaps at the internal level they will > communicate more natively to reduce overhead, but in general we will be > looking to organize its pieces in a way that corresponds closely to > this "generalized template scheme". This kind of recursive call is definitely considered in my spec. find_template (I call it find_resource, but we can ignore the "resource" stuff too) gets relative_to_name and relative_to_object optional arguments. You can use these for whatever resolution logic you want. templates/resources get their name, a stream representing their content (e.g., an opened file), and a reference to find_resource/find_template. They should (maybe optionally) use find_resource to find any templates they refer to. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From chrism at plope.com Mon Feb 6 12:47:45 2006 From: chrism at plope.com (Chris McDonough) Date: Mon, 6 Feb 2006 06:47:45 -0500 Subject: [Web-SIG] My original template API proposal In-Reply-To: <5.1.1.6.0.20060205170548.021522d8@mail.telecommunity.com> References: <43E53709.4040007@colorstudy.com> <43E53709.4040007@colorstudy.com> <5.1.1.6.0.20060205170548.021522d8@mail.telecommunity.com> Message-ID: Although I've been trying to follow this thread, I'm finding it difficult to get a handle on what is meant to *call* the template API (e.g. what typically calls "render" in Ian's ITemplatePlugin interface at http://svn.pythonpaste.org/home/ianb/templateapi/ interface.py)? Is the framework meant to call "render"? Sorry for the remedial question ;-) - C On Feb 5, 2006, at 5:19 PM, Phillip J. Eby wrote: > At 02:46 PM 2/5/2006 -0600, Ian Bicking wrote: >> Ian Bicking wrote: >>> def render(template_instance, vars, format="html", >>> fragment=False): >> >> Here I can magically turn this into a WEB templating spec: >> >> def render(template_instance, vars, format="html", fragment=False, >> wsgi_environ=None, set_header_callback=None) >> >> wsgi_environ is the environ dictionary if this is being called in >> a WSGI >> context. set_header_callback can be called like >> set_header_callback(header_name, header_value) to write such a >> header to >> the response. Frameworks may or may not allow for setting >> headers. If >> they don't allow for it, they shouldn't provide that callback (thus >> headers will not be mysteriously thrown away -- instead they will be >> rejected immediately). [Should set_header_callback('Status', '404 >> Not >> Found') be used, or a separate callback, or...?] >> >> This follows what all "server pages" templates I know of do. That >> is, >> they do not have special syntax related to any metadata (i.e., >> headers) >> or even any special syntax related to web requests. Instead the web >> request is represented through some set of variables available in the >> template. > > Yes, but different template systems offer different APIs based on > it; the > idea of using WSGI here was to make it possible for them to offer > their > *own*, native APIs under this spec, not to force the use of the host > framework's API. > > The only thing that's missing from your proposal is streaming > control or > large file support. I'll agree that it's an edge use case, but it > seems to > me just as easy to just offer a plain WSGI interface and not have to > document a bunch of differences and limitations. OTOH, if this is > what it > takes to get consensus, so be it. > > The additional advantage to using plain ol' WSGI as the calling > interface, > however, is that it also lets you embed *anything* as a template, > including > whole applications if they provide a "template engine" whose syntax is > actually the application's configuration. > > Anyway, the only differences I'm aware of between what you're > proposing and > what I'm proposing are: > > 1. Syntax sugar (each proposal sweetens different use cases) > 2. Feature restrictions (yours takes away streaming) > 3. What's optional (you consider WSGI optional, I want strings to > be optional) > > It would be better, I think, to address further discussion to > addressing > the actual points of difference. > > Regarding #2, I'm willing to compromise to get consensus. > Regarding #3, > I'd be willing to compromise by making *both* optional, with clearly > defined variations of the spec so that plugins and frameworks that > support > each are clearly distinguishable. This would also mean that we'd > both be > able to get the syntaxes we want under #1. > > _______________________________________________ > Web-SIG mailing list > Web-SIG at python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: http://mail.python.org/mailman/options/web-sig/chrism% > 40plope.com > From ianb at colorstudy.com Mon Feb 6 17:41:53 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 06 Feb 2006 10:41:53 -0600 Subject: [Web-SIG] My original template API proposal In-Reply-To: References: <43E53709.4040007@colorstudy.com> <43E53709.4040007@colorstudy.com> <5.1.1.6.0.20060205170548.021522d8@mail.telecommunity.com> Message-ID: <43E77C51.6070609@colorstudy.com> Chris McDonough wrote: > Although I've been trying to follow this thread, I'm finding it > difficult to get a handle on what is meant to *call* the template API > (e.g. what typically calls "render" in Ian's ITemplatePlugin interface > at http://svn.pythonpaste.org/home/ianb/templateapi/ interface.py)? Is > the framework meant to call "render"? Yes, it would be the framework. There shouldn't need to be too much code between the framework and application code, but at least a little (e.g., to create a find_resource/find_template callable). The framework will probably also load the template plugins. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From pje at telecommunity.com Mon Feb 6 17:53:38 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 06 Feb 2006 11:53:38 -0500 Subject: [Web-SIG] Bowing out (was Re: A trivial template API counter-proposal) In-Reply-To: <43E659BB.9000301@xhaus.com> References: <5.1.1.6.0.20060205124025.021885b0@mail.telecommunity.com> <5.1.1.6.0.20060204131722.02179008@mail.telecommunity.com> <5.1.1.6.0.20060205124025.021885b0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> At 08:02 PM 2/5/2006 +0000, Alan Kennedy wrote: >Looking at this in an MVC context, the application is responsible for >populating the Model (user namespace), and selecting which View >(template<->media-type) is suitable for return to the user. Templates >should not vary media types. HTTP headers do need to be set for >different templates/media-types. But that should be the responsibility >of the HTTP application, not the template, which should be unaware of >the application contect in which it is running, except for the contents >of the Model/user-namespace. As soon as you start talking about what templates should or should not do (as opposed to what they *already* do), you've stopped writing an inclusive spec and have wandered off into evangelizing a particular framework philosophy. OTOH, before I first proposed WSGI in 2003, nobody here seemed especially interested in writing inclusive specs anyway, and I rather get the impression they still aren't now, my "insistence" (as Ian calls it) notwithstanding. What I've been trying to do with both WSGI and with this spec is to create something that deals with the *actual* complexity and diversity of Python web frameworks as they exist today, rather than reducing the diversity to match whatever the currently popular paradigm is. This wasn't a popular approach when I introduced WSGI, either, but in the case of WSGI it was easy to point to all the previously-failed attempts at standardizing request/response objects due to people not taking backward compatibility into account. At this point it has become clear to me that even if I spent my days and nights writing a compelling spec of what I'm proposing and then trying to sell it to the Web SIG, it would be at best a 50/50 chance of getting through, and in the process it appears that I'd be burning through every bit of goodwill I might have previously possessed here. And, although I believe that the approach currently being taken to this spec is divisive of the community, I have to admit that since my attempts at education about the issues hasn't been particularly successful, it would appear that continuing to argue about it is no less divisive than what I'm arguing against. (For that matter, it's not even clear to me any more that most of the people on whose behalf I'm fighting would even realize yet why the future I want would be beneficial for them.) I really expected more people to see the benefits of the WSGI embedding approach, though, and although I've gotten a few private mails of support, it isn't anywhere near the level I thought it would be. Given the intense pressure that some parties are putting on having a spec *right* now, I don't feel that I can reasonably deliver a competing spec without interfering with my work and personal commitments in the next few weeks. Since I've already been using most of my "Python community contribution" time in the last week arguing about this, at this point it seems the community would be better served by me devoting that time to working on setuptools, rather than continuing to fight for a vision that hardly anybody else believes in. And, I'd rather save whatever karma I have left here for something with a better chance of success. Good luck with the spec. From mike_mp at zzzcomputing.com Mon Feb 6 18:10:53 2006 From: mike_mp at zzzcomputing.com (Michael Bayer) Date: Mon, 6 Feb 2006 12:10:53 -0500 (EST) Subject: [Web-SIG] A trivial template API counter-proposal In-Reply-To: <43E6E9E4.3030805@colorstudy.com> References: <5.1.1.6.0.20060205121213.021562b8@mail.telecommunity.com> <62388AF2-7B46-42AB-ADC2-A008BD87BE91@groovie.org> <5.1.1.6.0.20060205173119.0385b718@mail.telecommunity.com> <42884277-B14A-43B4-9ED0-3D2935A04A4A@zzzcomputing.com> <43E6E9E4.3030805@colorstudy.com> Message-ID: <31103.66.192.34.8.1139245853.squirrel@www.geekisp.com> Ian Bicking wrote: >> I would say that >> the argument or arguments passed to find_template() need to be pretty >> open ended, since you cant predict what kinds of things might be >> meaningful to a template finder, such as in myghty's case, the resolver >> wants to know if you are asking for this template to be used in a >> request, or an "embedded" template call, since there may be rules set >> up to return different components based on that context. > > Do these need to be arguments, or can the context be instance variables > of a find_template callable? That is, you might do: > > def myghty_entry_point(environ, start_response): > find_template = MyghtyTemplateFinder(environ) > ... Well this is the question- say im using an implementation of "the spec", whatever it is. I say, OK, I want to use the PylonsResolver with the KidCompiler with the DjangoCodeCache with the MyghtyInterpreter, and we snap those together. We fire it up, send a request for template "xxxxx", PylonsResolver calls it up and sends it off to the KidCompiler/DjangoCodeCache to get a template object, then off to MyghtyInterpreter to run it. Then , inside the template, we print out some html, then we say, "include_template('xxxx')", meaning the template wants to call another one. Does the "include_template" call the resolver that is internal to the MyghtyInterpreter ? Or does it have to use the Resolver that was sent into the beginning of the request ? (id say definitely the latter). My instinct in this kind of situation is, cant we just add a **kwargs to find_template() ? :) Im guessing not (but why not ?). But some kind of dictionary where the template's runtime environment can pass messages through to the resolver would be handy...else, Mygthy would have to hack something together, like serializing arguments in a string or something inefficient like that. Also, since this kind of call will happen on the order of N based on the complexity of the template, it would be nice if there wasnt too much (or really any) "create object" overhead to that...since it shouldnt have to use up a lot of resources for a very frequent operation. > > At least in my spec, find_template is not meant to be a global value. > (However, I believe this complicates caching substantially, unless we > leave it up to the find_template callable to handle that). it should be local to some specific template-lookup object that was specified to the context of this particular template call. there might be lots of template-lookup implementations happening within the same thread even. > One substantial one that I think comes up in Myghty is that you get a > template, and then you get all its implicit parents (I'm not sure what > the term is in Myghty). I don't even know what to call that, and > certainly not how to deal with it in find_template. This is not just in Myghty (its called "inheritance" in the general sense, and the "implicit" parent is just called an "autohandler" which is basically just, "the default parent"). Spyce and Django templates also support inheritance....and it is my opinion (a former version of me would say "i can guarantee you") that once you use template inheritance, you'll never go back. From a find_template() perspective, its not different from a template embedding another - its just that the executional relationship of those templates is reversed (or turned inside-out, depending on how you look at it). > > One possible workaround for that would be for the plugin's load_template > to return an object that encompasses all the implicit parents. But that > takes some functionality from find_template that seems like it should be > in find_template. Unless there was some really simple rule, like: I dont think find_template has to be organized differently for inheritance vs. embedded components...the only thing that I would say is needed is the ability for find_template to get passed that "context" of, "i need this component to use as my parent"/ "i need to use this component as an 'include'", since its nice to enable your resolution to look in different places for those two things. The four built-in contexts in myghty are "request", "subrequest", "component", and "inherit". Plus you can define your own contexts, such as in one demo I build a "front" controller which then forwards onto a secondary controller, and it uses different contexts to indicate different search paths (therefore nobody can hit a secondary file directly from the outside). although those arent even templates, those are controller functions. are we considering the finder being able to return arbitrary callables and not just "templates" (since I like that) ? > This kind of recursive call is definitely considered in my spec. > > find_template (I call it find_resource, but we can ignore the "resource" > stuff too) gets relative_to_name and relative_to_object optional > arguments. You can use these for whatever resolution logic you want. > I am not even ready to look at the actual specs yet :) I am hoping that if Ben and I, and whoever else wants to chime in, can say, "hey heres an execution model you should probably support", the specs will mold themselves to fit all those models and then we can start hypothesizing how to stretch our work into them. From janssen at parc.com Mon Feb 6 20:25:13 2006 From: janssen at parc.com (Bill Janssen) Date: Mon, 6 Feb 2006 11:25:13 PST Subject: [Web-SIG] WSGI in standard library In-Reply-To: Your message of "Sun, 05 Feb 2006 05:12:42 PST." <43E5F9CA.9010001@xhaus.com> Message-ID: <06Feb6.112517pst."58633"@synergy1.parc.xerox.com> > Instead, I think the right approach is to continue with the existing > approach: put the most basic possible WSGI server in the standard > library, for educational purposes only, and a warning that it shouldn't > really be used for production purposes. I strongly disagree with this thinking. Non-production code shouldn't go into the stdlib; instead, Alan's proposed module should go onto some pedagogical website somewhere with appropriate tutorial documentation. Bill From dangoor at gmail.com Mon Feb 6 20:50:43 2006 From: dangoor at gmail.com (Kevin Dangoor) Date: Mon, 6 Feb 2006 14:50:43 -0500 Subject: [Web-SIG] My original template API proposal In-Reply-To: <43E687A6.3070900@colorstudy.com> References: <43E53709.4040007@colorstudy.com> <43E6642B.50006@colorstudy.com> <43E687A6.3070900@colorstudy.com> Message-ID: <3f085ecd0602061150r175faf96va6fbe3525749be11@mail.gmail.com> >From your original spec: def __init__(extra_vars_func=None, options=None): """ [Is ``extra_vars_func`` necessary? Couldn't it be handled by a wrapper around ``render``?] I think you're right. I don't see any need for extra_vars_func. I think extra_vars_func was a side effect of refactoring the template plugins to not be TurboGears-specific in any way. But, TurboGears can do the equivalent from the place where it calls render(). On 2/5/06, Ian Bicking wrote: > def render(template_instance, vars, wsgi_environ=None, > set_header_callback=None, **kwargs): I'm not keen on the wsgi_environ and set_header_callback options, because I don't perceive a true need for *this* API to be tied to the web. Of course, you need these if the template itself is going to set any of the headers, but there is some added complexity that results. TurboGears skirts this by having the Python code outside of the template decide what content-type (and other headers) are appropriate. Since these are optional, I'm not strongly against them, but they just feel like they add a bit of complexity to an API that is dealing with a simple problem. Kevin From ben at groovie.org Mon Feb 6 21:08:31 2006 From: ben at groovie.org (Ben Bangert) Date: Mon, 6 Feb 2006 12:08:31 -0800 Subject: [Web-SIG] My original template API proposal In-Reply-To: <3f085ecd0602061150r175faf96va6fbe3525749be11@mail.gmail.com> References: <43E53709.4040007@colorstudy.com> <43E6642B.50006@colorstudy.com> <43E687A6.3070900@colorstudy.com> <3f085ecd0602061150r175faf96va6fbe3525749be11@mail.gmail.com> Message-ID: <9B615E3D-277F-439F-9347-3A659F01C8D5@groovie.org> On Feb 6, 2006, at 11:50 AM, Kevin Dangoor wrote: > I'm not keen on the wsgi_environ and set_header_callback options, > because I don't perceive a true need for *this* API to be tied to the > web. Of course, you need these if the template itself is going to set > any of the headers, but there is some added complexity that results. > TurboGears skirts this by having the Python code outside of the > template decide what content-type (and other headers) are appropriate. Of course, quite a few people have all mentioned conflicting 'needs' they perceived. This option keeps flexibility so that whatever your perception, if you *need* the template language to work like a WSGI app, you *could* do that with an API including these options. > Since these are optional, I'm not strongly against them, but they just > feel like they add a bit of complexity to an API that is dealing with > a simple problem. Many people won't ever see them, its only those of us using template languages where it can handle a full WSGI environment that will want this. Having that option enables wrapping the render function in a way that will be identical to a WSGI call. I think it's a good compromise that still lets template languages have the freedom to provide their own request/response API independent of the framework, in addition to accommodating existing template languages. I don't see a problem with Ian's spec, as it keeps the up-front simple appearance, while leaving the door open to a WSGI wrapper for those of us that want it. Cheers, Ben From gary at zope.com Mon Feb 6 21:40:53 2006 From: gary at zope.com (Gary Poster) Date: Mon, 6 Feb 2006 15:40:53 -0500 Subject: [Web-SIG] A trivial template API counter-proposal In-Reply-To: References: <5.1.1.6.0.20060205121213.021562b8@mail.telecommunity.com> <5.1.1.6.0.20060205135745.0371d938@mail.telecommunity.com> <43E66763.5010902@colorstudy.com> Message-ID: <6208A0A7-0C79-40F8-BA6B-115FB006986E@zope.com> On Feb 5, 2006, at 11:32 PM, Guido van Rossum wrote: > Phillip described the workflow for Django/Cheetah style templates > as follows: > >> * framework calls some Python code you wrote >> * your code returns a dictionary of values you want rendered >> * framework passes your returned values to the template, gets back >> a string >> * framework sends the string back to the browser > > But there's an even lower-level variant, where the Python code > referenced invokes the template (with variables) and receives a string > back, which it possibly munges further and then passes to the > framework. (For example, my first attempt at using Django templates > had one piece of Python code that concatenated the results from > several templates.) > > I'd like someone to write up a similar list explaining how e.g. Zope3 > differs -- I suppose at one point in the past I used to know, but that > knowledge is on a back-up tape that was lost after I moved to > California. :-( Zope 3 usually works like this: - framework calls your view code, usually with a context object and a request (which has a response). - your code can set headers on the response, and should return either unicode or a string (with encoding already set in an explicit ContentType). Often the unicode is generated by a template. Often, but less often, the template is a ZPT. If calculating data, setting headers, and rendering a template are done in one, two, or X steps, the code here is in charge. We have a variety of approaches to dividing the tasks up, sometimes quite finely. Zope 3 can alternately work like this. This approach is in flux, but illustrates a direction we are going: - (same first step as before:) framework calls your view code, usually with a context object and a request (which has a response). - your code can return any object that provides or can be adapted to IResult, where IResult has a `headers` attribute and a `body` attribute. We've also talked about internal pipelines, where something like the IResult hook might help; or "external" pipelines with a WSGI middleware stack. I hope that helps. I haven't followed this thread very closely for lack of time, and I am nervous that Phil Eby is right that an "official" template spec is going to be exclusive, not inclusive. If someone replies to Chris McDonough's request for summary, perhaps that will also let me and others catch up with the proposals. Gary From ianb at colorstudy.com Mon Feb 6 22:02:07 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 06 Feb 2006 15:02:07 -0600 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <06Feb6.112517pst."58633"@synergy1.parc.xerox.com> References: <06Feb6.112517pst."58633"@synergy1.parc.xerox.com> Message-ID: <43E7B94F.6030601@colorstudy.com> Bill Janssen wrote: >>Instead, I think the right approach is to continue with the existing >>approach: put the most basic possible WSGI server in the standard >>library, for educational purposes only, and a warning that it shouldn't >>really be used for production purposes. > > > I strongly disagree with this thinking. Non-production code shouldn't > go into the stdlib; instead, Alan's proposed module should go onto > some pedagogical website somewhere with appropriate tutorial > documentation. SimpleHTTPServer has always been pseudo-production. People use it successfully in production environments, but typically in fairly constrained environments. For instance, if you are proxying from Apache, Apache isn't going to act crazy with your application. If your application is serving the public, who knows what kind of connections you are going to get. So it works to a degree. Anyway, I think general rules about this -- choosing simple or complete -- is only of limited utility. There are features that I think can and probably should be left out of the standard library. For instance, support for Keep-Alive and Continue. The server is 100% functional without some of these features, it just won't perform as well. However, that doesn't mean we should deliberately choose an implementation that takes less into account. Some things should be supported. All methods should be supported (and I now note that Paste's doesn't do that, but CP's and wsgiref's does, though from what I can tell CP's might not support PUT or other methods properly, as it special cases POST wrt request bodies). The server should be reasonable easy to use with HTTPS -- maybe not natively, but at least providing a mix-in that can be used easily enough. Anyway, there's some details that should be worked out, and I don't think we should slavishly ignore possible enhancements. But I also agree that possibly problematic features (specifically optional HTTP/1.1 features) can be left out, especially if they can be re-added in subclasses (subclasses that would probably have more frequent releases). -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From ianb at colorstudy.com Mon Feb 6 22:23:14 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 06 Feb 2006 15:23:14 -0600 Subject: [Web-SIG] My original template API proposal In-Reply-To: <3f085ecd0602061150r175faf96va6fbe3525749be11@mail.gmail.com> References: <43E53709.4040007@colorstudy.com> <43E6642B.50006@colorstudy.com> <43E687A6.3070900@colorstudy.com> <3f085ecd0602061150r175faf96va6fbe3525749be11@mail.gmail.com> Message-ID: <43E7BE42.4090205@colorstudy.com> Kevin Dangoor wrote: > On 2/5/06, Ian Bicking wrote: > >> def render(template_instance, vars, wsgi_environ=None, >> set_header_callback=None, **kwargs): > > > I'm not keen on the wsgi_environ and set_header_callback options, > because I don't perceive a true need for *this* API to be tied to the > web. Of course, you need these if the template itself is going to set > any of the headers, but there is some added complexity that results. > TurboGears skirts this by having the Python code outside of the > template decide what content-type (and other headers) are appropriate. > > Since these are optional, I'm not strongly against them, but they just > feel like they add a bit of complexity to an API that is dealing with > a simple problem. The benefit I see, particularly with wsgi_environ, is that we can define exactly what it is -- it is a WSGI environment, with a supporting spec to define exactly what that is. We could also put it in a specific key in vars, but that doesn't feel as clean to me. Anyway, that doesn't change that it's very optional, and a template plugin is completely free to ignore it, and a framework is free not to pass it in. But in the case where both template and framework are interested in that sort of thing, the spec documents how that information is passed around because we've given it an argument name. set_header_callback is very kludgy-feeling, though, and I'm not very comfortable with it. OTOH, I like at least that you know if anyone is listening (where a value of None means no one is) -- having all templates generate headers, and then substantial portion of frameworks throw those headers away, seems like a bad solution as well. And there's lots of contexts when a template simply shouldn't be allowed to set headers, because that's not meaningful in the context it is being used (even if it is in a web request, and wsgi_environ still applies). Though content-type has some real utility even in embedded contexts -- smart templates could fix up encodings of included content, or even do markup transformations -- like XHTML to HTML (or vice versa). -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From fumanchu at amor.org Mon Feb 6 22:45:41 2006 From: fumanchu at amor.org (Robert Brewer) Date: Mon, 6 Feb 2006 13:45:41 -0800 Subject: [Web-SIG] WSGI in standard library Message-ID: <6949EC6CD39F97498A57E0FA55295B21016A4B80@ex9.hostedexchange.local> Ian Bicking wrote: > Anyway, I think general rules about this -- choosing simple > or complete -- is only of limited utility. There are features > that I think can and probably should be left out of the > standard library. For instance, support for Keep-Alive and > Continue. The server is 100% functional without some of > these features, it just won't perform as well. However, > that doesn't mean we should deliberately choose an > implementation that takes less into account. FWIW, CherryPy has gotten a lot of criticism on this front. Our servers are "conditionally compliant" with HTTP/1.1 (specifically relating to Continue), but others prefer to call them broken. Just be aware you'll get the same flak over any candidate for the stdlib. > Some things should be supported. All methods should be > supported (and I now note that Paste's doesn't do that, > but CP's and wsgiref's does, though from what I can tell > CP's might not support PUT or other methods properly, > as it special cases POST wrt request bodies). CP's _cpwsgiserver isn't a party to that special-casing, and handles any method, even fictional ones, equally. However, one thing CP's WSGI server currently does *not* have is the ability for the user to configure SCRIPT_NAME. Christian Wyglendowski is working on a patch for that at the moment, and I think that would have to be built, released, and stable in the field before CP's WSGI server could be considered a candidate for the stdlib. Given how recent *that* feature-request is (only a month or two), I think it's premature for any server to be considered for the stdlib; WSGI needs another year to shake out other, hidden spec-interpretation differences. Robert Brewer System Architect Amor Ministries fumanchu at amor.org From ianb at colorstudy.com Mon Feb 6 23:08:26 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 06 Feb 2006 16:08:26 -0600 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <6949EC6CD39F97498A57E0FA55295B21016A4B80@ex9.hostedexchange.local> References: <6949EC6CD39F97498A57E0FA55295B21016A4B80@ex9.hostedexchange.local> Message-ID: <43E7C8DA.2010001@colorstudy.com> Robert Brewer wrote: > Ian Bicking wrote: > >>Anyway, I think general rules about this -- choosing simple >>or complete -- is only of limited utility. There are features >>that I think can and probably should be left out of the >>standard library. For instance, support for Keep-Alive and >>Continue. The server is 100% functional without some of >>these features, it just won't perform as well. However, >>that doesn't mean we should deliberately choose an >>implementation that takes less into account. > > > FWIW, CherryPy has gotten a lot of criticism on this front. Our servers > are "conditionally compliant" with HTTP/1.1 (specifically relating to > Continue), but others prefer to call them broken. Just be aware you'll > get the same flak over any candidate for the stdlib. I think the standard library avoids criticism on these kinds of points by underselling itself. Not that people don't complain about SimpleHTTPServer as well... Is what CherryPy does (which I think it just inherits from SimpleHTTPServer) compliant with the HTTP spec? >>Some things should be supported. All methods should be >>supported (and I now note that Paste's doesn't do that, >>but CP's and wsgiref's does, though from what I can tell >>CP's might not support PUT or other methods properly, >>as it special cases POST wrt request bodies). > > > CP's _cpwsgiserver isn't a party to that special-casing, and handles any > method, even fictional ones, equally. > > However, one thing CP's WSGI server currently does *not* have is the > ability for the user to configure SCRIPT_NAME. Christian Wyglendowski is > working on a patch for that at the moment, and I think that would have > to be built, released, and stable in the field before CP's WSGI server > could be considered a candidate for the stdlib. Given how recent *that* > feature-request is (only a month or two), I think it's premature for any > server to be considered for the stdlib; WSGI needs another year to shake > out other, hidden spec-interpretation differences. I think this is a separate issue. The HTTP server should always set SCRIPT_NAME to ''. If it was receiving requests proxied from another server, and got a custom header that somehow indicated some other SCRIPT_NAME, then maybe -- but I don't think that's a necessary feature, nor even a convention that belongs in the standard library at this time. I think the WSGI server discussion lately has been with CherryPy the *framework* acting as a WSGI server, not the CherryPy HTTP server. Other discussions have been around requests coming from non-HTTP-server WSGI sources, where SCRIPT_NAME might not be ''. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From christian at dowski.com Tue Feb 7 01:49:08 2006 From: christian at dowski.com (Christian Wyglendowski) Date: Mon, 06 Feb 2006 18:49:08 -0600 Subject: [Web-SIG] On HTTP Servers and WSGI (was "WSGI in standard library") Message-ID: <43E7EE84.4060208@dowski.com> Robert Brewer said (in reference to CherryPy's WSGI server): >> However, one thing CP's WSGI server currently does *not* have is the >> ability for the user to configure SCRIPT_NAME. Christian Wyglendowski is >> working on a patch for that at the moment Ian Bicking said: > I think this is a separate issue. The HTTP server should always set > SCRIPT_NAME to ''. If it is a full blown WSGI compliant HTTP server, why should it do that? Shouldn't it support mounting applications at various points (SCRIPT_NAME locations)? That makes sense to me at least. Christian http://www.dowski.com From renesd at gmail.com Tue Feb 7 01:51:03 2006 From: renesd at gmail.com (Rene Dudfield) Date: Tue, 7 Feb 2006 11:51:03 +1100 Subject: [Web-SIG] Bowing out (was Re: A trivial template API counter-proposal) In-Reply-To: <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> References: <5.1.1.6.0.20060204131722.02179008@mail.telecommunity.com> <5.1.1.6.0.20060205124025.021885b0@mail.telecommunity.com> <43E659BB.9000301@xhaus.com> <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> Message-ID: <64ddb72c0602061651t208f02f5j7f47e74695184b32@mail.gmail.com> I have no idea what any of these template specifications are about yet. I do value your approach to being inclusive, and allowing as many as possible styles to play. As you have said yourself, I think many people can not understand what everyone is talking about. So until people have something written down most of the rest of us will have no idea what anyone is really talking about. Being able to keep up with 100ish emails is no easy task. At least a gathering of the use cases has been happening. Which is nice. Once a spec is written up, we can all look at which use cases are not allowed with whatever spec is proposed. On 2/7/06, Phillip J. Eby wrote: > At 08:02 PM 2/5/2006 +0000, Alan Kennedy wrote: > >Looking at this in an MVC context, the application is responsible for > >populating the Model (user namespace), and selecting which View > >(template<->media-type) is suitable for return to the user. Templates > >should not vary media types. HTTP headers do need to be set for > >different templates/media-types. But that should be the responsibility > >of the HTTP application, not the template, which should be unaware of > >the application contect in which it is running, except for the contents > >of the Model/user-namespace. > > As soon as you start talking about what templates should or should not do > (as opposed to what they *already* do), you've stopped writing an inclusive > spec and have wandered off into evangelizing a particular framework philosophy. > > OTOH, before I first proposed WSGI in 2003, nobody here seemed especially > interested in writing inclusive specs anyway, and I rather get the > impression they still aren't now, my "insistence" (as Ian calls it) > notwithstanding. > > What I've been trying to do with both WSGI and with this spec is to create > something that deals with the *actual* complexity and diversity of Python > web frameworks as they exist today, rather than reducing the diversity to > match whatever the currently popular paradigm is. This wasn't a popular > approach when I introduced WSGI, either, but in the case of WSGI it was > easy to point to all the previously-failed attempts at standardizing > request/response objects due to people not taking backward compatibility > into account. > > At this point it has become clear to me that even if I spent my days and > nights writing a compelling spec of what I'm proposing and then trying to > sell it to the Web SIG, it would be at best a 50/50 chance of getting > through, and in the process it appears that I'd be burning through every > bit of goodwill I might have previously possessed here. And, although I > believe that the approach currently being taken to this spec is divisive of > the community, I have to admit that since my attempts at education about > the issues hasn't been particularly successful, it would appear that > continuing to argue about it is no less divisive than what I'm arguing > against. (For that matter, it's not even clear to me any more that most of > the people on whose behalf I'm fighting would even realize yet why the > future I want would be beneficial for them.) > > I really expected more people to see the benefits of the WSGI embedding > approach, though, and although I've gotten a few private mails of support, > it isn't anywhere near the level I thought it would be. Given the intense > pressure that some parties are putting on having a spec *right* now, I > don't feel that I can reasonably deliver a competing spec without > interfering with my work and personal commitments in the next few > weeks. Since I've already been using most of my "Python community > contribution" time in the last week arguing about this, at this point it > seems the community would be better served by me devoting that time to > working on setuptools, rather than continuing to fight for a vision that > hardly anybody else believes in. And, I'd rather save whatever karma I > have left here for something with a better chance of success. > > Good luck with the spec. > > > _______________________________________________ > Web-SIG mailing list > Web-SIG at python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: http://mail.python.org/mailman/options/web-sig/renesd%40gmail.com > From ianb at colorstudy.com Tue Feb 7 02:53:49 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 06 Feb 2006 19:53:49 -0600 Subject: [Web-SIG] On HTTP Servers and WSGI (was "WSGI in standard library") In-Reply-To: <43E7EE84.4060208@dowski.com> References: <43E7EE84.4060208@dowski.com> Message-ID: <43E7FDAD.4060903@colorstudy.com> Christian Wyglendowski wrote: > Robert Brewer said (in reference to CherryPy's WSGI server): > >>>However, one thing CP's WSGI server currently does *not* have is the >>>ability for the user to configure SCRIPT_NAME. Christian Wyglendowski is >>>working on a patch for that at the moment > > > Ian Bicking said: > >>I think this is a separate issue. The HTTP server should always set >>SCRIPT_NAME to ''. > > > If it is a full blown WSGI compliant HTTP server, why should it do that? > Shouldn't it support mounting applications at various points > (SCRIPT_NAME locations)? That makes sense to me at least. It doesn't really need to do that, it's easy to do that in the WSGI application itself. For instance: def dispatch(app_map): app_map = app_map.items() app_map.sort(lambda a, b: -cmp(len(a[0]), len(b[0]))) def application(environ, start_response): path_info = environ.get('PATH_INFO', '') for app_prefix, app in app_map: app_prefix = app_prefix.rstrip('/')+'/' if path_info.startswith(app_prefix): environ['SCRIPT_NAME'] += app_prefix[:-1] environ['PATH_INFO'] = environ.get( 'PATH_INFO', '')[len(app_prefix)-1:] return app(environ, start_response) else: start_response('404 Not Found', []) return [] return application dispatching_app = dispatch({ '/blog': my_blog_app, '/': root_app, '/admin': admin_app, }) I can understand this is a common desire and it is not obvious that you can do this, so it might be useful to include some middleware like this. But the server itself does not need to do this. And there's lots of ways you might want to do this (e.g., virtual hosts, which do the same basic thing but match against environ['HTTP_HOST']). Paste includes code along these lines in paste.urlmap: http://pythonpaste.org/module-paste.urlmap.html -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ianb at colorstudy.com Tue Feb 7 03:16:03 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 06 Feb 2006 20:16:03 -0600 Subject: [Web-SIG] A trivial template API counter-proposal In-Reply-To: <31103.66.192.34.8.1139245853.squirrel@www.geekisp.com> References: <5.1.1.6.0.20060205121213.021562b8@mail.telecommunity.com> <62388AF2-7B46-42AB-ADC2-A008BD87BE91@groovie.org> <5.1.1.6.0.20060205173119.0385b718@mail.telecommunity.com> <42884277-B14A-43B4-9ED0-3D2935A04A4A@zzzcomputing.com> <43E6E9E4.3030805@colorstudy.com> <31103.66.192.34.8.1139245853.squirrel@www.geekisp.com> Message-ID: <43E802E3.1020108@colorstudy.com> Michael Bayer wrote: > Ian Bicking wrote: > >>>I would say that >>>the argument or arguments passed to find_template() need to be pretty >>>open ended, since you cant predict what kinds of things might be >>>meaningful to a template finder, such as in myghty's case, the resolver >>>wants to know if you are asking for this template to be used in a >>>request, or an "embedded" template call, since there may be rules set >>>up to return different components based on that context. >> >>Do these need to be arguments, or can the context be instance variables >>of a find_template callable? That is, you might do: >> >>def myghty_entry_point(environ, start_response): >> find_template = MyghtyTemplateFinder(environ) >> ... > > > Well this is the question- say im using an implementation of "the spec", > whatever it is. I say, OK, I want to use the PylonsResolver with the > KidCompiler with the DjangoCodeCache with the MyghtyInterpreter, and we > snap those together. We fire it up, send a request for template "xxxxx", > PylonsResolver calls it up and sends it off to the > KidCompiler/DjangoCodeCache to get a template object, then off to > MyghtyInterpreter to run it. > > Then , inside the template, we print out some html, then we say, > "include_template('xxxx')", meaning the template wants to call another > one. Does the "include_template" call the resolver that is internal to > the MyghtyInterpreter ? Or does it have to use the Resolver that was sent > into the beginning of the request ? (id say definitely the latter). It should use the resolver passed in. Getting it to do so is the hard part to implement ;) > My instinct in this kind of situation is, cant we just add a **kwargs to > find_template() ? :) Im guessing not (but why not ?). But some kind of > dictionary where the template's runtime environment can pass messages > through to the resolver would be handy...else, Mygthy would have to hack > something together, like serializing arguments in a string or something > inefficient like that. Also, since this kind of call will happen on the > order of N based on the complexity of the template, it would be nice if > there wasnt too much (or really any) "create object" overhead to > that...since it shouldnt have to use up a lot of resources for a very > frequent operation. What things do you pass through? What things can you ignore? Should the template pass through kwargs unless it knows better for some reason? If these are state held in find_template, then this isn't a problem. And potentially find_template can have other (non-standard) methods on it, which the template could try to use. This seems clearer in terms of what is an extension, and what is a core method. The WSGI environment does offer an advantage here, because it is easier to put extra items in the environment -- keys get namespaces, where attributes are flatter, and there are easier techniques for introspecting a dictionary, and easier ways to filter, copy, or recreate a dictionary. So... maybe that's an argument for WSGIness. I do appreciate these aspects of the WSGI dictionary for web requests. >>At least in my spec, find_template is not meant to be a global value. >>(However, I believe this complicates caching substantially, unless we >>leave it up to the find_template callable to handle that). > > > it should be local to some specific template-lookup object that was > specified to the context of this particular template call. there might be > lots of template-lookup implementations happening within the same thread > even. In the spec I give, it is assumed that the template object keeps a reference to the name it was fetched with, and the find_template it was found with. And so it gets passed through that way. Maybe it should be more explicit. Hrm... I'm feeling myself coming back around to WSGI thoughts. Oh irony! (What about WSGI-like, but not WSGI?) >>One substantial one that I think comes up in Myghty is that you get a >>template, and then you get all its implicit parents (I'm not sure what >>the term is in Myghty). I don't even know what to call that, and >>certainly not how to deal with it in find_template. > > > This is not just in Myghty (its called "inheritance" in the general sense, > and the "implicit" parent is just called an "autohandler" which is > basically just, "the default parent"). Spyce and Django templates also > support inheritance....and it is my opinion (a former version of me would > say "i can guarantee you") that once you use template inheritance, you'll > never go back. From a find_template() perspective, its not different from > a template embedding another - its just that the executional relationship > of those templates is reversed (or turned inside-out, depending on how you > look at it). The inversion of control is the tricky part. In something like Cheetah, Kid, or ZPT the template (as represented solely by the file contents) gets control first, and then explicitly hands it off to whatever it is "inheriting" from (they each have substantially different notions of how to compose templates). This implies another call to find_template, which gives control back to the framework, which is what we want. But that doesn't seem to work for Myghty. >>One possible workaround for that would be for the plugin's load_template >>to return an object that encompasses all the implicit parents. But that >>takes some functionality from find_template that seems like it should be >>in find_template. Unless there was some really simple rule, like: > > > I dont think find_template has to be organized differently for inheritance > vs. embedded components...the only thing that I would say is needed is the > ability for find_template to get passed that "context" of, "i need this > component to use as my parent"/ "i need to use this component as an > 'include'", since its nice to enable your resolution to look in different > places for those two things. The four built-in contexts in myghty are > "request", "subrequest", "component", and "inherit". > > Plus you can define your own contexts, such as in one demo I build a > "front" controller which then forwards onto a secondary controller, and it > uses different contexts to indicate different search paths (therefore > nobody can hit a secondary file directly from the outside). although > those arent even templates, those are controller functions. are we > considering the finder being able to return arbitrary callables and not > just "templates" (since I like that) ? Hmm... I'm going to think about the dictionary/callable WSGI-ish approach, because maybe that will make this easier. Umm... circles... -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From christian at dowski.com Tue Feb 7 04:00:53 2006 From: christian at dowski.com (Christian Wyglendowski) Date: Mon, 06 Feb 2006 21:00:53 -0600 Subject: [Web-SIG] On HTTP Servers and WSGI (was "WSGI in standard library") In-Reply-To: <43E7FDAD.4060903@colorstudy.com> References: <43E7EE84.4060208@dowski.com> <43E7FDAD.4060903@colorstudy.com> Message-ID: <43E80D65.8010002@dowski.com> Ian Bicking wrote: > It doesn't really need to do that, it's easy to do that in the WSGI > application itself. For instance: > > def dispatch(app_map): .... > I can understand this is a common desire and it is not obvious that you > can do this, so it might be useful to include some middleware like this. > But the server itself does not need to do this. And there's lots of > ways you might want to do this (e.g., virtual hosts, which do the same > basic thing but match against environ['HTTP_HOST']). Ok, gotcha. Thanks for the example code. I had basically written something like that right in the CP WSGI server. I guess as long as the SCRIPT_NAME is '', then everything else should be cool. Christian http://www.dowski.com From mike_mp at zzzcomputing.com Tue Feb 7 05:46:19 2006 From: mike_mp at zzzcomputing.com (Michael Bayer) Date: Mon, 6 Feb 2006 23:46:19 -0500 Subject: [Web-SIG] A trivial template API counter-proposal In-Reply-To: <43E802E3.1020108@colorstudy.com> References: <5.1.1.6.0.20060205121213.021562b8@mail.telecommunity.com> <62388AF2-7B46-42AB-ADC2-A008BD87BE91@groovie.org> <5.1.1.6.0.20060205173119.0385b718@mail.telecommunity.com> <42884277-B14A-43B4-9ED0-3D2935A04A4A@zzzcomputing.com> <43E6E9E4.3030805@colorstudy.com> <31103.66.192.34.8.1139245853.squirrel@www.geekisp.com> <43E802E3.1020108@colorstudy.com> Message-ID: On Feb 6, 2006, at 9:16 PM, Ian Bicking wrote: > > The inversion of control is the tricky part. In something like > Cheetah, Kid, or ZPT the template (as represented solely by the > file contents) gets control first, and then explicitly hands it off > to whatever it is "inheriting" from (they each have substantially > different notions of how to compose templates). This implies > another call to find_template, which gives control back to the > framework, which is what we want. > > But that doesn't seem to work for Myghty. > Well no, I think it will work; the template that is the one youre calling is always the first thing loaded, just like a non-inherited template environment. it is loaded by the runtime interpreter (Ben - the "m" object). the runtime interpreter then, before running the template, looks at the template for an "inherit" attribute (or defaults to the special name 'autohandler'), and then uses the resolver again to find the inherited template. this process continues up the chain until no more parent templates are found. *then*, it passes control to the bottommost template which executes, that template then instructs the runtime interpreter to go to the next in the chain, and so on. if you were monitoring the usage of the find_template() function during this procedure, it would not be recognizable if the initial template were simply including all those other templates within it, or if they were being used as inherited wrappers...its the runtime interpreter that is making them execute themselves in a different order. in Myghty we just might have to slightly change the external view of "the runtime environment" and "the template" so that they look like the same thing even though they are different. but like the ones you mention, the "first" template gets "control", and uses find_template to locate successive templates. >>> One possible workaround for that would be for the plugin's >>> load_template >>> to return an object that encompasses all the implicit parents. >>> But that >>> takes some functionality from find_template that seems like it >>> should be >>> in find_template. Unless there was some really simple rule, like: >> I dont think find_template has to be organized differently for >> inheritance >> vs. embedded components...the only thing that I would say is >> needed is the >> ability for find_template to get passed that "context" of, "i need >> this >> component to use as my parent"/ "i need to use this component as an >> 'include'", since its nice to enable your resolution to look in >> different >> places for those two things. The four built-in contexts in myghty >> are >> "request", "subrequest", "component", and "inherit". >> Plus you can define your own contexts, such as in one demo I build a >> "front" controller which then forwards onto a secondary >> controller, and it >> uses different contexts to indicate different search paths (therefore >> nobody can hit a secondary file directly from the outside). although >> those arent even templates, those are controller functions. are we >> considering the finder being able to return arbitrary callables >> and not >> just "templates" (since I like that) ? > > Hmm... I'm going to think about the dictionary/callable WSGI-ish > approach, because maybe that will make this easier. Umm... circles... > > > -- > Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From fumanchu at amor.org Tue Feb 7 08:12:07 2006 From: fumanchu at amor.org (Robert Brewer) Date: Mon, 6 Feb 2006 23:12:07 -0800 Subject: [Web-SIG] Who is responsible for SCRIPT_NAME? References: <43E7EE84.4060208@dowski.com> <43E7FDAD.4060903@colorstudy.com> Message-ID: <6949EC6CD39F97498A57E0FA55295B214115FB@ex9.hostedexchange.local> Ian Bicking said: > The HTTP server should always set SCRIPT_NAME to ''. Christian asked: > If it is a full blown WSGI compliant HTTP server, > why should it do that? Shouldn't it support mounting > applications at various points (SCRIPT_NAME locations)? and Ian replied: > It doesn't really need to do that, it's easy to do > that in the WSGI application itself. For instance: > > def dispatch(app_map): > ... > > I can understand this is a common desire and it > is not obvious that you can do this, so it might > be useful to include some middleware like this. > But the server itself does not need to do this. > And there's lots of ways you might want to do > this (e.g., virtual hosts, which do the same > basic thing but match against environ['HTTP_HOST']). > Paste includes code along these lines in paste.urlmap Aargh. This is getting really confusing for me as the author of WSGI gateways (wrappers around non-WSGI HTTP servers, like mod_python + Apache), one of the maintainers of a pure WSGI HTTP server (in CherryPy) and the primary developer of a WSGI interface for a popular app framework (CherryPy). On the one hand, I'm being told that WSGI apps need to "obey" SCRIPT_NAME no matter where it comes from, and on the other, I'm being told that SCRIPT_NAME is essentially ignorable. I'm getting tired of the run-around. Dear Web-SIG, please weigh in. Who is responsible for SCRIPT_NAME (setting and/or interpreting it) and what does it "mean"? I've been told that PEP 333 requires that the origin server (like my modpython_gateway, for example) must be told the "mount point" of every WSGI app it serves and set SCRIPT_NAME to that, always, so the app can trust it. Now (by the same person, as far as I can tell) I'm being told that the origin server should always set SCRIPT_NAME to "", which IMO denies the reason for its inclusion in the spec. Regardless of the answer, that answer needs to be more clear in the spec--even if the answer is "it depends" or "do what you want". So please, let's decide so I can make compliant all the code for which I'm responsible. I'm so very tired of interpretation games. Ian and I keep talking past each other, so I, at least, need others to chime in and explain things. Robert Brewer System Architect Amor Ministries fumanchu at amor.org -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/web-sig/attachments/20060206/98503780/attachment.html From titus at caltech.edu Sun Feb 5 02:11:37 2006 From: titus at caltech.edu (titus at caltech.edu) Date: Sat, 4 Feb 2006 17:11:37 -0800 Subject: [Web-SIG] A trivial template API counter-proposal In-Reply-To: References: <5.1.1.6.0.20060204131722.02179008@mail.telecommunity.com> Message-ID: <20060205011137.GA27469@caltech.edu> -> > Sure. I'd suggest that we might want to expand it a little to include some -> > of the fine contributions of other authors, such as Ian's "lint" middleware -> > (which sits between an app and a server and checks both for compliance) and -> > maybe even his "Python debugger in the browser" middleware. There have -> > also been people who've written a few utilities for CGI-in-WSGI, WSGI in -> > mod_python, etc. And it'd be nice to squeeze in a FastCGI implementation -> > if somebody's got a suitable one. Docs are also nonexistent at the -> > moment. I'd be happy to co-ordinate and consolidate the additions, and of -> > course I'll write documentation, though any assistance people can offer -> > would be greatly appreciated. Some people have written nice WSGI tutorials -> > or explanations, so if anybody wants to offer those up for consolidation -> > into docs, that would be good too. -> -> Adding support for mod_python in the stdlib seems excessive -- -> shouldn't that be included with mod_python instead? FastCGI OTOH seems -> handy to have. -> -> See my response to Ian recommending a conservative approach here. A *very* conservative approach would be to include a simple WSGI_HTTPHandler class based on BaseHTTPServer (much like SimpleHTTPServer), and that's it. This is my favorite option... While there are many additional handlers one *could* add in, it's not all that tough to include the necessary machinery in one's own package. I'd shy away from FastCGI and SCGI (even though I like them both) simply because it's easy enough to get adapters and it seems like an undue burden on *someone* to put them into the Python distribution. cheers, --titus From pje at telecommunity.com Tue Feb 7 12:50:37 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 07 Feb 2006 06:50:37 -0500 Subject: [Web-SIG] Who is responsible for SCRIPT_NAME? In-Reply-To: <6949EC6CD39F97498A57E0FA55295B214115FB@ex9.hostedexchange. local> References: <43E7EE84.4060208@dowski.com> <43E7FDAD.4060903@colorstudy.com> Message-ID: <5.1.1.6.0.20060207064306.02154fc0@mail.telecommunity.com> At 11:12 PM 2/6/2006 -0800, Robert Brewer wrote: >Ian Bicking said: > > The HTTP server should always set SCRIPT_NAME to ''. > >Christian asked: > > If it is a full blown WSGI compliant HTTP server, > > why should it do that? Shouldn't it support mounting > > applications at various points (SCRIPT_NAME locations)? > >and Ian replied: > > It doesn't really need to do that, it's easy to do > > that in the WSGI application itself. For instance: > > > > def dispatch(app_map): > > ... > > > > I can understand this is a common desire and it > > is not obvious that you can do this, so it might > > be useful to include some middleware like this. > > But the server itself does not need to do this. > > And there's lots of ways you might want to do > > this (e.g., virtual hosts, which do the same > > basic thing but match against environ['HTTP_HOST']). > > Paste includes code along these lines in paste.urlmap > >Aargh. This is getting really confusing for me as the author of WSGI >gateways (wrappers around non-WSGI HTTP servers, like mod_python + >Apache), one of the maintainers of a pure WSGI HTTP server (in CherryPy) >and the primary developer of a WSGI interface for a popular app framework >(CherryPy). On the one hand, I'm being told that WSGI apps need to "obey" >SCRIPT_NAME no matter where it comes from, and on the other, I'm being >told that SCRIPT_NAME is essentially ignorable. I'm getting tired of the >run-around. > >Dear Web-SIG, please weigh in. Who is responsible for SCRIPT_NAME (setting >and/or interpreting it) and what does it "mean"? I've been told that PEP >333 requires that the origin server (like my modpython_gateway, for >example) must be told the "mount point" of every WSGI app it serves and >set SCRIPT_NAME to that, always, so the app can trust it. Now (by the same >person, as far as I can tell) I'm being told that the origin server should >always set SCRIPT_NAME to "", which IMO denies the reason for its >inclusion in the spec. The only time you should set SCRIPT_NAME to "" is if the application object being executed is mounted at the server root. So, this is correct behavior for a server like wsgiref's SimpleHTTPServer, which does not provide any mount points. If your server implements mount points, you should indeed set SCRIPT_NAME to appropriate path for the app being executed. As far as I can tell, the only confusion in the conversation was that Ian was trying to point out that middleware can be used to implement mount points, rather than building mount points directly into the server. That is, he was making an implementation suggestion that a server (especially one destined for stdlib inclusion) can be simpler if it just provides a single mount point at the root and leaves the routing to a middleware component -- which of course would then be reusable and interchangeable. And if that wasn't what he meant by saying the server should set SCRIPT_NAME to "", then he's wrong. ;) I don't think the *spec* is ambiguous on this, but if you can point out a bit that makes it seem uncertain, please feel free to suggest additional or alternate wording. Thanks. From jim at zope.com Tue Feb 7 15:08:53 2006 From: jim at zope.com (Jim Fulton) Date: Tue, 07 Feb 2006 09:08:53 -0500 Subject: [Web-SIG] Bowing out (was Re: A trivial template API counter-proposal) In-Reply-To: <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> References: <5.1.1.6.0.20060205124025.021885b0@mail.telecommunity.com> <5.1.1.6.0.20060204131722.02179008@mail.telecommunity.com> <5.1.1.6.0.20060205124025.021885b0@mail.telecommunity.com> <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> Message-ID: <43E8A9F5.2040807@zope.com> I'm not sure what you're talking about here. Are you talking about WSGI? Or the templating effort? I've tuned out the templating discussion. I think you and others did a fantastic job with WSGI. (Based on my experience I do think it needs more work in the future, but that's beside the point.) Despite some skepticism about the templating effort, I certainly planned to evaluate it when it settled down. For the record, I am very interested in inclusive specs and other means to collaborate. Keep up the good work! I do think that the pioneering work you are doing with setuptools is far more important for Python, so I'd be happy to see you focus on that. :) Jim Phillip J. Eby wrote: > At 08:02 PM 2/5/2006 +0000, Alan Kennedy wrote: > >>Looking at this in an MVC context, the application is responsible for >>populating the Model (user namespace), and selecting which View >>(template<->media-type) is suitable for return to the user. Templates >>should not vary media types. HTTP headers do need to be set for >>different templates/media-types. But that should be the responsibility >>of the HTTP application, not the template, which should be unaware of >>the application contect in which it is running, except for the contents >>of the Model/user-namespace. > > > As soon as you start talking about what templates should or should not do > (as opposed to what they *already* do), you've stopped writing an inclusive > spec and have wandered off into evangelizing a particular framework philosophy. > > OTOH, before I first proposed WSGI in 2003, nobody here seemed especially > interested in writing inclusive specs anyway, and I rather get the > impression they still aren't now, my "insistence" (as Ian calls it) > notwithstanding. > > What I've been trying to do with both WSGI and with this spec is to create > something that deals with the *actual* complexity and diversity of Python > web frameworks as they exist today, rather than reducing the diversity to > match whatever the currently popular paradigm is. This wasn't a popular > approach when I introduced WSGI, either, but in the case of WSGI it was > easy to point to all the previously-failed attempts at standardizing > request/response objects due to people not taking backward compatibility > into account. > > At this point it has become clear to me that even if I spent my days and > nights writing a compelling spec of what I'm proposing and then trying to > sell it to the Web SIG, it would be at best a 50/50 chance of getting > through, and in the process it appears that I'd be burning through every > bit of goodwill I might have previously possessed here. And, although I > believe that the approach currently being taken to this spec is divisive of > the community, I have to admit that since my attempts at education about > the issues hasn't been particularly successful, it would appear that > continuing to argue about it is no less divisive than what I'm arguing > against. (For that matter, it's not even clear to me any more that most of > the people on whose behalf I'm fighting would even realize yet why the > future I want would be beneficial for them.) > > I really expected more people to see the benefits of the WSGI embedding > approach, though, and although I've gotten a few private mails of support, > it isn't anywhere near the level I thought it would be. Given the intense > pressure that some parties are putting on having a spec *right* now, I > don't feel that I can reasonably deliver a competing spec without > interfering with my work and personal commitments in the next few > weeks. Since I've already been using most of my "Python community > contribution" time in the last week arguing about this, at this point it > seems the community would be better served by me devoting that time to > working on setuptools, rather than continuing to fight for a vision that > hardly anybody else believes in. And, I'd rather save whatever karma I > have left here for something with a better chance of success. > > Good luck with the spec. > > > _______________________________________________ > Web-SIG mailing list > Web-SIG at python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: http://mail.python.org/mailman/options/web-sig/jim%40zope.com -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From matt at pollenation.net Tue Feb 7 16:08:51 2006 From: matt at pollenation.net (Matt Goodall) Date: Tue, 07 Feb 2006 15:08:51 +0000 Subject: [Web-SIG] My original template API proposal In-Reply-To: References: <43E53709.4040007@colorstudy.com> <43E6642B.50006@colorstudy.com> <43E687A6.3070900@colorstudy.com> <43E68DF0.9030906@colorstudy.com> Message-ID: <43E8B803.6030406@pollenation.net> Guido van Rossum wrote: > On 2/5/06, Ian Bicking wrote: > >>I suspect most templates will buffer their output internally, unless >>somehow configured or dynamically set not to do so. > > > Why would they? Isn't that a function that the web server typically does? I've not seen anyone respond to this, and I'm not sure if it was meant entirely in the context of rendering templates but ... I don't think a web server should buffer anything except for perhaps small blocks of data on their way to a socket. It's perfectly valid to stream HTML from the application code and most browsers will have a go at rendering it as it arrives (although to be honest, it's of dubious use ;-). However, there are plenty of file formats that support streaming, progressive rendering, frames, etc. It's possible to explicitly make use of this from application code (i.e. sending images in "real-time" as GIF frames). Buffering to avoid sending any headers out of sequence is something the templating engine or (perhaps) the web framework should do. The web server should stick to shuffling bytes around the internet. - Matt -- __ / \__ Matt Goodall, Pollenation Internet Ltd \__/ \ w: http://www.pollenation.net __/ \__/ e: matt at pollenation.net / \__/ \ t: +44 (0)113 2252500 \__/ \__/ / \ Any views expressed are my own and do not necessarily \__/ reflect the views of my employer. From pje at telecommunity.com Tue Feb 7 16:20:43 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 07 Feb 2006 10:20:43 -0500 Subject: [Web-SIG] Bowing out (was Re: A trivial template API counter-proposal) In-Reply-To: <43E8A9F5.2040807@zope.com> References: <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> <5.1.1.6.0.20060205124025.021885b0@mail.telecommunity.com> <5.1.1.6.0.20060204131722.02179008@mail.telecommunity.com> <5.1.1.6.0.20060205124025.021885b0@mail.telecommunity.com> <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> At 09:08 AM 2/7/2006 -0500, Jim Fulton wrote: >I'm not sure what you're talking about here. Are you talking about >WSGI? Or the templating effort? I've tuned out the templating discussion. Just the templating effort, and that only for the time being. I just don't feel I have enough time to devote to trying to compete in that area, since my take on it is very much against the grain of the popular approach. >Despite some skepticism about the templating >effort, I certainly planned to evaluate it when it settled down. I'm not complaining about you personally tuning out; it's just that I ended up being a sole advocate for stuff I thought Zope would need in order to utilize the template standard as a basis for views, without being certain of the details or whether you (i.e. zope.com and .org) actually cared (due to you having disappeared after your initial comment). (This of course also goes for other view-based and "active page" frameworks that have similar issues, but whose architects weren't around to comment in the first place.) >I do think that the pioneering work you are doing with setuptools >is far more important for Python, so I'd be happy to see you focus on >that. :) I'd be especially interested in your thoughts on my current "API for finding plugins" posts on the distutils-sig, as I gather you're working in that area with Zope at the moment. From ianb at colorstudy.com Tue Feb 7 17:51:45 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 07 Feb 2006 10:51:45 -0600 Subject: [Web-SIG] My original template API proposal In-Reply-To: <43E8B803.6030406@pollenation.net> References: <43E53709.4040007@colorstudy.com> <43E6642B.50006@colorstudy.com> <43E687A6.3070900@colorstudy.com> <43E68DF0.9030906@colorstudy.com> <43E8B803.6030406@pollenation.net> Message-ID: <43E8D021.5020308@colorstudy.com> Matt Goodall wrote: > Guido van Rossum wrote: > >>On 2/5/06, Ian Bicking wrote: >> >> >>>I suspect most templates will buffer their output internally, unless >>>somehow configured or dynamically set not to do so. >> >> >>Why would they? Isn't that a function that the web server typically does? > > > I've not seen anyone respond to this, and I'm not sure if it was meant > entirely in the context of rendering templates but ... Lots of parallel threads... but yes, I was only speaking about templates. In practice I believe ZPT does not stream output, and Cheetah can but does not by default. Kid does not either, as it constructs a complete ElementTree representation and does modifications of that then serializes (though maybe the serialization is streamed). > I don't think a web server should buffer anything except for perhaps > small blocks of data on their way to a socket. Certainly, and WSGI supports that well. The templating spec (any of the forms) can support that as well, I just expect to see most results as [complete_body]. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From srichter at cosmos.phy.tufts.edu Tue Feb 7 18:02:40 2006 From: srichter at cosmos.phy.tufts.edu (Stephan Richter) Date: Tue, 7 Feb 2006 12:02:40 -0500 Subject: [Web-SIG] Bowing out (was Re: A trivial template API counter-proposal) In-Reply-To: <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> References: <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> Message-ID: <200602071202.40777.srichter@cosmos.phy.tufts.edu> On Tuesday 07 February 2006 10:20, Phillip J. Eby wrote: > >Despite some skepticism about the templating > >effort, I certainly planned to evaluate it when it settled down. > > I'm not complaining about you personally tuning out; it's just that I ended > up being a sole advocate for stuff I thought Zope would need in order to > utilize the template standard as a basis for views, without being certain > of the details or whether you (i.e. zope.com and .org) actually cared (due > to you having disappeared after your initial comment). ?(This of course > also goes for other view-based and "active page" frameworks that have > similar issues, but whose architects weren't around to comment in the first > place.) I phased out as well and decided to comment on a draft. The amount of E-mails just overwhelmed me. But I agree as well, that the egg work is more important. :-) BTW, did we reach a conclusion on the user logging issue. We really, really need to solve that somehow. Anything you can come up with is fine by me; I'll trust you do the right thing. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training From ianb at colorstudy.com Tue Feb 7 18:12:04 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 07 Feb 2006 11:12:04 -0600 Subject: [Web-SIG] Bowing out (was Re: A trivial template API counter-proposal) In-Reply-To: <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> References: <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> <5.1.1.6.0.20060205124025.021885b0@mail.telecommunity.com> <5.1.1.6.0.20060204131722.02179008@mail.telecommunity.com> <5.1.1.6.0.20060205124025.021885b0@mail.telecommunity.com> <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> Message-ID: <43E8D4E4.3060104@colorstudy.com> Phillip J. Eby wrote: >>Despite some skepticism about the templating >>effort, I certainly planned to evaluate it when it settled down. > > > I'm not complaining about you personally tuning out; it's just that I ended > up being a sole advocate for stuff I thought Zope would need in order to > utilize the template standard as a basis for views, without being certain > of the details or whether you (i.e. zope.com and .org) actually cared (due > to you having disappeared after your initial comment). (This of course > also goes for other view-based and "active page" frameworks that have > similar issues, but whose architects weren't around to comment in the first > place.) Maybe the reason those voices are missing -- I now realize -- is that there aren't many "active page" frameworks left. Spyce was, but since then I believe a more traditional controller-driven API has been added -- I don't know if the "active" part has been extracted from Spyce or not, but at least that portion is optional (and if you are embedding Spyce into another framework it is likely you won't want that part). Cheetah supports both models, but the active page model has long been discouraged. Webware's PSP is unmaintained. I suppose mod_python's PSP is similar as well, but I never got the impression anyone was championing that for anything. Both Myghty and Django have active-page-like ways of working with them, but because they have both techniques available there's still a fairly solid conceptual separation of the framework/controller and the template language. That's not to say active pages shouldn't be supported, but I think most of the people here see templates as generic content-builders as more fundamental than templates as web apps. Active pages then are an application of templates, not to be confused with the template itself. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From pje at telecommunity.com Tue Feb 7 18:28:09 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 07 Feb 2006 12:28:09 -0500 Subject: [Web-SIG] Logging the authenticated user (was Re: Bowing out) In-Reply-To: <200602071202.40777.srichter@cosmos.phy.tufts.edu> References: <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060207120658.038909e8@mail.telecommunity.com> At 12:02 PM 2/7/2006 -0500, Stephan Richter wrote: >BTW, did we reach a conclusion on the user logging issue. We really, really >need to solve that somehow. Anything you can come up with is fine by me; I'll >trust you do the right thing. As I mentioned, you can do this right now by offering and documenting an extension API. You don't need a modification to the PEP to do this; it's sufficient for you to get together with the authors of the web servers you care about and hash out a working agreement. To get it into the 'wsgi.*' namespace and included in the PEP as a "standard" optional extension, the bar is a bit higher, and it does require coming to some resolution about the remaining issues. If I recall correctly, the question of which side (server or app) should be responsible for encoding issues was the remaining open item, aside from the name that should be used for the extension API. Also IIRC, Ian Bicking had proposed that the server should be responsible for making sure that what it writes to the log is in a valid format for that log. I'm inclined to agree, and think we should use a name like 'wsgi.set_authenticated_user_name', with the API allowed to be called multiple times. For future versions of WSGI, I'd prefer to send this kind of thing in response headers, since that would preserve functional composability better. Indeed, I'd prefer to do it that way now, but too many people have objected to this as a security risk. OTOH, an easy way to fix that would be for us to define a 'wsgi.response_filtering' key that means the server will drop any 'X-Internal-Foo' headers. You can then simply avoid generating any internal communication headers unless the server promises to fix them. And, as a side effect, this would close the encoding issue by definition; the format of response headers is already dictated by the WSGI and HTTP specs. So, if there are no objections, I propose that we: * Add an optional 'wsgi.response_filtering' key to the spec. If its value is present and true, the server promises to prevent 'X-Internal-*' headers from being transmitted. * Add an optional 'X-Internal-WSGI-Authenticated-User' header to the spec, that indicates the authenticated user name. This should only be inserted into the response headers if 'wsgi.response_filtering' is in effect. * Require that any user-defined X-Internal headers include a product name, e.g. 'X-Internal-Zope-Foo', to avoid conflict with WSGI-defined or other products' user-defined headers. This would all be placed under a new section entitled "Internal Response Headers" and defined as an optional extension. Any thoughts? From pje at telecommunity.com Tue Feb 7 18:32:31 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 07 Feb 2006 12:32:31 -0500 Subject: [Web-SIG] Bowing out (was Re: A trivial template API counter-proposal) In-Reply-To: <43E8D4E4.3060104@colorstudy.com> References: <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> <5.1.1.6.0.20060205124025.021885b0@mail.telecommunity.com> <5.1.1.6.0.20060204131722.02179008@mail.telecommunity.com> <5.1.1.6.0.20060205124025.021885b0@mail.telecommunity.com> <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060207122925.038e52b8@mail.telecommunity.com> At 11:12 AM 2/7/2006 -0600, Ian Bicking wrote: >Maybe the reason those voices are missing -- I now realize -- is that >there aren't many "active page" frameworks left. All the more reason to encourage an approach that allows migrating existing code written using those systems! :) I think it's also the case that active page systems are a useful stepping stone for people seeking to "move up" from PHP and ASP while learning Python, but now you're going to get me doing advocacy again. :) From srichter at cosmos.phy.tufts.edu Tue Feb 7 18:38:39 2006 From: srichter at cosmos.phy.tufts.edu (Stephan Richter) Date: Tue, 7 Feb 2006 12:38:39 -0500 Subject: [Web-SIG] Logging the authenticated user (was Re: Bowing out) In-Reply-To: <5.1.1.6.0.20060207120658.038909e8@mail.telecommunity.com> References: <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> <5.1.1.6.0.20060207120658.038909e8@mail.telecommunity.com> Message-ID: <200602071238.39611.srichter@cosmos.phy.tufts.edu> On Tuesday 07 February 2006 12:28, Phillip J. Eby wrote: > * Add an optional 'wsgi.response_filtering' key to the spec. ?If its value > is present and true, the server promises to prevent 'X-Internal-*' headers > from being transmitted. > > * Add an optional 'X-Internal-WSGI-Authenticated-User' header to the spec, > that indicates the authenticated user name. ?This should only be inserted > into the response headers if 'wsgi.response_filtering' is in effect. > > * Require that any user-defined X-Internal headers include a product name, > e.g. 'X-Internal-Zope-Foo', to avoid conflict with WSGI-defined or other > products' user-defined headers. > > This would all be placed under a new section entitled "Internal Response > Headers" and defined as an optional extension. > > Any thoughts? This sounds really good! Thanks for the great summary and suggestions. As far as I can tell it solves all of our use cases and addresses our security concerns; i.e. not sending the username to the client. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training From jonathan at carnageblender.com Tue Feb 7 18:47:04 2006 From: jonathan at carnageblender.com (Jonathan Ellis) Date: Tue, 07 Feb 2006 09:47:04 -0800 Subject: [Web-SIG] Bowing out (was Re: A trivial template API counter-proposal) In-Reply-To: <43E8D4E4.3060104@colorstudy.com> References: <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> <5.1.1.6.0.20060205124025.021885b0@mail.telecommunity.com> <5.1.1.6.0.20060204131722.02179008@mail.telecommunity.com> <5.1.1.6.0.20060205124025.021885b0@mail.telecommunity.com> <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> <43E8D4E4.3060104@colorstudy.com> Message-ID: <1139334424.14874.253767792@webmail.messagingengine.com> On Tue, 07 Feb 2006 11:12:04 -0600, "Ian Bicking" said: > Maybe the reason those voices are missing -- I now realize -- is that > there aren't many "active page" frameworks left. Spyce was, but since > then I believe a more traditional controller-driven API has been added Yes, but you can still write plain active-page stuff if you wanted to. > -- I don't know if the "active" part has been extracted from Spyce or > not, but at least that portion is optional (and if you are embedding > Spyce into another framework it is likely you won't want that part). I'm not exactly sure what you have in mind here, but the active-page- ness of Spyce isn't really optional the way I think of it. Views still correspond one-to-one with .spy active pages. The new part is it's easy to put the controller in a separate .py file. (Spyce has always been able to import vanilly .py code, so separating out the model isn't new.) -Jonathan -- C++ is history repeated as tragedy. Java is history repeated as farce. --Scott McKay From smulloni at smullyan.org Tue Feb 7 18:32:18 2006 From: smulloni at smullyan.org (Jacob Smullyan) Date: Tue, 7 Feb 2006 12:32:18 -0500 Subject: [Web-SIG] Bowing out (was Re: A trivial template API counter-proposal) In-Reply-To: <43E8D4E4.3060104@colorstudy.com> References: <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> <5.1.1.6.0.20060205124025.021885b0@mail.telecommunity.com> <5.1.1.6.0.20060204131722.02179008@mail.telecommunity.com> <5.1.1.6.0.20060205124025.021885b0@mail.telecommunity.com> <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> <43E8D4E4.3060104@colorstudy.com> Message-ID: <20060207173218.GA4291@smullyan.org> On Tue, Feb 07, 2006 at 11:12:04AM -0600, Ian Bicking wrote: > Maybe the reason those voices are missing -- I now realize -- is that > there aren't many "active page" frameworks left. Yup. SkunkWeb too, which like Myghty also started out as a port of Mason, now has a controller framework, with active pages as a fallback. But for that purpose -- for little content-oriented crusts left out of the application -- active pages sure come in handy, so giving them some extra consideration here is probably still worthwhile. -- Jacob Smullyan -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/web-sig/attachments/20060207/cea52be6/attachment.pgp From ianb at colorstudy.com Tue Feb 7 18:50:49 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 07 Feb 2006 11:50:49 -0600 Subject: [Web-SIG] Logging the authenticated user (was Re: Bowing out) In-Reply-To: <5.1.1.6.0.20060207120658.038909e8@mail.telecommunity.com> References: <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> <5.1.1.6.0.20060207120658.038909e8@mail.telecommunity.com> Message-ID: <43E8DDF9.7000902@colorstudy.com> Phillip J. Eby wrote: > So, if there are no objections, I propose that we: > > * Add an optional 'wsgi.response_filtering' key to the spec. If its value > is present and true, the server promises to prevent 'X-Internal-*' headers > from being transmitted. > > * Add an optional 'X-Internal-WSGI-Authenticated-User' header to the spec, > that indicates the authenticated user name. This should only be inserted > into the response headers if 'wsgi.response_filtering' is in effect. > > * Require that any user-defined X-Internal headers include a product name, > e.g. 'X-Internal-Zope-Foo', to avoid conflict with WSGI-defined or other > products' user-defined headers. > > This would all be placed under a new section entitled "Internal Response > Headers" and defined as an optional extension. > > Any thoughts? Sounds good to me, and wsgi.response_filtering seems to address the backward compatibility well. It would be easy, for instance, to apply the filtering in the logging middleware if the server was not already filtering the response, and set the key to represent that the filtering was in place. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From pje at telecommunity.com Tue Feb 7 19:23:56 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 07 Feb 2006 13:23:56 -0500 Subject: [Web-SIG] Logging the authenticated user (was Re: Bowing out) In-Reply-To: <43E8DDF9.7000902@colorstudy.com> References: <5.1.1.6.0.20060207120658.038909e8@mail.telecommunity.com> <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> <5.1.1.6.0.20060207120658.038909e8@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060207131755.0397efe8@mail.telecommunity.com> At 11:50 AM 2/7/2006 -0600, Ian Bicking wrote: >Phillip J. Eby wrote: >>So, if there are no objections, I propose that we: >>* Add an optional 'wsgi.response_filtering' key to the spec. If its >>value is present and true, the server promises to prevent 'X-Internal-*' >>headers from being transmitted. >>* Add an optional 'X-Internal-WSGI-Authenticated-User' header to the >>spec, that indicates the authenticated user name. This should only be >>inserted into the response headers if 'wsgi.response_filtering' is in effect. >>* Require that any user-defined X-Internal headers include a product >>name, e.g. 'X-Internal-Zope-Foo', to avoid conflict with WSGI-defined or >>other products' user-defined headers. >>This would all be placed under a new section entitled "Internal Response >>Headers" and defined as an optional extension. >>Any thoughts? > >Sounds good to me, and wsgi.response_filtering seems to address the >backward compatibility well. It would be easy, for instance, to apply the >filtering in the logging middleware if the server was not already >filtering the response, and set the key to represent that the filtering >was in place. This brings up an interesting point, which is to what extent middleware can or should filter internal headers it doesn't understand, and what kinds of functionality are acceptable to expose via response headers. That issue deserves some more thought before the spec addition is made, although for this particular application I'm thinking that folks are probably safe prototyping based on just the info above. The only issue remaining is whether there are any objecting parties. I believe that Clark Evans eventually agreed that response headers were the best long-term way to address middleware composition issues in upward communication like this, but we should leave the proposed extension open to objections for a few days in any case. And of course the issue of what's acceptable to expose via headers and what's acceptable for middleware to ignore or pass through is very much open for discussion, since it needs to be resolved before the actual spec can be written. From fumanchu at amor.org Tue Feb 7 21:18:38 2006 From: fumanchu at amor.org (Robert Brewer) Date: Tue, 7 Feb 2006 12:18:38 -0800 Subject: [Web-SIG] Who is responsible for SCRIPT_NAME? Message-ID: <6949EC6CD39F97498A57E0FA55295B21017192C0@ex9.hostedexchange.local> Phillip J. Eby wrote: > The only time you should set SCRIPT_NAME to "" is if the > application object being executed is mounted at the server > root. So, this is correct behavior for a server like > wsgiref's SimpleHTTPServer, which does not provide any > mount points. If your server implements mount points, you > should indeed set SCRIPT_NAME to appropriate path for > the app being executed. Thanks; that makes sense, and is how I've always interpreted the spec. > As far as I can tell, the only confusion in the conversation > was that Ian was trying to point out that middleware can be > used to implement mount points, rather than building mount > points directly into the server. That is, he was making an > implementation suggestion that a server (especially one > destined for stdlib inclusion) can be simpler if it just > provides a single mount point at the root and leaves the > routing to a middleware component -- which of course would > then be reusable and interchangeable. And if that wasn't > what he meant by saying the server should set SCRIPT_NAME > to "", then he's wrong. ;) Here's my continuing issue: if we allow middleware to implement mount points, then your first statement (above) needs to be modified to: The only time [the origin server] should set SCRIPT_NAME to "" is if the application object being executed is mounted at the server root OR you use a dumb server and instead rely on middleware to do the routing. > I don't think the *spec* is ambiguous on this, but if you can > point out a bit that makes it seem uncertain, please feel > free to suggest additional or alternate wording. Thanks. SCRIPT_NAME The initial portion of the request URL's "path" that corresponds to the application object, so that the application knows its virtual "location". This may be an empty string, if the application corresponds to the "root" of the server. This section is becoming increasingly ambiguous to me. If you're using middleware to do the routing, I don't know whether "the application" refers to the middleware or to the end app which the middleware wraps. 1. If "the application" refers to the end app or framework, then I think many people (me included) are expecting that whatever SCRIPT_NAME is set by the origin server should pass through unchanged to the end app. If true, then it seems to me we're saying that this part of the spec simply doesn't apply to the origin-server-to-routing-middleware part of the conversation, because the origin server doesn't know whether the application corresponds to the "root" of the server or not (only the middleware knows). So the first half of the WSGI conversation (server-to-middleware) doesn't strictly follow the spec. I don't mind that, but the spec could be clearer by saying SHOULD or MUST in the above block, and (if it's SHOULD instead of MUST) detailing the "middleware routing scenario", in which the origin server isn't responsible to set SCRIPT_NAME to the end application's virtual location if middleware is going to do that instead. 2. "The application" could refer to the routing-middleware (because it implements both the application side and the server side of the spec). In other words, to the origin server, the routing-middleware's callable is the only WSGI application to care about, and it's always "mounted at root" so SCRIPT_NAME can always be "". I haven't adopted this interpretation, because in my mind, middleware isn't mounted anywhere--it's a filter for the end app(s). So which one is the correct interpretation, 1 or 2? Robert Brewer System Architect Amor Ministries fumanchu at amor.org From pje at telecommunity.com Tue Feb 7 21:35:38 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 07 Feb 2006 15:35:38 -0500 Subject: [Web-SIG] Who is responsible for SCRIPT_NAME? In-Reply-To: <6949EC6CD39F97498A57E0FA55295B21017192C0@ex9.hostedexchang e.local> Message-ID: <5.1.1.6.0.20060207152531.021594c8@mail.telecommunity.com> At 12:18 PM 2/7/2006 -0800, Robert Brewer wrote: >Here's my continuing issue: if we allow middleware to implement mount >points, then your first statement (above) needs to be modified to: > > The only time [the origin server] should set SCRIPT_NAME > to "" is if the application object being executed is > mounted at the server root OR you use a dumb server > and instead rely on middleware to do the routing. I don't follow you here. My original statement was precisely correct; SCRIPT_NAME should be set to "" if and only if the application to be invoked is mounted at the origin server's root. That is, if the URL path that addresses the application is '/'. This statement applies to middleware as well as to the origin server, as the middleware is obliged to set SCRIPT_NAME correctly for any application that *it* calls. Nor is this a distinction without a difference; it's possible for a middleware component to exist only to filter or handle certain conditions, but be a passthrough for others, in which case it might leave an empty SCRIPT_NAME alone. > > I don't think the *spec* is ambiguous on this, but if you can > > point out a bit that makes it seem uncertain, please feel > > free to suggest additional or alternate wording. Thanks. > >SCRIPT_NAME > The initial portion of the request URL's "path" that corresponds to >the application object, so that the application knows its virtual >"location". This may be an empty string, if the application corresponds >to the "root" of the server. > >This section is becoming increasingly ambiguous to me. If you're using >middleware to do the routing, I don't know whether "the application" >refers to the middleware or to the end app which the middleware wraps. "The application" always refers to the application that is *receiving* the WSGI environment. >1. If "the application" refers to the end app or framework >2. "The application" could refer to the routing-middleware (because it "Application" simply refers to any code that is reading an 'environ' that it received from *its* server. So *both* of these are "applications", and each should see the correct value of SCRIPT_NAME. The simple explanation is that SCRIPT_NAME must always reflect the path that was followed to get to the *current* application. >In other words, to the origin server, the routing-middleware's callable >is the only WSGI application to care about, and it's always "mounted at >root" so SCRIPT_NAME can always be "". That's correct. > I haven't adopted this >interpretation, because in my mind, middleware isn't mounted >anywhere--it's a filter for the end app(s). As far as any server knows, a given piece of middleware *is* an application. Ergo, it has a location at which it must be considered "mounted" in the server. There is really very little that's special about middleware; it's simply an app that's also a server. There are some additional rules to make sure that middleware composition behaves sanely, but apart from those extra rules, middleware has all of the freedoms (and restrictions) of both a server and an application. From ianb at colorstudy.com Tue Feb 7 21:39:06 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 07 Feb 2006 14:39:06 -0600 Subject: [Web-SIG] Who is responsible for SCRIPT_NAME? In-Reply-To: <6949EC6CD39F97498A57E0FA55295B21017192C0@ex9.hostedexchange.local> References: <6949EC6CD39F97498A57E0FA55295B21017192C0@ex9.hostedexchange.local> Message-ID: <43E9056A.50203@colorstudy.com> Robert Brewer wrote: > Phillip J. Eby wrote: > >>The only time you should set SCRIPT_NAME to "" is if the >>application object being executed is mounted at the server >>root. So, this is correct behavior for a server like >>wsgiref's SimpleHTTPServer, which does not provide any >>mount points. If your server implements mount points, you >>should indeed set SCRIPT_NAME to appropriate path for >>the app being executed. > > > Thanks; that makes sense, and is how I've always interpreted the spec. > > >>As far as I can tell, the only confusion in the conversation >>was that Ian was trying to point out that middleware can be >>used to implement mount points, rather than building mount >>points directly into the server. That is, he was making an >>implementation suggestion that a server (especially one >>destined for stdlib inclusion) can be simpler if it just >>provides a single mount point at the root and leaves the >>routing to a middleware component -- which of course would >>then be reusable and interchangeable. And if that wasn't >>what he meant by saying the server should set SCRIPT_NAME >>to "", then he's wrong. ;) > > > Here's my continuing issue: if we allow middleware to implement mount > points, then your first statement (above) needs to be modified to: > > The only time [the origin server] should set SCRIPT_NAME > to "" is if the application object being executed is > mounted at the server root OR you use a dumb server > and instead rely on middleware to do the routing. I can mount a routing middleware anywhere. So I can have /blog routed to my WSGI application, and then have /blog/admin routed one place and /blog/ routed another place. Using the dispatching example I gave earlier, that would be {'/admin': admin_app, '/': other_app}, as it only routes based on the PATH_INFO. (It can be okay to route based on SCRIPT_NAME too, but you are more likely to encounter confusing situations in that case) Of course, it is also always the prerogative of the developer to fix something that is broken, and some servers do pass an incorrect SCRIPT_NAME, and some protocols have no way to express SCRIPT_NAME. It is best to fix this as early as possible in the request chain, and then assume it is correct from there on. >>I don't think the *spec* is ambiguous on this, but if you can >>point out a bit that makes it seem uncertain, please feel >>free to suggest additional or alternate wording. Thanks. > > > SCRIPT_NAME > The initial portion of the request URL's "path" that corresponds to > the application object, so that the application knows its virtual > "location". This may be an empty string, if the application corresponds > to the "root" of the server. > > This section is becoming increasingly ambiguous to me. If you're using > middleware to do the routing, I don't know whether "the application" > refers to the middleware or to the end app which the middleware wraps. > > 1. If "the application" refers to the end app or framework, then I think > many people (me included) are expecting that whatever SCRIPT_NAME is set > by the origin server should pass through unchanged to the end app. If > true, then it seems to me we're saying that this part of the spec simply > doesn't apply to the origin-server-to-routing-middleware part of the > conversation, because the origin server doesn't know whether the > application corresponds to the "root" of the server or not (only the > middleware knows). So the first half of the WSGI conversation > (server-to-middleware) doesn't strictly follow the spec. In the case of routing middleware there are two WSGI requests happening. The first is the server talking to the middleware (the middleware is the application in this case), and the second is the middleware talking to the application (the middleware is a server in this case). "server" always means "the thing that is calling the application", and "application" always means "the thing being called by the server". The two names refer to a relationship during a particular call sequence, nothing more. > I don't mind that, but the spec could be clearer by saying SHOULD or > MUST in the above block, and (if it's SHOULD instead of MUST) detailing > the "middleware routing scenario", in which the origin server isn't > responsible to set SCRIPT_NAME to the end application's virtual location > if middleware is going to do that instead. > > 2. "The application" could refer to the routing-middleware (because it > implements both the application side and the server side of the spec). > In other words, to the origin server, the routing-middleware's callable > is the only WSGI application to care about, and it's always "mounted at > root" so SCRIPT_NAME can always be "". I haven't adopted this > interpretation, because in my mind, middleware isn't mounted > anywhere--it's a filter for the end app(s). The middleware is an application -- the server doesn't and can't know the difference between middleware and a terminal application. The distinction is fuzzy anyway, as it is fairly easy to find yourself writing something that plays both roles (application and middleware). In turn, an application can't know if the thing calling it is a "server" that is actually talking to a socket, or some middleware that is passing along a request. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From foom at fuhm.net Tue Feb 7 22:29:01 2006 From: foom at fuhm.net (James Y Knight) Date: Tue, 7 Feb 2006 16:29:01 -0500 Subject: [Web-SIG] Logging the authenticated user (was Re: Bowing out) In-Reply-To: <5.1.1.6.0.20060207120658.038909e8@mail.telecommunity.com> References: <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> <5.1.1.6.0.20060207120658.038909e8@mail.telecommunity.com> Message-ID: On Feb 7, 2006, at 12:28 PM, Phillip J. Eby wrote: > * Add an optional 'wsgi.response_filtering' key to the spec. If > its value > is present and true, the server promises to prevent 'X-Internal-*' > headers > from being transmitted. > > * Add an optional 'X-Internal-WSGI-Authenticated-User' header to > the spec, > that indicates the authenticated user name. This should only be > inserted > into the response headers if 'wsgi.response_filtering' is in effect. > > * Require that any user-defined X-Internal headers include a > product name, > e.g. 'X-Internal-Zope-Foo', to avoid conflict with WSGI-defined or > other > products' user-defined headers. > > This would all be placed under a new section entitled "Internal > Response > Headers" and defined as an optional extension. I like it. James From fumanchu at amor.org Tue Feb 7 22:54:47 2006 From: fumanchu at amor.org (Robert Brewer) Date: Tue, 7 Feb 2006 13:54:47 -0800 Subject: [Web-SIG] Who is responsible for SCRIPT_NAME? Message-ID: <6949EC6CD39F97498A57E0FA55295B210171949A@ex9.hostedexchange.local> Phillip J. Eby wrote: > > 1. If "the application" refers to the end app or framework > > 2. "The application" could refer to the routing-middleware > > "Application" simply refers to any code that is reading an > 'environ' that it received from *its* server. So *both* > of these are "applications", and each should see the correct > value of SCRIPT_NAME. OK. Thanks for the clarification. Based on this, I can now go modify CherryPy's WSGI server, so that it can serve multiple WSGI applications in the same process, and set a different SCRIPT_NAME for each one, if requested. Robert Brewer System Architect Amor Ministries fumanchu at amor.org From cce at clarkevans.com Wed Feb 8 01:30:55 2006 From: cce at clarkevans.com (Clark C. Evans) Date: Tue, 7 Feb 2006 19:30:55 -0500 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <43E7B94F.6030601@colorstudy.com> References: <06Feb6.112517pst."58633"@synergy1.parc.xerox.com> <43E7B94F.6030601@colorstudy.com> Message-ID: <20060208003055.GA35427@prometheusresearch.com> I'd very much like to see a _usable_ WSGI server included in the standard Python library: I don't care how it is done. It should have a few requirements: - it should obviously implement all of WSGI that is required by a server implementation; - it should be stand-alone, only dependent upon Python's built-in BaseHTTPServer (not SimpleHTTPServer, for obvious reasons); - it should _not_ have any functionality beyond the minimums required by WSGI: specificially, it should not handle multiple WSGI applications (routing), nor should it include file downloading; - it should be implemented as a Mix-In so that you can use it /w any derivitive of BaseHTTPServer (for example one using SSL); - it should be accompanied by a SSL enabled server built upon BaseHTTPServer (and perhaps fix a few bugs in BaseHTTPServer); - it should fully support HTTP/1.1, including Chunked-Encoding, 100-Continue, and degrade to HTTP/1.0 if Content-Length is not provided in a Client's request; - it should take into account design "insights" found in various currently competing WSGI servers; and finally, - it *must* be heavily tested -- ideally, already in production setting to identify issues that can be missed. I wrote paste.httpserver not beacuse I needed to kill a week of my life, I did it beacuse most of the servers out there didn't do this sort of stuff and beacause I got tired of dealing with broken code. You're welcome to use paste.httpserver, or not use it. I'd suggest looking at all possible servers out there, and stealing the best peices/ideas from each one. I will gladly lead such an effort if it is deemed appropriate. Best, Clark From fumanchu at amor.org Wed Feb 8 01:38:02 2006 From: fumanchu at amor.org (Robert Brewer) Date: Tue, 7 Feb 2006 16:38:02 -0800 Subject: [Web-SIG] WSGI in standard library Message-ID: <6949EC6CD39F97498A57E0FA55295B21017197FE@ex9.hostedexchange.local> Clark C. Evans wrote: > I'd very much like to see a _usable_ WSGI server included in the > standard Python library: I don't care how it is done. > > It should have a few requirements: > ... > - it should _not_ have any functionality beyond the > minimums required by WSGI: specificially, it should > not handle multiple WSGI applications (routing) Having just added routing to CherryPy's WSGI server in less than a dozen lines of code, I'd be interested to know why you feel routing shouldn't be included. Robert Brewer System Architect Amor Ministries fumanchu at amor.org From cce at clarkevans.com Wed Feb 8 02:22:33 2006 From: cce at clarkevans.com (Clark C. Evans) Date: Tue, 7 Feb 2006 20:22:33 -0500 Subject: [Web-SIG] Logging the authenticated user (was Re: Bowing out) In-Reply-To: <5.1.1.6.0.20060207120658.038909e8@mail.telecommunity.com> References: <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> <5.1.1.6.0.20060207120658.038909e8@mail.telecommunity.com> Message-ID: <20060208012233.GA35806@prometheusresearch.com> Phillip, You are correct, I'm convinced that response headers are the place to include this sort of inter-middleware communication. On Tue, Feb 07, 2006 at 12:28:09PM -0500, Phillip J. Eby wrote: | * Add an optional 'wsgi.response_filtering' key to the spec. If its value | is present and true, the server promises to prevent 'X-Internal-*' headers | from being transmitted. I absolutely love wsgi.response_filtering, only I feel that it should be a *mutable* listing of the headers that the server should strip. If the list is not present, then response filtering is not available. In particular, I think that requring 'X-Internal-' is not quite explicit enough and, at the same time, too limiting. | * Add an optional 'X-Internal-WSGI-Authenticated-User' header to the spec, | that indicates the authenticated user name. This should only be inserted | into the response headers if 'wsgi.response_filtering' is in effect. | | * Require that any user-defined X-Internal headers include a product name, | e.g. 'X-Internal-Zope-Foo', to avoid conflict with WSGI-defined or other | products' user-defined headers. | | This would all be placed under a new section entitled "Internal Response | Headers" and defined as an optional extension. Sure; these are good suggestions. Best, Clark From cce at clarkevans.com Wed Feb 8 02:27:10 2006 From: cce at clarkevans.com (Clark C. Evans) Date: Tue, 7 Feb 2006 20:27:10 -0500 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <6949EC6CD39F97498A57E0FA55295B21017197FE@ex9.hostedexchange.local> References: <6949EC6CD39F97498A57E0FA55295B21017197FE@ex9.hostedexchange.local> Message-ID: <20060208012710.GB35806@prometheusresearch.com> On Tue, Feb 07, 2006 at 04:38:02PM -0800, Robert Brewer wrote: | > - it should _not_ have any functionality beyond the | > minimums required by WSGI: specificially, it should | > not handle multiple WSGI applications (routing) | | Having just added routing to CherryPy's WSGI server in less than a dozen | lines of code, I'd be interested to know why you feel routing shouldn't | be included. Routing, in particular, can be done just as easily in a WSGI middleware component where alternative approaches can be taken without bias. In my opinion, just beacuse something can have a feature doesn't mean it should: too many levers make an interface complicated. Kind Regards, Clark From pje at telecommunity.com Wed Feb 8 03:20:38 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 07 Feb 2006 21:20:38 -0500 Subject: [Web-SIG] Logging the authenticated user (was Re: Bowing out) In-Reply-To: <20060208012233.GA35806@prometheusresearch.com> References: <5.1.1.6.0.20060207120658.038909e8@mail.telecommunity.com> <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> <5.1.1.6.0.20060207120658.038909e8@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060207211958.0215d288@mail.telecommunity.com> At 08:22 PM 2/7/2006 -0500, Clark C. Evans wrote: >I absolutely love wsgi.response_filtering, only I feel that it should be >a *mutable* listing of the headers that the server should strip. If >the list is not present, then response filtering is not available. In >particular, I think that requring 'X-Internal-' is not quite explicit >enough and, at the same time, too limiting. I don't understand what you want to do with this. Can you be more specific about what's limiting and how, not to mention what you expect to do with this mutable list? Thanks. From cce at clarkevans.com Wed Feb 8 16:47:26 2006 From: cce at clarkevans.com (Clark C. Evans) Date: Wed, 8 Feb 2006 10:47:26 -0500 Subject: [Web-SIG] Logging the authenticated user (was Re: Bowing out) In-Reply-To: <5.1.1.6.0.20060207211958.0215d288@mail.telecommunity.com> References: <5.1.1.6.0.20060207120658.038909e8@mail.telecommunity.com> <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> <5.1.1.6.0.20060207120658.038909e8@mail.telecommunity.com> <5.1.1.6.0.20060207211958.0215d288@mail.telecommunity.com> Message-ID: <20060208154726.GA43205@prometheusresearch.com> On Tue, Feb 07, 2006 at 09:20:38PM -0500, Phillip J. Eby wrote: | At 08:22 PM 2/7/2006 -0500, Clark C. Evans wrote: | >I absolutely love wsgi.response_filtering, only I feel that it should be | >a *mutable* listing of the headers that the server should strip. If | >the list is not present, then response filtering is not available. In | >particular, I think that requring 'X-Internal-' is not quite explicit | >enough and, at the same time, too limiting. | | I don't understand what you want to do with this. Can you be more | specific about what's limiting and how, not to mention what you expect | to do with this mutable list? Thanks. | I think it is unnecessarly limiting since you might want the server to filter (not send to the client) other headers, such as something like the Meter header defined in RFC 2227. This seems like a generally useful feature; I don't see the need to restrict it to only headers starting with 'X-Internal-'. At the same time, explicitly listing the headers you wish to filter isn't that much of a burden -- and actually helps document (paste.lint could, for example, issue warnings of all 'X-Internal-' headers that arn't listed for filtering; this could be used to catch spelling errors). Kind Regards, Clark From pje at telecommunity.com Wed Feb 8 18:38:22 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 08 Feb 2006 12:38:22 -0500 Subject: [Web-SIG] Logging the authenticated user (was Re: Bowing out) In-Reply-To: <20060208154726.GA43205@prometheusresearch.com> References: <5.1.1.6.0.20060207211958.0215d288@mail.telecommunity.com> <5.1.1.6.0.20060207120658.038909e8@mail.telecommunity.com> <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> <5.1.1.6.0.20060207101303.02155748@mail.telecommunity.com> <5.1.1.6.0.20060207120658.038909e8@mail.telecommunity.com> <5.1.1.6.0.20060207211958.0215d288@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060208123242.02275630@mail.telecommunity.com> At 10:47 AM 2/8/2006 -0500, Clark C. Evans wrote: >On Tue, Feb 07, 2006 at 09:20:38PM -0500, Phillip J. Eby wrote: >| At 08:22 PM 2/7/2006 -0500, Clark C. Evans wrote: >| >I absolutely love wsgi.response_filtering, only I feel that it should be >| >a *mutable* listing of the headers that the server should strip. If >| >the list is not present, then response filtering is not available. In >| >particular, I think that requring 'X-Internal-' is not quite explicit >| >enough and, at the same time, too limiting. >| >| I don't understand what you want to do with this. Can you be more >| specific about what's limiting and how, not to mention what you expect >| to do with this mutable list? Thanks. >| > >I think it is unnecessarly limiting since you might want the server to >filter (not send to the client) other headers, such as something like >the Meter header defined in RFC 2227. This seems like a generally >useful feature; I don't see the need to restrict it to only headers >starting with 'X-Internal-'. > >At the same time, explicitly listing the headers you wish to filter >isn't that much of a burden -- and actually helps document (paste.lint >could, for example, issue warnings of all 'X-Internal-' headers that >arn't listed for filtering; this could be used to catch spelling errors). My concerns about this are that it introduces unnecessary coupling and doesn't reflect the intended use case, which is for allowing internal communication between WSGI servers and applications. These are all internal headers, and I don't see how or why 'Meter' would be used with this. It also seems to me that the spellcheck use case would be better handled by defining a 'wsgi.known_internal_headers' provided by the server to the application, so that the application can determine whether a particular piece of functionality is supported. In other words, I still don't see the purpose of having a mutable; the environ key is for the server to communicate to the app, not the other way around. (Which is done by the headers themselves.) From pywebsig at xhaus.com Sun Feb 12 12:39:58 2006 From: pywebsig at xhaus.com (Alan Kennedy) Date: Sun, 12 Feb 2006 11:39:58 +0000 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <06Feb6.112517pst."58633"@synergy1.parc.xerox.com> References: <06Feb6.112517pst."58633"@synergy1.parc.xerox.com> Message-ID: <43EF1E8E.3080804@xhaus.com> [Alan Kennedy] >>Instead, I think the right approach is to continue with the existing >>approach: put the most basic possible WSGI server in the standard >>library, for educational purposes only, and a warning that it shouldn't >>really be used for production purposes. [Bill Janssen] > I strongly disagree with this thinking. Non-production code shouldn't > go into the stdlib; instead, Alan's proposed module should go onto > some pedagogical website somewhere with appropriate tutorial > documentation. I still disagree ;-) IMO, the primary reason for not including production servers in the standard library is that servers need to be maintained much more fastidiously than the standard library, and need to be released on a timescale that is independent of python releases. Note the security hole incovered in the standard library xml-rpc lib last year. PSF-2005-001 - SimpleXMLRPCServer.py allows unrestricted traversal http://www.python.org/security/PSF-2005-001/ This particular security hole is the very reason why the Python Security response team had to be founded, and required point-releases of the entire python distribution to fix, i.e. python 2.3.5 and python 2.4.1 were released simply to fix this bug. There are two primary areas of the python distro that can result in such significant security holes. 1. Crypto libraries. Fortunately, the Timbot has been carefully watching over us, and ensuring the excellence of the python crypto libraries (as witnessed by the appearance of Ron Rivest on python-dev (!) last December: http://mail.python.org/pipermail/python-dev/2005-December/058850.html 2. Internet-exposed servers. No matter how careful developers are, it is very difficult to avoid designing security holes into such servers. Therefore, IMHO, it is wrong to include such servers into the standard distribution. Instead, production-ready servers should be independent of the standard distribution, have their own development teams, have independent release-cycles, etc, etc: think Twisted, mod_python, etc. So, I still think that only basic servers educational/playpen servers should go in the standard library, with an indication that the user should pick an openly server from outside the distro if they require to do serious server work. Maybe if there were no "production-ready" servers in the standard library, there would be no need for a "Python Security Response Team". Just my ?0,02. Regards, Alan. From grahamd at dscpl.com.au Sun Feb 12 12:58:24 2006 From: grahamd at dscpl.com.au (Graham Dumpleton) Date: Sun, 12 Feb 2006 22:58:24 +1100 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <43EF1E8E.3080804@xhaus.com> References: <06Feb6.112517pst."58633"@synergy1.parc.xerox.com> <43EF1E8E.3080804@xhaus.com> Message-ID: On 12/02/2006, at 10:39 PM, Alan Kennedy wrote: > Note the security hole incovered in the standard library xml-rpc lib > last year. > > PSF-2005-001 - SimpleXMLRPCServer.py allows unrestricted traversal > http://www.python.org/security/PSF-2005-001/ > > This particular security hole is the very reason why the Python > Security > response team had to be founded, and required point-releases of the > entire python distribution to fix, i.e. python 2.3.5 and python 2.4.1 > were released simply to fix this bug. FWIW, that isn't entirely true. Python 2.3.5 was about to be released at that time anyway for other reasons. Because of this issue it was though delayed a little bit to add the change. As to Python 2.4.1 I can't find the exact details. There was going to be a 2.4.1 release a few weeks later, again for other reasons, so I think the fix got rolled into the first release candidate. Anyway, not that it matters, but the security fix was not the only thing in those releases. Graham From pywebsig at xhaus.com Sun Feb 12 13:36:14 2006 From: pywebsig at xhaus.com (Alan Kennedy) Date: Sun, 12 Feb 2006 12:36:14 +0000 Subject: [Web-SIG] WSGI in standard library In-Reply-To: References: <06Feb6.112517pst."58633"@synergy1.parc.xerox.com> <43EF1E8E.3080804@xhaus.com> Message-ID: <43EF2BBE.50705@xhaus.com> [Graham Dumpleton] > Anyway, not that it matters, but the security fix was not the only thing > in those releases. Still, I think my point stands that internet-facing servers in the standard lilbrary are currently the only source of security advisories in python. http://www.python.org/security/ How sure are we that any proposed production WSGI server in the standard library will not become a source of further holes, especially if it tries to cover all the bases of a true production server, i.e. security, flexibility, efficiency, full http1.1 compliance, etc? Regards, Alan. From lists at theorganization.net Sun Feb 12 14:01:29 2006 From: lists at theorganization.net (Jos Yule) Date: Sun, 12 Feb 2006 08:01:29 -0500 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <43EF1E8E.3080804@xhaus.com> References: <06Feb6.112517pst."58633"@synergy1.parc.xerox.com> <43EF1E8E.3080804@xhaus.com> Message-ID: <43EF31A9.8060709@theorganization.net> Alan Kennedy wrote: > So, I still think that only basic servers educational/playpen servers > should go in the standard library, with an indication that the user > should pick an openly server from outside the distro if they require to > do serious server work. +1 j -- jos yule Digital Hyakugei mailto:jos at theorganization.net http://www.theorganization.net aim:josyule msn:hyakugei at hotmail.com From pywebsig at xhaus.com Sun Feb 12 15:06:03 2006 From: pywebsig at xhaus.com (Alan Kennedy) Date: Sun, 12 Feb 2006 14:06:03 +0000 Subject: [Web-SIG] Bowing out (was Re: A trivial template API counter-proposal) In-Reply-To: <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> References: <5.1.1.6.0.20060205124025.021885b0@mail.telecommunity.com> <5.1.1.6.0.20060204131722.02179008@mail.telecommunity.com> <5.1.1.6.0.20060205124025.021885b0@mail.telecommunity.com> <5.1.1.6.0.20060206112120.02156208@mail.telecommunity.com> Message-ID: <43EF40CB.7000804@xhaus.com> [Alan Kennedy] >> Looking at this in an MVC context ....... [Phillip J. Eby] > As soon as you start talking about what templates should or should not > do (as opposed to what they *already* do), you've stopped writing an > inclusive spec and have wandered off into evangelizing a particular > framework philosophy. Sorry if my message seemed unreasonable. My approach to such matters is to attempt to start from best design practice, keeping a keen focus on the best way to do things in the future, relegating poorly-architected legacy systems, e.g. active page systems, to being a secondary concern. Also, my take on active page systems is that they could easily be encompassed by an MVC model. The View is the active page, the Model is the namespace in which the active page is rendered and the Controller is the thing that does the rendering. [Phillip J. Eby] > At this point it has become clear to me that even if I spent my days > and nights writing a compelling spec of what I'm proposing and then > trying to sell it to the Web SIG, it would be at best a 50/50 chance > of getting through, and in the process it appears that I'd be burning > through every bit of goodwill I might have previously possessed here. > .. I'd rather save whatever karma I > have left here for something with a better chance of success. I'm sorry to hear that. [Phillip J. Eby] > Good luck with the spec. Well, I'm currently designing and implementing a View and ViewResolver in Spring for a customer, so I'll be keeping a note of requirements as I go, and will attempt to come up with a generic design which is suitable for a a templating standard. But it will be a few weeks before I can spec that, document it and start doing sample implementations which I can open source. Regards, Alan. From chrism at plope.com Sun Feb 12 17:20:53 2006 From: chrism at plope.com (Chris McDonough) Date: Sun, 12 Feb 2006 11:20:53 -0500 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <43EF1E8E.3080804@xhaus.com> References: <06Feb6.112517pst."58633"@synergy1.parc.xerox.com> <43EF1E8E.3080804@xhaus.com> Message-ID: <26610A50-9F9D-4B4F-A24F-F02EB0F6500B@plope.com> On Feb 12, 2006, at 6:39 AM, Alan Kennedy wrote: > So, I still think that only basic servers educational/playpen servers > should go in the standard library, with an indication that the user > should pick an openly server from outside the distro if they > require to > do serious server work. I agree 100%. > > Maybe if there were no "production-ready" servers in the standard > library, there would be no need for a "Python Security Response Team". As an example, it's currently possible to perform denial of service on any framework/server that uses the cgi.FieldStorage module. See http://sourceforge.net/tracker/? func=detail&aid=1112549&group_id=5470&atid=105470 . That module probably doesn't belong in the stdlib in the first place, but it's in there, and now things depend on it. In the meantime, this patch *really* should have been applied by now but hasn't been. If anyone has checkin access, or can help me poke the appropriate person, it would help... this was reported to the SRT at the time. - C From wilk-ml at flibuste.net Mon Feb 13 08:51:29 2006 From: wilk-ml at flibuste.net (William Dode) Date: Mon, 13 Feb 2006 07:51:29 +0000 (UTC) Subject: [Web-SIG] WSGI in standard library References: <06Feb6.112517pst."58633"@synergy1.parc.xerox.com> <43EF1E8E.3080804@xhaus.com> <43EF31A9.8060709@theorganization.net> Message-ID: On 12-02-2006, Jos Yule wrote: > Alan Kennedy wrote: >> So, I still think that only basic servers educational/playpen servers >> should go in the standard library, with an indication that the user >> should pick an openly server from outside the distro if they require to >> do serious server work. Anyway if there is a server in the standard library it will be used in production, like all others part of the library, even if there is this kind of indication. Specialy for intranet or under proxy. -- William Dod? - http://flibuste.net From cce at clarkevans.com Mon Feb 13 16:16:34 2006 From: cce at clarkevans.com (Clark C. Evans) Date: Mon, 13 Feb 2006 10:16:34 -0500 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <26610A50-9F9D-4B4F-A24F-F02EB0F6500B@plope.com> References: <06Feb6.112517pst."58633"@synergy1.parc.xerox.com> <43EF1E8E.3080804@xhaus.com> <26610A50-9F9D-4B4F-A24F-F02EB0F6500B@plope.com> Message-ID: <20060213151634.GB85989@prometheusresearch.com> If it isn't production quality, it does not deserve to go in the standard library. If this means not having WSGI in the standard library, so be it. Clark On Sun, Feb 12, 2006 at 11:20:53AM -0500, Chris McDonough wrote: | On Feb 12, 2006, at 6:39 AM, Alan Kennedy wrote: | > So, I still think that only basic servers educational/playpen servers | > should go in the standard library, with an indication that the user | > should pick an openly server from outside the distro if they | > require to | > do serious server work. | | I agree 100%. | | > | > Maybe if there were no "production-ready" servers in the standard | > library, there would be no need for a "Python Security Response Team". | | As an example, it's currently possible to perform denial of service | on any framework/server that uses the cgi.FieldStorage module. See | http://sourceforge.net/tracker/? | func=detail&aid=1112549&group_id=5470&atid=105470 | . That module probably doesn't belong in the stdlib in the first | place, but it's in there, and now things depend on it. | | In the meantime, this patch *really* should have been applied by now | but hasn't been. If anyone has checkin access, or can help me poke | the appropriate person, it would help... this was reported to the SRT | at the time. | | - C | | _______________________________________________ | Web-SIG mailing list | Web-SIG at python.org | Web SIG: http://www.python.org/sigs/web-sig | Unsubscribe: http://mail.python.org/mailman/options/web-sig/cce%40clarkevans.com | From guido at python.org Mon Feb 13 21:49:00 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 13 Feb 2006 12:49:00 -0800 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <20060213151634.GB85989@prometheusresearch.com> References: <43EF1E8E.3080804@xhaus.com> <26610A50-9F9D-4B4F-A24F-F02EB0F6500B@plope.com> <20060213151634.GB85989@prometheusresearch.com> Message-ID: On 2/13/06, Clark C. Evans wrote: > If it isn't production quality, it does not deserve to go in the > standard library. If this means not having WSGI in the standard > library, so be it. There are many different ways to judge "production quality". If we're talking about correct, (standards-compliant, even) code, I wholly agree. If we're talking about having the level of performance, configurability, feature-fullness typically associated with production quality software, I think the standard library is not the place for it. After all, one person's required set of features is another person's feature bloat (and I'm usually that other person :-). Given a particular standard, it's usually not so hard to agree on the most straightforward way to implement it using Python. However, it's *very* hard to agree on the most performant way (since not everybody has the same performance requirements -- e.g. space/time, single/multi-processor etc.), let alone on features and configuration options. This, plus the radically different release cycle needed for production quality software, makes inclusion in the standard library and production quality pretty much mutually exclusive requirements (except for the obvious correctness requirements, which are only a tiny fraction of the set of properties going into production quality). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ianb at colorstudy.com Mon Feb 13 22:52:32 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 13 Feb 2006 15:52:32 -0600 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <43EF2BBE.50705@xhaus.com> References: <06Feb6.112517pst."58633"@synergy1.parc.xerox.com> <43EF1E8E.3080804@xhaus.com> <43EF2BBE.50705@xhaus.com> Message-ID: <43F0FFA0.6010005@colorstudy.com> Alan Kennedy wrote: >>Anyway, not that it matters, but the security fix was not the only thing >>in those releases. > > > Still, I think my point stands that internet-facing servers in the > standard lilbrary are currently the only source of security advisories > in python. > > http://www.python.org/security/ > > How sure are we that any proposed production WSGI server in the standard > library will not become a source of further holes, especially if it > tries to cover all the bases of a true production server, i.e. security, > flexibility, efficiency, full http1.1 compliance, etc? Note that the scope of a WSGI server is very very limited. It is quite distinct from an XMLRPC server from that perspective -- an XMLRPC server actually *does* something. A WSGI server does nothing but delegate. Still, there are certainly some DoS possibilities, but I don't think that's something we're handling very gracefully at the moment anyhow. Maybe we should, but we're not. CherryPy's HTTP server does seem to have code for that, but then that might just be a cover for the bug in the cgi module. Anyway, even there most DoS issues are out of scope for a WSGI server. I'm not set on "production" quality code, but I think the general sentiment against that is entirely premature. The implementations brought up -- CherryPy's (http://svn.cherrypy.org/trunk/cherrypy/_cphttpserver.py) and Paste's (http://svn.pythonpaste.org/Paste/trunk/paste/httpserver.py) and wsgiref's (http://cvs.eby-sarna.com/wsgiref/src/wsgiref/simple_server.py?rev=1.2&view=markup) are all pretty short. It would be better to discuss the particulars. Is there a code path in one or more of these servers which you think is unneeded and problematic? -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From pywebsig at xhaus.com Tue Feb 14 20:34:58 2006 From: pywebsig at xhaus.com (Alan Kennedy) Date: Tue, 14 Feb 2006 19:34:58 +0000 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <43F0FFA0.6010005@colorstudy.com> References: <06Feb6.112517pst."58633"@synergy1.parc.xerox.com> <43EF1E8E.3080804@xhaus.com> <43EF2BBE.50705@xhaus.com> <43F0FFA0.6010005@colorstudy.com> Message-ID: <43F230E2.1090101@xhaus.com> [Ian Bicking] > Note that the scope of a WSGI server is very very limited. It is quite > distinct from an XMLRPC server from that perspective -- an XMLRPC server > actually *does* something. A WSGI server does nothing but delegate. and > I'm not set on "production" quality code, but I think the general > sentiment against that is entirely premature. The implementations > brought up -- CherryPy's > (http://svn.cherrypy.org/trunk/cherrypy/_cphttpserver.py) and Paste's > (http://svn.pythonpaste.org/Paste/trunk/paste/httpserver.py) and > wsgiref's > (http://cvs.eby-sarna.com/wsgiref/src/wsgiref/simple_server.py?rev=1.2&view=markup) > are all pretty short. It would be better to discuss the particulars. Is > there a code path in one or more of these servers which you think is > unneeded and problematic? A few points. 1. My opinion is not relevant to whether/which WSGI server goes into the standard library. What's required is for someone to propose to python-dev that a particular WSGI server should go into the standard library. I imagine that the response on python-dev to the proposer is going to be along the lines of "Will you be maintaining this?" If/when python-dev is happy, then it'll go into the distribution. 2. What's wrong with leaving the current situation as-is, i.e. the available WSGI implementations are listed on the WSGI Moin page http://wiki.python.org/moin/WSGIImplementations 3. If I had to pick one of the 3 you suggested, I'd pick the last one, i.e. PJE's, because it fulfills exactly the criteria I listed - It's pretty much the simplest possible implementation, meaning it's easiest to understand. - It's based on the existing *HttpServer hierarchy - It's got a big notice at the top saying """This is both an example of how WSGI can be implemented, and a basis for running simple web applications on a local machine, such as might be done when testing or debugging an application. It has not been reviewed for security issues, however, and we strongly recommend that you use a "real" web server for production use.""" Regards, Alan. From guido at python.org Tue Feb 14 21:00:57 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 14 Feb 2006 12:00:57 -0800 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <43F230E2.1090101@xhaus.com> References: <43EF1E8E.3080804@xhaus.com> <43EF2BBE.50705@xhaus.com> <43F0FFA0.6010005@colorstudy.com> <43F230E2.1090101@xhaus.com> Message-ID: On 2/14/06, Alan Kennedy wrote: > [Ian Bicking] > > Note that the scope of a WSGI server is very very limited. It is quite > > distinct from an XMLRPC server from that perspective -- an XMLRPC server > > actually *does* something. A WSGI server does nothing but delegate. > > and > > > I'm not set on "production" quality code, but I think the general > > sentiment against that is entirely premature. The implementations > > brought up -- CherryPy's > > (http://svn.cherrypy.org/trunk/cherrypy/_cphttpserver.py) and Paste's > > (http://svn.pythonpaste.org/Paste/trunk/paste/httpserver.py) and > > wsgiref's > > (http://cvs.eby-sarna.com/wsgiref/src/wsgiref/simple_server.py?rev=1.2&view=markup) > > are all pretty short. It would be better to discuss the particulars. Is > > there a code path in one or more of these servers which you think is > > unneeded and problematic? > > A few points. > > 1. My opinion is not relevant to whether/which WSGI server goes into the > standard library. What's required is for someone to propose to > python-dev that a particular WSGI server should go into the standard > library. I imagine that the response on python-dev to the proposer is > going to be along the lines of "Will you be maintaining this?" If/when > python-dev is happy, then it'll go into the distribution. > > 2. What's wrong with leaving the current situation as-is, i.e. the > available WSGI implementations are listed on the WSGI Moin page > > http://wiki.python.org/moin/WSGIImplementations > > 3. If I had to pick one of the 3 you suggested, I'd pick the last one, > i.e. PJE's, because it fulfills exactly the criteria I listed > > - It's pretty much the simplest possible implementation, meaning it's > easiest to understand. > - It's based on the existing *HttpServer hierarchy > - It's got a big notice at the top saying """This is both an example > of how WSGI can be implemented, and a basis for running simple web > applications on a local machine, such as might be done when testing or > debugging an application. It has not been reviewed for security issues, > however, and we strongly recommend that you use a "real" web server for > production use.""" Let's make it so. I propose to add wsgiref to the standard library and nothing more. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ianb at colorstudy.com Tue Feb 14 21:28:16 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 14 Feb 2006 14:28:16 -0600 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <43F230E2.1090101@xhaus.com> References: <06Feb6.112517pst."58633"@synergy1.parc.xerox.com> <43EF1E8E.3080804@xhaus.com> <43EF2BBE.50705@xhaus.com> <43F0FFA0.6010005@colorstudy.com> <43F230E2.1090101@xhaus.com> Message-ID: <43F23D60.7020500@colorstudy.com> Alan Kennedy wrote: >> I'm not set on "production" quality code, but I think the general >> sentiment against that is entirely premature. The implementations >> brought up -- CherryPy's >> (http://svn.cherrypy.org/trunk/cherrypy/_cphttpserver.py) and Paste's >> (http://svn.pythonpaste.org/Paste/trunk/paste/httpserver.py) and >> wsgiref's >> (http://cvs.eby-sarna.com/wsgiref/src/wsgiref/simple_server.py?rev=1.2&view=markup) >> are all pretty short. It would be better to discuss the particulars. >> Is there a code path in one or more of these servers which you think >> is unneeded and problematic? > > > A few points. > > 1. My opinion is not relevant to whether/which WSGI server goes into the > standard library. What's required is for someone to propose to > python-dev that a particular WSGI server should go into the standard > library. I imagine that the response on python-dev to the proposer is > going to be along the lines of "Will you be maintaining this?" If/when > python-dev is happy, then it'll go into the distribution. > > 2. What's wrong with leaving the current situation as-is, i.e. the > available WSGI implementations are listed on the WSGI Moin page > > http://wiki.python.org/moin/WSGIImplementations I think a WSGI HTTP server is considerably more useful than SimpleHTTPServer, for a whole bunch of reasons. With a WSGI HTTP server, I see little reason for anyone to use any of the other Simple*Server implementations (unless perhaps they are using async processing, which WSGI does not facilitate, but I think is doable with SimpleHTTPServer). > 3. If I had to pick one of the 3 you suggested, I'd pick the last one, > i.e. PJE's, because it fulfills exactly the criteria I listed > > - It's pretty much the simplest possible implementation, meaning it's > easiest to understand. In part this is because it uses the other parts of wsgiref, which the other servers do not. > - It's based on the existing *HttpServer hierarchy All three share this property. > - It's got a big notice at the top saying """This is both an example of > how WSGI can be implemented, and a basis for running simple web > applications on a local machine, such as might be done when testing or > debugging an application. It has not been reviewed for security issues, > however, and we strongly recommend that you use a "real" web server for > production use.""" And of course this is one of the easiest features to port ;) But anyway, maybe the standard library isn't the right place for any of these, maybe it wasn't the right place for SimpleHTTPServer (though it's hard to actually argue that, since in practical terms SimpleHTTPServer has been very successful). But we could also just create a stand-alone package that provides just an HTTP server, and reference that from wherever appropriate (including the standard library documentation). Right now the standard library documentation doesn't often reference external packages. I don't know if there's any real reason for that. I will posit that no, there is no good reason that the stdlib documentation shouldn't link directly to projects (at least in certain situations). I think we have enough implementations to take from to make such a canonical HTTP package, at least within the scope of threaded HTTP implementations based on SimpleHTTPServer. Though then there's the whole Twisted question -- but right now packaging and scope make it hard to actually use Twisted in these same situations. But given a specific API, at least that would give Twisted a clear target so that instead of "from httpserver import HTTPServer" you'd do "from twisted.httpserver import HTTPServer" and then you'd be running under Twisted (ignoring scenarios where Twisted does more than HTTP serving, which is out of scope here). Of course, then you could say "lets create a multi-protocol app server package" integrating perhaps portions of Flup, an HTTP server, and a couple other things (but there's not a whole lot else I imagine going into this stuff). And then the scope starts to match Twisted, except perhaps with a simplification that you'd only handle request/response-style protocols with blocking WSGI-style responders. Then you could say, hey, isn't WSGI meant to keep us from having to do these combined efforts, because multiple packages are okay if they all implement the same standard? Then you could say, sure, but it also means that we can combine packages on a different timescale and structure than higher-level frameworks (WSGI applications), waiting until the time is ripe, and maybe the time is ripe. It feels lame (to me) to tell people to use one package or another as they switch from HTTP to SCGI to whatever. Then-you-could-saying'ly y'rs... -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From fumanchu at amor.org Tue Feb 14 22:09:00 2006 From: fumanchu at amor.org (Robert Brewer) Date: Tue, 14 Feb 2006 13:09:00 -0800 Subject: [Web-SIG] WSGI in standard library Message-ID: <6949EC6CD39F97498A57E0FA55295B21019D2484@ex9.hostedexchange.local> Alan Kennedy wrote: > 3. If I had to pick one of the 3 you suggested, I'd pick the > last one, i.e. PJE's, because it fulfills exactly the criteria > I listed > > - It's pretty much the simplest possible implementation, > meaning it's easiest to understand. I have to disagree (having examined/unraveled it quite a bit recently, to remove modpython_gateway's dependency on it). The server class seems simple enough, but the handlers module is IMO horribly convoluted, mostly to support too many options: async vs. threaded, origin and non-origin servers, various HTTP versions, file sending hooks, etc. There are simply too many variables involved in building a WSGI handler appropriate for your environment; trying to do that by subclassing wsgiref.handlers.* results in extremely complicated and slow code. Robert Brewer System Architect Amor Ministries fumanchu at amor.org From ianb at colorstudy.com Tue Feb 14 22:45:06 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 14 Feb 2006 15:45:06 -0600 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <6949EC6CD39F97498A57E0FA55295B21019D2484@ex9.hostedexchange.local> References: <6949EC6CD39F97498A57E0FA55295B21019D2484@ex9.hostedexchange.local> Message-ID: <43F24F62.30607@colorstudy.com> Robert Brewer wrote: > Alan Kennedy wrote: > >>3. If I had to pick one of the 3 you suggested, I'd pick the >>last one, i.e. PJE's, because it fulfills exactly the criteria >>I listed >> >> - It's pretty much the simplest possible implementation, >>meaning it's easiest to understand. > > > I have to disagree (having examined/unraveled it quite a bit recently, > to remove modpython_gateway's dependency on it). The server class seems > simple enough, but the handlers module is IMO horribly convoluted, > mostly to support too many options: async vs. threaded, origin and > non-origin servers, various HTTP versions, file sending hooks, etc. > There are simply too many variables involved in building a WSGI handler > appropriate for your environment; trying to do that by subclassing > wsgiref.handlers.* results in extremely complicated and slow code. I think it also tries to enforce a lot of the details of WSGI, and thus guide a WSGI implementor into creating a compliant server. But in the process it creates a framework that has to be correctly used, so it's swapping a well-understood set of bugs (doing something you aren't supposed to with WSGI) for a different set (using the handler incorrectly). paste.lint is reasonably good at checking WSGI compliance (those parts that are actually detectable) without caring about what your code actually looks like. I think it is easier to use, with largely the same effect. For the actual server a framework seems unnecessary, as there will only be two servers in the standard library (I assume) -- CGI and HTTP (and hopefully HTTPS will be easy to build off of HTTP). -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From guido at python.org Tue Feb 14 23:44:29 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 14 Feb 2006 14:44:29 -0800 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <43F24319.8000200@latte.ca> References: <43EF1E8E.3080804@xhaus.com> <43EF2BBE.50705@xhaus.com> <43F0FFA0.6010005@colorstudy.com> <43F230E2.1090101@xhaus.com> <43F24319.8000200@latte.ca> Message-ID: On 2/14/06, Blake Winton wrote: > > Let's make it so. I propose to add wsgiref to the standard library and > > nothing more. > > Will you be maintaining this? ;) I'd expect we could twist Phillip's arm to maintain it; he's not expecting much maintenance. But if a maintainer is all that's needed to make everybody happy, I'd be happy to volunteer if no-one else does. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Tue Feb 14 23:59:28 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 14 Feb 2006 17:59:28 -0500 Subject: [Web-SIG] WSGI in standard library In-Reply-To: References: <43F24319.8000200@latte.ca> <43EF1E8E.3080804@xhaus.com> <43EF2BBE.50705@xhaus.com> <43F0FFA0.6010005@colorstudy.com> <43F230E2.1090101@xhaus.com> <43F24319.8000200@latte.ca> Message-ID: <5.1.1.6.0.20060214175602.01fc13f0@mail.telecommunity.com> At 02:44 PM 2/14/2006 -0800, Guido van Rossum wrote: >On 2/14/06, Blake Winton wrote: > > > Let's make it so. I propose to add wsgiref to the standard library and > > > nothing more. > > > > Will you be maintaining this? ;) > >I'd expect we could twist Phillip's arm to maintain it; he's not >expecting much maintenance. Yes, and yes. >But if a maintainer is all that's needed to make everybody happy, Make everybody *happy*? Now *that*'s wishful thinking. ;) I usually settle for "doesn't annoy anybody enough to cause a fork." From guido at python.org Wed Feb 15 00:15:04 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 14 Feb 2006 15:15:04 -0800 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <5.1.1.6.0.20060214175602.01fc13f0@mail.telecommunity.com> References: <43EF1E8E.3080804@xhaus.com> <43EF2BBE.50705@xhaus.com> <43F0FFA0.6010005@colorstudy.com> <43F230E2.1090101@xhaus.com> <43F24319.8000200@latte.ca> <5.1.1.6.0.20060214175602.01fc13f0@mail.telecommunity.com> Message-ID: On 2/14/06, Phillip J. Eby wrote: > Make everybody *happy*? Now *that*'s wishful thinking. ;) I usually > settle for "doesn't annoy anybody enough to cause a fork." Oh, I'll gladly make them fork. Good riddance. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pywebsig at xhaus.com Wed Feb 15 00:17:37 2006 From: pywebsig at xhaus.com (Alan Kennedy) Date: Tue, 14 Feb 2006 23:17:37 +0000 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <43F24F62.30607@colorstudy.com> References: <6949EC6CD39F97498A57E0FA55295B21019D2484@ex9.hostedexchange.local> <43F24F62.30607@colorstudy.com> Message-ID: <43F26511.1010609@xhaus.com> [Alan Kennedy] >>>3. If I had to pick one of the 3 you suggested, I'd pick the >>>last one, i.e. PJE's, because it fulfills exactly the criteria >>>I listed [Robert Brewer] >>I have to disagree (having examined/unraveled it quite a bit recently, >>to remove modpython_gateway's dependency on it). [Ian Bicking] > I think it also tries to enforce a lot of the details of WSGI, and thus > guide a WSGI implementor into creating a compliant server. Well, I'm sure we all want our favourite server in the stdlib ;-) But a few things have to happen first. Priority #1: Make the requisite server a single standalone module. Anticipating PJE's willingness to have WSGIRef included in the stdlib, I've taken the liberty of putting it all into one big file. And I think it looks pretty damn good: fully WSGI compliant, with code to represent every single aspect of the spec. Take a look for yourself: the file is attached. If the attachment doesn't make it to the list, I'll upload it somewhere. But that doesn't mean the decision's over. It means that the bar has been raised. Anyone else who wants their module to be a contender has to get it all into the one file, i.e. eliminating all framework dependencies, etc. Here's a few comments I put together about the three contenders that have been proposed so far. They're just my own comments from reading the code: feel free to treat them as the ravings of a madman if you so wish. 1. CherryPy server - 407 lines (non-code lines: ~80) - Depends on cherrypy, cherryp._cputil, cherryp.lib.httptools - Depends on cherrypy.config - Implements HTTP header length limit checking - Implements HTTP body length limit checking - Uses own logging handler - Subclasses SocketServer.BaseServer, not BaseHTTPServer.HTTPServer - Therefore does low-level socket mucking-about - Provides 2 server implementations - CherryHTTPServer - PooledThreadServer - Explicitly checks for KeyboardInterrupt exceptions - PooledThreadServer has clean shutdown through Queue.Queue messaging - Does not detect hop-by-hop headers - No demo application My gut feeling: too complex, works to hard to be "production-ready", at the expense of readability. 2. Paste Server - 450 lines - Supports 100 continue responses - No imports from outside stdlib - Provides HTTPS/SSL server, with fallback if no SSL - Supports socket timeout - Demo application is (imported) paste.wsgilib.dump_environ - Does not detect hop-by-hop headers My gut feeling: Ignores many parts of the WSGI spec (sendfile, strict error checking), supports unnecessary stuff for stdlib, i.e. Continue support, HTTPS. 3. WSGIRef_onefile.py - 660 lines - No imports from outside stdlib - Detects hop-by-hop headers - Has WSGI sendfile support - Has dedicated class to manage WSGI headers list as dictionary - Has builtin demo app My gut feeling: WSGIRef is the sweetspot in terms of simplicity vs. usability. Covers all aspects of WSGI (which is what it was designed for, IIRC ;-) The ball's in yizzir court now...... Alan. -------------- next part -------------- A non-text attachment was scrubbed... Name: wsgiref_onefile.py Type: text/x-python Size: 23564 bytes Desc: not available Url : http://mail.python.org/pipermail/web-sig/attachments/20060214/7d64639a/attachment-0001.py From pywebsig at xhaus.com Wed Feb 15 00:20:54 2006 From: pywebsig at xhaus.com (Alan Kennedy) Date: Tue, 14 Feb 2006 23:20:54 +0000 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <5.1.1.6.0.20060214175602.01fc13f0@mail.telecommunity.com> References: <43F24319.8000200@latte.ca> <43EF1E8E.3080804@xhaus.com> <43EF2BBE.50705@xhaus.com> <43F0FFA0.6010005@colorstudy.com> <43F230E2.1090101@xhaus.com> <43F24319.8000200@latte.ca> <5.1.1.6.0.20060214175602.01fc13f0@mail.telecommunity.com> Message-ID: <43F265D6.0@xhaus.com> [Guido van Rossum] >>>>Let's make it so. I propose to add wsgiref to the standard library and >>>>nothing more. [Blake Winton] >>>Will you be maintaining this? ;) [Guido van Rossum] >>I'd expect we could twist Phillip's arm to maintain it; he's not >>expecting much maintenance. [Phillip J. Eby] > Yes, and yes. Whew! :-) Phillip: Hope you don't mind me taking the liberty of rearranging your code? And before we go finalising anything, please let's give the other contenders a chance to come up with something competitive. Alan. From guido at python.org Wed Feb 15 00:32:28 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 14 Feb 2006 15:32:28 -0800 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <43F26511.1010609@xhaus.com> References: <6949EC6CD39F97498A57E0FA55295B21019D2484@ex9.hostedexchange.local> <43F24F62.30607@colorstudy.com> <43F26511.1010609@xhaus.com> Message-ID: On 2/14/06, Alan Kennedy wrote: > Well, I'm sure we all want our favourite server in the stdlib ;-) > > But a few things have to happen first. > > Priority #1: Make the requisite server a single standalone module. Huh? What makes you think this? There are plenty of packages in the standard library. It obviously needs to be free from dependencies on things that aren't in the standard library. But I don't see the need to remove the package structure. (The only cleanup I'd like to see is to remove the strange occurrences of *many* consecutive blank lines here and there. What are these for?) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Wed Feb 15 00:40:51 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 14 Feb 2006 18:40:51 -0500 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <43F265D6.0@xhaus.com> References: <5.1.1.6.0.20060214175602.01fc13f0@mail.telecommunity.com> <43F24319.8000200@latte.ca> <43EF1E8E.3080804@xhaus.com> <43EF2BBE.50705@xhaus.com> <43F0FFA0.6010005@colorstudy.com> <43F230E2.1090101@xhaus.com> <43F24319.8000200@latte.ca> <5.1.1.6.0.20060214175602.01fc13f0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060214182637.047d74f8@mail.telecommunity.com> At 11:20 PM 2/14/2006 +0000, Alan Kennedy wrote: >Phillip: Hope you don't mind me taking the liberty of rearranging your code? You can do whatever you want with it; just don't expect me to maintain your rearranged version. :) Among other issues, I've got a pending patch from Guido regarding date formatting for the server. Note that Guido has proposed to include wsgiref in the stdlib, which makes the issue of having separate imports for the handler code and such moot, so there would be no need in that case to reorganize the code. A lot of people seem to have missed the part where Guido's original and ongoing proposal is to add wsgiref to the stdlib. The present discussion about what WSGI server (if any) should be in the stdlib is really a question of whether there should be another server *in addition* to the one in wsgiref -- unless you want to try to argue that neither wsgiref.handlers nor wsgiref.simple_server should be included, in which case Guido's simple proposal devolves into the chaos of redesigning the wsgiref library by committee. (Although I suppose it already has become that.) The reasoning for having wsgiref in the stdlib isn't so people can have a server, it's to have a reference implementation of WSGI functionality and some standards-compliant utilities for working with the data structures (request/response manipulation). There would possibly be value in adding other similar utilities, middleware (e.g. paste.lint), etc. But any proposal for adding a WSGI server that was for the purpose of *having* a server, would be independent of that, and indeed independent of wsgiref AFAICT. From pje at telecommunity.com Wed Feb 15 00:50:33 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 14 Feb 2006 18:50:33 -0500 Subject: [Web-SIG] WSGI in standard library In-Reply-To: References: <43F26511.1010609@xhaus.com> <6949EC6CD39F97498A57E0FA55295B21019D2484@ex9.hostedexchange.local> <43F24F62.30607@colorstudy.com> <43F26511.1010609@xhaus.com> Message-ID: <5.1.1.6.0.20060214184431.047af5e0@mail.telecommunity.com> At 03:32 PM 2/14/2006 -0800, Guido van Rossum wrote: >(The only cleanup I'd like to see is to remove the >strange occurrences of *many* consecutive blank lines here and there. >What are these for?) Personal quirk: I prefer to edit files in screen-sized pages, as it allows me to navigate quickly and see things in units. The blank lines align units of code to the logical "pages". Naturally, these won't stay in when this goes to the stdlib, as I recognize that my taste in paging is, um, unique. :) From pywebsig at xhaus.com Wed Feb 15 00:50:57 2006 From: pywebsig at xhaus.com (Alan Kennedy) Date: Tue, 14 Feb 2006 23:50:57 +0000 Subject: [Web-SIG] WSGI in standard library In-Reply-To: References: <6949EC6CD39F97498A57E0FA55295B21019D2484@ex9.hostedexchange.local> <43F24F62.30607@colorstudy.com> <43F26511.1010609@xhaus.com> Message-ID: <43F26CE1.7040307@xhaus.com> [Alan Kennedy] >>Priority #1: Make the requisite server a single standalone module. [Guido van Rossum] > Huh? What makes you think this? My bad :-( Two things made me think like that 1. BaseHttpServer -> BaseHttpServer.py SimpleHttpServer -> SimpleHttpServer.py WSGIHttpServer -> WSGIHttpServer.py 2. The comment was more aimed at the CherryPy entry, which imports a fair amount of CherryPy support code. i'll-get-me-coat-ly'yrs, Alan. From fumanchu at amor.org Wed Feb 15 00:52:26 2006 From: fumanchu at amor.org (Robert Brewer) Date: Tue, 14 Feb 2006 15:52:26 -0800 Subject: [Web-SIG] WSGI in standard library Message-ID: <6949EC6CD39F97498A57E0FA55295B21019D27D8@ex9.hostedexchange.local> Alan Kennedy wrote: > Here's a few comments I put together about the three contenders that > have been proposed so far. They're just my own comments from > reading the code: feel free to treat them as the ravings of > a madman if you so wish. > > 1. CherryPy server - 407 lines (non-code lines: ~80) > > - Depends on cherrypy, cherryp._cputil, cherryp.lib.httptools > - Depends on cherrypy.config ?? Perhaps you're assuming the wrapper in _cpwsgi is being proposed, but it isn't. That has all of the CherryPy-specific "stuff" in it. The base file (_cpwsgiserver) has no dependencies except for stdlib modules. > - Implements HTTP header length limit checking > - Implements HTTP body length limit checking > - Uses own logging handler These are not present in the base _cpwsgiserver. > - Subclasses SocketServer.BaseServer, not BaseHTTPServer.HTTPServer > - Therefore does low-level socket mucking-about > - Provides 2 server implementations > - CherryHTTPServer > - PooledThreadServer Okay, now you're really looking in the wrong place. Ignore _cphttpserver; it's deprecated and has nothing to do with WSGI. Look at _cpwsgiserver and only _cpwsgiserver. > - Explicitly checks for KeyboardInterrupt exceptions > - PooledThreadServer has clean shutdown through Queue.Queue > messaging > - Does not detect hop-by-hop headers > - No demo application > > My gut feeling: too complex, works to hard to be > "production-ready", at the expense of readability. Look at the right code and see if your gut feeling changes. ;) Robert Brewer System Architect Amor Ministries fumanchu at amor.org From pywebsig at xhaus.com Wed Feb 15 00:59:01 2006 From: pywebsig at xhaus.com (Alan Kennedy) Date: Tue, 14 Feb 2006 23:59:01 +0000 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <6949EC6CD39F97498A57E0FA55295B21019D27D8@ex9.hostedexchange.local> References: <6949EC6CD39F97498A57E0FA55295B21019D27D8@ex9.hostedexchange.local> Message-ID: <43F26EC5.3030101@xhaus.com> [Robert Brewer] > Look at the right code and see if your gut feeling changes. ;) I looked at http://svn.cherrypy.org/trunk/cherrypy/_cphttpserver.py As indicated by Ian in this message http://mail.python.org/pipermail/web-sig/2006-February/002074.html Sorry if that was the wrong one to look at, I'm not at all familiar with CherryPy. Alan. From fumanchu at amor.org Wed Feb 15 01:01:57 2006 From: fumanchu at amor.org (Robert Brewer) Date: Tue, 14 Feb 2006 16:01:57 -0800 Subject: [Web-SIG] WSGI in standard library Message-ID: <6949EC6CD39F97498A57E0FA55295B21019D2805@ex9.hostedexchange.local> Alan Kennedy wrote: > [Robert Brewer] > > Look at the right code and see if your gut feeling changes. ;) > > I looked at > > http://svn.cherrypy.org/trunk/cherrypy/_cphttpserver.py > > As indicated by Ian in this message > > http://mail.python.org/pipermail/web-sig/2006-February/002074.html > > Sorry if that was the wrong one to look at, I'm not at all > familiar with CherryPy. Sorry about that; I should've provided a link. Here's the latest _cpwsgiserver.py: http://www.cherrypy.org/file/trunk/cherrypy/_cpwsgiserver.py Robert Brewer System Architect Amor Ministries fumanchu at amor.org From ianb at colorstudy.com Wed Feb 15 01:03:03 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 14 Feb 2006 18:03:03 -0600 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <43F26EC5.3030101@xhaus.com> References: <6949EC6CD39F97498A57E0FA55295B21019D27D8@ex9.hostedexchange.local> <43F26EC5.3030101@xhaus.com> Message-ID: <43F26FB7.7080304@colorstudy.com> Alan Kennedy wrote: >> Look at the right code and see if your gut feeling changes. ;) > > > I looked at > > http://svn.cherrypy.org/trunk/cherrypy/_cphttpserver.py > > As indicated by Ian in this message > > http://mail.python.org/pipermail/web-sig/2006-February/002074.html > > Sorry if that was the wrong one to look at, I'm not at all familiar with > CherryPy. Sorry about that. Peter just proposed CherryPy and _cphttpserver was my first guess (and got a hit). Looking at _cpwsgiserver I already prefer it over _cphttpserver purely on the basis of indentation ;) I also notice it doesn't use SimpleHTTPServer. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From guido at python.org Wed Feb 15 01:07:58 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 14 Feb 2006 16:07:58 -0800 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <43F26CE1.7040307@xhaus.com> References: <6949EC6CD39F97498A57E0FA55295B21019D2484@ex9.hostedexchange.local> <43F24F62.30607@colorstudy.com> <43F26511.1010609@xhaus.com> <43F26CE1.7040307@xhaus.com> Message-ID: On 2/14/06, Alan Kennedy wrote: > [Alan Kennedy] > Two things made me think like that > > 1. BaseHttpServer -> BaseHttpServer.py > SimpleHttpServer -> SimpleHttpServer.py > WSGIHttpServer -> WSGIHttpServer.py Actually BaseHTTPServer.py and friends use a deprecated naming scheme -- just as StringIO, UserDict and many other fine standard library modules. If you read PEP 8, the current best practice is for module names to be all-lowercase and *different* from the class name. The main reason for this is the following common mistake: import StringIO . . (100s of lines of code) . ...f = StringIO() # oops, should've been StringIO.StringIO(). vs. from String import StringIO . . (100s of lines of code) . ...f = StringIO.StringIO() # oops, should've been StringIO(). Since both import styles are common, it's easy to forget which import style was used and to use the wrong invocation style. While the error doesn't pass silently, it's annoying and a bit jarring; the error message isn't all that clear. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From floydophone at gmail.com Wed Feb 15 02:02:52 2006 From: floydophone at gmail.com (Peter Hunt) Date: Tue, 14 Feb 2006 20:02:52 -0500 Subject: [Web-SIG] WSGI in standard library Message-ID: <6654eac40602141702g5daf6d8dn72e0ce6fa7a3097d@mail.gmail.com> Alan, you're looking at cherrypy._cphttpserver. You should be looking at cherrypy._cpwsgiserver. That's the primary server nowadays; cherrypy._cphttpserver is an artifact for the most part. Peter Hunt -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/web-sig/attachments/20060214/22afa7c7/attachment-0001.html From cce at clarkevans.com Wed Feb 15 06:50:11 2006 From: cce at clarkevans.com (Clark C. Evans) Date: Wed, 15 Feb 2006 00:50:11 -0500 Subject: [Web-SIG] WSGI in standard library In-Reply-To: References: <43EF1E8E.3080804@xhaus.com> <43EF2BBE.50705@xhaus.com> <43F0FFA0.6010005@colorstudy.com> <43F230E2.1090101@xhaus.com> Message-ID: <20060215055011.GA46861@prometheusresearch.com> On Mon, Feb 13, 2006 at 12:49:00PM -0800, Guido van Rossum wrote: | There are many different ways to judge "production quality". If we're | talking about correct, (standards-compliant, even) code, I wholly | agree. Fantastic. I just don't think it is appropriate to have a "toy" in the standard library. On Tue, Feb 14, 2006 at 12:00:57PM -0800, Guido van Rossum wrote: | Let's make it so. I propose to add wsgiref to the standard library and | nothing more. I propose we add wsgiref, but look at other implementations and steal what ever you can from them. This is not a huge chunk of code -- no reason why you can't have the best combination of features and correctness. On Tue, Feb 14, 2006 at 11:17:37PM +0000, Alan Kennedy wrote: | 1. CherryPy server - 407 lines (non-code lines: ~80) | 2. Paste Server - 450 lines | | - Supports 100 continue responses | - No imports from outside stdlib | - Provides HTTPS/SSL server, with fallback if no SSL | - Supports socket timeout | - Demo application is (imported) paste.wsgilib.dump_environ (dump_environ is trivial, something like it would be useful for a wsgilibrary | - Does not detect hop-by-hop headers I'd add in one more important item: - Implemented as a MixIn | My gut feeling: Ignores many parts of the WSGI spec (sendfile, strict | error checking), supports unnecessary stuff for stdlib, i.e. Continue | support, HTTPS. HTTPS is pretty standard requirement, its a small amount of code for a "battery"; 100 Continue is easy to drop; adding hop-by-hop header support and sendfile isn't hard (it can be copied from wsgiref or the reverse) | 3. WSGIRef_onefile.py - 660 lines | | - No imports from outside stdlib | - Detects hop-by-hop headers | - Has WSGI sendfile support | - Has dedicated class to manage WSGI headers list as dictionary -> this thing is quite huge, is it necessary; I do WSGI all the time and have not found it useful | - Has builtin demo app | | My gut feeling: WSGIRef is the sweetspot in terms of simplicity vs. | usability. Covers all aspects of WSGI (which is what it was designed | for, IIRC ;-) Best, Clark From exarkun at divmod.com Wed Feb 15 14:39:03 2006 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Wed, 15 Feb 2006 08:39:03 -0500 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <20060215055011.GA46861@prometheusresearch.com> Message-ID: <20060215133903.20077.181425073.divmod.quotient.2183@ohm> On Wed, 15 Feb 2006 00:50:11 -0500, "Clark C. Evans" wrote: >On Mon, Feb 13, 2006 at 12:49:00PM -0800, Guido van Rossum wrote: > >| My gut feeling: Ignores many parts of the WSGI spec (sendfile, strict >| error checking), supports unnecessary stuff for stdlib, i.e. Continue >| support, HTTPS. > >HTTPS is pretty standard requirement, its a small amount of >code for a "battery"; HTTPS is orthogonal. Besides, how would you support it in the stdlib? It's currently not possible to write an SSL server in Python without a third-party library. Maybe someone would be interested in rectifying /that/? :) Jean-Paul From cce at clarkevans.com Wed Feb 15 17:22:18 2006 From: cce at clarkevans.com (Clark C. Evans) Date: Wed, 15 Feb 2006 11:22:18 -0500 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <20060215133903.20077.181425073.divmod.quotient.2183@ohm> References: <20060215055011.GA46861@prometheusresearch.com> <20060215133903.20077.181425073.divmod.quotient.2183@ohm> Message-ID: <20060215162218.GA51995@prometheusresearch.com> On Wed, Feb 15, 2006 at 08:39:03AM -0500, Jean-Paul Calderone wrote: | On Wed, 15 Feb 2006 00:50:11 -0500, "Clark C. Evans" wrote: | >On Mon, Feb 13, 2006 at 12:49:00PM -0800, Guido van Rossum wrote: | > | >| My gut feeling: Ignores many parts of the WSGI spec (sendfile, strict | >| error checking), supports unnecessary stuff for stdlib, i.e. Continue | >| support, HTTPS. | > | >HTTPS is pretty standard requirement, its a small amount of | >code for a "battery"; | | HTTPS is orthogonal. Besides, how would you support it in the stdlib? | It's currently not possible to write an SSL server in Python without | a third-party library. Maybe someone would be interested in | rectifying /that/? :) I'd start by making sure that the WSGI implementation adopted is a "MixIn" to permit its usage in any HTTP Server that has the same protocol as HTTPServer: including, but not limited to an SSL variant. I hope that wsgiref can be refactored in this way before it is added to the standard library (see my implementation, steal at will). Making it hard-coded to work with HTTPServer is an unnecessary restriction. I'd also say that the base HTTPServer could use smarter code to deal with dropped connections: they really arn't errors. Any resonable WSGI server implementation will need to catch socket.error (and the bastard variant that the SSL library uses) so that you're not spamming your log files with ill-behaved clients who use the STOP button. Further, it would be nice if the "version" string put forth by the BaseHTTPServer wasn't so intelligent; a simple string would do just fine. Beyond that, an SSL version of the base HTTPServer is a whole 30 lines of code, (and much most of this is patching the SSL library's makefile() method); this could easily go in an "except ImportError" block. I don't really care one way or the other with this one - I can keep this code in my own trunk just as happily. However, isn't the idea of the python standard library to include common "batteries"? Kind Regards, Clark From guido at python.org Wed Feb 15 18:30:14 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 15 Feb 2006 09:30:14 -0800 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <20060215133903.20077.181425073.divmod.quotient.2183@ohm> References: <20060215055011.GA46861@prometheusresearch.com> <20060215133903.20077.181425073.divmod.quotient.2183@ohm> Message-ID: On 2/15/06, Jean-Paul Calderone wrote: > HTTPS is orthogonal. Besides, how would you support it in the stdlib? It's currently not possible to write an SSL server in Python without a third-party library. Maybe someone would be interested in rectifying /that/? :) Yes, why do't you volunteer? Theer's absolutely no fundamental reason why we can't support server-side SSL when we do support client-side SSL in httplib.py, except that someone has to write the code -- the same OpenSSL library can be used. Well, I can see one *practical* reason -- server-side SSL use probably requires a lot more OpenSSL APIs to be exposed. Also, managing an SSL server *securely* requires much care. So perhaps having to use an external library (pyopenssl or m2crypto) could be appropriate. But it's nothing fundamental. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Feb 15 19:26:41 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 15 Feb 2006 10:26:41 -0800 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <20060215055011.GA46861@prometheusresearch.com> References: <43EF1E8E.3080804@xhaus.com> <43EF2BBE.50705@xhaus.com> <43F0FFA0.6010005@colorstudy.com> <43F230E2.1090101@xhaus.com> <20060215055011.GA46861@prometheusresearch.com> Message-ID: On 2/14/06, Clark C. Evans wrote: > On Mon, Feb 13, 2006 at 12:49:00PM -0800, Guido van Rossum wrote: > | There are many different ways to judge "production quality". If we're > | talking about correct, (standards-compliant, even) code, I wholly > | agree. > > Fantastic. I just don't think it is appropriate to have a "toy" > in the standard library. So we disagree fundamentally -- IMO sometimes a toy is right for the standard library, for example when anything considered a non-toy would have vast reams of platform-specific code, or when the design space for non-toys is simply too vast to pick just one solution. The standard library should only have one of each thing it supports. Example: I doubt that the standard library will ever contain a non-toy GUI library or a non-toy web framework. Developing one of these is a huge effort, and there are serious trade-offs in choosing one. For example, professionals disagree on whether Django, Cheetah or Myghty is the best templating language. Until one has "won" (an unlikely outcome) none of these should be promoted to standard library status, because it would do a disservice to those users for whom the chosen one is the wrong choice. So instead, the standard library gets something like string.Template, which is markedly less powerful, but also less controversial. > On Tue, Feb 14, 2006 at 12:00:57PM -0800, Guido van Rossum wrote: > | Let's make it so. I propose to add wsgiref to the standard library and > | nothing more. > > I propose we add wsgiref, but look at other implementations and > steal what ever you can from them. This is not a huge chunk of > code -- no reason why you can't have the best combination of > features and correctness. But it would need to be done *before* it is submitted to the standard library. What you propose sounds like a big task, while what I'm proposing is a simple matter of slightly cleaning up a few files and checkin them in. Also, "stealing whatever you can" might easily be considered a license for feature bloat, which would be unpythonic. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From cce at clarkevans.com Wed Feb 15 20:12:28 2006 From: cce at clarkevans.com (Clark C. Evans) Date: Wed, 15 Feb 2006 14:12:28 -0500 Subject: [Web-SIG] WSGI in standard library In-Reply-To: References: <43EF1E8E.3080804@xhaus.com> <43EF2BBE.50705@xhaus.com> <43F0FFA0.6010005@colorstudy.com> <43F230E2.1090101@xhaus.com> <20060215055011.GA46861@prometheusresearch.com> Message-ID: <20060215191228.GA53436@prometheusresearch.com> On Wed, Feb 15, 2006 at 10:26:41AM -0800, Guido van Rossum wrote: | On 2/14/06, Clark C. Evans wrote: | > On Mon, Feb 13, 2006 at 12:49:00PM -0800, Guido van Rossum wrote: | > | There are many different ways to judge "production quality". If we're | > | talking about correct, (standards-compliant, even) code, I wholly | > | agree. | > | > Fantastic. I just don't think it is appropriate to have a "toy" | > in the standard library. | | So we disagree fundamentally -- IMO sometimes a toy is right for the | standard library I'm seriously surprised to hear this. What other standard library items are "toys"? If you really think that WSGI and its implementation are a "toy", then let's put it in the Python documentation -- not in the standard library. | > On Tue, Feb 14, 2006 at 12:00:57PM -0800, Guido van Rossum wrote: | > | Let's make it so. I propose to add wsgiref to the standard library and | > | nothing more. | > | > I propose we add wsgiref, but look at other implementations and | > steal what ever you can from them. This is not a huge chunk of | > code -- no reason why you can't have the best combination of | > features and correctness. | | But it would need to be done *before* it is submitted to the standard | library. Yes. | What you propose sounds like a big task, while what I'm | proposing is a simple matter of slightly cleaning up a few files and | checkin them in. I disagree. Given the amount of effort gone into WSGI and the amount of experience/expertise accumulated, I think it would be foolish to incorporate any single implementation without seriously considering the impact of competing implementations. | Also, "stealing whatever you can" might easily be | considered a license for feature bloat, which would be unpythonic. I think you're going off the rails here Guido. There is nothing preventing a SVN repository for a week or so; with people experienced with real-live WSGI deployments to work on a quality (but minimal) module for Python's standard library. If you're worried about "feature bloat" have you even looked at the 190 line headers.py? I rest my case. I'm talking about a Sufficient _and_ Necessary implementaion of WSGI, I'm more than happy to help, and I'm more than happy to leave final discretion to Phillip Eby as he has demonstrated a serious command over the problem domain. Kind Regards, Clark From guido at python.org Wed Feb 15 20:27:01 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 15 Feb 2006 11:27:01 -0800 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <20060215191228.GA53436@prometheusresearch.com> References: <43EF1E8E.3080804@xhaus.com> <43EF2BBE.50705@xhaus.com> <43F0FFA0.6010005@colorstudy.com> <43F230E2.1090101@xhaus.com> <20060215055011.GA46861@prometheusresearch.com> <20060215191228.GA53436@prometheusresearch.com> Message-ID: On 2/15/06, Clark C. Evans wrote: > On Wed, Feb 15, 2006 at 10:26:41AM -0800, Guido van Rossum wrote: > | So we disagree fundamentally -- IMO sometimes a toy is right for the > | standard library > > I'm seriously surprised to hear this. What other standard library items > are "toys"? BaseHTTPServer and its subclasses. Probably even SocketServer -- a non-toy version would be something like Twisted. > If you really think that WSGI and its implementation are a > "toy", then let's put it in the Python documentation -- not in the > standard library. Maybe we have different definitions of toys? Working toys are quite useful -- I test most of my servers with BaseHTTPServer, but I would never dream of deploying that way. > I disagree. Given the amount of effort gone into WSGI and the amount > of experience/expertise accumulated, I think it would be foolish to > incorporate any single implementation without seriously considering > the impact of competing implementations. I think we have exhausted the amount of disagreement. Let's just agree we disagree. It would be more productive if you started discussing *specific* cases where you think wsgiref can be improved upon by borrowing code/features/ideas from other WSGI servers without damaging its "as simple as possible but no simpler" approach. > | Also, "stealing whatever you can" might easily be > | considered a license for feature bloat, which would be unpythonic. > > I think you're going off the rails here Guido. There is nothing > preventing a SVN repository for a week or so; with people experienced > with real-live WSGI deployments to work on a quality (but minimal) > module for Python's standard library. If you're worried about "feature > bloat" have you even looked at the 190 line headers.py? I rest my case. What case? It's totally reasonable code implementing a fairly complete mapping API. Smells similar to rfc822.py actually. > I'm talking about a Sufficient _and_ Necessary implementaion of WSGI, > I'm more than happy to help, and I'm more than happy to leave final > discretion to Phillip Eby as he has demonstrated a serious command > over the problem domain. If there's anything in wsgiref you think should be changed, please be specific. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From cce at clarkevans.com Wed Feb 15 21:17:42 2006 From: cce at clarkevans.com (Clark C. Evans) Date: Wed, 15 Feb 2006 15:17:42 -0500 Subject: [Web-SIG] WSGI in standard library In-Reply-To: References: <43EF1E8E.3080804@xhaus.com> <43EF2BBE.50705@xhaus.com> <43F0FFA0.6010005@colorstudy.com> <43F230E2.1090101@xhaus.com> <20060215055011.GA46861@prometheusresearch.com> <20060215191228.GA53436@prometheusresearch.com> Message-ID: <20060215201742.GB54188@prometheusresearch.com> On Wed, Feb 15, 2006 at 11:27:01AM -0800, Guido van Rossum wrote: | > If you're worried about "featurebloat" have you even looked at | > the 190 line headers.py? I rest my case. | | What case? It's totally reasonable code implementing a fairly complete | mapping API. Smells similar to rfc822.py actually. Then perhaps this should be included in the ``list`` or ``dict`` interface. If it is WSGI specific, then it's "feature bloat". I have a very large WSGI application, and I've not seen the need for this module even once. While I don't mind it, it is unnecessary baggage: it is not required by the specification and does little to help its usage. | > I'm talking about a Sufficient _and_ Necessary implementation of WSGI, | > I'm more than happy to help, and I'm more than happy to leave final | > discretion to Phillip Eby as he has demonstrated a serious command | > over the problem domain. | | If there's anything in wsgiref you think should be changed, please | be specific. * __init__.py should be updated to remove future plans such as validate, cgi_gateway, cgi_wrapper and router * wsgiref/handlers should be removed in its entirety; it adds little to the reference implementation * as I've stated above, wsgiref/headers should be re-factored into a shared library; or dropped altogether -- it really has very little to do with WSGI * wsgiref/simple_server should be re-written as mix-ins, allowing for implementations other than BaseHTTPRequestHandler and HTTPServer; a demonstration using SSL should be provided in documentation string or its equivalent * wsgiref/simple_serve/demo_app and the __name__ == '__main__' test suite should be moved into documentation and not be included in the standard distribution * either the WSGI specification should be amended to not support (not talk about?) 100-continue and chucked-encoding, or the implementation should provide a valid interpretation of the requirements specified * some attempt at handling connection-dropping clients should be done in either HTTPServer or at a higher level in WSGI; I don't care which * since this is a *DEMO* implementation, it should be heavily commented... otherwise it fails its duty of being instructive Kind Regards, Clark From ianb at colorstudy.com Wed Feb 15 21:52:15 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 15 Feb 2006 14:52:15 -0600 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <20060215201742.GB54188@prometheusresearch.com> References: <43EF1E8E.3080804@xhaus.com> <43EF2BBE.50705@xhaus.com> <43F0FFA0.6010005@colorstudy.com> <43F230E2.1090101@xhaus.com> <20060215055011.GA46861@prometheusresearch.com> <20060215191228.GA53436@prometheusresearch.com> <20060215201742.GB54188@prometheusresearch.com> Message-ID: <43F3947F.5040102@colorstudy.com> Clark C. Evans wrote: > On Wed, Feb 15, 2006 at 11:27:01AM -0800, Guido van Rossum wrote: > | > If you're worried about "featurebloat" have you even looked at > | > the 190 line headers.py? I rest my case. > | > | What case? It's totally reasonable code implementing a fairly complete > | mapping API. Smells similar to rfc822.py actually. > > Then perhaps this should be included in the ``list`` or ``dict`` > interface. If it is WSGI specific, then it's "feature bloat". I have a > very large WSGI application, and I've not seen the need for this module > even once. While I don't mind it, it is unnecessary baggage: it is not > required by the specification and does little to help its usage. I've used this several times (well, not wsgiref's implementation, but paste.response.HeaderDict). rfc822 is heavier than this dictionary-like object, and apparently is also deprecated. Too bad, the email module is much heavier still. Anyway, I'm +1 on the object going somewhere. I don't know if the parent package has to be named "wsgi" -- and "wsgiref" seems even stranger to me, as anything in the standard library isn't a "reference implementation" anymore, but an actual implementation. I personally like a package name like "web". Everyone will know what that means (though it would start with most of the web related modules not in it, which is a problem). If it was refactored a little, a similar object could be used for URL parameters, which are ordered/multivalue but case sensitive. Maybe something to consider. OTOH, maybe it should just be a similar object. And maybe both should use UserDict.DictMixin; much simpler then. Oh, and I don't like that __getitem__() and .get() are the same. __getitem__ should raise KeyError. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From pywebsig at alan.kennedy.name Thu Feb 16 11:45:32 2006 From: pywebsig at alan.kennedy.name (Alan Kennedy) Date: Thu, 16 Feb 2006 10:45:32 +0000 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <43F3947F.5040102@colorstudy.com> References: <43EF1E8E.3080804@xhaus.com> <43F0FFA0.6010005@colorstudy.com> <43F230E2.1090101@xhaus.com> <20060215055011.GA46861@prometheusresearch.com> <20060215191228.GA53436@prometheusresearch.com> <20060215201742.GB54188@prometheusresearch.com> <43F3947F.5040102@colorstudy.com> Message-ID: <4a951aa00602160245g7815180bn249c9d0718d660d@mail.gmail.com> [Ian Bicking] > Anyway, I'm +1 on the object [wsgiref's wsgi header manipulation class] > going somewhere. I don't know if the > parent package has to be named "wsgi" -- and "wsgiref" seems even > stranger to me, as anything in the standard library isn't a "reference > implementation" anymore, but an actual implementation. I personally > like a package name like "web". Everyone will know what that means > (though it would start with most of the web related modules not in it, > which is a problem). While we're on the subject, can we find a better home for the HTTP status codes->messages mapping? Integer status codes. http://mail.python.org/pipermail/web-sig/2004-September/000764.html Adding status code constants to httplib http://mail.python.org/pipermail/web-sig/2004-September/000842.html Regards, Alan. From pywebsig at alan.kennedy.name Thu Feb 16 13:16:04 2006 From: pywebsig at alan.kennedy.name (Alan Kennedy) Date: Thu, 16 Feb 2006 12:16:04 +0000 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <4a951aa00602160245g7815180bn249c9d0718d660d@mail.gmail.com> References: <43EF1E8E.3080804@xhaus.com> <43F230E2.1090101@xhaus.com> <20060215055011.GA46861@prometheusresearch.com> <20060215191228.GA53436@prometheusresearch.com> <20060215201742.GB54188@prometheusresearch.com> <43F3947F.5040102@colorstudy.com> <4a951aa00602160245g7815180bn249c9d0718d660d@mail.gmail.com> Message-ID: <4a951aa00602160416i1cb05c26s254b796d748b324e@mail.gmail.com> [Guido Van Rossum] > Actually BaseHTTPServer.py and friends use a deprecated naming scheme > -- just as StringIO, UserDict and many other fine standard library > modules. > If you read PEP 8, the current best practice is for module names to be > all-lowercase and *different* from the class name. [Clark C Evans] > I propose we add wsgiref, but look at other implementations and > steal what ever you can from them. This is not a huge chunk of > code -- no reason why you can't have the best combination of > features and correctness. [Jean Paul Calderone] > HTTPS is orthogonal. Besides, how would you support it in the stdlib? It's currently not > possible to write an SSL server in Python without a third-party library. Maybe someone > would be interested in rectifying /that/? :) [Ian Bicking] > I've used this several times (well, not wsgiref's implementation, but > paste.response.HeaderDict). rfc822 is heavier than this dictionary-like > object, and apparently is also deprecated. [Alan Kennedy] > While we're on the subject, can we find a better home for the HTTP > status codes->messages mapping? Folks, Thinking about this some more, it's beginning to sound to me like the server-side web support in the standard library needs a proper review and possible rework: it's slowly decohering/kipplizing. Maybe we need a PEP, so that we can all discuss the subject (rationally ;-) and sort out all of the issues before we go ahead and commit anything? Just a thought. Feel free to dis-regard Alan. From janssen at parc.com Thu Feb 16 17:30:13 2006 From: janssen at parc.com (Bill Janssen) Date: Thu, 16 Feb 2006 08:30:13 PST Subject: [Web-SIG] WSGI in standard library In-Reply-To: Your message of "Thu, 16 Feb 2006 04:16:04 PST." <4a951aa00602160416i1cb05c26s254b796d748b324e@mail.gmail.com> Message-ID: <06Feb16.083019pst."58633"@synergy1.parc.xerox.com> > Thinking about this some more, it's beginning to sound to me like the > server-side web support in the standard library needs a proper review > and possible rework: it's slowly decohering/kipplizing. > > Maybe we need a PEP, so that we can all discuss the subject > (rationally ;-) and sort out all of the issues before we go ahead and > commit anything? Great idea! That's exactly what I thought when I organized this SIG a couple of years ago. Bill From guido at python.org Thu Feb 16 20:05:47 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 16 Feb 2006 11:05:47 -0800 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <6836180321904295534@unknownmsgid> References: <4a951aa00602160416i1cb05c26s254b796d748b324e@mail.gmail.com> <6836180321904295534@unknownmsgid> Message-ID: At first I was going to respond "+1". But the fact that a couple of years haven't led to much suggests that it's unlikely to be fruitful; there just are too many diverging ideas on what is right. (Which makes sense since it's a huge and fast developing field.) So unless someone (Alan Kennedy?) actually puts forward a PEP and gets it through a review of the major players on web-sig, I'm skeptical. I certainly don't want this potential effort to keep us from adding the low-hanging fruit (wsgiref, with perhaps some tweaks as PJE can manage based on recent feedback here) to the 2.5 stdlib. --Guido On 2/16/06, Bill Janssen wrote: > > Thinking about this some more, it's beginning to sound to me like the > > server-side web support in the standard library needs a proper review > > and possible rework: it's slowly decohering/kipplizing. > > > > Maybe we need a PEP, so that we can all discuss the subject > > (rationally ;-) and sort out all of the issues before we go ahead and > > commit anything? > > Great idea! That's exactly what I thought when I organized this SIG a > couple of years ago. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Feb 16 20:06:17 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 16 Feb 2006 11:06:17 -0800 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <4a951aa00602160245g7815180bn249c9d0718d660d@mail.gmail.com> References: <43EF1E8E.3080804@xhaus.com> <43F230E2.1090101@xhaus.com> <20060215055011.GA46861@prometheusresearch.com> <20060215191228.GA53436@prometheusresearch.com> <20060215201742.GB54188@prometheusresearch.com> <43F3947F.5040102@colorstudy.com> <4a951aa00602160245g7815180bn249c9d0718d660d@mail.gmail.com> Message-ID: On 2/16/06, Alan Kennedy wrote: > [Ian Bicking] > > Anyway, I'm +1 on the object [wsgiref's wsgi header manipulation class] > > going somewhere. I don't know if the > > parent package has to be named "wsgi" -- and "wsgiref" seems even > > stranger to me, as anything in the standard library isn't a "reference > > implementation" anymore, but an actual implementation. I personally > > like a package name like "web". Everyone will know what that means > > (though it would start with most of the web related modules not in it, > > which is a problem). > > While we're on the subject, can we find a better home for the HTTP > status codes->messages mapping? > > Integer status codes. > http://mail.python.org/pipermail/web-sig/2004-September/000764.html > > Adding status code constants to httplib > http://mail.python.org/pipermail/web-sig/2004-September/000842.html +1. Feel free to submit a patch to SF and assign it to me. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From wilk-ml at flibuste.net Thu Feb 16 20:41:14 2006 From: wilk-ml at flibuste.net (William Dode) Date: Thu, 16 Feb 2006 19:41:14 +0000 (UTC) Subject: [Web-SIG] middleware in stantard library ? Message-ID: Hi, In the same goal as wsgiserver, what do you think to add two middleware to can really make quickly a meta-framework and "play" immediatly with it without any external dependency. I think about cookies and session. bye -- William Dod? - http://flibuste.net From ianb at colorstudy.com Thu Feb 16 21:08:02 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 16 Feb 2006 14:08:02 -0600 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <4a951aa00602160416i1cb05c26s254b796d748b324e@mail.gmail.com> References: <43EF1E8E.3080804@xhaus.com> <43F230E2.1090101@xhaus.com> <20060215055011.GA46861@prometheusresearch.com> <20060215191228.GA53436@prometheusresearch.com> <20060215201742.GB54188@prometheusresearch.com> <43F3947F.5040102@colorstudy.com> <4a951aa00602160245g7815180bn249c9d0718d660d@mail.gmail.com> <4a951aa00602160416i1cb05c26s254b796d748b324e@mail.gmail.com> Message-ID: <43F4DBA2.8090605@colorstudy.com> Alan Kennedy wrote: > Thinking about this some more, it's beginning to sound to me like the > server-side web support in the standard library needs a proper review > and possible rework: it's slowly decohering/kipplizing. > > Maybe we need a PEP, so that we can all discuss the subject > (rationally ;-) and sort out all of the issues before we go ahead and > commit anything? What would the PEP(s) cover? (Anyone can answer that, of course) I can imagine several intersecting PEPs. E.g., a process PEP about consolidating web-related modules in the stdlib. Or a process PEP about what we should look for in stdlib modules. Or, I can imagine an implementation-focused PEP on several of the things discussed. Which things? -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From ianb at colorstudy.com Thu Feb 16 23:46:54 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 16 Feb 2006 16:46:54 -0600 Subject: [Web-SIG] WSGI in standard library In-Reply-To: References: <4a951aa00602160416i1cb05c26s254b796d748b324e@mail.gmail.com> <6836180321904295534@unknownmsgid> Message-ID: <43F500DE.3080506@colorstudy.com> Guido van Rossum wrote: > At first I was going to respond "+1". But the fact that a couple of > years haven't led to much suggests that it's unlikely to be fruitful; > there just are too many diverging ideas on what is right. (Which makes > sense since it's a huge and fast developing field.) > > So unless someone (Alan Kennedy?) actually puts forward a PEP and gets > it through a review of the major players on web-sig, I'm skeptical. I > certainly don't want this potential effort to keep us from adding the > low-hanging fruit (wsgiref, with perhaps some tweaks as PJE can manage > based on recent feedback here) to the 2.5 stdlib. What would we be PEPing? wsgiref, for instance, could get a PEP. Should it have one? If there was an HTTP server beyond what is in wsgiref now, should that have a PEP? I one point I thought it would be too hard to come to agreement, but these days I think a request object would now be easy to define, with a constructor taking a WSGI environment, and just a couple attributes (params, cookies, environ, maybe headers, method, body). In the standard library that could provide some utility, but since it would be strictly derivative of the WSGI environment it would not detract from anyone else's request object. The request object would be limited to the information present in the WSGI environment, which simplifies the design process substantially. I would not propose a response object. I think a basic routing middleware (prefix-based URL match) would have some utility in concert with the HTTP server, both functional and educational. Would people be interested in that? Some current servers could be implemented as WSGI applications: CGIHTTPServer, SimpleHTTPServer, SimpleXMLRPCServer, DocXMLRPCServer. In each case the result would be substantially more useful than it is current, IMHO. But it would also not be my first choice of functionality to implement, it's only that these parts are already in the standard library. Some simple method to simulate a WSGI call would also be useful. If other WSGI apps and middleware were included anywhere, I'd want to at least include such a simulation in the tests. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From janssen at parc.com Fri Feb 17 01:28:34 2006 From: janssen at parc.com (Bill Janssen) Date: Thu, 16 Feb 2006 16:28:34 PST Subject: [Web-SIG] WSGI in standard library In-Reply-To: Your message of "Thu, 16 Feb 2006 14:46:54 PST." <43F500DE.3080506@colorstudy.com> Message-ID: <06Feb16.162834pst."58633"@synergy1.parc.xerox.com> > What would we be PEPing? Well, when we started, I made up a list of various things that seem to be missing in the standard library, like server-side support for SSL-encrypted socket connections. http://www.parc.com/janssen/web-sig/needed.html Bill From ianb at colorstudy.com Fri Feb 17 01:55:55 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 16 Feb 2006 18:55:55 -0600 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <06Feb16.162834pst."58633"@synergy1.parc.xerox.com> References: <06Feb16.162834pst."58633"@synergy1.parc.xerox.com> Message-ID: <43F51F1B.4010005@colorstudy.com> Bill Janssen wrote: >>What would we be PEPing? > > > Well, when we started, I made up a list of various things that seem to > be missing in the standard library, like server-side support for > SSL-encrypted socket connections. > > http://www.parc.com/janssen/web-sig/needed.html Here's my thoughts on those: - Real HTML parser Really hard. Current state-of-the-art is maybe Tidy (sloppy), Beautiful Soup (precise, but still not a tree structure), and some parsers embedded in some template languages (ZPT and meld3 -- precise and create a tree, but require quite well formed HTML). If the HTML 5 spec works out, I expect a quality parser can be written, but until that happens it's just really hard. - CSS Parser I think John Lee has done some work on this? Beats me. I've never felt any need for CSS parsing personally. - post-multipart You mean like posting a file upload? I suppose current interfaces don't stop you, but they don't make it easy. The email module will probably help a lot with this. Maybe it could go in urllib, next to urlencode. - Asynchronous fetch Not sure how this would work. What kind of async networking can we do in a cross-platform manner? - Connection caching Seems somewhat complex. I'm not sure it is appropriate for the standard library. Maybe httplib2 or another such project could take this on, but it doesn't seem like a standard library task. - Traffic monitor Eh. Clark wrote an upload monitor as WSGI middleware, so that's a kind of proof of concept. I would consider this doable enough with WSGI, and it doesn't need any special support in the stdlib. On the client side I really don't have a clear idea of the use case. - Server-side SSL support What is the current state of this? It's mostly there, isn't it? - A standard server framework on the order of Medusa Well, we have BaseHTTPServer, and now a WSGI based server. - A standard interface to request data We have this with WSGI. It's not a pretty request object, but it's standardized. I think there's room for a standard object-like interface to the WSGI environment (not comprehensive, but it doesn't need to be). - Improved support for authentication I haven't tried doing this in a long time, I'm not sure what the status is. - Explicit cookie handling I guess I brought this one up? I'm not sure what I mean ;) - Form value validation I believe this is rather difficult, but not impossible. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From janssen at parc.com Fri Feb 17 02:11:19 2006 From: janssen at parc.com (Bill Janssen) Date: Thu, 16 Feb 2006 17:11:19 PST Subject: [Web-SIG] WSGI in standard library In-Reply-To: Your message of "Thu, 16 Feb 2006 16:55:55 PST." <43F51F1B.4010005@colorstudy.com> Message-ID: <06Feb16.171124pst."58633"@synergy1.parc.xerox.com> > - CSS Parser > > I think John Lee has done some work on this? Beats me. I've never > felt any need for CSS parsing personally. > If you are doing any work with the Web (spidering, for instance), and need to do rendering of the Web pages (say, for pop-out prism, or building an ebook from it), a CSS parser is pretty much necessary. I tend to think that you really can't understand the Web programmatically without a CSS parser. An ECMAscript interpreter would be a big help, too. > - Asynchronous fetch > > Not sure how this would work. What kind of async networking can we > do in a cross-platform manner? I ran into this building the Plucker web spider. You need to be able to issue requests for web pages (or other resources) and come back from time to time to see if they had loaded yet. I think the addition of generators might be useful here. > - Connection caching > > Seems somewhat complex. I'm not sure it is appropriate for the > standard library. Maybe httplib2 or another such project could take > this on, but it doesn't seem like a standard library task. My philosophy of the standard library is that it should keep working when you lean on it, not break and force you to find something else when you lean on it. Apparently (and oddly) that seems to be a somewhat unusual way of thinking about it. > - Server-side SSL support > > What is the current state of this? It's mostly there, isn't it? Not there at all, as far as I know. The standard library only supports client-side SSL. > - A standard interface to request data > > We have this with WSGI. It's not a pretty request object, but it's > standardized. I think there's room for a standard object-like interface > to the WSGI environment (not comprehensive, but it doesn't need to be). Yes, I think this is covered by WSGI. > - A standard server framework on the order of Medusa > > Well, we have BaseHTTPServer, and now a WSGI based server. So, does either work as well as Medusa? Surely BaseHTTPServer doesn't. > - Explicit cookie handling > > I guess I brought this one up? I'm not sure what I mean ;) I had to do some client-side cookie handling lately... There are some missing pieces, like cookie databases that automatically sync with the user's browser cookie repositories. I don't believe there's any support for server-side cookie handling either. Bill From ianb at colorstudy.com Fri Feb 17 03:02:59 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 16 Feb 2006 20:02:59 -0600 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <06Feb16.171124pst."58633"@synergy1.parc.xerox.com> References: <06Feb16.171124pst."58633"@synergy1.parc.xerox.com> Message-ID: <43F52ED3.9070604@colorstudy.com> Bill Janssen wrote: >> I think John Lee has done some work on this? Beats me. I've never >>felt any need for CSS parsing personally. >> > > > If you are doing any work with the Web (spidering, for instance), and > need to do rendering of the Web pages (say, for pop-out prism, or > building an ebook from it), a CSS parser is pretty much necessary. I > tend to think that you really can't understand the Web > programmatically without a CSS parser. > > An ECMAscript interpreter would be a big help, too. Yes, but what is the advantage of having these in the standard library? These seem fairly niche, in that not that many people actually do this kind of work; isn't it better to simply organize an effort among the interested parties? That said, I think it would be perhaps an appropriate middleground for the standard library documentation to refer to specific projects. >>- Asynchronous fetch >> >> Not sure how this would work. What kind of async networking can we >>do in a cross-platform manner? > > > I ran into this building the Plucker web spider. You need to be able > to issue requests for web pages (or other resources) and come back > from time to time to see if they had loaded yet. I think the addition > of generators might be useful here. Yes, but there's still the cross-platform asynchronous socket handling issues. These seem to require fiddling and details that can't be responded to well in the standard library. Or, at least, this has to be built on a lower-level network layer that (AFAIK) does not exist in the standard library. >>- Connection caching >> >> Seems somewhat complex. I'm not sure it is appropriate for the >>standard library. Maybe httplib2 or another such project could take >>this on, but it doesn't seem like a standard library task. > > > My philosophy of the standard library is that it should keep working > when you lean on it, not break and force you to find something else > when you lean on it. Apparently (and oddly) that seems to be a > somewhat unusual way of thinking about it. The standard library only gets substantial updates every year or two, and there's no real user feedback process (except predictive in the case of a PEP, or by way of using an already mature library). >>- Server-side SSL support >> >> What is the current state of this? It's mostly there, isn't it? > > > Not there at all, as far as I know. The standard library only > supports client-side SSL. I suppose the problem is the certificate management, or something along those lines? It's not that hard to do as it is, and I believe Clark has specifically mentioned what he would like so that HTTPS would be fairly easy to apply to any WSGI server (even if it may not be built in). >>- A standard server framework on the order of Medusa >> >> Well, we have BaseHTTPServer, and now a WSGI based server. > > > So, does either work as well as Medusa? Surely BaseHTTPServer doesn't. I dunno. I'm more concerned that such a server have the same interface as a better server could have. WSGI alone may be enough of a standard there. BaseHTTPServer has been very successful at getting people doing things very quickly, in a way that is satisfying and motivating. It did that without being a really robust web server. I think it's okay that people have to move on to other software at a certain point, so long as it is not jarring, and the stdlib interface is not so hopelessly naive that any "real" alternative feels very different. I don't think the real alternative(s) to BaseHTTPServer have to feel that different. >>- Explicit cookie handling >> >> I guess I brought this one up? I'm not sure what I mean ;) > > > I had to do some client-side cookie handling lately... There are some > missing pieces, like cookie databases that automatically sync with the > user's browser cookie repositories. I think cookielib.CookieJar gives you your database, and reads browser databases. > I don't believe there's any support for server-side cookie handling either. On the server side the Cookie module is crude but workable. If I was to make a request object with a .cookies attribute, it would use but not expose any part of that module, since the cookies the server recieves can be fully described by a dictionary (unlike the cookies that a server sends). -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From pywebsig at xhaus.com Sun Feb 19 21:39:10 2006 From: pywebsig at xhaus.com (Alan Kennedy) Date: Sun, 19 Feb 2006 20:39:10 +0000 Subject: [Web-SIG] WSGI in standard library In-Reply-To: References: <4a951aa00602160416i1cb05c26s254b796d748b324e@mail.gmail.com> <6836180321904295534@unknownmsgid> Message-ID: <43F8D76E.5010802@xhaus.com> [Alan Kennedy] >>>Maybe we need a PEP [Bill Janssen] >>Great idea! That's exactly what I thought when I organized this SIG a >>couple of years ago. [Guido van Rossum] > At first I was going to respond "+1". But the fact that a couple of > years haven't led to much suggests that it's unlikely to be fruitful; > there just are too many diverging ideas on what is right. (Which makes > sense since it's a huge and fast developing field.) Having considered the area for a couple of days, I think you're right: the generic concept "web", as in web-sig, covers far too much ground, and there are too many schools of thought. > So unless someone (Alan Kennedy?) actually puts forward a PEP and gets > it through a review of the major players on web-sig, I'm skeptical. But there is a subset which I think is achievable, namely http support, which IMO is the subset that most needs a rework. And now that we have a nice web standard, WSGI, it would be nice to make use of it to refactor the current http support. The following are important omissions in the current stdlib. - Asynchronous http client/server support (use asyncore? twisted?) - SSL support in threaded http servers - Asynchronous SSL support - Simple client file upload support - HTTP header parsing support, e.g. language codes, quality lists, etc - Simple object publishing framework? Addressing all of the above would be significant piece of work. And IMHO, it is only achievable by staying focussed on http and NOT addressing requirements such as - Content processing, e.g. html tidy, html parsing, css parsing - Foreign script language parsing or execution - Page templating API I think it would be a good idea to address these concerns in separate PEPs. [Guido van Rossum] > I certainly don't want this potential effort to keep us from adding > the low-hanging fruit (wsgiref, with perhaps some tweaks as PJE can > manage based on recent feedback here) to the 2.5 stdlib. Completely agreed. Any web-related PEPs are going to take a long time, and are unlikely to be ready in time for 2.5. Regards, Alan. From ianb at colorstudy.com Sun Feb 19 22:22:56 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 19 Feb 2006 15:22:56 -0600 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <43F8D76E.5010802@xhaus.com> References: <4a951aa00602160416i1cb05c26s254b796d748b324e@mail.gmail.com> <6836180321904295534@unknownmsgid> <43F8D76E.5010802@xhaus.com> Message-ID: <43F8E1B0.2000501@colorstudy.com> Alan Kennedy wrote: > [Alan Kennedy] > >>>Maybe we need a PEP > > [Bill Janssen] > >>Great idea! That's exactly what I thought when I organized this SIG a > >>couple of years ago. > > [Guido van Rossum] > > At first I was going to respond "+1". But the fact that a couple of > > years haven't led to much suggests that it's unlikely to be fruitful; > > there just are too many diverging ideas on what is right. (Which makes > > sense since it's a huge and fast developing field.) > > Having considered the area for a couple of days, I think you're right: > the generic concept "web", as in web-sig, covers far too much ground, > and there are too many schools of thought. > > > So unless someone (Alan Kennedy?) actually puts forward a PEP and gets > > it through a review of the major players on web-sig, I'm skeptical. > > But there is a subset which I think is achievable, namely http support, > which IMO is the subset that most needs a rework. And now that we have a > nice web standard, WSGI, it would be nice to make use of it to refactor > the current http support. The following are important omissions in the > current stdlib. > > - Asynchronous http client/server support (use asyncore? twisted?) > - SSL support in threaded http servers > - Asynchronous SSL support I lack any opinion on these. These don't matter much to me personally. > - Simple client file upload support The addition of another function like urllib.urlencode could do this, e.g., urllib.fileencode. This seems fairly straight-forward. > - HTTP header parsing support, e.g. language codes, quality lists, etc This would be very nice. Clark has done some work in paste.httpheaders for these; but I don't think any of us (Clark included) are really satisfied with how that module is layed out. Maybe it is because headers in general don't have that much in common with each other. I'm not sure. But nevertheless, it's all very mechanical, and based on existing and stable HTTP standards. So I think it is an excellent candidate for the standard library. > - Simple object publishing framework? That's harder. I think path-prefix based routing would be a nice addition to compliment any WSGI http server, but mostly because it lives above the level of any particular framework. There's small issues to any object publishing framework that make it a bad candidate for the standard library. > Addressing all of the above would be significant piece of work. And > IMHO, it is only achievable by staying focussed on http and NOT > addressing requirements such as > > - Content processing, e.g. html tidy, html parsing, css parsing Indeed; all implementations for content processing are flawed in one way or another, which I think implies no canonical is available, and so it's not a good candidate for the standard library. > - Foreign script language parsing or execution Like content but even harder. > - Page templating API Nothing stable and popular enough here to even consider. > I think it would be a good idea to address these concerns in separate PEPs. I don't think they all need PEPs. urllib.fileencode is pretty simple, for instance. > [Guido van Rossum] > > I certainly don't want this potential effort to keep us from adding > > the low-hanging fruit (wsgiref, with perhaps some tweaks as PJE can > > manage based on recent feedback here) to the 2.5 stdlib. > > Completely agreed. Any web-related PEPs are going to take a long time, > and are unlikely to be ready in time for 2.5. I don't think anything will work that requires coming up with implementations where there currently are none close to appropriate, or where we have to synthesize agreement from widely different frameworks. Maybe at sometime in the future -- beyond 2.5 -- the environment will change; an implementation will emerge, or some new consensus on best practice or some new refactorings of a problem will emerge. But lots of what we're talking about here isn't like that, so we shouldn't be pessimistic, just focused. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From foom at fuhm.net Sun Feb 19 22:33:49 2006 From: foom at fuhm.net (James Y Knight) Date: Sun, 19 Feb 2006 16:33:49 -0500 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <43F8E1B0.2000501@colorstudy.com> References: <4a951aa00602160416i1cb05c26s254b796d748b324e@mail.gmail.com> <6836180321904295534@unknownmsgid> <43F8D76E.5010802@xhaus.com> <43F8E1B0.2000501@colorstudy.com> Message-ID: <6575E296-FE36-4AFA-9BCC-D0AE199B6E32@fuhm.net> On Feb 19, 2006, at 4:22 PM, Ian Bicking wrote: >> - HTTP header parsing support, e.g. language codes, quality lists, >> etc > > This would be very nice. Clark has done some work in > paste.httpheaders > for these; but I don't think any of us (Clark included) are really > satisfied with how that module is layed out. Maybe it is because > headers in general don't have that much in common with each other. > I'm > not sure. > > But nevertheless, it's all very mechanical, and based on existing and > stable HTTP standards. So I think it is an excellent candidate for > the > standard library. I've done a bunch of work on this in twisted.web2, and it is pretty much completely separable from the rest of the server (it doesn't import any other twisted modules). I have parsers for almost all of the standard HTTP headers. There is a standard header format that many of the headers conform to. http://svn.twistedmatrix.com/cvs/trunk/twisted/web2/http_headers.py? view=auto James From cce at clarkevans.com Mon Feb 20 03:43:21 2006 From: cce at clarkevans.com (Clark C. Evans) Date: Sun, 19 Feb 2006 21:43:21 -0500 Subject: [Web-SIG] WSGI in standard library In-Reply-To: <6575E296-FE36-4AFA-9BCC-D0AE199B6E32@fuhm.net> References: <4a951aa00602160416i1cb05c26s254b796d748b324e@mail.gmail.com> <6836180321904295534@unknownmsgid> <43F8D76E.5010802@xhaus.com> <43F8E1B0.2000501@colorstudy.com> <6575E296-FE36-4AFA-9BCC-D0AE199B6E32@fuhm.net> Message-ID: <20060220024321.GA38394@prometheusresearch.com> On Sun, Feb 19, 2006 at 04:33:49PM -0500, James Y Knight wrote: | On Feb 19, 2006, at 4:22 PM, Ian Bicking wrote: | >> - HTTP header parsing support, e.g. language codes, quality lists, | >> etc | > | > This would be very nice. Clark has done some work in | > paste.httpheaders for these; but I don't think any of us | > (Clark included) are really satisfied with how that module | > is layed out. Maybe it is because headers in general don't | > have that much in common with each other. I'm not sure. | > | > But nevertheless, it's all very mechanical, and based on existing and | > stable HTTP standards. So I think it is an excellent candidate for | > the standard library. | | I've done a bunch of work on this in twisted.web2, and it is pretty | much completely separable from the rest of the server (it doesn't | import any other twisted modules). I have parsers for almost all of | the standard HTTP headers. There is a standard header format that | many of the headers conform to. | | http://svn.twistedmatrix.com/cvs/trunk/twisted/web2/http_headers.py? | view=auto Something like this would be excellent in the standard library; the sheer coverage of header parsing in your Twisted module is more complete than what I did (for example, your date parsing and cookie handling code). We have a solid overlap in the general pattern which points to the need of a standard python version of this stuff: For each header, there are two primary functions: a 'parse' function for reading headers, and a 'compose' function for generating them. The rest of the module is more or less implementation details and user-interface issues: - a class to store a collection of headers (found in your module and in wsgiref, but not in my implementation); - a mechanism to enumerate and list headers; both of our implementations do this; - specific objects for more complicated headers (ETag and Cookie) - your module provides this (mine doesn't); - providing for a mechanism to "extend" the header parsing mechanism so that header parse/compose pairs for custom or less common headers can be provided by "plug-in" libraries (I use objects, you use a dict of tuples); - nice doc strings (your module could use a bit of help in this regard - plus cross reference to the particular RFC for the given header) - handling the fundmental distinction between headers that have a single value, have multiple values as described by section 4.2 of RFC 2616, and headers which are not compliant with RFC 2616; and - integration /w WSGI; my module provides a uniform mechanism for accessing/setting headers in a WSGI ``environ`` dict or ``response_headers`` list. In any case, it probably wouldn't be too much effort (a few days?) to integrate the modules and document them for inclusion. Kind Regards, Clark From janssen at parc.com Mon Feb 20 05:39:16 2006 From: janssen at parc.com (Bill Janssen) Date: Sun, 19 Feb 2006 20:39:16 PST Subject: [Web-SIG] WSGI in standard library In-Reply-To: Your message of "Sun, 19 Feb 2006 12:39:10 PST." <43F8D76E.5010802@xhaus.com> Message-ID: <06Feb19.203919pst."58633"@synergy1.parc.xerox.com> Nice list, Alan. I think you make a lot of sense here. Bill > [Alan Kennedy] > >>>Maybe we need a PEP > > [Bill Janssen] > >>Great idea! That's exactly what I thought when I organized this SIG a > >>couple of years ago. > > [Guido van Rossum] > > At first I was going to respond "+1". But the fact that a couple of > > years haven't led to much suggests that it's unlikely to be fruitful; > > there just are too many diverging ideas on what is right. (Which makes > > sense since it's a huge and fast developing field.) > > Having considered the area for a couple of days, I think you're right: > the generic concept "web", as in web-sig, covers far too much ground, > and there are too many schools of thought. > > > So unless someone (Alan Kennedy?) actually puts forward a PEP and gets > > it through a review of the major players on web-sig, I'm skeptical. > > But there is a subset which I think is achievable, namely http support, > which IMO is the subset that most needs a rework. And now that we have a > nice web standard, WSGI, it would be nice to make use of it to refactor > the current http support. The following are important omissions in the > current stdlib. > > - Asynchronous http client/server support (use asyncore? twisted?) > - SSL support in threaded http servers > - Asynchronous SSL support > - Simple client file upload support > - HTTP header parsing support, e.g. language codes, quality lists, etc > - Simple object publishing framework? > > Addressing all of the above would be significant piece of work. And > IMHO, it is only achievable by staying focussed on http and NOT > addressing requirements such as > > - Content processing, e.g. html tidy, html parsing, css parsing > - Foreign script language parsing or execution > - Page templating API > > I think it would be a good idea to address these concerns in separate PEPs. > > [Guido van Rossum] > > I certainly don't want this potential effort to keep us from adding > > the low-hanging fruit (wsgiref, with perhaps some tweaks as PJE can > > manage based on recent feedback here) to the 2.5 stdlib. > > Completely agreed. Any web-related PEPs are going to take a long time, > and are unlikely to be ready in time for 2.5. > > Regards, > > Alan. > _______________________________________________ > Web-SIG mailing list > Web-SIG at python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: http://mail.python.org/mailman/options/web-sig/janssen%40parc.com From chad at zetaweb.com Mon Feb 27 18:07:52 2006 From: chad at zetaweb.com (Chad Whitacre) Date: Mon, 27 Feb 2006 12:07:52 -0500 Subject: [Web-SIG] httpy -- raise Response(409) In-Reply-To: <43FDF0EF.5030702@zetaweb.com> References: <43FDF0EF.5030702@zetaweb.com> Message-ID: Dear Web-SIG, First thing's first: > class Responder: > def respond(request): Yes, I forgot 'self'. Sorry. I rushed the announcement out 2.5 hrs before my plane to Dallas took off. BTW, Ian: take that as a +1 for your Python 4k, I guess. :^) Second thing's second: > raise httpy.Response(200, "Greetings, program!") From conversations this weekend (briefly with Jim; also with Guido) it seems that the most controversial aspect of httpy's API is the use of raise for flow control. I wonder if you all wouldn't mind helping me get to the bottom of this issue? (BTW, for background on the project, see here: http://www.zetadev.com/software/httpy/ ) I annoyed Guido at length about why he thought it a bad idea to end transactions with raise, and the answer was that it makes code less readable because you don't know whether to expect a subroutine (that someone else probably wrote) to raise a Response or not. I suggested that this could be documented, but that was thought not to be a strong enough assurance. (Accurate, Guido?) But in fact, isn't this how exceptions work in Python already? There is nothing formal in a function call that indicates what exceptions that call might raise. We rely on the documentation for that, no? I think the trick is that in HTTP, success and error conditions both produce the same result: a Response message. In Python we distinguish the two: successful "requests" (i.e., function calls) return something; bad requests raise something. So perhaps the answer is that respond(request) may *either* return *or* raise a Response? Guido: would that answer your objection? What about the further step of validating the response code for each case (you may only raise a non-2xx Response)? Thanks for the good times this weekend, and for any further thoughts anyone might have. chad From jonathan at carnageblender.com Mon Feb 27 19:08:34 2006 From: jonathan at carnageblender.com (Jonathan Ellis) Date: Mon, 27 Feb 2006 10:08:34 -0800 Subject: [Web-SIG] httpy -- raise Response(409) In-Reply-To: References: <43FDF0EF.5030702@zetaweb.com> Message-ID: <1141063714.4441.255382191@webmail.messagingengine.com> On Mon, 27 Feb 2006 12:07:52 -0500, "Chad Whitacre" said: > From conversations this weekend (briefly with Jim; also with Guido) it > seems that the most controversial aspect of httpy's API is the use of > raise for flow control. I wonder if you all wouldn't mind helping me get > to the bottom of this issue? I agree with the general sentiment that raise/except to pass Response objects around feels clunky. > I think the trick is that in HTTP, success and error conditions both > produce the same result: a Response message. In Python we distinguish > the two: successful "requests" (i.e., function calls) return something; > bad requests raise something. If the source of your unease is the handler returning both "success" and "error" responses itself, I'm more comfortable with the abstraction of a handler raising some NotFoundError, for instance, and translating that into a 404 response farther up the stack. (Spyce takes this approach because centralizing exception handling like this makes it easy to allow the user to plug in a custom function to handle them if necessary.) -Jonathan -- C++ is history repeated as tragedy. Java is history repeated as farce. --Scott McKay