[Web-SIG] Standardized template API

[Phillip J. Eby]

At 01:34 PM 2/3/2006 -0800, Ben Bangert wrote:
>On Feb 3, 2006, at 11:57 AM, Phillip J. Eby wrote:
>
>>I think maybe there's some confusion here as to what I'm suggesting
>>SCRIPT_NAME be set to.  I probably should've said that SCRIPT_NAME
>>should represent the "consumed" part of the URL, and PATH_INFO
>>should be empty unless there's some unconsumed path portion.  So if
>>the URL path is /comment/view/4, then SCRIPT_NAME should be / 
>>comment/view/4 appended to whatever the original SCRIPT_NAME was,
>>and PATH_INFO would be empty.
>
>What would be an example where PATH_INFO would not be empty?

When the "template" is responsible for the URL space underneath it.  This 
is more of a use case for "active page" frameworks, and for application 
embedding scenarios.


>>I'd like to preserve the normal WSGI invariants instead, because
>>this allows entire mini-applications with their own URL space to be
>>embedded as "templates" within a containing application.  Such apps
>>can then rely on SCRIPT_NAME as being their root, even if they
>>weren't originally written for embedding.
>
>Sure, again, I see no issue with that, though I'm sure many might
>wonder why you'd want to. :)

It lets you create mount points from one WSGI app/server into another 
app.  Like having a configuration file that Apache "serves" by treating the 
app's config file as a "template".


>>I don't see a use for including a template path anywhere, and that
>>really wasn't what I was suggesting.  Locating templates'
>>implementation data is a job for the enclosing framework, and it's
>>not really the template's business to know about that.
>
>Wait, then how does the template engine know what template to render?

The framework gives the engine the source of the template, and asks for a 
WSGI app object in return.  The framework is allowed to hold on to this 
application object and reuse it as often as desired.



>>The template engine wouldn't; the framework simply hands the engine
>>the "template" source (as a string or a stream) and gets back an
>>app object like the above.  See my "trivial counterproposal API"
>>for a summary of the engine interface.
>
>Ok, so in my case, I'd pass the Myghty template engine the string
>indicating the template I want rendered. Mainly because Myghty has
>its own powerful compiling/caching system for templates, I wouldn't
>want to try loading them myself and passing them as streams. But I
>see how this affords that flexibility should you want this.

Well, the idea I was trying to do here was to allow the embedder to control 
how the data is stored.  I think of this as a bit like OLE embedding, where 
the enclosing application just hands off a stream to the class that 
implements the embedded object or document.

What I'd suggest is that this is actually an opportunity for Myghty to put 
its lookup and caching on one side of the interface, and an actual template 
rendering facility on the other side.  That is, treat Myghty as an 
embedd*er* rather than the embedd*ee*, so that the engine can be used with 
arbitrary apps or templates on the embedded side.

I'm not really familiar with Myghty, though, so I might not be saying 
something sensible here.  I'm just saying that if you have a mechanism for 
locating, compiling, and caching templates, then it should live on the 
framework side of the "engine" interface.  An embedding engine offers the 
"compile_string, compile_stream, write_compiled, read_compiled" methods, 
and your locator/compiler/cache system would just call down to those 
instead of hardwiring the implementation of compiling and serializing.  You 
would of course have to add in a way to tell what embedding engine to 
select in that case, but now Myghty itself (if I understand you correctly) 
becomes a WSGI embedding host that can run any kind of template using its 
lookup/cache/etc. facilities.  Does that make sense?


>>So, we're basically solving the old "How do you deploy/configure a
>>WSGI app?" question, not by having a single file format for
>>deployment, but by allowing pluggable file formats, some of which
>>just happen to be "templates".
>>
>>Meanwhile, view-oriented frameworks like Zope 3 and peak.web can
>>basically embed entire applications or sub-applications wrapping
>>published objects, and even simple servers like the mod_python WSGI
>>server could serve templates and applications straight from the
>>file system.  And even controller-style frameworks that use
>>templates strictly for postprocessing will end up being able to
>>embed fairly sophisticated things.
>
>Ok, so.... I'm going to try and summarize to ensure that I have a
>firm grasp on the proposal.
>
>1) The template spec will use a WSGI call to ask the template engine
>to return a WSGI application (iterable) of the rendered template
>(resource).

No; what happens is that the framework locates the serialized form of the 
template it wants to use, and requests that a specific engine deserialize 
it to get a WSGI app (callable, not iterable).  The framework is allowed to 
cache the returned app object and reuse it for multiple requests.  The 
engine doesn't get any data about the current request.

The app object is used to "render" the template.  It's just called like a 
normal WSGI app object.


>2) The spec shall declare 2 environ keys to be used:
>     2a) wti.source  - either a stream or string which is the
>template source (source as in code, or as in the name of the template
>is up to the engine)

The idea of wti.source was that it would be either the "published object" 
or the "vars", depending on the kind of framework that was involved.  It's 
whatever is the subject of the request.  For "active page" style 
frameworks, there would be no object.  I don't think that source is the 
best name for this, though; it should probably be "target" or "param" or 
something like that.

Hell, maybe it shouldn't be just one key; maybe there should be a "params" 
key and a "self" key, such that a given framework can define one or 
both.  That way, the template implementations will know what they're 
dealing with.


>     2b) wsgi.application_root - shall be used to access the original
>SCRIPT_NAME if desired, ie, the SCRIPT_NAME of the application
>calling the template engine

Yeah, this needs to be nailed down a bit more clearly, because some 
frameworks don't have such a concept.  For example, in a pure "active page" 
model running under mod_python there is really no "calling application", so 
there would be no application_root set.


>That's it. For a template language like Myghty, it already has a WSGI
>handler that parses the template to render given the PATH_INFO, so
>I'd merely sub-class its WSGIHandler to have it use wti.source as the
>name of the template to render.

Hrm.  Well, it's a bit different than that, as you can see.  I guess we'll 
revisit this after you've read the above notes.


>Now, there's a few things this doesn't touch at all, that the
>TurboGears plug-in does address. Because wti.source is left open as
>either a string, or a stream, and the meaning of said string could be
>that its a string to be rendered as if it was a template, or it could
>be as I'm used it here, as the string representing what template
>Myghty should search for and render.

This actually isn't an issue; the engine interface has 'compile_string' and 
'compile_stream' methods, and wti.source has nothing to do with it.  See my 
earlier post at:

http://mail.python.org/pipermail/web-sig/2006-February/001933.html


>Though a small layer on top of the spec could handle it in whatever
>way it preferred to, definitely a lot of flexibility involved. A
>common question given the minimality of the spec as I'm summarizing
>would be, "How do I tell my framework to pass a dict of variables
>into the template so it can use them?"
>
>This would probably be best done as adding a key to the environ. So I
>propose in addition to the 2 keys:
>     2c) wti.vars - (optional) A dict representing variables you'd
>like access to in the template should it provide ways to make them
>available

Yeah, this is actually what the wti.source was supposed to be, as you see 
above.


>Anyways, hopefully this helps. Assuming I'm interpreting the WSGI as
>template spec thing properly, I think it'll work great, and provide a
>solid base for powerful template languages, as well as more minimal
>ones (they can use a small WSGI wrapper).

I think you'll find that the actual design is even better than the one you 
interpreted it as, so I think we're doing good.  :)

I also think I'm really going to need to carve out some time soon to write 
up a proper spec, or at least something a lot closer to one than the 
current tangled threads of explanation.  It's all pretty clear in my mind, 
but that's mainly because I have only a *vague* understanding of most 
frameworks, so I know how little I can rely on things working a particular 
way.  Thus, my design is such that it doesn't rely on much of anything 
about either the frameworks or templates; it just draws a sharp dividing 
line between three distinct sets of responsibilities: the embedding host, 
the embedded app, and the application factory.

These divisions may cut things at slightly different points than what 
people were originally thinking.  For example, it sounds from your 
description like Myghty might have parts in all three areas.  So, if it 
were to implement WSGI embedding, it would become componentized along those 
lines, making each part potentially interchangeable or reusable.

I appreciate your feedback and questions a lot, as they're helping me 
improve how I communicate the ideas.  Hopefully, I'll soon be able to 
explain it with examples and metaphors so that it doesn't take this much 
back-and-forth.  Despite Ian's fears, I'm not really worried that it's too 
complex; it's hugely simpler than WSGI itself, and is probably less complex 
than even the idea of WSGI middleware.  I am just not that good at 
*communicating* it simply yet.