[Web-SIG] A trivial template API counter-proposal

[Phillip J. Eby]

At 10:40 AM 2/5/2006 -0800, Guido van Rossum wrote:
>[Me]
> > >  Or static HTML pages that are written
> > >to the filesystem and then served by a classic Apache setup? Or source
> > >code (program generators are fun toys!)?
>
>[Phillip]
> > Use the output file's write() method as the "write" callback returned by
> > start_response(), or use one of the wsgiref handlers that you give
> > arbitrary file-like objects to.
>
>I didn't see an output file in the proposed standard. All I saw was a
>WSGI interface. The output file is a figment of your implementation.

You said you wanted to generate static HTML files in the file system; 
that's the "output file" I meant.


>It is my strong preference that the standard API we are attempting to
>design here does *not* tie you strongly to WSGI or the generation of
>dynamic web content.

That seems to be a popular position, to be sure.  However, since there is 
already a de facto standard support for that simpler use case, that is 
already supported by two frameworks for at least 6 templating systems(!) it 
seems that it's already both a solved problem and outside the scope of the 
Web-SIG.  You already *have* this standard API; why shouldn't the web-sig 
then focus on something that supports *web* templating, not just strings?


> > For ASP/PHP-style frameworks, it's routine for the "template" to make these
> > kinds of response-control decisions, and it's certainly seen a lot in Zope
> > 2 applications using DTML as well.  Even if only for backward
> > compatibility, it seems wrong to me to leave all these applications unable
> > to operate under the new standard.
>
>There's nothing wrong with having some things that can invoked from
>the template that set or add headers behind your back, if the template
>engine supports such a thing. But I think the API to let it do so
>might not be part of the standard, or it might be an optional part of
>the standard; I want to be able to use a templating engine in a
>non-web context and still use it (of course I won't be able to set
>headers then, but I'd assume this functionality is optional and I can
>just elect not to use it).

My goal here is to make it so that the standard encourages web frameworks 
to offer the WSGI capabilities to the template.  If this is not part of the 
standard, there is no encouragement, there are simply scattered 
non-standard APIs, if anything at all.

An example of why I'd want this: legacy code written in DTML and Python 
ASP.  If I'm migrating a bunch of it, I'd like the option to stick it in a 
new framework en masse, but actually port/convert one template at a time to 
newer approaches.  Any framework that uses only a string-based approach, 
isn't going to allow porting this code.

A WSGI interface makes it possible to have the template use its native web 
APIs.  One could create Zope request/response objects wrapping WSGI for use 
in the template, thus allowing the template to initially remain unchanged.


>I'm okay if the proposed standard doesn't provide access to every
>aspect of Zope 3 templates. If I were intereste in using the
>web-specific features of Zope3 templates, I should probably be using
>Zope 3.

And what if you're trying to port something away from Zope 3? or Zope 
2?  What if you are porting Python Server Pages *to* Zope 3?  DTML to 
TurboGears?  I'm trying to make it so that these kinds of embeddings are 
possible, so that we can start consolidating frameworks.

This is a Prisoner's Dilemna situation to some extent.  Nobody wants to do 
extra work, even though we all want the framework situation to 
improve.  Few people want *their* framework to be the one that goes away, 
either.


> > >and
> > >for that, an API that produces a string or writes to a stream seems to
> > >make more sense. What am I missing? Are you proposing that I should
> > >fake out the wsgi API and capture the strings passed to write() and
> > >the sequence returned?
> >
> > In that case, why don't you just use the template engine directly?
>
>Because every template engine has a @#$%! different API!!!!

There's a bunch now that use the TG/Buffet plugin API.  Why invent a *new* 
standard if all you want is strings?


> > But I
> > don't see how such an API is a *web* templating API in that
> > case.  Shouldn't such things just follow the stdlib string template
> > API?  Why do we need a *web* standard for string templating?
>
>I didn't ask for a *web* templating API; I suspect you've been reading
>too much into my request.

Well, it's being discussed on the Web-SIG.  Note that this discussion 
started before your web framework posts, as did the TG/Buffet template API, 
so it's not really "your" request that we're talking about here.


>But there *must* be a standard way yo invoke the
>template engine (context in, string or stream out) without assuming
>it's an HTTP response.

See http://www.turbogears.org/docs/plugins/template.html for the 
documentation on how to do this now.  Not to mention: 
http://projects.dowski.com/projects/buffet , which lists the 
currently-available plugins for Stan, Cheetah, Kid, string.Template, 
Myghty, and XSLT.


> > Currently, the progress of the discussion I've been having with Ben Bangert
> > is that we would split the templating API into two or three layers.  The
> > layer I'm describing now would be part of "WSGI Linking" and the kind of
> > API you're describing would be an optional extension to "WSGI
> > Embedding".  Since not all template systems can reasonably be used without
> > access to a request/response, it seems to me that the string-only facility
> > should be considered an optional extension.
>
>How can it be considered a templating system if it doesn't have access
>to a request/response?

But that's what you're proposing.  If there are only variables in and a 
string out, there is no request/response, and no ability to use the 
request/response API that would normally be used with that templating language.

With my approach, if you were using a DTML template plugin, you would get 
to use the REQUEST/RESPONSE objects you normally use, since the template 
engine would be wrapping the WSGI call with them.  This would make porting 
templates between frameworks *much* easier.


>  Surely somewhere in the bowels is an engine
>that renders a template in a context to a text stream. The rest ought
>to be considered part of the web framework that hots the templating
>engine. ISTM we may be confused (or simply disagreeing) about the
>boundary between the templating engine and the framework.

In my discussion with Ben Bangert and others to date, it's very clear that 
there is no common terminology or conceptual model as soon as you get 
outside of a relatively narrow range of frameworks.  The frameworks 
involved in the TG/Buffet standard have a model that looks like this:

    * framework calls some Python code you wrote
    * your code returns a dictionary of values you want rendered
    * framework passes your returned values to the template, gets back a string
    * framework sends the string back to the browser

There is *no* request/response API available from the template in this 
model, which is why the folks who use this model don't see a reason why 
anybody would want them.  Meanwhile, it seems like you're implicitly 
assuming that *of course* the template would have some kind of 
request/response, or maybe I'm misunderstanding you now.

The point is, we've barely explored this enough to understand what models 
are in use.  I think I have enough info now to draft a proposal that covers 
all the bases that have been brought up so far, but of course there's no 
guarantee that that's all the bases that exist.


> > While it means that there will be at least three different parts to the
> > spec, it will hopefully be clearer what different frameworks and templates
> > do and don't do with regard to using each other's templates or vice versa.
>
>Please keep coming up with proposals and rationales for them. I will
>certainly admit that I haven't used a Python web framework in a while
>(not since I left Zope :-). But when I see Cheetah and Django and some
>competitors that were mentioned in responses to my blogs, a certain
>"Platonic ideal" of a templating engine emerges in my brain, and I'd
>like at least the API for that standardardized, if at all possible, so
>people writing frameworks can choose from the many templating engines
>available, and people with templating needs *outside* a web context
>can have the same selection.

A non-web de facto standard already exists, and I would certainly encourage 
people to support it.  I think the *web* standard should include it or 
something very close to it as an optional extension.  I just view that part 
of the problem as mostly solved, but not very germane to actual template 
portability.

The problem that I see is that everybody seems to me to be saying, "real 
template portability is impossible", and when I say, "but just support WSGI 
and everything is there", they go, "but that's too hard".  It seems to me 
that this is a reflection of the a priori belief that it's impossible, 
rather than an actual reflection of the difficulty.  Most frameworks now 
already have code to turn WSGI into whatever their existing 
request/response API is.  And some frameworks already have WSGI passthrough 
capability to bring WSGI down to where they would need it to call a 
template using it.


> > The idea of course was to allow existing framework APIs to wrap WSGI, so
> > that you can have request.path or REQUEST.path_info or whatever suits your
> > fancy.  It was never intended that anybody but framework developers write
> > "bare metal" WSGI code, and such developers have to touch either CGI or
> > HTTP in their code anyway.
>
>Yeah, but potentially this causes several translations of the data
>(especially since WSGI requires a *real* dict, not something with
>mapping behavior) if the server's native API doesn't come in he form
>of CGI-compatible environment variables. This seems counter to the
>desire for it to be fast.

Note that using "WSGI all the way down" to the template level means you 
have at most two translations: HTTP->WSGI->API, and the API can be a lazy 
wrapper over WSGI.

However, being *fast* was never a WSGI goal; interoperability and 
portability are more important if we're ever to reduce the chaos around web 
frameworks.  Even if you were to now Pronounce a One True Framework, there 
are still millions of lines of code already out there, and migration paths 
are essential.  That has always been my vision for WSGI, and I always 
thought it was going to be a *lot* easier to agree on and write than it 
actually was.  Even on this second go-round, I'm being surprised by lots of 
interesting little twists that I hadn't thought of.