[Python-Dev] ANN: PEP 335: Overloadable Boolean Operators

[Phillip J. Eby]

At 11:39 AM 9/14/04 -0700, Robert Brewer wrote:
>We already have a de facto "code literal syntax": lambdas.
>
>db.query(lambda x: x.y==z and foo*bar<27)

Right, but that requires non-portable bytecode disassembly.  If there was a 
simple way to convert from a function object to an AST, I'd be happy with 
that too.


> > But, even if PEP 335 *were* implemented, creating a query
> > system using Python expressions would *still* be kludgy,
> > because you still need "seed variables" in the current
> > scope to write a query expression.
> > In my example above, I didn't need to bind 'x' or 'y'
> > or 'z' or 'foo' or 'bar', because the db.query() method
> > is going to interpret those in some context.  If I
> > were using a PEP 335-based query system, I'd have to
> > initialize those variables to special querying objects first.
>
>A lot of that becomes a non-issue if you bind early. Once the constants
>are bound, you're left with attribute access on your core objects (x.y)
>and special functions (see logic.ieq or logic.today for example). Again,
>too, I can use the lambda to evaluate Python objects, the 'Object' side
>of "ORM". In that situation, the binding is a benefit.

I'm not following what you mean by "bind early".  My point was that in 
order to have bindings for seeds like 'x' and 'z' and 'foo', most query 
languages end up with hacks like 'tables.tablename.columname' or 
'_.table.column' or other rigamarole, and that this is usually more awkward 
to deal with than the &/|/~ operator spelling.


> > That's why I say that an AST literal syntax would be much
> > more useful to me than PEP 335 for this type of use case.
>
>I seem to recall my AST version was quite slow, in pure Python. Can't
>recall whether that was all the tuple-unpacking or just my naive
>function-call overhead at the time.

When I say AST, I just mean "some kind of syntax representation", not 
necessarily the 'parser' module's current AST implementation.  However, I 
have found that it's possible to translate parser-module AST's to query 
specifications quite efficiently in pure Python, such that the overhead is 
minor compared to whatever actual computation you're doing.  The key is 
that the vast majority of AST nodes are a trivial wrapper around another 
AST node.  The core of my AST-handling engine, therefore, looks like this:

     def build(builder, nodelist):
         while len(nodelist)==2:
             nodelist = nodelist[1]
         return production[nodelist[0]](builder,nodelist)

Where 'production' is a table mapping symbol IDs to helper functions that 
invoke methods on 'builder', which then may recursively invoke 'build' on 
items in 'nodelist'.  The first two lines of this function eliminate 
enormous amounts of overhead by ignoring all the zillions of trivial 
wrapper nodes.  (Note that you must include line number information in the 
generated AST, or it will mistake tokens for unnecessary symbols.)


>Anyway, for those reasons, I'm -0.5.

On what?  AST literals, or PEP 335?