[Python-3000] LINQ

[tomer filiba]

i read the references fredrik posted several days back, and it got
me thinking: why not add a "co_ast" attribute to code objects?

i.e., since (virtually) all code objects are constructed from source
code by compilation, the compiler could just emit the AST as an
attribute to the generated code object.

yes, there's an issue of memory consumption, but bear with me
for a moment. also, maybe co_ast could be a lazy attribute, reading
the ast from the pyc file on demand.

having the AST as part of the code object would allow third-party tools,
like a LINQ library, to take a normal code object and walk it's AST,
generating the relevant SQL text or what not.

for those who are not familiar with LINQ, it's an integrated query
mechanism, not unlike list comprehensions:
from X where Y select Z --> [Z for Z in X if Y]

but (and that's a big but) since it's integrated into the language
through some special interfaces, classes can implement custom
"iterables", i.e., a database.

instead of going over the DB in a O(N) fashion (as list-comp does),
the LINQ expression can generate SQL text, send it to the DB,
and yield the results one at a time.

if we combine co_ast with the frame info (or the function's globals),
we can do really impressive things:

db = sql.connection(...)
MIN_AGE = 17
rows = query(firstname for firstname, age in db if age > MIN_AGE)

instead of the cumbersome and error-prone version of

db.query("select ... from ... where age > %s" % (MIN_AGE,))

we can extract the SELECT and WHERE lines from co_ast,
and take the value of MIN_AGE from the frame's globals,
thus generating SQL text from an expression, just like LINQ.

my intuition tells me there are many possible paths to explore
using this method...

================================

[Fredrik Lundh]
> (And frankly, it would be *wonderful* if someone could come up with a
> new proposal that actually enabled Python programmers to do things they
> *cannot* do today, instead of just rehashing old "let's move the sofa
> over there" threads.  How about doing something along the lines of
> LINQ's "give me this expression as an AST so I can optimize it myself"
> model or looking at COmega's parallelization/synchronization stuff or
> digging deeper into how PJE's RuleDispatch could be fit into Python or
> stealing some cool idea from some functional research language that I
> haven't even heard of.  I want to get *new* things when I upgrade from
> 2.X to 3.0, not just silly syntax tweaks that would only give me the
> ability to change two lines of Python code to one line of code plus a
> comment that explains what the heck that line is doing.  Let's do some
> *hard* stuff, for a change.)

but then again, if we go this far, why not just introduce real, first class
code objects? first class code is something LISP has got right
from the very beginning (although i hate LISP :)

i know metasyntax is considered a taboo here, but i thought i'd share
this idea i had for an extensible language:
* the compiler's AST could be altered at runtime (a la boo, only better)
* bytecode instructions could be added at runtime

to show it off a little:

with_statement = Statement(
    Literal("with"),
    Expression,
    Literal("as"),
    Name,
    Suite
)
# also define the code generating visitors for this node ...

import compiler
compiler.add_statement(with_statement)

# now add a new instruction for the WITH statement
import sys

def enter_with(stack, obj):
    obj = stack.pop()
    stack.push( obj.__enter__() )

bytecode = sys.add_instruction(enter_with, 0)

---

this way, you could really do from "__future__ import X"...
the language can be self-extending, which means we can try new
features out before adding them to the C implementation
(kinda like PyPy)

but i guess it's too far fetched for py3k...
maybe i should aim it at py4k :)

g'night.


-tomer