[Python-3000] LINQ

[Robert Brewer]

tomer filiba wrote:
> i read the references fredrik posted several days back, and it got
> me thinking: why not add a "co_ast" attribute to code objects?
> 
> i.e., since (virtually) all code objects are constructed from source
> code by compilation, the compiler could just emit the AST as an
> attribute to the generated code object.
> 
> yes, there's an issue of memory consumption, but bear with me
> for a moment. also, maybe co_ast could be a lazy attribute, reading
> the ast from the pyc file on demand.
> 
> having the AST as part of the code object would allow 
> third-party tools,
> like a LINQ library, to take a normal code object and walk it's AST,
> generating the relevant SQL text or what not.
> 
> for those who are not familiar with LINQ, it's an integrated query
> mechanism, not unlike list comprehensions:
> from X where Y select Z --> [Z for Z in X if Y]
> 
> but (and that's a big but) since it's integrated into the language
> through some special interfaces, classes can implement custom
> "iterables", i.e., a database.
> 
> instead of going over the DB in a O(N) fashion (as list-comp does),
> the LINQ expression can generate SQL text, send it to the DB,
> and yield the results one at a time.
> 
> if we combine co_ast with the frame info (or the function's globals),
> we can do really impressive things:
> 
> db = sql.connection(...)
> MIN_AGE = 17
> rows = query(firstname for firstname, age in db if age > MIN_AGE)
> 
> instead of the cumbersome and error-prone version of
> 
> db.query("select ... from ... where age > %s" % (MIN_AGE,))
> 
> we can extract the SELECT and WHERE lines from co_ast,
> and take the value of MIN_AGE from the frame's globals,
> thus generating SQL text from an expression, just like LINQ.

This is exactly what I do in Dejavu [1], only using the co_code instead,
and first-class expression objects [2] instead of full-blown code
objects. If co_ast were present, I'd probably use it instead; the only
reason I use bytecode now (Python 2.4 and under) is because it's faster.

> my intuition tells me there are many possible paths to explore
> using this method...

No intuition necessary; I've already explored them in Dejavu. ;) Read
through the short Querying doc [3] for a summary.

I have a proposal in for PyCon 2007 to explain the first-class
expression objects in detail, and how they enable LINQ-style code.


Robert Brewer
System Architect
Amor Ministries
fumanchu at amor.org

[1] http://projects.amor.org/dejavu
[2] http://projects.amor.org/dejavu/browser/trunk/logic.py
[3] http://projects.amor.org/docs/dejavu/managing.html#Querying