[Python-ideas] New __reference__ hook

[Masklinn]

On 2012-12-05, at 21:09 , Sturla Molden wrote:

> 
> Den 5. des. 2012 kl. 19:51 skrev Masklinn <masklinn at masklinn.net>:
> 
>> 
>> Why? z could just be a "lazy value" at this point, basically a manual
>> building of thunks, only reifying them when necessary (whenever that
>> is). It's not like numpy *has* to create three temporary arrays, just
>> that it *does*.
>> 
> 
> It has to, because it does not know when to flush an expression.

That tends to be the hard thing to decide, but it should be possible to
find out most cases e.g. evaluate the thunks when elements are requested
(similar to generators, but do the whole thunk at once), when printing,
etc…

Or use the numexpr approach and perform the reification explicitly.

> But perhaps it is possible to do this with AST magic? I don't know.

I'm not sure there's even a need for AST magic (although you could also
play with that by writing operations within lambdas I guess, I've never
done much AST analysis/rewriting), it could simply use an approach
similar to SQLAlchemy's's ClauseElement: when applying an operation to
e.g. an array, rather than perform it just return a representation of
the operation itself (effectively rebuild some sort of AST), new
operations on *that* would simply build the tree further (composing the
thunk), and an explicit evaluation call or implicit evaluation due to
e.g. accessing stuff would compile the "potential" operation and perform
the actual computations.

> A binary operator x+y would just return a symbolic representation of the
> expression, but when the full expression needs to be flushed we can e.g.
> ask OpenCL or LLVM to generate the code on the fly.

Indeed.