[IronPython] LINQ from IronPython

[Jeff Hardy]

On Tue, Dec 22, 2009 at 11:49 AM, Dino Viehland <dinov at microsoft.com> wrote:
> Personally I like the 2nd option.  I think only the 2nd and 3rd are
> really viable.  The one important thing to keep in mind here is that
> importlib may become the default importer in the future.  So any hooks
> here need to be based off of the processing of what the import machinery
> brings back - not something buried deep in the import machinery.  The 2nd
> form means we only need to check the type going back vs. it's entire
> namespace which will be more expensive.
>
> The 3rd option could also be done w/o myObj, e.g. you just say bring
> in this type or namespace.  It could also be global (not very nice) or
> it could be local to the module like the other forms presumably are.

Sounds like option #2 is the preferred one, so far. I do like the idea
of using `clr.ExtendFrom` (or whatever) as a prototype, however,
before trying to muck with the import machinery.


> We certainly have the raw AST available to us at all times except for
> when we've pre-compiled code.  We don't expose this directly today just
> because it's a big implementation detail to bleed out.  Exposing a
> conversion of functions to Experssion<T> though might be a really good
> way to expose it w/o exposing all of the implementation details.

I think even just lambdas (and, for the IR guys, blocks) would be
sufficient, as they are limited to expressions; arbitrary functions
are unlikely to be handled by the various LINQ providers out there
(per Keith's comment about generating C#-compatible expression trees).

>
> The biggest problem w/ exposing our AST though is can anyone else do
> something useful with it?  If they're just going to compile the AST
> and run it then that's fine.  But if they want to take the AST and
> send it off to the database as a query things aren't going to work
> so well at this point simply because they won't recognize the dynamic
> nodes.  There's also some situations where we open code to the dynamic
> nodes as call sites then things are really confusing.  This may
> be fine for this case where the conversion is explicit and the user
> can make sure it works.

I can think of all sorts of uses for being to manipulate the parse
tree at compile time; I'm just not sure any of them are good ideas :).
Much like C#, supporting a tightly restricted subset of cases is
probably the best approach.

> This is actually a really cool idea.  But because it'd occur implicitly
> it has the same problems as the function ASTs do but they might otherwise
> break working code.

If generator expressions (or anything else...) were convertible back
to their parse trees, you could have a method in C#:
    Expression query(GeneratorExpression genexp) { ... }

that would re-write the genexp to use LINQ's extension methods instead
- basically, an extension point for the IronPython compiler.

- Jeff