[IronPython] LINQ from IronPython

[Jeff Hardy]

This is one of those things that I've been pondering for a while, so I
thougt I'd throw it out there and see what everyone else thinks (I
fully expect to hear that I'm crazy...). I'd like to be able to query
an existing data model (Entity Framework in this case, but that's not
important) from IronPython without pulling all of the data to the
client. By my read there are two necessary things and one nice-to-have
to enable this.

This is just a bunch of random thoughts about how to get LINQ support
into IronPython; I know nothing about the details of the IronPython
compiler, so I'm making no assumptions about the difficulty of any of
this.

Extension Methods
--
The biggest hurdle (but probably the most fruitful) is enabling
extension methods. Unfortunately, I don't know enough about how
IronPython looks up methods on .NET classes to know how easy this is,
so I'll just ramble about possible syntaxes and assume the lookups
will work.

The first option is identical to C#:
   from System import Linq    # or...
   import System.Linq

Either form pulls in all extension methods in all public, static types
in that namespace.

The second option is similar to C#, but requires knowing the extension
class as well:
   from System.Linq import Queryable

This form would only pull in extension methods defined in the Queryable class.

I prefer the second option over the first ('explicit is better than
implicit'), but both have problems in not being completely clear what
is happening.

Another option is to be completely explicit:
   clr.Extend(myObj, System.Linq.Queryable)

It's nice and explicit about what is happening, but could get tedious quickly.

Expression Trees (C# lambdas)
--
Trying to do
   Queryable.Where(entities.Customers, lambda c: c.Id % 2 == 0))
gives (not surprisingly)
   The type arguments for method 'Where' cannot be inferred from the usage.

Putting in the generic argument manually
   Queryable.Where[Customer](entities.Customers, lambda c: c.Id % 2 == 0))
I get (again, not surprisingly)
   `expected Expression[Func[Course, int, bool]], got function`

So, at the very least lambdas would have to be convertible to
Expression<> objects. I know the DLR uses expression trees internally,
but I don't know if they're available at the proper time for that
conversion to be possible. The type inference doesn't seem anymore
difficult than what's already done, but who knows...

Generator Expressions as LINQ exprssions
--
With extension methods and expression trees, the Q in LINQ is handled;
now the LIN ('Language INtegrated') needs to be filled in.

Most of the LINQ queries I do are of the form:
   from customer in entities.Customers
   where customer.Name == name
   select customer

This just so happens to look a lot like
   customer for customer in entities.Customers if customer.Name == name

i.e. a generator expression.

Generator expressions and LINQ queries have a lot in common - chief
amongst them is that they're both lazy. Would it be possible, if
`entities.Customers ` implements IQueryable, for a generator
expression to be converted into a LINQ query? Would it even be
desirable?

The alternative is
   entities.Customers.Where(lambda c: c.Name == name)

which is perfectly fine, but something more would be nice.
--

One thing all of this is that it doesn't change the syntax of Python
at all, just the semantics. I'm not sure if that's good or bad,
though.

Thoughts?

- Jeff