[Python-3000] Adding sorting to generator comprehension

[Josiah Carlson]

Talin <talin at acm.org> wrote:
> Ian Bicking <ianb <at> colorstudy.com> writes:
> > Josiah Carlson wrote:
> > > One of the features of generator expressions which makes it desireable
> > > instead of list comprehensions is that generator expressions may use
> > > less memory *now*, and may be able to start returning results *now*.
> > > 
> > > Using (<genexp> orderby ...) as a replacement for sorted((genexp), key=...)
> > > is a bit misleading because while the original generator expression
> > > could have been space and time efficient, the orderby version certainly may
> > > not be.
> > 
> > Certainly it changes the performance substantially (of course if the 
> > expression is translated and executed elsewhere the performance can 
> > actually be improved, so it can go both ways).  Since list 
> > comprehensions are planned to just be syntactic sugar for generator 
> > comprehension, generator comprehensions are now the more fundamental 
> > construct.
> > 
> > But yeah, it is a little awkward, since something that is sorted can be 
> > returned as a list anyway, except for the fact that the expression 
> > itself could be ported off elsewhere which isn't normal Python.  (But 
> > should be normal Python!)
> > 
> 
> I think that I am (mostly) in agreement with Ian on this one, but perhaps with
> different reasoning.
> 
> There seems to be a common feeling on py-dev that "list comprehensions are just
> a special case of generators expressions, so really, we don't need them".
> 
> But then we turn around and say "Well, this feature might be handy for list
> comprehensions, but since list comprehensions are based on generator
> expressions, and since this feature would make generator expressions
> inefficient, we won't do it."

No.  I was only responding to the question of orderby in relation to
generator expressions.  In generator expressions, it is further
unnecessary because one can always wrap the generator expression up with
a sorted(genexp, ...) to get your sorted version of the generator (in
list form).

In the case of list comprehensions, it is doubly unnecessary, because
you can again use sorted([genexp], ...) or even list.sort(...) .


> In other words, we dismiss any evidence that disproves our initial assumption.
> This is sort of like saying that we don't need the 'def' statement, since you
> can define functions with "func = lambda"; and also we shouldn't add multi-line
> statements, since lambda doesn't support them.

You are letting the tail wag the dog.  From what I understand, lambda
was a post-facto introduction to the language along with map and filter
to allow for a more functional approach with Python.  def already exists. 
I would also point out that all of those modifications to lambda
(multi-line function bodies, statements, etc.) have all been proposed
and *rejected*.

Indeed, it is a similar case for list comprehensions and generator
expressions.  Generator expressions did come after list comprehensions;
but note that Ian's original request was in relation to generator
expressions, _not_ list comprehensions, and in the realm of generator
expressions, I believe, orderby is a non-starter (for the reasons I've
already expressed).


> The fact is that sorting happens a *lot*. Sorting with key= happens a lot. A
> large proportion of my list comprehensions involve sorting or ordering of one
> form or another, and the number of ugly sorted + key + lambda expressions is
> enough to convince me that an "orderby" or "ascending" or whatever clause would
> be a welcome addition.

Sorting may happen "a *lot*" with what you write, but in the software
that I write, sorting *rarely* happens.  If you would like to lessen
your use of lambda, you should consider discovering the
operator.attrgetter() and operator.itemgetter() functions to be passed
to sorted().

 - Josiah