Performance of list comprehensions vs. map

[Chris Barker]

Skip Montanaro wrote:

>     Chris> Because it is important for hte design that it work exactly that
>     Chris> the same way: which I don't understand at all.
> 
> It wasn't important for the design to work exactly like for loops, but given
> Python's semantics it is hard to see how to have done it some other way.
> Also, it was certainly the easiest route.
> 
>     Chris> Again, is this a design decision? that the expression in a list
>     Chris> comprehension shuld be able to be changes mid-comprehension? In
>     Chris> another thread to boot? This sounds like a the worst of confusing
>     Chris> and unpredictable behaviour to me!!!
> 
> No, this is Python's principle of least astonishment at work.  Python's
> language is dynamic all over the place.  Placing arbitrary restrictions on
> those semantics because they happen to occur between "[" and "]" would have
> been wrong.

but it's not wrong to put in the same restrictions when map is called?
The list comprehension syntax is new.. it's not legal in older version
of Python. Therefore, you could put any restrictions you wanted in
there, and the restiction that the lists being comprehended don't change
in the middle ofhte process sounds like a pretty darn good one to me.

>     Chris> If it's OK from a design standpoint to require that the function
>     Chris> in a map is bound once, it seems pretty reasonable to have the
>     Chris> same requirement for a list comprehension. In fact, it seems like
>     Chris> a darn good idea! The fact that the current implimentation
>     Chris> doesn't enforce it is a pretty poor design justification...or am
>     Chris> I missing something?
> 
> map() is like any other function written in C.  It does its thing, then
> returns.

The fact that it is written in C should ideally not determine it's
behaviour. Ideally the behavious or map(), and list comprehensions,
should be determined by some sensible design criteria, including
performance, of course.

> It would be impossible for map() to check back to see if the name
> "str" was rebound during its execution.  All it knows is that it is passed a
> callable object.  It has no idea what that object was bound to in the
> calling scope.  It calls it once for each element of its various argument
> lists.  That would be true if map() was written in Python as well.

All I'm saying is that list comprehensions shluld behave the same way. I
havn't done any multi-threaded programming, but it sounds like all this
re-binding of objects in the middle function calls could get really
messy. 

>     >> But there is no guarantee that the list you are operating on isn't
>     >> modified by another thread during execution.
> 
>     Chris> but there should be !!
> 
> Why here and not elsewhere in the language?  

I thought you said that map() doesn't allow it?

>I'm afraid you're tilting at windmills.

Perhaps I am. It may be that a highly dynamic language isn't well suited
to multi-threading, but this stuff sounds pretty scary to me... I'll
have to remember to be very careful with any multi-threaded code I
write.

> You're more than welcome to write a PEP.

The reason I havn't is that I am in no position to impliment even a
prototype of the idea (both time and skills), and unless someone who is
in that position takes some interest, there really isn't any point. It
also may be a just plain bad idea, so I would like more feedback before
I went that far. Thanks for your input, I have certainly learned from
this discussion.

> One approach might be to propose
> that the array object be "brought into the fold" as a builtin object

This is in PEP 209 (with NumPY arrays, whcih is much better that the
library array). I'm looking forward to it, I wish I could do more to
help out. Note that PEP 211 and 225 are relevent too. The three together
would do a lot to make Python more "array-oriented" which I think would
be wonderful.

> and
> modifications be made to the virtual machine so that homogeneity can be
> assumed when operating on arrays.

That would be a great next step.... it's not in the PEP  yet, maybe I'll
send a note to authors.


>     Chris> further off. Personally, I think a little static type
>     Chris> declarations could help Py2C and the like a lot.
> 
> Sure, but then it wouldn't be Python. ;-)

I'm not so sure about that... be definintion any change or enhancement
makes something that is not python. As long as full dynamic typing is
the default, it would still be Python.

-Chris


-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------