PEP 201 - Parallel iteration
I've made a final pass through PEP 201, Parallel Iteration. For those of you without easy access to the CVS tree, I'm including it below. You can also access it through the web at http://cvs.sourceforge.net/cgi-bin/cvsweb.cgi/python/nondist/peps/pep-0201.txt?rev=1.2&content-type=text/x-cvsweb-markup&cvsroot=python High bit: Guido strongly prefers zip() as the name of the built-in. All the suggestions so far have pluses and minuses, and in the absence of a clear winner, he'd just as soon go with the Haskell name. I agree. Please see the new reference implementation, which includes a __len__() method, and the discussion of the rejected elaborations. There's still the open issue of what zip(sequence_a) should return. I'd like to reach closure on this sometime soon after the guys get back from California. -Barry -------------------- snip snip -------------------- PEP: 201 Title: Parallel Iteration Version: $Revision: 1.2 $ Owner: bwarsaw@beopen.com (Barry A. Warsaw) Python-Version: 2.0 Status: Draft Introduction This PEP describes the `parallel iteration' proposal for Python 2.0, previously known as `parallel for loops'. This PEP tracks the status and ownership of this feature, slated for introduction in Python 2.0. It contains a description of the feature and outlines changes necessary to support the feature. This PEP summarizes discussions held in mailing list forums, and provides URLs for further information, where appropriate. The CVS revision history of this file contains the definitive historical record. Standard For-Loops Motivation for this feature has its roots in a concept described as `parallel for loops'. A standard for-loop in Python iterates over every element in the sequence until the sequence is exhausted. A `break' statement inside the loop suite causes an explicit loop exit. For-loops also have else: clauses which get executed when the loop exits normally (i.e. not by execution of a break). For-loops can iterate over built-in types such as lists and tuples, but they can also iterate over instance types that conform to an informal sequence protocol. This protocol states that the instance should implement the __getitem__() method, expecting a monotonically increasing index starting at 0, and this method should raise an IndexError when the sequence is exhausted. This protocol is currently undocumented -- a defect in Python's documentation hopefully soon corrected. For-loops are described in the Python language reference manual[1]. An example for-loop: >>> for i in (1, 2, 3): print i ... 1 2 3 In this example, the variable `i' is called the `target', and is assigned the next element of the list, each time through the loop. Parallel For-Loops Parallel for-loops are non-nested iterations over two or more sequences, such that at each pass through the loop, one element from each sequence is taken to compose the target. This behavior can already be accomplished in Python through the use of the map() built-in function: >>> a = (1, 2, 3) >>> b = (4, 5, 6) >>> for i in map(None, a, b): print i ... (1, 4) (2, 5) (3, 6) Here, map() returns a list of N-tuples, where N is the number of sequences in map()'s argument list (after the initial `None'). Each tuple is constructed of the i-th elements from each of the argument lists, specifically in this example: >>> map(None, a, b) [(1, 4), (2, 5), (3, 6)] The for-loop simply iterates over this list as normal. While the map() idiom is a common one in Python, it has several disadvantages: - It is non-obvious to programmers without a functional programming background. - The use of the magic `None' first argument is non-obvious. - It has arbitrary, often unintended, and inflexible semantics when the lists are not of the same length: the shorter sequences are padded with `None'. >>> c = (4, 5, 6, 7) >>> map(None, a, c) [(1, 4), (2, 5), (3, 6), (None, 7)] For these reasons, several proposals were floated in the Python 2.0 beta time frame for providing a better spelling of parallel for-loops. The initial proposals centered around syntactic changes to the for statement, but conflicts and problems with the syntax were unresolvable, especially when parallel for-loops were combined with another proposed feature called `list comprehensions' (see pep-0202.txt). The Proposed Solution The proposed solution is to introduce a new built-in sequence generator function, available in the __builtin__ module. This function is to be called `zip' and has the following signature: zip(seqa, [seqb, [...]], [pad=<value>]) zip() takes one or more sequences and weaves their elements together, just as map(None, ...) does with sequences of equal length. The optional keyword argument `pad', if supplied, is a value used to pad all shorter sequences to the length of the longest sequence. If `pad' is omitted, then weaving stops when the shortest sequence is exhausted. It is not possible to pad short lists with different pad values, nor will zip() ever raise an exception with lists of different lengths. To accomplish either behavior, the sequences must be checked and processed before the call to zip(). Lazy Execution For performance purposes, zip() does not construct the list of tuples immediately. Instead it instantiates an object that implements a __getitem__() method and conforms to the informal for-loop protocol. This method constructs the individual tuples on demand. Examples Here are some examples, based on the reference implementation below. >>> a = (1, 2, 3, 4) >>> b = (5, 6, 7, 8) >>> c = (9, 10, 11) >>> d = (12, 13) >>> zip(a, b) [(1, 5), (2, 6), (3, 7), (4, 8)] >>> zip(a, d) [(1, 12), (2, 13)] >>> zip(a, d, pad=0) [(1, 12), (2, 13), (3, 0), (4, 0)] >>> zip(a, d, pid=0) Traceback (most recent call last): File "<stdin>", line 1, in ? File "/usr/tmp/python-iKAOxR", line 11, in zip TypeError: unexpected keyword arguments >>> zip(a, b, c, d) [(1, 5, 9, 12), (2, 6, 10, 13)] >>> zip(a, b, c, d, pad=None) [(1, 5, 9, 12), (2, 6, 10, 13), (3, 7, 11, None), (4, 8, None, None)] >>> map(None, a, b, c, d) [(1, 5, 9, 12), (2, 6, 10, 13), (3, 7, 11, None), (4, 8, None, None)] Reference Implementation Here is a reference implementation, in Python of the zip() built-in function and helper class. These would ultimately be replaced by equivalent C code. class _Zipper: def __init__(self, args, kws): # Defaults self.__padgiven = 0 if kws.has_key('pad'): self.__padgiven = 1 self.__pad = kws['pad'] del kws['pad'] # Assert no unknown arguments are left if kws: raise TypeError('unexpected keyword arguments') self.__sequences = args self.__seqlen = len(args) def __getitem__(self, i): ret = [] exhausted = 0 for s in self.__sequences: try: ret.append(s[i]) except IndexError: if not self.__padgiven: raise exhausted = exhausted + 1 if exhausted == self.__seqlen: raise ret.append(self.__pad) return tuple(ret) def __len__(self): # If we're padding, then len is the length of the longest sequence, # otherwise it's the length of the shortest sequence. if not self.__padgiven: shortest = -1 for s in self.__sequences: slen = len(s) if shortest < 0 or slen < shortest: shortest = slen return shortest longest = 0 for s in self.__sequences: slen = len(s) if slen > longest: longest = slen return longest def __str__(self): ret = [] i = 0 while 1: try: ret.append(self[i]) except IndexError: break i = i + 1 return str(ret) __repr__ = __str__ def zip(*args, **kws): return _Zipper(args, kws) Rejected Elaborations Some people have suggested that the user be able to specify the type of the inner and outer containers for the zipped sequence. This would be specified by additional keyword arguments to zip(), named `inner' and `outer'. This elaboration is rejected for several reasons. First, there really is no outer container, even though there appears to be an outer list container the example above. This is simply an artifact of the repr() of the zipped object. User code can do its own looping over the zipped object via __getitem__(), and build any type of outer container for the fully evaluated, concrete sequence. For example, to build a zipped object with lists as an outer container, use >>> list(zip(sequence_a, sequence_b, sequence_c)) for tuple outer container, use >>> tuple(zip(sequence_a, sequence_b, sequence_c)) This type of construction will usually not be necessary though, since it is expected that zipped objects will most often appear in for-loops. Second, allowing the user to specify the inner container introduces needless complexity and arbitrary decisions. You might imagine that instead of the default tuple inner container, the user could prefer a list, or a dictionary, or instances of some sequence-like class. One problem is the API. Should the argument to `inner' be a type or a template object? For flexibility, the argument should probably be a type object (i.e. TupleType, ListType, DictType), or a class. For classes, the implementation could just pass the zip element to the constructor. But what about built-in types that don't have constructors? They would have to be special-cased in the implementation (i.e. what is the constructor for TupleType? The tuple() built-in). Another problem that arises is for zips greater than length two. Say you had three sequences and you wanted the inner type to be a dictionary. What would the semantics of the following be? >>> zip(sequence_a, sequence_b, sequence_c, inner=DictType) Would the key be (element_a, element_b) and the value be element_c, or would the key be element_a and the value be (element_b, element_c)? Or should an exception be thrown? This suggests that the specification of the inner container type is needless complexity. It isn't likely that the inner container will need to be specified very often, and it is easy to roll your own should you need it. Tuples are chosen for the inner container type due to their (slight) memory footprint and performance advantages. Open Issues - What should "zip(a)" do? Given a = (1, 2, 3); zip(a) three outcomes are possible. 1) Returns [(1,), (2,), (3,)] Pros: no special casing in the implementation or in user code, and is more consistent with the description of it's semantics. Cons: this isn't what map(None, a) would return, and may be counter to user expectations. 2) Returns [1, 2, 3] Pros: consistency with map(None, a), and simpler code for for-loops, e.g. for i in zip(a): instead of for (i,) in zip(a): Cons: too much complexity and special casing for what should be a relatively rare usage pattern. 3) Raises TypeError Pros: None Cons: needless restriction Current scoring seems to generally favor outcome 1. - The name of the built-in `zip' may cause some initial confusion with the zip compression algorithm. Other suggestions include (but are not limited to!): marry, weave, parallel, lace, braid, interlace, permute, furl, tuples, lists, stitch, collate, knit, plait, and with. All have disadvantages, and there is no clear unanimous choice, therefore the decision was made to go with `zip' because the same functionality is available in other languages (e.g. Haskell) under the name `zip'[2]. References [1] http://www.python.org/doc/devel/ref/for.html [2] http://www.haskell.org/onlinereport/standard-prelude.html#$vzip TBD: URL to python-dev archives Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil End:
On Mon, Jul 17, 2000 at 03:06:13PM -0400, Barry A. Warsaw wrote:
There's still the open issue of what zip(sequence_a) should return.
I'd vote for [(a,), (b,), (c,)], that is, still return tuples. It's consistent with the use in the case of multiple sequences. We already have map(None, sequence_a) that does it the non-obvious way ;) In the map case it's easily explained: it turns the sequence into a list. In the case of zip(), that isn't necessary (and not the point of the function, anyway.) I-still-wish-it-were-marry()-ly y'rs, ;) -- Thomas Wouters <thomas@xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
Barry A. Warsaw wrote:
I've made a final pass through PEP 201, Parallel Iteration.
Nice! I think that a broader audience needs to read this. Why not asking for reactions on c.l.py?
High bit: Guido strongly prefers zip() as the name of the built-in. All the suggestions so far have pluses and minuses, and in the absence of a clear winner, he'd just as soon go with the Haskell name. I agree.
I don't feel comfortable with it. Period. You might want to see what http://www.dict.org has to say about it. Looking at the examples for this builtin function, and without thinking too much about the name, I'd call it - fold(). zip-gun'ly y'rs -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252
Vladimir Marangozov wrote:
I don't feel comfortable with it. Period. You might want to see what http://www.dict.org has to say about it.
Looking at the examples for this builtin function, and without thinking too much about the name, I'd call it - fold().
Please, don't!!! As far as I can remember, fold is the "reduce of the functional languages". At least it is that in Haskell. Here's a quote from the Journal of Functional Programming (July 1999): "In functional programming, fold is a standard operator that encapsulates a simple pattern of recursion for processing lists." BTW: I still like zip() and as someone said: dir and del have the same problem/not-a-problem Peter -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de
Peter Schneider-Kamp wrote:
[me]
Looking at the examples for this builtin function, and without thinking too much about the name, I'd call it - fold().
Please, don't!!! As far as I can remember, fold is the "reduce of the functional languages". At least it is that in Haskell.
Here's a quote from the Journal of Functional Programming (July 1999): "In functional programming, fold is a standard operator that encapsulates a simple pattern of recursion for processing lists."
Okay, but weren't we paying less attention to historical facts in language design? Guido regrets map() that slipped through, for instance. My opinion is a newbie opinion. I just can't believe that we can't find a more intuitive name for this function!
I just can't believe that we can't find a more intuitive name for this function!
...and I cannot believe that you're trying to reopen this discussion once again :-( (or is this a trick by Guido to make sure we don't do any real work while he's on the conference? ;-) </F> "After consultation with Guido, zip() is chosen as the name of this built-in." -- seen on the python-checkins list.
Fredrik Lundh wrote:
I just can't believe that we can't find a more intuitive name for this function!
...and I cannot believe that you're trying to reopen this discussion once again :-(
:-) Sorry, but it was discussed during a short period of time (despite the huge number of posts in that time slice) and I wasn't really available for participating seriously in the discussion. So now I'm taking the opportunity to read the PEP without much external pression, as well as to say what I think and to think what I say. Note that I suggested Barry to sumbit this PEP to a broader audience for collection external opinions on all aspects of the PEP. If there are several people with CS background that don't feel good with this name, there's something wrong with it. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252
Vladimir.Marangozov wrote:
Fredrik Lundh wrote:
I just can't believe that we can't find a more intuitive name for this function!
...and I cannot believe that you're trying to reopen this discussion once again :-(
:-) Sorry, but it was discussed during a short period of time (despite the huge number of posts in that time slice) and I wasn't really available for participating seriously in the discussion. So now I'm taking the opportunity to read the PEP without much external pression, as well as to say what I think and to think what I say.
Note that I suggested Barry to sumbit this PEP to a broader audience for collection external opinions on all aspects of the PEP. If there are several people with CS background that don't feel good with this name, there's something wrong with it.
To me, "zip" in the context of computers means compress. But it's a stupid name for compress. And one look at the args makes it obvious it's not a compress. So I expect that your typical newbie will do one, and only one "Huh??? Oh - zip as in zipper, not zip as in PKZIP" and never have to post again. wishfully-y'rs - Gordon
Gordon McMillan wrote:
To me, "zip" in the context of computers means compress. But it's a stupid name for compress. And one look at the args makes it obvious it's not a compress. So I expect that your typical newbie will do one, and only one "Huh??? Oh - zip as in zipper, not zip as in PKZIP" and never have to post again.
Right.
wishfully-y'rs
My wish is to add fold() as one of the suggested names in the PEP, then let this PEP spend some time in c.l.py. Give people a chance to catch up, please. That's how the PEP process is intended to work. The name hasn't been cast in stone. It's in the PEP, not in CVS. end-of-story-ly y'rs -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252
"VM" == Vladimir Marangozov <Vladimir.Marangozov@inrialpes.fr> writes:
VM> My wish is to add fold() as one of the suggested names in the VM> PEP Done. VM> then let this PEP spend some time in c.l.py. Will do. I just updated the reference implementation. I want to finish going through the comments in this forum and then I'll post to c.l.py. -Barry
Vladimir Marangozov wrote:
What's the problem with `folding' sequences element-wise in Python?
What's the problem with `zipping' sequences element-wise in Python?
Which one of the 2 questions above makes more sense?
The first. But it gives me the wrong impression: fold the list [1, 2, 3]: \ / f(1,f(2,f(3))) for some function f fold the lists [1, 2, 3] and [4, 5, 6] elementwise: \ / f(f(3,f(6)),f(f(2,f(5)),f(1,f(4)))) for some function f I'd just wonder what to fold the lists with. Even if my judgement is biased by my Haskell organ, I still think that taking the name of an important list operator from different languages for a completely different list operation is way worse than using the name of a file compression utility as a list operator. just-my-cents(0.02)-ly y'rs Peter -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de
I realize it's late in the day, but could the name of the concept (not the function) be changed from "parallel iteration" to something that doesn't imply concurrent execution quite so strongly? I think the use of "parallel" is very likely to mislead, particularly scientists and engineers who actually have parallel loops in their jumped-up Fortran and C++ systems. Thanks, Greg
Barry A. Warsaw wrote: [PEP 201, Parallel Iteration]
Open Issues
- What should "zip(a)" do? Given
a = (1, 2, 3); zip(a)
three outcomes are possible.
How about a fourth: zip(a) is the same as zip(a, []) ? Look at it this way: what should zip(a, pad=None) return? Obviously: [(1, None), (2, None), (3, None)] So what should zip(a) return? Don't special case it, just return []. With only one arg, zip has clearly lost meaning. That is, it's clearly a degenerate case, and not worth a bunch of special casing. although-we-do-special-case-Barry-ly y'rs - Gordon
(assume a = [1, 2, 3]) On Mon, 17 Jul 2000, Gordon McMillan wrote:
How about a fourth: zip(a) is the same as zip(a, []) ?
Huh???????????????????
Look at it this way: what should zip(a, pad=None) return?
[(1,), (2,), (3,)], of course
Obviously: [(1, None), (2, None), (3, None)]
Why? -- Moshe Zadka <moshez@math.huji.ac.il> There is no GOD but Python, and HTTP is its prophet. http://advogato.org/person/moshez
Moshe Zadka wrote:
(assume a = [1, 2, 3])
On Mon, 17 Jul 2000, Gordon McMillan wrote:
How about a fourth: zip(a) is the same as zip(a, []) ?
Huh???????????????????
I think he meant: zip((a,[]))
Look at it this way: what should zip(a, pad=None) return?
[(1,), (2,), (3,)], of course
Obviously: [(1, None), (2, None), (3, None)]
Why?
Good question ;-) Here's another one: why should zip() have this special case at all ? I mean, have you ever seen a zipper with only one part ? Ok, there is one-hand clapping, but I wouldn't consider this the normal mode of operation ;-) IMHO, all the special casing does is mask programming errors. And another one: zip() as defined is an operation which is -- in some respects -- the inverse of itself, yet the name does not carry this information. You can't zip a zipper twice. The mathematical operation behind it is called transposition (you view the sequence of sequences as matrix), so in that light the natural name would be transpose()... tranpose(transpose(matrix))) has the same structure as matrix itself. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/
M.-A. Lemburg wrote:
Moshe Zadka wrote:
(assume a = [1, 2, 3])
On Mon, 17 Jul 2000, Gordon McMillan wrote:
How about a fourth: zip(a) is the same as zip(a, []) ?
Huh???????????????????
I think he meant: zip((a,[]))
No I didn't. Sigh.
pad([1,2,3], [1], pad=None) [(1,1), (2,None), (3,None)]
pad([1,2,3], [], pad=None) [(1,None), (2,None), (3,None)]
(If you don't agree with that, you don't agree with the PEP). So I'm proposing that the 2nd positional arg, if not present, be treated as []. That makes:
zip([1,2,3], pad=None) [(1,None), (2,None), (3,None) zip([1,2,3]) []
The other possibility is to make Evan Simpson's dream come true, so
zip([(1,None), (2,None), (3,None)]) [[1,2,3],[None,None,None]]
But then you'd have to throw an exception if the (single) arg wasn't a rectangular sequence of sequences. Also, since lately Guido's on a kick to get rid of auto-tupling/untupling of args, I think you need to use apply(zip, tuple(...)) or the *args syntax to accomplish this.
Here's another one: why should zip() have this special case at all ? I mean, have you ever seen a zipper with only one part ? Ok, there is one-hand clapping, but I wouldn't consider this the normal mode of operation ;-) IMHO, all the special casing does is mask programming errors.
Which doesn't answer the question posed in the PEP. All of the proposed responses are special cases. The one I've seen mentioned most often:
zip([1,2,3]) [(1,),(2,),(3,)] is very much as special case, as you can see if you ask "what happens when I add a pad arg?".
However, if the dev list is having so much trouble deciding how it should decay, I'll vote for throwing an exception.
And another one: zip() as defined is an operation which is -- in some respects -- the inverse of itself, yet the name does not carry this information. You can't zip a zipper twice. The mathematical operation behind it is called transposition (you view the sequence of sequences as matrix), so in that light the natural name would be transpose()... tranpose(transpose(matrix))) has the same structure as matrix itself.
But zip takes some number of sequences, not necessarily of the same length. That's not a matrix. I think if you go that route, the user has to first insure they're all the same length and then package them into a sequence. It might have its uses, but not as a replacement for map(None, ...). - Gordon
On Tue, 18 Jul 2000, Gordon McMillan wrote:
(If you don't agree with that, you don't agree with the PEP).
I agree with that. Luckily, I've snipped it, so it's not valid as evidence <wink>
So I'm proposing that the 2nd positional arg, if not present, be treated as []. That makes:
That's a special case!
Which doesn't answer the question posed in the PEP. All of the proposed responses are special cases. The one I've seen mentioned most often:
zip([1,2,3]) [(1,),(2,),(3,)] is very much as special case, as you can see if you ask "what happens when I add a pad arg?".
I disagree: you agree that if you add a pad argument when the lists are all the same size it doesn't matter (if you don't agree with this, you don't agree with the PEP). Now, when there is only one list, the lists are all the same size. QED.
However, if the dev list is having so much trouble deciding how it should decay, I'll vote for throwing an exception.
I'm starting to agree -- I'm thinking like a mathematician, which not everyone in the world are (sadly) -- Moshe Zadka <moshez@math.huji.ac.il> There is no GOD but Python, and HTTP is its prophet. http://advogato.org/person/moshez
Moshe wrote:
I disagree: you agree that if you add a pad argument when the lists are all the same size it doesn't matter (if you don't agree with this, you don't agree with the PEP). Now, when there is only one list, the lists are all the same size. QED.
I would agree if we were talking about transpose(matrix, [pad]). Then: transpose([(1,2,3)]) -> [(1,), (2,), (3,)] and transpose([(1,), (2,), (3,)]) -> [(1,2,3)] That is, a 3X1 transposes to a 1X3 and vice versa. But zip doesn't take a matrix, it takes some number of sequences. Oh heck, I'll agree anyway. But I dread explaining why 1) it doesn't behave like map(None, [1,2,3]) 2) why zip(((1,2),(3,4))) doesn't yield [(1,3), (2,4)] but [((1,2),), ((3,4),)] instead! So: document zip as taking 2 or more sequences, and throw an exception if fed only one. - Gordon
On Tue, 18 Jul 2000, Gordon McMillan wrote:
So: document zip as taking 2 or more sequences, and throw an exception if fed only one.
Agreed: it seems too much ambiguos. I mean, I didn't think it ambiguous, and neither did Gordon and /F -- only each of us saw a different unambiguouity <wink> which show it is. -- Moshe Zadka <moshez@math.huji.ac.il> There is no GOD but Python, and HTTP is its prophet. http://advogato.org/person/moshez
"MZ" == Moshe Zadka <moshez@math.huji.ac.il> writes:
MZ> I disagree: you agree that if you add a pad argument when the MZ> lists are all the same size it doesn't matter (if you don't MZ> agree with this, you don't agree with the PEP). Now, when MZ> there is only one list, the lists are all the same size. QED. Bingo. -Barry
Moshe Zadka wrote:
(assume a = [1, 2, 3])
On Mon, 17 Jul 2000, Gordon McMillan wrote:
How about a fourth: zip(a) is the same as zip(a, []) ?
Huh???????????????????
Look at it this way: what should zip(a, pad=None) return?
[(1,), (2,), (3,)], of course
Then why does the PEP say:
zip(a, b, c, d, pad=None) [(1, 5, 9, 12), (2, 6, 10, 13), (3, 7, 11, None), (4, 8, None, None)] >>> map(None, a, b, c, d) [(1, 5, 9, 12), (2, 6, 10, 13), (3, 7, 11, None), (4, 8, None, None)]
If you want no padding, you don't specify a pad arg. pad=None means "pad with None".
Obviously: [(1, None), (2, None), (3, None)]
Why?
To be consistent. - Gordon
On Tue, 18 Jul 2000, Gordon McMillan wrote: [about zip(a, pad=None) ]
Obviously: [(1, None), (2, None), (3, None)]
Why?
To be consistent.
But when there is only one list, all lists are of the same size, so the pad argument is moot. (Consistent with all other cases where all lists are the same size) Or maybe we're talking about a different situation? Can you show a fake Python session which demonstrates your point? confused-ly y'rs, Z. -- Moshe Zadka <moshez@math.huji.ac.il> There is no GOD but Python, and HTTP is its prophet. http://advogato.org/person/moshez
"Gordo" == Gordon McMillan <gmcm@hypernet.com> writes:
Gordo> How about a fourth: zip(a) is the same as zip(a, []) ? I don't like it. The way I view zip() is that the length of the elements returned by __getitem__() should always be equal to the number of sequences passed in as parameters. Thus it seems most natural to me that >>> zip((1, 2, 3)) [(1,), (2,), (3,)] >>> zip((1, 2, 3), pad=None) [(1,), (2,), (3,)] When I talk about "special casing", there's two ways to look at it. One possibility is that the implementation of zip has to explicitly check that the number of sequences is 1 and then `do something special' if so. To support Gordon's proposal, or the map()-equivalent, or the raise-an-exception proposal, that would be the case. The second possibility is that user code must `appear' to special case the situation when the number of sequences is exactly 1. I.e. >>> for (i,) in zip((1, 2, 3)): instead of the more natural (because it's less to type and because of the map() heritage): >>> for i in zip((1, 2, 3)): The first example isn't really special casing though. Personally, I think this is going to come up so infrequently that it's not worth it. -Barry
"Barry A. Warsaw" wrote:
...
The second possibility is that user code must `appear' to special case the situation when the number of sequences is exactly 1. I.e.
>>> for (i,) in zip((1, 2, 3)):
instead of the more natural (because it's less to type and because of the map() heritage):
>>> for i in zip((1, 2, 3)):
Why are you using zip in this case? If you *know* that you are going to get back 1-tuples then you must know that the input is a single list (i.e. you aren't using "apply"). So why are you using zip at all? -- Paul Prescod - Not encumbered by corporate consensus Just how compassionate can a Republican get before he has to leave the GOP and join Vegans for Global Justice? ... One moment, George W. Bush is holding a get-to-know-you meeting with a bunch of gay Republicans. The next he is holding forth on education or the environment ... It is enough to make a red-blooded conservative choke on his spotted-owl drumstick. - April 29th, Economist
"PP" == Paul Prescod <paul@prescod.net> writes:
PP> Why are you using zip in this case? If you *know* that you are PP> going to get back 1-tuples then you must know that the input PP> is a single list (i.e. you aren't using "apply"). So why are PP> you using zip at all? It was just an example, but I agree, you probably wouldn't use zip in this situation. My question is, why would you ever use zip with one parameter? To me, that's an argument for letting zip(a) return 1-tuples, not raising an exception. -Barry
Barry A. Warsaw" wrote:
...
It was just an example, but I agree, you probably wouldn't use zip in this situation. My question is, why would you ever use zip with one parameter? To me, that's an argument for letting zip(a) return 1-tuples, not raising an exception.
Your logic is impeccable. Also, consider this: zip( "mybigfile.xml" ) -- Paul Prescod - Not encumbered by corporate consensus "Hardly anything more unwelcome can befall a scientific writer than having the foundations of his edifice shaken after the work is finished. I have been placed in this position by a letter from Mr. Bertrand Russell..." - Frege, Appendix of Basic Laws of Arithmetic (of Russell's Paradox)
On Wed, Jul 19, 2000 at 02:47:42PM -0500, Paul Prescod wrote:
Also, consider this: zip( "mybigfile.xml" )
That's kinda like 'dir()', isn't it ? "I don't have those files in my directory!!?" Nevertheless, yet another reason to use marry() <wink> No confusion there! -- Thomas Wouters <thomas@xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
On Wed, 19 Jul 2000, Barry A. Warsaw wrote:
"PP" == Paul Prescod <paul@prescod.net> writes:
PP> Why are you using zip in this case? If you *know* that you are PP> going to get back 1-tuples then you must know that the input PP> is a single list (i.e. you aren't using "apply"). So why are PP> you using zip at all?
It was just an example, but I agree, you probably wouldn't use zip in this situation. My question is, why would you ever use zip with one parameter?
Or, in other words, anyone using zip() with one parameter is plain nuts or made a bug.
To me, that's an argument for letting zip(a) return 1-tuples, not raising an exception.
I fail to see the logic. To me, that points at the exception route: if it's probably a bug, yell loudly. -- Moshe Zadka <moshez@math.huji.ac.il> There is no GOD but Python, and HTTP is its prophet. http://advogato.org/person/moshez
"MZ" == Moshe Zadka <moshez@math.huji.ac.il> writes:
MZ> I fail to see the logic. To me, that points at the exception MZ> route: if it's probably a bug, yell loudly. Hmm, thinking about the ambiguity your brought up with zip-with-no-args, which I agree with more than zip-with-one-arg, I might be willing to concede the exception for anything less than 2 sequences provided. At the very least, I will work this into the PEP so Guido can rule by (informed) decree. -Barry
Barry A. Warsaw <bwarsaw@beopen.com>:
High bit: Guido strongly prefers zip() as the name of the built-in. All the suggestions so far have pluses and minuses, and in the absence of a clear winner, he'd just as soon go with the Haskell name. I agree.
Grumble. Not too happy with the conclusion, but can't argue with the logic. I somewhat reluctantly agree. -- <a href="http://www.tuxedo.org/~esr">Eric S. Raymond</a> Gun Control: The theory that a woman found dead in an alley, raped and strangled with her panty hose, is somehow morally superior to a woman explaining to police how her attacker got that fatal bullet wound. -- L. Neil Smith
I would say that you should make the ambiguous single-param case illegal. For one of the examples you say:
for i in zip(a):
Why would you ever do that??? The only way that would make sense to me if zip([[1],[2]])==zip([1],[2]), which I would dislike. Use zip(*[[1],[2]]) instead. -- Paul Prescod - Not encumbered by corporate consensus It's difficult to extract sense from strings, but they're the only communication coin we can count on. - http://www.cs.yale.edu/~perlis-alan/quotes.html
"Barry A. Warsaw" wrote:
I've made a final pass through PEP 201, Parallel Iteration. For those of you without easy access to the CVS tree, I'm including it below. You can also access it through the web at
Great work. It's a pleasure to read.
High bit: Guido strongly prefers zip() as the name of the built-in. All the suggestions so far have pluses and minuses, and in the absence of a clear winner, he'd just as soon go with the Haskell name. I agree.
I agree (giving +1 would be blasphemy, right?<wink>). One important remark: What about unzip()? Shouldn't there be the reverse function? One example (which I really wrote not so long ago in an ANN context): def normalize(seq): # scale to values in 0..1 return map(lambda x, max=max(seq), min=min(seq): (x-min)/(max-min), seq) # from file with: x1 y1 z1\nx2 y2 z2\n etc. data = [[1.0, 2.0, 3.0], [-1.0, 1.0, -1.0], [-1.0, -2.0, -3.0], [2.0, 3.0, 1.0]] # zip related data fields, normalize that field, unzip data = unzip(map(normalize,zip(data)))
Please see the new reference implementation, which includes a __len__() method, and the discussion of the rejected elaborations. There's still the open issue of what zip(sequence_a) should return.
I think zip([1, 2, 3]) should give ((1), (2), (3)). My reasons? 1) It feels right to me. 2) It is the way Lisp (or Scheme, don't remember) deals with it. [Haskell's zip-unzip is limited to two lists] By the way, yet another example for unzip(): mylist = [(1,), (2,), (3,)] What looks better? a) newlist = map(lambda x:x[0], mylist) b) newlist = unzip(mylist) advocating-unzip-as-an-overridable-builtin-ly y'rs Peter -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de
On Tue, 18 Jul 2000, Peter Schneider-Kamp wrote:
One important remark: What about unzip()? Shouldn't there be the reverse function? One example (which I really wrote not so long ago in an ANN context):
Evan Simpson (a colleague here at digicool, who'd be a good addition to python-dev) noticed that unzip is unnecessary - zip is its own inverse. Since this is a worthwhile observation, i'll indulge a bit and present it in evan's terms - particularly noting a different packaging that i think is pretty cool, as methods on sequences.
From evan@digicool.com Mon Jul 17 20:17:30 2000 Date: Mon, 17 Jul 2000 14:36:18 -0400 From: Evan Simpson <evan@digicool.com> To: Ken Manheimer <klm@digicool.com> Subject: Next Braid
How's this:
([1, 2, 3], ['a', 'b', 'c']).braid() == ((1, 'a'), (2, 'b'), (3, 'c')) [[1, 2, 3], ['a', 'b', 'c']].braid() == [(1, 'a'), (2, 'b'), (3, 'c')]
and
[[1,2,3], ['a']].braid(None) == [(1, 'a'), (2, None), (3, None)]
Also, notice that if 's' is a rectangular sequence of sequences, then for all 'a' and 'b':
s.braid()[a][b] == s[b][a] by definition, so
s.braid().braid()[a][b] == s.braid()[b][a] == s[a][b] and therefore,
braid is its own inverse, and we don't need unbraid.
Cheers,
Evan @ digicool & 4-am
I can't do better than that, but i have one further elaboration i'd like to see. If the subject sequence already contains some elements, the result sequence is a copy of the originals with the new braided, whoops zipped elements appended - this way, braids can be cumulative.
[(1, 3), (2, 4)].braid(('a', 'b'), ('x', 'y')) [(1, 3, 'a', 'x'), (2, 4, 'b', 'y')]
Anyway, we don't need an unzip/unbraid. ! Ken klm@digicool.com
Whoops, i got my section wrong. Disregard the last part. On Mon, 17 Jul 2000, Ken Manheimer wrote:
On Tue, 18 Jul 2000, Peter Schneider-Kamp wrote:
One important remark: What about unzip()? Shouldn't there be the reverse function? One example (which I really wrote not so long ago in an ANN context):
Evan Simpson (a colleague here at digicool, who'd be a good addition to python-dev) noticed that unzip is unnecessary - zip is its own inverse. Since this is a worthwhile observation, i'll indulge a bit and present it in evan's terms - particularly noting a different packaging that i think is pretty cool, as methods on sequences.
From evan@digicool.com Mon Jul 17 20:17:30 2000 Date: Mon, 17 Jul 2000 14:36:18 -0400 From: Evan Simpson <evan@digicool.com> To: Ken Manheimer <klm@digicool.com> Subject: Next Braid
How's this:
([1, 2, 3], ['a', 'b', 'c']).braid() == ((1, 'a'), (2, 'b'), (3, 'c')) [[1, 2, 3], ['a', 'b', 'c']].braid() == [(1, 'a'), (2, 'b'), (3, 'c')]
and
[[1,2,3], ['a']].braid(None) == [(1, 'a'), (2, None), (3, None)]
Also, notice that if 's' is a rectangular sequence of sequences, then for all 'a' and 'b':
s.braid()[a][b] == s[b][a] by definition, so
s.braid().braid()[a][b] == s.braid()[b][a] == s[a][b] and therefore,
braid is its own inverse, and we don't need unbraid.
Cheers,
Evan @ digicool & 4-am
I can't do better than that, but i have one further elaboration i'd like to see. If the subject sequence already contains some elements, the result sequence is a copy of the originals with the new braided, whoops zipped elements appended - this way, braids can be cumulative.
[(1, 3), (2, 4)].braid(('a', 'b'), ('x', 'y')) [(1, 3, 'a', 'x'), (2, 4, 'b', 'y')]
Anyway, we don't need an unzip/unbraid. !
Ken klm@digicool.com
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://www.python.org/mailman/listinfo/python-dev
Ken Manheimer <klm@digicool.com>:
([1, 2, 3], ['a', 'b', 'c']).braid() == ((1, 'a'), (2, 'b'), (3, 'c')) [[1, 2, 3], ['a', 'b', 'c']].braid() == [(1, 'a'), (2, 'b'), (3, 'c')]
and
[[1,2,3], ['a']].braid(None) == [(1, 'a'), (2, None), (3, None)]
This is rather cool -- I especially like the implied feature of deriving the type of the result from the type of the outermost level of the object. I would be +1 on making zip/braid a sequence method rather than a function. -- <a href="http://www.tuxedo.org/~esr">Eric S. Raymond</a> I don't like the idea that the police department seems bent on keeping a pool of unarmed victims available for the predations of the criminal class. -- David Mohler, 1989, on being denied a carry permit in NYC
Ken Manheimer wrote:
Evan Simpson (a colleague here at digicool, who'd be a good addition to python-dev) noticed that unzip is unnecessary - zip is its own inverse. Since this is a worthwhile observation, i'll indulge a bit and present it in evan's terms - particularly noting a different packaging that i think is pretty cool, as methods on sequences.
Yeah, but this is only true if we let zip() be a sequence method (or a function taking a sequence): for x,y in [[1, 2, 3], [4, 5, 6]].zip(): for x,y in zip([[1, 2, 3], [4, 5, 6]]): for x,y in zip([1, 2, 3], [4, 5, 6]): To be fair I'll do second one: a = [1, 2, 3] b = [4, 5, 6] c = [7, 8, 9] for x,y,z in [a, b, c].zip(): for x,y,z in zip([a, b, c]): for x,y,z in zip(a, b, c): I still think the third form is more explicit, but I am +0 on making zip() a sequence method. Peter -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de
peter wrote:
I still think the third form is more explicit, but I am +0 on making zip() a sequence method.
Remember the "string join" debate? There is no sequence base class in Python (it's an informal inter- face, not a type), so where would you put that method? As specified, "zip" has a *single* implementation which takes any number of objects implementing the sequence interface, and return an object implementing the same interface. Can it be much simpler? -1 on a method. Python is moving towards more genericness (if you accept an integer/sequence/string, you should be able to deal with any kind of integer/sequence/string). Adding a method would be a move in the opposite direction. </F>
On Tue, Jul 18, 2000 at 11:23:38AM +0000, Peter Schneider-Kamp wrote:
Ken Manheimer wrote:
Evan Simpson (a colleague here at digicool, who'd be a good addition to python-dev) noticed that unzip is unnecessary - zip is its own inverse. Since this is a worthwhile observation, i'll indulge a bit and present it in evan's terms - particularly noting a different packaging that i think is pretty cool, as methods on sequences.
for x,y,z in [a, b, c].zip():
-1. Completely counter-intuitive, in my opinion... It would be nice if having a list of lists to marry() is a common operation, but I doubt it is. Having to create one just for the sake of zip() looks silly to me.
for x,y,z in zip([a, b, c]):
-1. What's the point ? What's the difference ? What would 'zip([a,b,c],[d,e,f])' do ?
for x,y,z in zip(a, b, c):
Still +0 on this. I still prefer the altered syntax 'for' loop :-) And marry() over zip(). But I can live with this. -- Thomas Wouters <thomas@xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
"KM" == Ken Manheimer <klm@digicool.com> writes:
KM> Evan Simpson (a colleague here at digicool, who'd be a good KM> addition to python-dev) noticed that unzip is unnecessary - KM> zip is its own inverse. Only if the sequences are of the same length. I've added this to the PEP, using the extended call syntax. Note that when the sequences are of the same length, zip() is reversible: >>> a = (1, 2, 3) >>> b = (4, 5, 6) >>> x = zip(a, b) >>> y = zip(*x) # alternatively, apply(zip, x) >>> z = zip(*y) # alternatively, apply(zip, y) >>> x [(1, 4), (2, 5), (3, 6)] >>> y [(1, 2, 3), (4, 5, 6)] >>> z [(1, 4), (2, 5), (3, 6)] >>> x == z 1 It is not possible to reverse zip this way when the sequences are not all the same length. -Barry
On Mon, 17 Jul 2000, Barry A. Warsaw wrote: [On the builtin that resembles lambda *args: map(None, *args)]
High bit: Guido strongly prefers zip() as the name of the built-in. All the suggestions so far have pluses and minuses, and in the absence of a clear winner, he'd just as soon go with the Haskell name. I agree.
Please see the new reference implementation, which includes a __len__() method, and the discussion of the rejected elaborations. There's still the open issue of what zip(sequence_a) should return.
I'm +1 on that. One thing that bugs me is that we're adding a builtin type. Why not stick with the Python implementation? -- Moshe Zadka <moshez@math.huji.ac.il> There is no GOD but Python, and HTTP is its prophet. http://advogato.org/person/moshez
"MZ" == Moshe Zadka <moshez@math.huji.ac.il> writes:
MZ> I'm +1 on that. One thing that bugs me is that we're adding a MZ> builtin type. Why not stick with the Python implementation? I've been thinking a little bit about the other generators, especially irange(). I wonder if we shouldn't just be putting zip() and friends in their own Python module and not make them builtins? -Barry
On Wed, 19 Jul 2000, Barry A. Warsaw wrote:
"MZ" == Moshe Zadka <moshez@math.huji.ac.il> writes:
MZ> I'm +1 on that. One thing that bugs me is that we're adding a MZ> builtin type. Why not stick with the Python implementation?
I've been thinking a little bit about the other generators, especially irange(). I wonder if we shouldn't just be putting zip() and friends in their own Python module and not make them builtins?
I'm +1. def-irange(seq):-zip(xrange(len(seq)), seq)-ly y'rs, Z. -- Moshe Zadka <moshez@math.huji.ac.il> There is no GOD but Python, and HTTP is its prophet. http://advogato.org/person/moshez
On Wed, Jul 19, 2000 at 12:00:17AM -0400, Barry A. Warsaw wrote:
"MZ" == Moshe Zadka <moshez@math.huji.ac.il> writes:
MZ> I'm +1 on that. One thing that bugs me is that we're adding a MZ> builtin type. Why not stick with the Python implementation?
I've been thinking a little bit about the other generators, especially irange(). I wonder if we shouldn't just be putting zip() and friends in their own Python module and not make them builtins?
Which are you proposing: 1) implement in a Python module. clients use some_module.zip() 2) implement in a Python module. insert into builtins. clients use zip(). ?? Cheers, -g -- Greg Stein, http://www.lyra.org/
"GS" == Greg Stein <gstein@lyra.org> writes:
>> other generators, especially irange(). I wonder if we >> shouldn't just be putting zip() and friends in their own Python >> module and not make them builtins? GS> Which are you proposing: GS> 1) implement in a Python module. clients use some_module.zip() Yes. GS> 2) implement in a Python module. insert into builtins. clients GS> use zip(). No. -Barry
Greg Stein wrote:
On Wed, Jul 19, 2000 at 12:00:17AM -0400, Barry A. Warsaw wrote:
> "MZ" == Moshe Zadka <moshez@math.huji.ac.il> writes:
MZ> I'm +1 on that. One thing that bugs me is that we're adding a MZ> builtin type. Why not stick with the Python implementation?
I've been thinking a little bit about the other generators, especially irange(). I wonder if we shouldn't just be putting zip() and friends in their own Python module and not make them builtins?
Which are you proposing:
1) implement in a Python module. clients use some_module.zip() 2) implement in a Python module. insert into builtins. clients use zip().
FYI, mx.Tools implements 1) + 2). I usually put the "rom mx.Tools import NewBuiltins" at the top of the module source file so that it becomes clear that the module uses the set of new builtins available through mx.Tools. YMMV. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/
"Barry A. Warsaw" wrote:
...
I've been thinking a little bit about the other generators, especially irange(). I wonder if we shouldn't just be putting zip() and friends in their own Python module and not make them builtins?
I prefer builtins (or in some cases methods) for this type of slice and dice functionality. It's not related to some particular datatype or domain. -- Paul Prescod - Not encumbered by corporate consensus Just how compassionate can a Republican get before he has to leave the GOP and join Vegans for Global Justice? ... One moment, George W. Bush is holding a get-to-know-you meeting with a bunch of gay Republicans. The next he is holding forth on education or the environment ... It is enough to make a red-blooded conservative choke on his spotted-owl drumstick. - April 29th, Economist
On Wed, 19 Jul 2000, Barry A. Warsaw wrote:
I've been thinking a little bit about the other generators, especially irange(). I wonder if we shouldn't just be putting zip() and friends in their own Python module and not make them builtins?
I'd say put both zip() and irange() in builtins. They're useful enough. (Ideally range(), zip(), and irange() would all make generators one day, and xrange() could be deprecated.) I would prefer zip() and irange() to make generators now -- because this means you could *pass* them generators, and they wouldn't force their arguments to enumerate everything first! For example, make readlines() a generator too and you've got for lnum, line in irange(file.readlines()): print "%5d: %s" % (lnum, line), How pretty is that? There's no issue with using generators in place of lists, as far as i can tell, since (as has already been suggested) you just turn the generator into a real list if somebody calls one of the methods on it or uses it where a tuple is expected. -- ?!ng
On Wed, 19 Jul 2000, Ka-Ping Yee wrote:
On Wed, 19 Jul 2000, Barry A. Warsaw wrote:
I've been thinking a little bit about the other generators, especially irange(). I wonder if we shouldn't just be putting zip() and friends in their own Python module and not make them builtins?
I'd say put both zip() and irange() in builtins. They're useful enough.
One of the things I love about Python is the clean namespaces. Each builtin function takes away some of the cleanliness. It's a tradeof, but I think we should be wary of tossing stuff into builtins just because we can't find a good place for them. -- Moshe Zadka <moshez@math.huji.ac.il> There is no GOD but Python, and HTTP is its prophet. http://advogato.org/person/moshez
Moshe Zadka wrote:
...
One of the things I love about Python is the clean namespaces. Each builtin function takes away some of the cleanliness. It's a tradeof, but I think we should be wary of tossing stuff into builtins just because we can't find a good place for them.
I feel the same way, but I hate importing basic functionality necessary for every single script, no matter what the domain. I wish there weren't so many exceptions directly in builtins. They should go in a sub-module __builtins__.exceptions . -- Paul Prescod - Not encumbered by corporate consensus Just how compassionate can a Republican get before he has to leave the GOP and join Vegans for Global Justice? ... One moment, George W. Bush is holding a get-to-know-you meeting with a bunch of gay Republicans. The next he is holding forth on education or the environment ... It is enough to make a red-blooded conservative choke on his spotted-owl drumstick. - April 29th, Economist
"KY" == Ka-Ping Yee <ping@lfw.org> writes:
KY> (Ideally range(), zip(), and irange() would all make KY> generators one day, and xrange() could be deprecated.) I agree! So maybe what we need is a general generator type written in C. Could the xrange type be subverted to this purpose? -Barry
On Wed, Jul 19, 2000 at 02:48:20PM -0400, Barry A. Warsaw wrote:
"KY" == Ka-Ping Yee <ping@lfw.org> writes: KY> (Ideally range(), zip(), and irange() would all make KY> generators one day, and xrange() could be deprecated.)
I agree! So maybe what we need is a general generator type written in C. Could the xrange type be subverted to this purpose?
Not likely. You'd have to write the assignment functions (changing the object into a list-type if that is done) and most of the other stuff has to be rewritten: almost all of xrange() is specific for PyInt's. It stores step, start, end and repeater (for 'xrange * 10') as longs, and calculates return values for getitem and repr and the like from those. Other than a standard object boilerplate, it's pretty useless for more generic generators. -- Thomas Wouters <thomas@xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
participants (15)
-
bwarsaw＠beopen.com -
esr＠thyrsus.com -
Fredrik Lundh -
Fredrik Lundh -
Gordon McMillan -
Greg Stein -
Greg Wilson -
Ka-Ping Yee -
Ken Manheimer -
M.-A. Lemburg -
Moshe Zadka -
Paul Prescod -
Peter Schneider-Kamp -
Thomas Wouters -
Vladimir.Marangozov＠inrialpes.fr