Syntax for tuple comprehension, something like: (i, for i in range(10)) This shouldn't result in ambiguity, since generators need to be in parentheses anyway: (i, for i in range(10)) vs (1, (i for i in range(10)))
On Mon, Nov 18, 2019 at 2:56 PM Daniel Zeng <daniel.zeng@gmail.com> wrote:
Syntax for tuple comprehension, something like: (i, for i in range(10))
Is this meant to produce a tuple that contains inner tuples, or is it the same as tuple(i for i in range(10)) ?
This shouldn't result in ambiguity, since generators need to be in parentheses anyway: (i, for i in range(10)) vs (1, (i for i in range(10)))
Technically no, but it would be very confusing. (i for i in range(10)) # genexp (i, for i in range(10)) # your proposal ((i,) for i in range(10)) # genexp (i,1 for i in range(10)) # still error Is the comma just magic that says "tuple comprehension, not genexp"? ChrisA
Is the comma just magic that says "tuple comprehension, not genexp"? I think that my proposal is consistent with Python's current tuple syntax, especially for one element:
((1,),) # all very similar, but ((1,),) ((1,)) # (1,) ((1),) # (1,) ((1)) # 1 I would say the comma says "tuple, not expression": (1+2)*3 # 9 (1+2,)*3 # (3, 3, 3)
To me, however, that would look like a one-element genexp tuple, and would be too easily confused with something like ((i for i in range(10),)
On Mon, 18 Nov 2019 at 03:57, Daniel Zeng <daniel.zeng@gmail.com> wrote:
I'm going to ignore the discussion on where precisely the commas should go in this (it's very relevant, but different from the point I want to make) and ask, is this really something that happens often enough to need dedicated syntax, rather than simply using tuple(i for i in range(10))? I don't think orthogonality/consistency arguments really work here (as they are the ones that get bogged down in questions about where the commas go) so I'd want to see some evidence that this solved a significant problem in real-world code. Paul
On 11/17/19 9:39 PM, Daniel Zeng wrote:
The existing comprehensions support the extensible collections (list, dict, set, generator). These are things with arbitrary, flexible, and often (at coding time) unknown lengths. Tuples are not like that: they are meant more for collections where the length is known at coding time because the positions of the elements have meaning: (name, address, phonenumber), or (host, port), etc. This is not an absolute rule, but is a very common distinction between lists and tuples. There are uses for comprehension syntax for making tuples, but they occur far far less frequently than the comprehensions we have. To me, the existing way to do it makes perfect sense, and exactly expresses what needs to happen: tuple(i for i in range(10)) (ignoring the two or three ways that this is an unrealistic toy example). There's no need to invent a tortured and confusing syntax to support making tuples with comprehensions. We already have a fine way to do it. --Ned.
On Nov 18, 2019, at 04:32, Ned Batchelder <ned@nedbatchelder.com> wrote:
There are uses for comprehension syntax for making tuples, but they occur far far less frequently than the comprehensions we have. To me, the existing way to do it makes perfect sense, and exactly expresses what needs to happen:
tuple(i for i in range(10))
It’s worth noting that this isn’t 100% identical to what a tuple comprehension would do—but only to know exactly what the differences are so we can conclude they don’t matter. def stop(): raise StopIteration a = list(stop() if i%3==2 else I for i in range(10)) b = [stop() if i%3==2 else I for i in range(10)] The second line will construct a = [0,1], but the third will raise a StopIteration to the user and not construct anything or bind b. You can also put the stop() in an if clause. Back around 3.2-ish, the way comprehensions were defined implied that these should be identical. I looked into whether we could add this behavior to listcomps (or actually implement a listcomp on top of a genexpr without a performance cost). But the overwhelming consensus was that this is an antipattern and the right fix was to change the docs. (Also, there used to be more varied opportunities for this kind of hackery, but now they raise RuntimeError.) So, a tuple comprehension would disallow this antipattern, it would avoid the problem that tuple is a plain old name that can be shadowed or rebound, it would be slightly faster (in 3.2 or wherever, IIRC, it was around a 30% difference in overhead, but the overhead isn’t significant except in trivial comprehensions), and the stack would look different from functions called from inside the comprehension. There’s also the readability difference, but I can’t see how a comma in an odd place is more readable than the name tuple is, unless people need to embed these in the middle of more complex expressions so brevity helps (as people definitely do need to do for, e.g., simple tuple literals). Is any of that a good enough reason for new display syntax? I don’t think so.
On Tue, Nov 19, 2019 at 4:41 AM Andrew Barnert via Python-ideas <python-ideas@python.org> wrote:
As of current versions of Python, the difference is now relatively insignificant: the third one halts with StopIteration, but the second will halt with RuntimeError chained to StopIteration. Neither of them will give you a truncated list :) ChrisA
On Mon, Nov 18, 2019, at 13:35, Serhiy Storchaka wrote:
It'd be (*(i for i in range(10)),) but point taken. In fact, after thinking about it some more, I'm not sure if there's any more efficient way to implement it anyway, considering that there's no way for the loop that would be generated to append to a tuple.
On Nov 18, 2019, at 13:14, Random832 <random832@fastmail.com> wrote:
At the C API level, tuples are mutable, and this is safe to use as long as you’re sure no Python code has a reference to the tuple. So there’s no reason we couldn’t have a special TUPLE_APPEND op that’s only used in tuple comprehensions, the same way LIST_APPEND is used in list comprehensions. (In a comprehension you can be sure that no Python code has a reference to the object that you’re building until after you’ve finished building it.) Of course this is all CPython-specific; you’d have to think about how, say, PyPy could implement the feature before declaring that it’s definitely not a problem. But I _think_ it’s not a problem.
On Mon, Nov 18, 2019 at 2:56 PM Daniel Zeng <daniel.zeng@gmail.com> wrote:
Syntax for tuple comprehension, something like: (i, for i in range(10))
Is this meant to produce a tuple that contains inner tuples, or is it the same as tuple(i for i in range(10)) ?
This shouldn't result in ambiguity, since generators need to be in parentheses anyway: (i, for i in range(10)) vs (1, (i for i in range(10)))
Technically no, but it would be very confusing. (i for i in range(10)) # genexp (i, for i in range(10)) # your proposal ((i,) for i in range(10)) # genexp (i,1 for i in range(10)) # still error Is the comma just magic that says "tuple comprehension, not genexp"? ChrisA
Is the comma just magic that says "tuple comprehension, not genexp"? I think that my proposal is consistent with Python's current tuple syntax, especially for one element:
((1,),) # all very similar, but ((1,),) ((1,)) # (1,) ((1),) # (1,) ((1)) # 1 I would say the comma says "tuple, not expression": (1+2)*3 # 9 (1+2,)*3 # (3, 3, 3)
To me, however, that would look like a one-element genexp tuple, and would be too easily confused with something like ((i for i in range(10),)
On Mon, 18 Nov 2019 at 03:57, Daniel Zeng <daniel.zeng@gmail.com> wrote:
I'm going to ignore the discussion on where precisely the commas should go in this (it's very relevant, but different from the point I want to make) and ask, is this really something that happens often enough to need dedicated syntax, rather than simply using tuple(i for i in range(10))? I don't think orthogonality/consistency arguments really work here (as they are the ones that get bogged down in questions about where the commas go) so I'd want to see some evidence that this solved a significant problem in real-world code. Paul
On 11/17/19 9:39 PM, Daniel Zeng wrote:
The existing comprehensions support the extensible collections (list, dict, set, generator). These are things with arbitrary, flexible, and often (at coding time) unknown lengths. Tuples are not like that: they are meant more for collections where the length is known at coding time because the positions of the elements have meaning: (name, address, phonenumber), or (host, port), etc. This is not an absolute rule, but is a very common distinction between lists and tuples. There are uses for comprehension syntax for making tuples, but they occur far far less frequently than the comprehensions we have. To me, the existing way to do it makes perfect sense, and exactly expresses what needs to happen: tuple(i for i in range(10)) (ignoring the two or three ways that this is an unrealistic toy example). There's no need to invent a tortured and confusing syntax to support making tuples with comprehensions. We already have a fine way to do it. --Ned.
On Nov 18, 2019, at 04:32, Ned Batchelder <ned@nedbatchelder.com> wrote:
There are uses for comprehension syntax for making tuples, but they occur far far less frequently than the comprehensions we have. To me, the existing way to do it makes perfect sense, and exactly expresses what needs to happen:
tuple(i for i in range(10))
It’s worth noting that this isn’t 100% identical to what a tuple comprehension would do—but only to know exactly what the differences are so we can conclude they don’t matter. def stop(): raise StopIteration a = list(stop() if i%3==2 else I for i in range(10)) b = [stop() if i%3==2 else I for i in range(10)] The second line will construct a = [0,1], but the third will raise a StopIteration to the user and not construct anything or bind b. You can also put the stop() in an if clause. Back around 3.2-ish, the way comprehensions were defined implied that these should be identical. I looked into whether we could add this behavior to listcomps (or actually implement a listcomp on top of a genexpr without a performance cost). But the overwhelming consensus was that this is an antipattern and the right fix was to change the docs. (Also, there used to be more varied opportunities for this kind of hackery, but now they raise RuntimeError.) So, a tuple comprehension would disallow this antipattern, it would avoid the problem that tuple is a plain old name that can be shadowed or rebound, it would be slightly faster (in 3.2 or wherever, IIRC, it was around a 30% difference in overhead, but the overhead isn’t significant except in trivial comprehensions), and the stack would look different from functions called from inside the comprehension. There’s also the readability difference, but I can’t see how a comma in an odd place is more readable than the name tuple is, unless people need to embed these in the middle of more complex expressions so brevity helps (as people definitely do need to do for, e.g., simple tuple literals). Is any of that a good enough reason for new display syntax? I don’t think so.
On Tue, Nov 19, 2019 at 4:41 AM Andrew Barnert via Python-ideas <python-ideas@python.org> wrote:
As of current versions of Python, the difference is now relatively insignificant: the third one halts with StopIteration, but the second will halt with RuntimeError chained to StopIteration. Neither of them will give you a truncated list :) ChrisA
On Mon, Nov 18, 2019, at 13:35, Serhiy Storchaka wrote:
It'd be (*(i for i in range(10)),) but point taken. In fact, after thinking about it some more, I'm not sure if there's any more efficient way to implement it anyway, considering that there's no way for the loop that would be generated to append to a tuple.
On Nov 18, 2019, at 13:14, Random832 <random832@fastmail.com> wrote:
At the C API level, tuples are mutable, and this is safe to use as long as you’re sure no Python code has a reference to the tuple. So there’s no reason we couldn’t have a special TUPLE_APPEND op that’s only used in tuple comprehensions, the same way LIST_APPEND is used in list comprehensions. (In a comprehension you can be sure that no Python code has a reference to the object that you’re building until after you’ve finished building it.) Of course this is all CPython-specific; you’d have to think about how, say, PyPy could implement the feature before declaring that it’s definitely not a problem. But I _think_ it’s not a problem.
participants (7)
-
Andrew Barnert -
Chris Angelico -
Daniel Zeng -
Ned Batchelder -
Paul Moore -
Random832 -
Serhiy Storchaka