On 10/10/05, Ron Adam wrote:

The problem is the '*' means different things depending on where it's located. In a function def, it means to group or to pack, but from the calling end it's used to unpack. I don't expect it to change as it's been a part of Python for a long time and as long as it's only used with argument passing it's not too difficult to keep straight.

My concern is if it's used outside of functions, then on the left hand side of assignments, it will be used to pack, but if used on the right hand side it will be to unpack. And if it becomes as common place as I think it will, it will present confusing uses and or situations where you may have to think, "oh yeah, it's umm... unpacking here and umm... packing there, but multiplying there". The point is it could be a stumbling block, especially for new Python users. So I think a certain amount of caution should be in order on this item. At least check that it's doesn't cause confusing situations.

This particular concern, I believe, is a fallacy. If you squint the right way, using *rest for both packing and unpacking is totally logical. If a, b, *rest = (1, 2, 3, 4, 5) puts 1 into a, 2 into b, and (3, 4, 5) into rest, then it's totally logical and symmetrical if after that x = a, b, *rest puts (1, 2, 3, 4, 5) into x. BTW, what should [a, b, *rest] = (1, 2, 3, 4, 5) do? Should it set rest to (3, 4, 5) or to [3, 4, 5]? Suppose the latter. Then should we allow [*rest] = x as alternative syntax for rest = list(x) ? And then perhaps *rest = x should mean rest = tuple(x) Or should that be disallowed and would we have to write *rest, = x analogous to singleton tuples? There certainly is a need for doing the same from the end: *rest, a, b = (1, 2, 3, 4, 5) could set rest to (1, 2, 3), a to 4, and b to 5. From there it's a simple step towards a, b, *rest, d, e = (1, 2, 3, 4, 5) meaning a, b, rest, d, e = (1, 2, (3,), 4, 5) and so on. Where does it stop? BTW, and quite unrelated, I've always felt uncomfortable that you have to write f(a, b, foo=1, bar=2, *args, **kwds) I've always wanted to write that as f(a, b, *args, foo=1, bar=2, **kwds) but the current grammar doesn't allow it. Still -0 on the whole thing, -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Greg Ewing wrote:

Guido van Rossum wrote:

...

BTW, what should

[a, b, *rest] = (1, 2, 3, 4, 5)

do? Should it set rest to (3, 4, 5) or to [3, 4, 5]?

Whatever type is chosen, it should be the same type, always. The rhs could be any iterable, not just a tuple or a list. Making a special case of preserving one or two types doesn't seem worth it to me.

And, for consistency with functions, the type chosen should be a tuple. I'm also trying to figure out why you would ever write: [a, b, c, d] = seq instead of: a, b, c, d = seq or: (a, b, c, d) = seq It's not like the square brackets generate different code: Py> def foo(): ... x, y = 1, 2 ... (x, y) = 1, 2 ... [x, y] = 1, 2 ... Py> dis.dis(foo) 2 0 LOAD_CONST 3 ((1, 2)) 3 UNPACK_SEQUENCE 2 6 STORE_FAST 1 (x) 9 STORE_FAST 0 (y) 3 12 LOAD_CONST 4 ((1, 2)) 15 UNPACK_SEQUENCE 2 18 STORE_FAST 1 (x) 21 STORE_FAST 0 (y) 4 24 LOAD_CONST 5 ((1, 2)) 27 UNPACK_SEQUENCE 2 30 STORE_FAST 1 (x) 33 STORE_FAST 0 (y) 36 LOAD_CONST 0 (None) 39 RETURN_VALUE So my vote would actually go for deprecating the use of square brackets to surround an assignment target list - it makes it look like an actual list object should be involved somewhere, but there isn't one.

...

? And then perhaps

*rest = x

should mean

rest = tuple(x)

Or should that be disallowed

Why bother? What harm would result from the ability to write that?

Given that: def foo(*args): print args is legal, I would have no problem with "*rest = x" being legal.

...

There certainly is a need for doing the same from the end:

*rest, a, b = (1, 2, 3, 4, 5)

I wouldn't mind at all if *rest were only allowed at the end. There's a pragmatic reason for that if nothing else: the rhs can be any iterable, and there's no easy way of getting "all but the last n" items from a general iterable.

Agreed. The goal here is to make the name binding rules consistent between for loops, tuple assigment and function entry, not to create different rules.

...

Where does it stop? For me, it stops with *rest only allowed at the end, and always yielding a predictable type (which could be either tuple or list, I don't care).

For me, it stops when the rules for positional name binding are more consistent across operations that bind names (although complete consistency isn't possible, given that function calls don't unpack sequences automatically). Firstly, let's list the operations that permit name binding to a list of identifiers: - binding of function parameters to function arguments - binding of assignment target list to assigned sequence - binding of iteration variables to iteration values However, that function argument case needs to be recognised as a two step operation, whereby the arguments are *always* packed into a tuple before being bound to the parameters. That is something very vaguely like: if numargs > 0: if numargs == 1: argtuple = args, # One argument gives singleton tuple else: argtuple = args # More arguments gives appropriate tuple argtuple += tuple(starargs) # Extended arguments are added to the tuple param1, param2, *rest = argtuple # Tuple is unpacked to parameters This means that the current behaviour of function parameters is actually the same as assignment target lists and iteration variables, in that the argument tuple is *always* unpacked into the parameter list - the only difference is that a single argument is always considered a singleton tuple. You can get the same behaviour with target lists and iteration variables by only using tuples of identifiers as targets (i.e., use "x," rather than just "x"). So the proposal at this stage is simply to mimic the unpacking of the argument tuple into the formal parameter list in the other two name list binding cases, such that the pseudocode above would actually do the same thing as building an argument list and binding it to its formal parameters does. Now, when it came to tuple *packing* syntax (i.e., extended call syntax) The appropriate behaviour would be for: 1, 2, 3, *range(10) to translate (roughly) to: (1, 2, 3) + tuple(range(10)) However, given that the equivalent code works just fine anywhere it really matters (assignment value, return value, yield value), and is clearer about what is going on, this option is probably worth avoiding.

...

BTW, and quite unrelated, I've always felt uncomfortable that you have to write

f(a, b, foo=1, bar=2, *args, **kwds)

I've always wanted to write that as

f(a, b, *args, foo=1, bar=2, **kwds)

Yes, I'd like that too, with the additional meaning that foo and bar can only be specified by keyword, not by position.

Indeed. It's a (minor) pain that optional flag variables and variable length argument lists are currently mutually exclusive. Although, if you had that rule, I'd want to be able to write: def f(a, b, *, foo=1, bar=2): pass to get a function which required exactly two positional arguments, but had a couple of optional keyword arguments, rather than having to do: def f(a, b, *args, foo=1, bar=2): if args: raise TypeError("f() takes exactly 2 positional arguments (%d given)", 2 + len(args)) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

Nick Coghlan wrote:

Greg Ewing wrote:

...

Guido van Rossum wrote:

...

BTW, what should

[a, b, *rest] = (1, 2, 3, 4, 5)

do? Should it set rest to (3, 4, 5) or to [3, 4, 5]?

Whatever type is chosen, it should be the same type, always. The rhs could be any iterable, not just a tuple or a list. Making a special case of preserving one or two types doesn't seem worth it to me.

And, for consistency with functions, the type chosen should be a tuple.

I'm also trying to figure out why you would ever write: [a, b, c, d] = seq

instead of: a, b, c, d = seq

or: (a, b, c, d) = seq

[...]

So my vote would actually go for deprecating the use of square brackets to surround an assignment target list - it makes it look like an actual list object should be involved somewhere, but there isn't one.

But don't forget that at present unpacking can be used at several levels:

...

...

((a, b), c) = ((1, 2), 3) a, b, c (1, 2, 3)

So presumably you'd need to be able to say ((a, *b), c, *d) = ((1, 2, 3), 4, 5, 6) and see a, b, c, d == 1, (2, 3), 4, (5, 6) if we are to retain today's multi-level consistency. And are you also proposing to allow a, *b = [1] to put the empty list into b, or is that an unpacking error?

...

...

? And then perhaps

*rest = x

should mean

rest = tuple(x)

Or should that be disallowed

Why bother? What harm would result from the ability to write that?

Given that: def foo(*args): print args

is legal, I would have no problem with "*rest = x" being legal.

Though presumably we'd still be raising TypeError is x weren't a sequence.

...

...

There certainly is a need for doing the same from the end:

*rest, a, b = (1, 2, 3, 4, 5)

I wouldn't mind at all if *rest were only allowed at the end. There's a pragmatic reason for that if nothing else: the rhs can be any iterable, and there's no easy way of getting "all but the last n" items from a general iterable.

Agreed. The goal here is to make the name binding rules consistent between for loops, tuple assigment and function entry, not to create different rules.

...

...

Where does it stop?

For me, it stops with *rest only allowed at the end, and always yielding a predictable type (which could be either tuple or list, I don't care).

For me, it stops when the rules for positional name binding are more consistent across operations that bind names (although complete consistency isn't possible, given that function calls don't unpack sequences automatically).

Hmm. Given that today we can write

...

...

def foo((a, b), c): ... print a, b, c ... foo((1, 2, 3)) Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: foo() takes exactly 2 arguments (1 given) foo((1, 2), 3) 1 2 3

does this mean that you'd also like to be able to write def foo((a, *b), *c): print a, b, c and then call it like foo((1, 2, 3, 4), 5, 6) to see 1, (2, 3, 4), (5, 6) [...]

...

...

BTW, and quite unrelated, I've always felt uncomfortable that you have to write

f(a, b, foo=1, bar=2, *args, **kwds)

I've always wanted to write that as

f(a, b, *args, foo=1, bar=2, **kwds)

Yes, I'd like that too, with the additional meaning that foo and bar can only be specified by keyword, not by position.

Indeed. It's a (minor) pain that optional flag variables and variable length argument lists are currently mutually exclusive. Although, if you had that rule, I'd want to be able to write:

def f(a, b, *, foo=1, bar=2): pass

to get a function which required exactly two positional arguments, but had a couple of optional keyword arguments, rather than having to do:

def f(a, b, *args, foo=1, bar=2): if args: raise TypeError("f() takes exactly 2 positional arguments (%d given)", 2 + len(args))

I do feel that for Python 3 it might be better to make a clean separation between keywords and positionals: in other words, of the function definition specifies a keyword argument then a keyword must be used to present it. This would allow users to provide an arbitrary number of positionals rather than having them become keyword arguments. At present it's difficult to specify that. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/

Steve Holden wrote:

So presumably you'd need to be able to say

((a, *b), c, *d) = ((1, 2, 3), 4, 5, 6)

Yes.

a, *b = [1]

to put the empty list into b, or is that an unpacking error?

Empty sequence in b (of whatever type is chosen).

does this mean that you'd also like to be able to write

def foo((a, *b), *c):

That would follow, yes.

I do feel that for Python 3 it might be better to make a clean separation between keywords and positionals: in other words, of the function definition specifies a keyword argument then a keyword must be used to present it.

But then how would you give a positional arg a default value without turning it into a keyword arg? It seems to me that the suggested extension covers all the possibilities quite nicely. You can have named positional args with or without default values, optional extra positional args with *, named keyword-only args with or without default values, and unnamed extra keyword-only args with **. The only thing it doesn't give you directly is mandatory positional-only args, and you can get that by catching them with * and unpacking them afterwards. This would actually synergise nicely with * in tuple unpacking: def f(*args): a, b, *rest = args And with one further small extension, you could even get that into the argument list as well: def f(*(a, b, *rest)): ... -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+

Nick Coghlan wrote:

So my vote would actually go for deprecating the use of square brackets to surround an assignment target list - it makes it look like an actual list object should be involved somewhere, but there isn't one.

I've found myself using square brackets a few times for more complicated unpacking, e.g.: try: x, y = args except ValueError: [x], y = args, None where I thought that (x,), y = args, None would have been more confusing. OTOH, I usually end up rewriting this to x, = args y = None because even the bracketed form is a bit confusing. So I wouldn't really be upset if the brackets went away. STeVe -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy

Greg Ewing wrote:

Guido van Rossum wrote:

...

BTW, what should

[a, b, *rest] = (1, 2, 3, 4, 5)

do? Should it set rest to (3, 4, 5) or to [3, 4, 5]?

Whatever type is chosen, it should be the same type, always. The rhs could be any iterable, not just a tuple or a list. Making a special case of preserving one or two types doesn't seem worth it to me.

I don't think that [a, b, c] = iterable is good style right now, so I'd say that [a, b, *rest] = iterable should be disallowed or be the same as with parentheses. It's not intuitive that rest could be a list here.

...

? And then perhaps

*rest = x

should mean

rest = tuple(x)

Or should that be disallowed

Why bother? What harm would result from the ability to write that?

...

There certainly is a need for doing the same from the end:

*rest, a, b = (1, 2, 3, 4, 5)

I wouldn't mind at all if *rest were only allowed at the end. There's a pragmatic reason for that if nothing else: the rhs can be any iterable, and there's no easy way of getting "all but the last n" items from a general iterable.

...

Where does it stop?

For me, it stops with *rest only allowed at the end, and always yielding a predictable type (which could be either tuple or list, I don't care).

+1. Tuple is more consistent.

...

BTW, and quite unrelated, I've always felt uncomfortable that you have to write

f(a, b, foo=1, bar=2, *args, **kwds)

I've always wanted to write that as

f(a, b, *args, foo=1, bar=2, **kwds)

Yes, I'd like that too, with the additional meaning that foo and bar can only be specified by keyword, not by position.

That would be a logical consequence. But one should also be able to give default values for positional parameters. So: foo(a, b, c=1, *args, d=2, e=5, **kwargs) ^^^^^^^^^ ^^^^^^^^ positional only with kw or with kw Reinhold -- Mail address is perfectly valid!

Ron Adam wrote:

I wonder if something like the following would fulfill the need?

Funny you should say that. . . A pre-PEP propsing itertools.iunpack (amongst other things): http://mail.python.org/pipermail/python-dev/2004-November/050043.html And the reason that PEP was never actually created: http://mail.python.org/pipermail/python-dev/2004-November/050068.html Obviouly, I've changed my views over the last year or so ;) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

Ron Adam wrote:

I wonder if you make '*' work outside of functions arguments lists, if requests to do the same for '**' would follow?

Only if keyword unpacking were to be permitted elsewhere first. That is: Py> data = dict(a=1, b=2, c=3) Py> (a, b, c) = **data Py> print a, b, c (1, 2, 3) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

Ron Adam wrote:

My concern is if it's used outside of functions, then on the left hand side of assignments, it will be used to pack, but if used on the right hand side it will be to unpack.

I don't see why that should be any more confusing than the fact that commas denote tuple packing on the right and unpacking on the left. Greg