Hi, I was just wondering why unpack of sequences did not follow same behavior of functions parameters. I mean: first, *rest = 'a b c'.split() should work in python, why doesn't it? -- Carlo Pires
On 29 août 2012, at 15:50, Carlo Pires <carlopires@gmail.com> wrote:
It does work in python 3, and following p3's improved unpacking semantics. Demo: http://pythonic.pocoo.org/2008/2/17/new-in-python-3:-extended-unpacking Pep: http://www.python.org/dev/peps/pep-3132/
On Wed, Aug 29, 2012 at 6:57 AM, Ned Batchelder <ned@nedbatchelder.com> wrote:
Note, however, that putting too much faith in the analogy between assignment and function calls leads to disappointment. E.g. there's a big difference between a = 42 and a = 42, while there is no difference between f(42) and f(42,) Also of course assignment has no equivalent to keyword parameters, nor does it (currently) allow a "lone star" -- although it would be handy to be able to say a, b, * = xs as a shorthand for a, b, *_ = xs del _ Also of note: after a, b, *z = xs z is a list, whereas in def foo(a, b, *z): ... z will be a tuple. (The latter is arguably a design mistake.) Plus in Pythin 3 you can't say def foo(a, (b, c), d): ... whereas the tuple unpacking even supports nested stars: a, (b, c, *d), e, *f = [1, range(10), 2, 3, 4, 5} PS. Asking "why does Python not do X" is generally considered a leading question. -- --Guido van Rossum (python.org/~guido)
On Wed, Aug 29, 2012 at 12:17 PM, Cesare Di Mauro <cesare.di.mauro@gmail.com> wrote:
a: x, b: y, c: z = {'a': 'x', 'b': 'y', 'c': 'z'}
Cesare
"a": x, "b": y, "c": z = {'a': 'x', 'b': 'y', 'c': 'z'} or {"a": x, "b": y, "c": z} = {'a': 'x', 'b': 'y', 'c': 'z'} would admit non-string keys. IMO, a more useful thing would be attribute-based unpacking--I feel like I do that a ton more often. Mike
Cesare Di Mauro wrote:
Would this assign 'a' to a or just use a as key for the lookup ? If the former, where would you take the lookup order from ? If the latter, what about keys that are not valid Python identifiers ? mxTools has a function extract() to extract values from a mapping or sequence object: extract(object,indices[,defaults]) Builds a list with entries object[index] for each index in the sequence indices. (see http://www.egenix.com/products/python/mxBase/mxTools/doc/)
mx.Tools.extract(d, ('a', 'b', 'c')) ['x', 'y', 'z']
IMO, that's a much cleaner way to express what you'd like Python to do. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 29 2012)
2012-10-23: Python Meeting Duesseldorf ... 55 days to go 2012-08-28: Released mxODBC 3.2.0 ... http://egenix.com/go31 2012-08-20: Released mxODBC.Connect 2.0.0 ... http://egenix.com/go30 ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
2012/8/29 M.-A. Lemburg <mal@egenix.com>
Would this assign 'a' to a or just use a as key for the lookup ?
If the former, where would you take the lookup order from ? If the latter, what about keys that are not valid Python identifiers ?
The latter. But we already have problems with Python identifiers:
It wasn't a feature blocker... mxTools has a function extract() to extract values from a mapping
Yes, may be an extract method for mappings will be finer. Cesare
On 30/08/12 02:10, Mike Graham wrote:
a, b, x, y = **mapping could be equivalent to: a, b, x, y = mapping['a'], mapping['b'], mapping['x'], mapping['y'] I don't have good syntax for the attribute equivalent, but I do have bad syntax for it: a, b, x, y = **.obj # like a, b, x, y = obj.a, obj.b, obj.x, obj.y Or you could use wrap your object in a helper class: class AttrGetter: def __init__(self, obj): self.obj = obj def __getitem__(self, key): return getattr(self.obj, key) a, b, x, y = **AttrGetter(obj) -- Steven
On 30/08/12 03:01, Steven D'Aprano wrote:
Oh, I forgot to mention... extra keys are ignored. That's because in my experience, you are likely to have a mapping with many different keys (say, a set of config options in a dict) and you only want to extract a few at a time. Unlike unpacking a tuple, you're unlikely to need *every* field at once. A potentially useful extension to the idea is to capture the extra items in a dict: a, b, x, y, **extras = **mapping which is like: a, b, x, y = [mapping[name] for name in ('a', 'b', 'x', 'y')] extras = dict((k, v) for k,v in mapping.items() if k not in ('a', 'b', 'x', 'y')) -- Steven
On Wed, Aug 29, 2012 at 10:18 AM, Steven D'Aprano <steve@pearwood.info> wrote:
Sounds to me like there are so many potential variations here that it's probably better not to add a language feature and let users write what they want. (My personal variation would be to use m.get(k) instead of m[k].) I think if I encountered the line a, b, c = **foo I would have only the vaguest intuition for its meaning -- and I'd be even less sure about something syntactically similarly plausible like self.a, self.b, self.c = **foo -- --Guido van Rossum (python.org/~guido)
On Wed, Aug 29, 2012 at 1:28 PM, Guido van Rossum <guido@python.org> wrote:
it's probably better not to add a language feature and let users write what they want.
How can users write a solution that does not require repetition of variable names? Of course I can write something like a, b, c = [m.get(x) for x in ('a', 'b', 'c')] but when I have more and longer names, this gets tedious. As far as syntax goes, I also find a, b, c = **m somewhat unintuitive. I would prefer {a, b, c} = m and {a, b, c, **rest} = m.
I'd be even less sure about something syntactically similarly plausible like
self.a, self.b, self.c = **foo
I don't think unpacking into attributes is as useful as unpacking into locals. Object attribute lists are often available programmatically and it is a simple matter to supply an _update() function that can be used as self._update(locals()) after values have been assigned to locals or simply use self._update(m) directly. (I recall that something like {'a': x, 'b': y} = m has been suggested and rejected in the past. That syntax also required explicit specification of the keys to be unpacked.)
Alexander Belopolsky wrote:
Not that I'd recommend doing this, but it's possible :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 29 2012)
2012-10-23: Python Meeting Duesseldorf ... 55 days to go 2012-08-28: Released mxODBC 3.2.0 ... http://egenix.com/go31 2012-08-20: Released mxODBC.Connect 2.0.0 ... http://egenix.com/go30 ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On 30/08/12 04:16, M.-A. Lemburg wrote:
Try it inside a function. py> def test(): ... d = dict(a=1, b=2, c=3) ... locals().update(d) ... print a ... py> test() Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 4, in test NameError: global name 'a' is not defined -- Steven
Steven D'Aprano wrote:
Yeah, that's because functions use fast locals, which locals() only mirrors as dictionary. You have to play some tricks to it work... def f(d): from test import * locals().update(d) print a,b,c d = dict(a=1, b=2, c=3) f(d) ...and because you're not supposed to do this, you get a SyntaxWarning :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 29 2012)
2012-10-23: Python Meeting Duesseldorf ... 55 days to go 2012-08-28: Released mxODBC 3.2.0 ... http://egenix.com/go31 2012-08-20: Released mxODBC.Connect 2.0.0 ... http://egenix.com/go30 ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
Guido van Rossum wrote:
Also it won't work in Python 3.
The star import is only used to trigger a call to PyFrame_LocalsToFast(). In Python 3, the only way to trigger such a call is by using a call level trace function... or by exposing the C function in Python.
-- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 29 2012)
2012-10-23: Python Meeting Duesseldorf ... 55 days to go 2012-08-28: Released mxODBC 3.2.0 ... http://egenix.com/go31 2012-08-20: Released mxODBC.Connect 2.0.0 ... http://egenix.com/go30 ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On Wed, Aug 29, 2012 at 12:30 PM, M.-A. Lemburg <mal@egenix.com> wrote:
I don't believe that's the whole story. In Python 2, the import * changes the semantics of locals. (So does 'exec' BTW.) Example:
Note the difference in generated bytecode:
4 13 LOAD_NAME 1 (x) 16 PRINT_ITEM 17 PRINT_NEWLINE 18 LOAD_CONST 0 (None) 21 RETURN_VALUE
3 13 LOAD_GLOBAL 1 (x) 16 PRINT_ITEM 17 PRINT_NEWLINE 18 LOAD_CONST 0 (None) 21 RETURN_VALUE
Compare line 13 in both: LOAD_NAME vs. LOAD_GLOBAL. Effectively, in f(), the locals are dynamic (this is how they were implemented in Python 0.0). In g() they are not, so the compiler decides that x can't be a local, and generates a LOAD_GLOBAL. In Python 3, all functions behave like g(): import * is no longer allowed, and exec() is no longer treated special (it's no longer a reserved keyword). -- --Guido van Rossum (python.org/~guido)
Guido van Rossum wrote:
You're right. The effect is not the calling of the PyFrame function, but that of the compiler generating different bytecode with the star import. In fact, the way the star import calls the PyFrame API overrides the locals update. It first copies the fast locals to the locals dictionary (overriding the updates applied before the import), then adds the symbols from the import and then copies the locals from the dictionary back to the fast slots. Here's a version that uses fast locals a,b,c: def h(d): a = 0 b = 0 c = 0 locals().update(d) from test import * print a,b,c It prints 0 0 0. The dis output: 20 0 LOAD_CONST 1 (0) 3 STORE_FAST 1 (a) 21 6 LOAD_CONST 1 (0) 9 STORE_FAST 2 (b) 22 12 LOAD_CONST 1 (0) 15 STORE_FAST 3 (c) 23 18 LOAD_NAME 0 (locals) 21 CALL_FUNCTION 0 24 LOAD_ATTR 1 (update) 27 LOAD_FAST 0 (d) 30 CALL_FUNCTION 1 33 POP_TOP 24 34 LOAD_CONST 2 (-1) 37 LOAD_CONST 3 (('*',)) 40 IMPORT_NAME 2 (test) 43 IMPORT_STAR 25 44 LOAD_FAST 1 (a) 47 PRINT_ITEM 48 LOAD_FAST 2 (b) 51 PRINT_ITEM 52 LOAD_FAST 3 (c) 55 PRINT_ITEM 56 PRINT_NEWLINE 57 LOAD_CONST 0 (None) 60 RETURN_VALUE
That's good. I don't think that any of this is really needed in Python. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 30 2012)
2012-10-23: Python Meeting Duesseldorf ... 54 days to go 2012-08-28: Released mxODBC 3.2.0 ... http://egenix.com/go31 2012-08-20: Released mxODBC.Connect 2.0.0 ... http://egenix.com/go30 ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On Wed, Aug 29, 2012 at 11:07 AM, Alexander Belopolsky <alexander.belopolsky@gmail.com> wrote:
Maybe they can't. So what? There's a limit to the applicability of DRY. Most solutions I've seen for this particular set of issues are worse than the problem they are trying to solve. Another antipattern IMO is sometimes seen in constructors: class C: def __init__(self, **kwds): self.__dict__.update(kwds) This has nothing to recommend it.
But {a, b, c} is already a set. I'd expect set-like semantics, perhaps assigning the keys of a 3-element set in arbitrary order to the variables a, b and c.
Another antipattern.
I don't believe a valid syntax proposal can come out of this thread. -- --Guido van Rossum (python.org/~guido)
On Wed, Aug 29, 2012 at 2:21 PM, Guido van Rossum <guido@python.org> wrote:
I considered this before posting and I think the potential for such confusion is rather low. We already allow [a,b,c] = and (a,b,c) = even though tuples are supposed to be immutable and (a,b,c) = may look like a syntax error to someone unfamiliar with unpacking. I don't see much of a problem with having the same syntactic elements have different meaning when they appear as in an assignment target.
On 2012-08-29, at 20:21 , Guido van Rossum wrote:
But {a, b, c} is already a set.
On the rhs, the lhs is expected to have different semantics: (a, b, c) = … matches any 3-items iterable (including sets), not necessarily a tuple. If there really is a need for disambiguation, colons could be added after the keys ({a:, b:, c:}), but I don't think the original syntax is confusing (let alone enough to warrant this). After all, {} is for dictionaries before it's for sets, I'm guessing most Python users still associate braces with dictionaries more than with sets.
On Wed, Aug 29, 2012 at 4:45 PM, Guido van Rossum <guido@python.org> wrote:
But {a, b, c} doesn't look like a dict.
locals().update(m) does not look like a dict either, but many people expect it to work and get bitten when it does not.
Please give it up.
Do you ask to give up the {a, b, c} = idea or any ideas that import mappings into local variables? For example some abuse of the with statement would allow with m in a, b, c: # a, b, and c are local here I am not really proposing this, just giving an example for the range of possibilities. I am sure an acceptable syntax can be found if DRY is considered important enough, but if any automation of a = m['a'] b = m['b'] c = m['c'] is deemed to be an anti-pattern, then this thread is better to stop.
Just backing Guido up here (not that he needs it): we've put a lot of work into making sure that the Python 3 compiler can see all local name definitions at compile time. Ideas that involve reverting that simply aren't going to be accepted. I'm personally less opposed to ideas for a new assignment statement that explicitly treats the lhs as a parameter definition and the rhs as an argument list and binds values to names accordingly, but even that would be a hard sell. -- Sent from my phone, thus the relative brevity :)
On Thu, Aug 30, 2012 at 8:18 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Expanding on this, now that I'm back at a real computer. Argument-parameter binding is actually a fairly powerful name binding operation, quite distinct from ordinary assignment and tuple unpacking. It consists of two separate (often confused) components: - the parameter specification - the argument list The parameter spec is what appears in the function *definition*, and is really the main item of interest reported by the new inspect.Signature objects in 3.3. A parameter spec allows you to do several things: - define the names that will be bound locally - define default values to be assigned to those names - provide named holders for excess positional and keyword arguments - indicate that certain values can *only* be supplied by name, and not by position The argument list is what appears in a function *call*, and also has several interesting features: - can provide explicit positional arguments - can provide explicit keyword arguments - can provide an iterable of positional arguments - can provide a mapping of additional keyword arguments It *may* make sense to decouple this name binding operation from function calls (and Signature.bind) and make it available as a language primitive in the form of a statement. However, it's *not* the same thing as an ordinary assignment statement or tuple unpacking, and we shouldn't try to jam it implicitly into the existing assignment statement. There are certainly valid use cases for such a construct. One example is support for positional only arguments. Currently, handling positional only arguments with reasonable error messages requires something like the following: def _bind_args(a=None, b=None): return a, b def f(*args): a, b = _bind_args(*args) Note that the "_bind_args" function exists solely to get access to the argument/parameter binding behaviour. Another example is unpacking values from a dictionary, which can be done using a similar technique: def _bind_kwds(a=None, b=None, **other): return a, b, other def f(mapping): a, b, other = _bind_kwds(**mapping) The obvious form for such a statement is "LHS OP RHS", however syntactic ambiguity in the evaluation of both the LHS and RHS (relative to normal assigment) would likely prevent that. As a sketch, I'll present a notation inspired by Ruby's block parameter syntax and emphasising the link to def statements: def f(*args): |a=None, b=None| def= *args def f(mapping): |a=None, b=None, **other| def= **mapping This is fairly magical, but hopefully the intent is clear: the LHS is enclosed in "|" characters to resolve the syntactic ambiguity problem for the LHS, while a form of augmented assignment "def=" is used to prevent assignment chaining and to resolve any syntactic ambiguity for the RHS. It may be that there's no solution to this problem that *doesn't* look magical. However, if people are interested in an enhanced form of name binding that's more powerful than the current assignment statement and tuple unpacking, then liberating parameter binding from function calls is the way to go. Unlike other proposals, it doesn't make the language more complicated, because parameter binding is something people learning Python already need to understand. Indeed, in some ways it would make the language *simpler*, since it would allow the explanation of parameter binding to be clearly decoupled from the explanation of function calls and definitions. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 8/29/2012 9:07 PM, Nick Coghlan wrote:
I have described this as cross-namespace assignment. The important point is that aliasing between and within namespaces has same effect vis-a-vis mutable objects.
Analogous to left and right sides of assignment. They were once more similar than they are now.
Interesting idea.
There seems to be too much to jam very well.
I would call it 'call assignment' or 'extended assignment' to clearly differentiate it from current augmented assignment, which it is not. (=) would be a possible syntax I have no idea whether the parser could handle this
-- Terry Jan Reedy
On 30 August 2012 02:07, Nick Coghlan <ncoghlan@gmail.com> wrote:
I had a similar thought. My initial idea was to introduce a new keyword "bind" to introduce a binding assignment: bind a=None, b=None: *args bind a=None, b=None, **other: **mapping You *might* be able to reuse the def keyword, but I suspect the ambiguity would be too difficult to resolve. Rereading the above, the only major issue I see with this is that the comma/colon distinction is visually a bit too light. Adding parens might help, but probably gets a bit too punctuation-heavy: bind (a=None, b=None): *args bind (a=None, b=None, **other): **mapping Paul.
On Thu, Aug 30, 2012 at 5:57 PM, Paul Moore <p.f.moore@gmail.com> wrote:
We use "bind" as a method name in the standard library. Other possible keywords are likely to have the same problem - it's not a space rich in suitable terminology. Opening the statement with "|" would suffice though - it's otherwise invalid at the start of an expression, so it gives a lot of freedom in the grammar details for the rest of the statement, including the spelling of the assignment details (you could even reuse a single "=", something like ":=" or "()=" or else Terry's TIE fighter notation "(=)") Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 30/08/12 18:38, Nick Coghlan wrote:
The Moby thesaurus lists 269 synonyms for "bind", and in my opinion not a single one other than bind itself is appropriate. There are 125 synonyms for "glue" (with some significant overlap), including "yoke" which I think holds promise. It's unlikely to exist in many people's code, and it has the right meaning of joining together. But I'm not sure that the associations with oxen yoked together is good or helpful. -- Steven
On 30/08/12 11:07, Nick Coghlan wrote:
You're talking about the *argument list* binding operations, yes? The stuff that happens when the function is called, not the parameter spec.
Not only shouldn't we, but we *can't*, due to backward compatibility. This is probably obvious to Nick, but for the benefit of anyone else reading this who struggled as much as I did to see why we couldn't just "upgrade" the assignment statement to work like function call binding, here's a sketch of why we can't. Suppose the assignment statement was upgraded. Then we could do things like this: a, b, c=DEFAULT, **kwargs = 1, 2, 3, d="extra" and it would bind: a = 1 b = 2 c = 3 kwargs = {"d": "extra"} but DEFAULT would necessarily be unchanged. Just as if you called a function def spam(a, b, c=DEFAULT, **kwargs). Similarly if you did this: a, b, c=DEFAULT = 1, 2, 3 you would expect to get the bindings: a = 1 b = 2 c = 3 also with the default value DEFAULT unchanged. But that second example is already legal Python, and it works like this: py> DEFAULT = 42 py> a, b, c=DEFAULT = 1, 2, 3 py> a, b, c (1, 2, 3) py> DEFAULT (1, 2, 3) Somebody is relying on this behaviour, and so we cannot change assignment to work like function argument binding without breaking backwards compatibility. There may be other problems too, but for me, this backwards compatibility issue convinced me that regular assignment cannot be made to match function argument binding. I'm not convinced that we *should* expose function argument binding as a language primitive, but if we do, we can't use the regular assignment statement, we would need something new.
I think you need to explain that in further detail. Suppose we used (making something up here) "::=" as the operator. Then e.g.: a, b, c=DEFAULT, d=None :== 1, 2, c=3 seems unambiguous to me, provided: 1) you can't chain multiple ::= operators; 2) tuples on either side have to be delimited by parentheses, exactly the same as is already the case for function parameter lists and function calls, e.g.: py> def f(a, b=1,2,3): File "<stdin>", line 1 def f(a, b=1,2,3): ^ SyntaxError: invalid syntax Can you explain where the ambiguity in things like this would lie, because I'm just not seeing it. -- Steven
On Fri, Aug 31, 2012 at 1:14 AM, Steven D'Aprano <steve@pearwood.info> wrote:
You're looking too far ahead - the (deliberately dumb) parser would choke because: a, b, c = DEFAULT, d = None is a valid assignment statement. It's one that can't work (due to the length mismatch in the tuple unpacking), but it's syntactically valid. Thus, by the time it reached the "::=", the parser wouldn't be able to backtrack and revise its opinion from "ordinary assignment statement" to "parameter binding operation". Instead, it would bail out, whinging about an unexpected token (in reality, you'd hit problems long before that stage - the parser generator would have quit on you, complaining about ambiguous syntax, probably with an entirely unhelpful error message). In some cases (such as augmented assignment) we can deal with that kind of ambiguity by making the parser permissive, and catching syntax errors at a later stage in the compilation pipeline (generally either symbol analysis or bytecode generation - I can't think of any reason error detection would ever be delayed until the peephole optimisation step). However, delayed validation would be a very poor approach in this case, since there are valid parameter specifications that are also completely valid assignment statements (as shown above). You'd have to maintain a complex set of "maybe this, maybe that" constructs in order to pull it off, which would be very, very, ugly (and make a mess of the AST). Furthermore, if something is hard for the *computer* to parse, odds are pretty good that humans are also going to struggle with it (natural language notwithstanding - in that case, computers are playing catchup with millions of years of evolutionary development). Fortunately, the entire ambiguity problem can go away *if* you can find a suitable prefixed or delimited syntax. Once you manage that, then the prefix or opening delimiter serves as a marker for both the compiler and the human reader that something different is happening. That's why I stole the |parameter-spec| notation directly from Ruby's block syntax for my syntactic sketch - it's a delimited syntax that doesn't conflict with other Python syntax ("|" can't start an expression - it can only appear as part of an operator or an augmented assignment statement). The choice of operator for the name binding operation itself would then be fairly arbitrary, since the parser would already know it was dealing with a parameter binding operation. "|param-spec| ::= arglist", "|param-spec| <- arglist", "|param-spec| def= arglist", "|param-spec| ()= arglist", "|param-spec| (=) arglist", "|param-spec|(arglist)" "|param-spec| from arglist" would all be viable options from a syntactic point of view (YMMV wildly from a readability point of view, of course) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Fri, Aug 31, 2012 at 1:14 AM, Steven D'Aprano <steve@pearwood.info> wrote:
You're talking about the *argument list* binding operations, yes? The stuff that happens when the function is called, not the parameter spec.
A function call binds parameter names to argument values or, equivalently, argument values to parameter names, so it doesn't really matter whether you call it "parameter (name) binding" or "argument (value) binding". It's the same operation, just emphasising either the source of the names or the source of the values. I normally talk about binding names to values, since a name can only refer to one value at a time, but there may be multiple names bound to any given value. Hence, I prefer to call this process "parameter binding" - it's the step of assigning a value to each of the parameter names before the body of the function starts executing. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 30/08/12 03:28, Guido van Rossum wrote:
The obvious problem is that without language support, they have to repeat the key names. We've been here before with decorators, where you had to repeat the function name three times before the @ syntax was introduced. It's not too bad for namedtuple, because you only have to repeat one name: spam = namedtuple('spam', field_names) although that's still a mild code smell. But once you have multiple targets, it gets ugly soon, e.g. the typical enumeration (anti-)pattern: RED, YELLOW, BLUE, GREEN = 'RED', 'YELLOW', 'BLUE', 'GREN' Maybe it's time to tackle this once and for all and find a way to access the left-hand side names from the right-hand side. Assuming that's even possible at all, here's a wild suggestion: expand @ on the right-hand side to a quoted list of the names from the left: # Enumerations RED, YELLOW, BLUE, GREEN = @ # Dict unpacking a, b, x, y = *[mapping.get(name) for name in @] # namedtuple spam = namedtuple(@, field_names) Obviously this isn't a fully-fleshed out idea. I'm not sure what the target @ should do here: dict['key'], list[0], obj.attr, x = [something(name) for name in @] 1) Expand to ['key', '0', 'attr', 'x'] ? 2) Expand to ["dict['key']", "list[0]", "obj.attr", "x"] ? Or what happens if you have multiple = signs? a, b, c = x, y, z = @ But the easy way out is to only allow @ for the simple case and raise an exception for anything else. We could always add support for the more complicated cases in the future. -- Steven
On Wed, Aug 29, 2012 at 3:10 PM, Mike Graham <mikegraham@gmail.com> wrote:
I should apologize for bringing this up, because the analogy is actually backwards. (Or maybe I could claim that this backwardness is a good warning against hypergeneralization. :-) In function definitions, it actually means *don't allow more positional arguments*. The equivalent already exists for unpacking assignment: a, b = xs The reason separate syntax is needed in function definitions is that we occasionally wish to say "and there are no more positional parameters" but also "but there are some additional keyword-only parameters". Until unpacking assignment support an equivalent to keyword parameters with default values we won't need * there to mean "there should be no more values". But giving it the *opposite* meaning of "and ignore subsequent values" would be just perverse given what it means in function declarations. -- --Guido van Rossum (python.org/~guido)
On Thu, Aug 30, 2012 at 11:46 AM, Serhiy Storchaka <storchaka@gmail.com> wrote:
The fact that it's currently a syntax error speaks in favour of the suggestion—if it was currently valid Python, you wouldn't want to make it do anything else. That being said, `a, b, c, *_ = d` or similar is probably better than introducing a new way. Mike
Mike Graham wrote:
That being said, `a, b, c, *_ = d` or similar is probably better than introducing a new way.
It's inefficient, though, because it results in iterating over the remainder of the sequence and building a new sequence that will never be used. There's currently no way to spell the most efficient way of doing this, which is simply to stop iterating and ignore the rest of the sequence. There's also no way to unpack the head of an infinite sequence without slicing it first. -- Greg
On 31/08/12 10:17, Greg Ewing wrote:
And if the sequence is large, that might even matter. I'm not being quite as dismissive as it might sound, but in practice, this inefficiency generally does not matter. It's unlikely to be a bottleneck.
For sequences: a, b, c, d = really_long_sequence[:4] which might not be as efficient as possible, but by estimation it is within a factor of two of the most efficient as possible. For iterators, use itertools.islice.
There's also no way to unpack the head of an infinite sequence without slicing it first.
Surely this is the obvious way? head = next(sequence) # actually an iterator -- Steven
Steven D'Aprano wrote:
For sequences:
a, b, c, d = really_long_sequence[:4]
Sure, there are any number of ways of doing it, but they all involve some degree of convolution or DRY-violation. The most direct and obvious way is denied to us, because there is no way to tell the unpacking operation *not* to check for the end of the sequence afterwards. Because that unwanted check is forced on us, we have to perform even more useless work to get around it. That seems perverse to me. -- Greg
On 29 août 2012, at 15:50, Carlo Pires <carlopires@gmail.com> wrote:
It does work in python 3, and following p3's improved unpacking semantics. Demo: http://pythonic.pocoo.org/2008/2/17/new-in-python-3:-extended-unpacking Pep: http://www.python.org/dev/peps/pep-3132/
On Wed, Aug 29, 2012 at 6:57 AM, Ned Batchelder <ned@nedbatchelder.com> wrote:
Note, however, that putting too much faith in the analogy between assignment and function calls leads to disappointment. E.g. there's a big difference between a = 42 and a = 42, while there is no difference between f(42) and f(42,) Also of course assignment has no equivalent to keyword parameters, nor does it (currently) allow a "lone star" -- although it would be handy to be able to say a, b, * = xs as a shorthand for a, b, *_ = xs del _ Also of note: after a, b, *z = xs z is a list, whereas in def foo(a, b, *z): ... z will be a tuple. (The latter is arguably a design mistake.) Plus in Pythin 3 you can't say def foo(a, (b, c), d): ... whereas the tuple unpacking even supports nested stars: a, (b, c, *d), e, *f = [1, range(10), 2, 3, 4, 5} PS. Asking "why does Python not do X" is generally considered a leading question. -- --Guido van Rossum (python.org/~guido)
On Wed, Aug 29, 2012 at 12:17 PM, Cesare Di Mauro <cesare.di.mauro@gmail.com> wrote:
a: x, b: y, c: z = {'a': 'x', 'b': 'y', 'c': 'z'}
Cesare
"a": x, "b": y, "c": z = {'a': 'x', 'b': 'y', 'c': 'z'} or {"a": x, "b": y, "c": z} = {'a': 'x', 'b': 'y', 'c': 'z'} would admit non-string keys. IMO, a more useful thing would be attribute-based unpacking--I feel like I do that a ton more often. Mike
Cesare Di Mauro wrote:
Would this assign 'a' to a or just use a as key for the lookup ? If the former, where would you take the lookup order from ? If the latter, what about keys that are not valid Python identifiers ? mxTools has a function extract() to extract values from a mapping or sequence object: extract(object,indices[,defaults]) Builds a list with entries object[index] for each index in the sequence indices. (see http://www.egenix.com/products/python/mxBase/mxTools/doc/)
mx.Tools.extract(d, ('a', 'b', 'c')) ['x', 'y', 'z']
IMO, that's a much cleaner way to express what you'd like Python to do. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 29 2012)
2012-10-23: Python Meeting Duesseldorf ... 55 days to go 2012-08-28: Released mxODBC 3.2.0 ... http://egenix.com/go31 2012-08-20: Released mxODBC.Connect 2.0.0 ... http://egenix.com/go30 ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
2012/8/29 M.-A. Lemburg <mal@egenix.com>
Would this assign 'a' to a or just use a as key for the lookup ?
If the former, where would you take the lookup order from ? If the latter, what about keys that are not valid Python identifiers ?
The latter. But we already have problems with Python identifiers:
It wasn't a feature blocker... mxTools has a function extract() to extract values from a mapping
Yes, may be an extract method for mappings will be finer. Cesare
On 30/08/12 02:10, Mike Graham wrote:
a, b, x, y = **mapping could be equivalent to: a, b, x, y = mapping['a'], mapping['b'], mapping['x'], mapping['y'] I don't have good syntax for the attribute equivalent, but I do have bad syntax for it: a, b, x, y = **.obj # like a, b, x, y = obj.a, obj.b, obj.x, obj.y Or you could use wrap your object in a helper class: class AttrGetter: def __init__(self, obj): self.obj = obj def __getitem__(self, key): return getattr(self.obj, key) a, b, x, y = **AttrGetter(obj) -- Steven
On 30/08/12 03:01, Steven D'Aprano wrote:
Oh, I forgot to mention... extra keys are ignored. That's because in my experience, you are likely to have a mapping with many different keys (say, a set of config options in a dict) and you only want to extract a few at a time. Unlike unpacking a tuple, you're unlikely to need *every* field at once. A potentially useful extension to the idea is to capture the extra items in a dict: a, b, x, y, **extras = **mapping which is like: a, b, x, y = [mapping[name] for name in ('a', 'b', 'x', 'y')] extras = dict((k, v) for k,v in mapping.items() if k not in ('a', 'b', 'x', 'y')) -- Steven
On Wed, Aug 29, 2012 at 10:18 AM, Steven D'Aprano <steve@pearwood.info> wrote:
Sounds to me like there are so many potential variations here that it's probably better not to add a language feature and let users write what they want. (My personal variation would be to use m.get(k) instead of m[k].) I think if I encountered the line a, b, c = **foo I would have only the vaguest intuition for its meaning -- and I'd be even less sure about something syntactically similarly plausible like self.a, self.b, self.c = **foo -- --Guido van Rossum (python.org/~guido)
On Wed, Aug 29, 2012 at 1:28 PM, Guido van Rossum <guido@python.org> wrote:
it's probably better not to add a language feature and let users write what they want.
How can users write a solution that does not require repetition of variable names? Of course I can write something like a, b, c = [m.get(x) for x in ('a', 'b', 'c')] but when I have more and longer names, this gets tedious. As far as syntax goes, I also find a, b, c = **m somewhat unintuitive. I would prefer {a, b, c} = m and {a, b, c, **rest} = m.
I'd be even less sure about something syntactically similarly plausible like
self.a, self.b, self.c = **foo
I don't think unpacking into attributes is as useful as unpacking into locals. Object attribute lists are often available programmatically and it is a simple matter to supply an _update() function that can be used as self._update(locals()) after values have been assigned to locals or simply use self._update(m) directly. (I recall that something like {'a': x, 'b': y} = m has been suggested and rejected in the past. That syntax also required explicit specification of the keys to be unpacked.)
Alexander Belopolsky wrote:
Not that I'd recommend doing this, but it's possible :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 29 2012)
2012-10-23: Python Meeting Duesseldorf ... 55 days to go 2012-08-28: Released mxODBC 3.2.0 ... http://egenix.com/go31 2012-08-20: Released mxODBC.Connect 2.0.0 ... http://egenix.com/go30 ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On 30/08/12 04:16, M.-A. Lemburg wrote:
Try it inside a function. py> def test(): ... d = dict(a=1, b=2, c=3) ... locals().update(d) ... print a ... py> test() Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 4, in test NameError: global name 'a' is not defined -- Steven
Steven D'Aprano wrote:
Yeah, that's because functions use fast locals, which locals() only mirrors as dictionary. You have to play some tricks to it work... def f(d): from test import * locals().update(d) print a,b,c d = dict(a=1, b=2, c=3) f(d) ...and because you're not supposed to do this, you get a SyntaxWarning :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 29 2012)
2012-10-23: Python Meeting Duesseldorf ... 55 days to go 2012-08-28: Released mxODBC 3.2.0 ... http://egenix.com/go31 2012-08-20: Released mxODBC.Connect 2.0.0 ... http://egenix.com/go30 ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
Guido van Rossum wrote:
Also it won't work in Python 3.
The star import is only used to trigger a call to PyFrame_LocalsToFast(). In Python 3, the only way to trigger such a call is by using a call level trace function... or by exposing the C function in Python.
-- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 29 2012)
2012-10-23: Python Meeting Duesseldorf ... 55 days to go 2012-08-28: Released mxODBC 3.2.0 ... http://egenix.com/go31 2012-08-20: Released mxODBC.Connect 2.0.0 ... http://egenix.com/go30 ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On Wed, Aug 29, 2012 at 12:30 PM, M.-A. Lemburg <mal@egenix.com> wrote:
I don't believe that's the whole story. In Python 2, the import * changes the semantics of locals. (So does 'exec' BTW.) Example:
Note the difference in generated bytecode:
4 13 LOAD_NAME 1 (x) 16 PRINT_ITEM 17 PRINT_NEWLINE 18 LOAD_CONST 0 (None) 21 RETURN_VALUE
3 13 LOAD_GLOBAL 1 (x) 16 PRINT_ITEM 17 PRINT_NEWLINE 18 LOAD_CONST 0 (None) 21 RETURN_VALUE
Compare line 13 in both: LOAD_NAME vs. LOAD_GLOBAL. Effectively, in f(), the locals are dynamic (this is how they were implemented in Python 0.0). In g() they are not, so the compiler decides that x can't be a local, and generates a LOAD_GLOBAL. In Python 3, all functions behave like g(): import * is no longer allowed, and exec() is no longer treated special (it's no longer a reserved keyword). -- --Guido van Rossum (python.org/~guido)
Guido van Rossum wrote:
You're right. The effect is not the calling of the PyFrame function, but that of the compiler generating different bytecode with the star import. In fact, the way the star import calls the PyFrame API overrides the locals update. It first copies the fast locals to the locals dictionary (overriding the updates applied before the import), then adds the symbols from the import and then copies the locals from the dictionary back to the fast slots. Here's a version that uses fast locals a,b,c: def h(d): a = 0 b = 0 c = 0 locals().update(d) from test import * print a,b,c It prints 0 0 0. The dis output: 20 0 LOAD_CONST 1 (0) 3 STORE_FAST 1 (a) 21 6 LOAD_CONST 1 (0) 9 STORE_FAST 2 (b) 22 12 LOAD_CONST 1 (0) 15 STORE_FAST 3 (c) 23 18 LOAD_NAME 0 (locals) 21 CALL_FUNCTION 0 24 LOAD_ATTR 1 (update) 27 LOAD_FAST 0 (d) 30 CALL_FUNCTION 1 33 POP_TOP 24 34 LOAD_CONST 2 (-1) 37 LOAD_CONST 3 (('*',)) 40 IMPORT_NAME 2 (test) 43 IMPORT_STAR 25 44 LOAD_FAST 1 (a) 47 PRINT_ITEM 48 LOAD_FAST 2 (b) 51 PRINT_ITEM 52 LOAD_FAST 3 (c) 55 PRINT_ITEM 56 PRINT_NEWLINE 57 LOAD_CONST 0 (None) 60 RETURN_VALUE
That's good. I don't think that any of this is really needed in Python. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 30 2012)
2012-10-23: Python Meeting Duesseldorf ... 54 days to go 2012-08-28: Released mxODBC 3.2.0 ... http://egenix.com/go31 2012-08-20: Released mxODBC.Connect 2.0.0 ... http://egenix.com/go30 ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On Wed, Aug 29, 2012 at 11:07 AM, Alexander Belopolsky <alexander.belopolsky@gmail.com> wrote:
Maybe they can't. So what? There's a limit to the applicability of DRY. Most solutions I've seen for this particular set of issues are worse than the problem they are trying to solve. Another antipattern IMO is sometimes seen in constructors: class C: def __init__(self, **kwds): self.__dict__.update(kwds) This has nothing to recommend it.
But {a, b, c} is already a set. I'd expect set-like semantics, perhaps assigning the keys of a 3-element set in arbitrary order to the variables a, b and c.
Another antipattern.
I don't believe a valid syntax proposal can come out of this thread. -- --Guido van Rossum (python.org/~guido)
On Wed, Aug 29, 2012 at 2:21 PM, Guido van Rossum <guido@python.org> wrote:
I considered this before posting and I think the potential for such confusion is rather low. We already allow [a,b,c] = and (a,b,c) = even though tuples are supposed to be immutable and (a,b,c) = may look like a syntax error to someone unfamiliar with unpacking. I don't see much of a problem with having the same syntactic elements have different meaning when they appear as in an assignment target.
On 2012-08-29, at 20:21 , Guido van Rossum wrote:
But {a, b, c} is already a set.
On the rhs, the lhs is expected to have different semantics: (a, b, c) = … matches any 3-items iterable (including sets), not necessarily a tuple. If there really is a need for disambiguation, colons could be added after the keys ({a:, b:, c:}), but I don't think the original syntax is confusing (let alone enough to warrant this). After all, {} is for dictionaries before it's for sets, I'm guessing most Python users still associate braces with dictionaries more than with sets.
On Wed, Aug 29, 2012 at 4:45 PM, Guido van Rossum <guido@python.org> wrote:
But {a, b, c} doesn't look like a dict.
locals().update(m) does not look like a dict either, but many people expect it to work and get bitten when it does not.
Please give it up.
Do you ask to give up the {a, b, c} = idea or any ideas that import mappings into local variables? For example some abuse of the with statement would allow with m in a, b, c: # a, b, and c are local here I am not really proposing this, just giving an example for the range of possibilities. I am sure an acceptable syntax can be found if DRY is considered important enough, but if any automation of a = m['a'] b = m['b'] c = m['c'] is deemed to be an anti-pattern, then this thread is better to stop.
Just backing Guido up here (not that he needs it): we've put a lot of work into making sure that the Python 3 compiler can see all local name definitions at compile time. Ideas that involve reverting that simply aren't going to be accepted. I'm personally less opposed to ideas for a new assignment statement that explicitly treats the lhs as a parameter definition and the rhs as an argument list and binds values to names accordingly, but even that would be a hard sell. -- Sent from my phone, thus the relative brevity :)
On Thu, Aug 30, 2012 at 8:18 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Expanding on this, now that I'm back at a real computer. Argument-parameter binding is actually a fairly powerful name binding operation, quite distinct from ordinary assignment and tuple unpacking. It consists of two separate (often confused) components: - the parameter specification - the argument list The parameter spec is what appears in the function *definition*, and is really the main item of interest reported by the new inspect.Signature objects in 3.3. A parameter spec allows you to do several things: - define the names that will be bound locally - define default values to be assigned to those names - provide named holders for excess positional and keyword arguments - indicate that certain values can *only* be supplied by name, and not by position The argument list is what appears in a function *call*, and also has several interesting features: - can provide explicit positional arguments - can provide explicit keyword arguments - can provide an iterable of positional arguments - can provide a mapping of additional keyword arguments It *may* make sense to decouple this name binding operation from function calls (and Signature.bind) and make it available as a language primitive in the form of a statement. However, it's *not* the same thing as an ordinary assignment statement or tuple unpacking, and we shouldn't try to jam it implicitly into the existing assignment statement. There are certainly valid use cases for such a construct. One example is support for positional only arguments. Currently, handling positional only arguments with reasonable error messages requires something like the following: def _bind_args(a=None, b=None): return a, b def f(*args): a, b = _bind_args(*args) Note that the "_bind_args" function exists solely to get access to the argument/parameter binding behaviour. Another example is unpacking values from a dictionary, which can be done using a similar technique: def _bind_kwds(a=None, b=None, **other): return a, b, other def f(mapping): a, b, other = _bind_kwds(**mapping) The obvious form for such a statement is "LHS OP RHS", however syntactic ambiguity in the evaluation of both the LHS and RHS (relative to normal assigment) would likely prevent that. As a sketch, I'll present a notation inspired by Ruby's block parameter syntax and emphasising the link to def statements: def f(*args): |a=None, b=None| def= *args def f(mapping): |a=None, b=None, **other| def= **mapping This is fairly magical, but hopefully the intent is clear: the LHS is enclosed in "|" characters to resolve the syntactic ambiguity problem for the LHS, while a form of augmented assignment "def=" is used to prevent assignment chaining and to resolve any syntactic ambiguity for the RHS. It may be that there's no solution to this problem that *doesn't* look magical. However, if people are interested in an enhanced form of name binding that's more powerful than the current assignment statement and tuple unpacking, then liberating parameter binding from function calls is the way to go. Unlike other proposals, it doesn't make the language more complicated, because parameter binding is something people learning Python already need to understand. Indeed, in some ways it would make the language *simpler*, since it would allow the explanation of parameter binding to be clearly decoupled from the explanation of function calls and definitions. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 8/29/2012 9:07 PM, Nick Coghlan wrote:
I have described this as cross-namespace assignment. The important point is that aliasing between and within namespaces has same effect vis-a-vis mutable objects.
Analogous to left and right sides of assignment. They were once more similar than they are now.
Interesting idea.
There seems to be too much to jam very well.
I would call it 'call assignment' or 'extended assignment' to clearly differentiate it from current augmented assignment, which it is not. (=) would be a possible syntax I have no idea whether the parser could handle this
-- Terry Jan Reedy
On 30 August 2012 02:07, Nick Coghlan <ncoghlan@gmail.com> wrote:
I had a similar thought. My initial idea was to introduce a new keyword "bind" to introduce a binding assignment: bind a=None, b=None: *args bind a=None, b=None, **other: **mapping You *might* be able to reuse the def keyword, but I suspect the ambiguity would be too difficult to resolve. Rereading the above, the only major issue I see with this is that the comma/colon distinction is visually a bit too light. Adding parens might help, but probably gets a bit too punctuation-heavy: bind (a=None, b=None): *args bind (a=None, b=None, **other): **mapping Paul.
On Thu, Aug 30, 2012 at 5:57 PM, Paul Moore <p.f.moore@gmail.com> wrote:
We use "bind" as a method name in the standard library. Other possible keywords are likely to have the same problem - it's not a space rich in suitable terminology. Opening the statement with "|" would suffice though - it's otherwise invalid at the start of an expression, so it gives a lot of freedom in the grammar details for the rest of the statement, including the spelling of the assignment details (you could even reuse a single "=", something like ":=" or "()=" or else Terry's TIE fighter notation "(=)") Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 30/08/12 18:38, Nick Coghlan wrote:
The Moby thesaurus lists 269 synonyms for "bind", and in my opinion not a single one other than bind itself is appropriate. There are 125 synonyms for "glue" (with some significant overlap), including "yoke" which I think holds promise. It's unlikely to exist in many people's code, and it has the right meaning of joining together. But I'm not sure that the associations with oxen yoked together is good or helpful. -- Steven
On 30/08/12 11:07, Nick Coghlan wrote:
You're talking about the *argument list* binding operations, yes? The stuff that happens when the function is called, not the parameter spec.
Not only shouldn't we, but we *can't*, due to backward compatibility. This is probably obvious to Nick, but for the benefit of anyone else reading this who struggled as much as I did to see why we couldn't just "upgrade" the assignment statement to work like function call binding, here's a sketch of why we can't. Suppose the assignment statement was upgraded. Then we could do things like this: a, b, c=DEFAULT, **kwargs = 1, 2, 3, d="extra" and it would bind: a = 1 b = 2 c = 3 kwargs = {"d": "extra"} but DEFAULT would necessarily be unchanged. Just as if you called a function def spam(a, b, c=DEFAULT, **kwargs). Similarly if you did this: a, b, c=DEFAULT = 1, 2, 3 you would expect to get the bindings: a = 1 b = 2 c = 3 also with the default value DEFAULT unchanged. But that second example is already legal Python, and it works like this: py> DEFAULT = 42 py> a, b, c=DEFAULT = 1, 2, 3 py> a, b, c (1, 2, 3) py> DEFAULT (1, 2, 3) Somebody is relying on this behaviour, and so we cannot change assignment to work like function argument binding without breaking backwards compatibility. There may be other problems too, but for me, this backwards compatibility issue convinced me that regular assignment cannot be made to match function argument binding. I'm not convinced that we *should* expose function argument binding as a language primitive, but if we do, we can't use the regular assignment statement, we would need something new.
I think you need to explain that in further detail. Suppose we used (making something up here) "::=" as the operator. Then e.g.: a, b, c=DEFAULT, d=None :== 1, 2, c=3 seems unambiguous to me, provided: 1) you can't chain multiple ::= operators; 2) tuples on either side have to be delimited by parentheses, exactly the same as is already the case for function parameter lists and function calls, e.g.: py> def f(a, b=1,2,3): File "<stdin>", line 1 def f(a, b=1,2,3): ^ SyntaxError: invalid syntax Can you explain where the ambiguity in things like this would lie, because I'm just not seeing it. -- Steven
On Fri, Aug 31, 2012 at 1:14 AM, Steven D'Aprano <steve@pearwood.info> wrote:
You're looking too far ahead - the (deliberately dumb) parser would choke because: a, b, c = DEFAULT, d = None is a valid assignment statement. It's one that can't work (due to the length mismatch in the tuple unpacking), but it's syntactically valid. Thus, by the time it reached the "::=", the parser wouldn't be able to backtrack and revise its opinion from "ordinary assignment statement" to "parameter binding operation". Instead, it would bail out, whinging about an unexpected token (in reality, you'd hit problems long before that stage - the parser generator would have quit on you, complaining about ambiguous syntax, probably with an entirely unhelpful error message). In some cases (such as augmented assignment) we can deal with that kind of ambiguity by making the parser permissive, and catching syntax errors at a later stage in the compilation pipeline (generally either symbol analysis or bytecode generation - I can't think of any reason error detection would ever be delayed until the peephole optimisation step). However, delayed validation would be a very poor approach in this case, since there are valid parameter specifications that are also completely valid assignment statements (as shown above). You'd have to maintain a complex set of "maybe this, maybe that" constructs in order to pull it off, which would be very, very, ugly (and make a mess of the AST). Furthermore, if something is hard for the *computer* to parse, odds are pretty good that humans are also going to struggle with it (natural language notwithstanding - in that case, computers are playing catchup with millions of years of evolutionary development). Fortunately, the entire ambiguity problem can go away *if* you can find a suitable prefixed or delimited syntax. Once you manage that, then the prefix or opening delimiter serves as a marker for both the compiler and the human reader that something different is happening. That's why I stole the |parameter-spec| notation directly from Ruby's block syntax for my syntactic sketch - it's a delimited syntax that doesn't conflict with other Python syntax ("|" can't start an expression - it can only appear as part of an operator or an augmented assignment statement). The choice of operator for the name binding operation itself would then be fairly arbitrary, since the parser would already know it was dealing with a parameter binding operation. "|param-spec| ::= arglist", "|param-spec| <- arglist", "|param-spec| def= arglist", "|param-spec| ()= arglist", "|param-spec| (=) arglist", "|param-spec|(arglist)" "|param-spec| from arglist" would all be viable options from a syntactic point of view (YMMV wildly from a readability point of view, of course) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Fri, Aug 31, 2012 at 1:14 AM, Steven D'Aprano <steve@pearwood.info> wrote:
You're talking about the *argument list* binding operations, yes? The stuff that happens when the function is called, not the parameter spec.
A function call binds parameter names to argument values or, equivalently, argument values to parameter names, so it doesn't really matter whether you call it "parameter (name) binding" or "argument (value) binding". It's the same operation, just emphasising either the source of the names or the source of the values. I normally talk about binding names to values, since a name can only refer to one value at a time, but there may be multiple names bound to any given value. Hence, I prefer to call this process "parameter binding" - it's the step of assigning a value to each of the parameter names before the body of the function starts executing. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 30/08/12 03:28, Guido van Rossum wrote:
The obvious problem is that without language support, they have to repeat the key names. We've been here before with decorators, where you had to repeat the function name three times before the @ syntax was introduced. It's not too bad for namedtuple, because you only have to repeat one name: spam = namedtuple('spam', field_names) although that's still a mild code smell. But once you have multiple targets, it gets ugly soon, e.g. the typical enumeration (anti-)pattern: RED, YELLOW, BLUE, GREEN = 'RED', 'YELLOW', 'BLUE', 'GREN' Maybe it's time to tackle this once and for all and find a way to access the left-hand side names from the right-hand side. Assuming that's even possible at all, here's a wild suggestion: expand @ on the right-hand side to a quoted list of the names from the left: # Enumerations RED, YELLOW, BLUE, GREEN = @ # Dict unpacking a, b, x, y = *[mapping.get(name) for name in @] # namedtuple spam = namedtuple(@, field_names) Obviously this isn't a fully-fleshed out idea. I'm not sure what the target @ should do here: dict['key'], list[0], obj.attr, x = [something(name) for name in @] 1) Expand to ['key', '0', 'attr', 'x'] ? 2) Expand to ["dict['key']", "list[0]", "obj.attr", "x"] ? Or what happens if you have multiple = signs? a, b, c = x, y, z = @ But the easy way out is to only allow @ for the simple case and raise an exception for anything else. We could always add support for the more complicated cases in the future. -- Steven
On Wed, Aug 29, 2012 at 3:10 PM, Mike Graham <mikegraham@gmail.com> wrote:
I should apologize for bringing this up, because the analogy is actually backwards. (Or maybe I could claim that this backwardness is a good warning against hypergeneralization. :-) In function definitions, it actually means *don't allow more positional arguments*. The equivalent already exists for unpacking assignment: a, b = xs The reason separate syntax is needed in function definitions is that we occasionally wish to say "and there are no more positional parameters" but also "but there are some additional keyword-only parameters". Until unpacking assignment support an equivalent to keyword parameters with default values we won't need * there to mean "there should be no more values". But giving it the *opposite* meaning of "and ignore subsequent values" would be just perverse given what it means in function declarations. -- --Guido van Rossum (python.org/~guido)
On Thu, Aug 30, 2012 at 11:46 AM, Serhiy Storchaka <storchaka@gmail.com> wrote:
The fact that it's currently a syntax error speaks in favour of the suggestion—if it was currently valid Python, you wouldn't want to make it do anything else. That being said, `a, b, c, *_ = d` or similar is probably better than introducing a new way. Mike
Mike Graham wrote:
That being said, `a, b, c, *_ = d` or similar is probably better than introducing a new way.
It's inefficient, though, because it results in iterating over the remainder of the sequence and building a new sequence that will never be used. There's currently no way to spell the most efficient way of doing this, which is simply to stop iterating and ignore the rest of the sequence. There's also no way to unpack the head of an infinite sequence without slicing it first. -- Greg
On 31/08/12 10:17, Greg Ewing wrote:
And if the sequence is large, that might even matter. I'm not being quite as dismissive as it might sound, but in practice, this inefficiency generally does not matter. It's unlikely to be a bottleneck.
For sequences: a, b, c, d = really_long_sequence[:4] which might not be as efficient as possible, but by estimation it is within a factor of two of the most efficient as possible. For iterators, use itertools.islice.
There's also no way to unpack the head of an infinite sequence without slicing it first.
Surely this is the obvious way? head = next(sequence) # actually an iterator -- Steven
Steven D'Aprano wrote:
For sequences:
a, b, c, d = really_long_sequence[:4]
Sure, there are any number of ways of doing it, but they all involve some degree of convolution or DRY-violation. The most direct and obvious way is denied to us, because there is no way to tell the unpacking operation *not* to check for the end of the sequence afterwards. Because that unwanted check is forced on us, we have to perform even more useless work to get around it. That seems perverse to me. -- Greg
participants (17)
-
Alexander Belopolsky -
Carlo Pires -
Cesare Di Mauro -
Ethan Furman -
Greg Ewing -
Guido van Rossum -
M.-A. Lemburg -
Masklinn -
Mathias Panzenböck -
Mike Graham -
MRAB -
Ned Batchelder -
Nick Coghlan -
Paul Moore -
Serhiy Storchaka -
Steven D'Aprano -
Terry Reedy