keyword arguments everywhere (stdlib) - issue8706
Something I'd like to bring up at the language summit next week if we have time (suggestion: limit it to 20 minutes), lets start discussion now: 1) Should _*all_* Python base types support keyword arguments on all of their methods. 2) Should _*all_* stdlib extension modules support keyword arguments on all of their functions and methods? Assumptions: any newly supported keyword arguments would have meaningful names (and the documentation updated to reflect their name). How should "_*all_*" be defined in each of the above. One example against _*all_* is trivial single-argument functions such as chr(). One example for is the str.find method's start parameter being more obvious if named. An example of a rule of thumb that could be proposed: * All optional arguments should also be accepted as keywords. * Any public function or method that require multiple arguments should accept them as keywords. Why propose this? For consistency and to promote readable code. A reasonably often heard complaint from people new to Python that I have heard is how some things work as keyword arguments and some don't. Pure Python code can't (easily without gross hacks) be written to refuse to accept keyword arguments, the standard types and library should not be different. Documentation and docstrings have at times been inconsistent in the past with respect to what they call arguments and would be cleaned up as part of this to match the actual keyword argument accepted (that is worth doing on its own even without changing what the code accepts). A decision with discussion notes on this may be PEP worthy. for reference - this is http://bugs.python.org/issue8706 -gps
Gregory P. Smith wrote:
+1 on adding keyword arguments to built-in methods and functions where they would help readability, e.g str.find(c, start=23), even if this happens in a ad-hoc fashion. +0 on forcing *all* built-in methods and functions to be updated to take keyword arguments out of a sense of purity, e.g. ord(char='c'). I think that "all built-ins should take keywords, so as to minimise the difference between them and pure-Python functions" is an admirable ideal. But it is an ideal, a nice-to-have rather than a must-have. -- Steven
On Fri, Mar 2, 2012 at 6:43 PM, Steven D'Aprano <steve@pearwood.info> wrote:
Indeed, this is the approach we have taken to date. For example, str.split() recently gained keyword support for 3.3 because "text.split(maxsplit=1)" is less cryptic than "text.split(None, 1)". It makes the most sense when at least one of the following holds: - the second argument accepts a number that is unclear if you're not familiar with the full function signature - the earlier arguments have sensible default values that you'd prefer not to override So +1 on declaring "make X support keyword arguments" non-controversial for multi-argument functions, +0 on also doing so for single argument functions, but -0 on attempting to boil the ocean and fix them wholesale. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Fri, Mar 2, 2012 at 4:13 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Hm. I think for many (most?) 1-arg and selected 2-arg functions (and rarely 3+-arg functions) this would reduce readability, as the example of ord(char=x) showed. I would actually like to see a syntactic feature to state that an argument *cannot* be given as a keyword argument (just as we already added syntax to state that it *must* be a keyword). One area where I think adding keyword args is outright wrong: Methods of built-in types or ABCs and that are overridable. E.g. consider the pop() method on dict. Since the argument name is currently undocumented, if someone subclasses dict and overrides this method, or if they create another mutable mapping class that tries to emulate dict using duck typing, it doesn't matter what the argument name is -- all the callers (expecting a dict, a dict subclass, or a dict-like duck) will be using positional arguments in the call. But if we were to document the argument names for pop(), and users started to use these, then most dict sublcasses and ducks would suddenly be broken (except if by luck they happened to pick the same name). -- --Guido van Rossum (python.org/~guido)
On 2 March 2012 19:28, Guido van Rossum <guido@python.org> wrote:
There was a discussion about this on this list in 2007. I wrote some decorators to implement it this functionality. Here's one at http://code.activestate.com/recipes/521874-functions-with-positional-only-ar... (note that it didn't attract a lot of attention !). The recipe also refers to the original discussion. -- Arnaud
On 2 March 2012 20:00, Guido van Rossum <guido@python.org> wrote:
On Mar 2, 2012 11:43 AM, "Arnaud Delobelle" <arnodel@gmail.com> wrote:
On 2 March 2012 19:28, Guido van Rossum <guido@python.org> wrote:
The one in the above recipe (which is for 2.X) doesn't incur any runtime overhead - although it is a bit hackish as it changes the 'co_varnames' attribute of the function's code object. -- Arnaud
On 2 March 2012 21:48, Guido van Rossum <guido@python.org> wrote:
True. OTOH if you decided to put such a decorator in CPython's standard library (and I'm not talking about this specific implementation of the decorator), then implementors of other Pythons would have to provide the same functionality. We would then get the ability to have positional only arguments free of overhead without having to make the syntax of function signatures even more complex. Also, a decorator would be a signal to users that positional only argument are not often necessary, whereas offering syntactical support for them may encourage over use of the feature. -- Arnaud
On Fri, Mar 2, 2012 at 12:00 PM, Guido van Rossum <guido@python.org> wrote:
I've written such decorators too, but they've got quite a bit of overhead...
yeah those fall into the gross hacks I alluded to in my original post. ;) I intentionally decided to leave out discussion of "should we allow positional-only arguments to be declared in Python" but it is a valid discussion and thing to consider... if we go that route, could it be possible to implement range([start=0, ] stop[, step=1]) such that they are positional only but mutliple arguments are treated different than strictly sequential without writing conditional code in Python to figure out each one's meaning at runtime. speaking of range... I think start and stop are plenty obvious, but I'd like to allow step to be specified as a keyword. -gps
On Fri, Mar 2, 2012 at 2:01 PM, Gregory P. Smith <greg@krypto.org> wrote:
I just want to remain realistic and acknowledge that positional arguments have their place.
Eew, I don't think this pattern is useful enough to support in syntax, even if one of the most popular builtins (but only one!) uses it.
speaking of range... I think start and stop are plenty obvious, but I'd like to allow step to be specified as a keyword.
That's fine, range() is not overloadable anyway. -- --Guido van Rossum (python.org/~guido)
On Fri, Mar 2, 2012 at 2:18 PM, Guido van Rossum <guido@python.org> wrote:
+1
Technically more than one, if you consider slice() separate from range()... but they are related so I'm willing to consider them "one" ;) anyways, agreed. keeping it simple makes sense. Though the syntax proposals so far aren't looking great to me. I need to stare at them longer. -gps
Guido van Rossum wrote:
I read this at first that you didn't approve of the range API. I agree that the API is too specialized to take syntactical support, but I'd just like to put my hand up and say I like range's API and have very occasionally used it for my own functions. I see no point in having special syntax for it: this is a specialized enough case that I don't mind handling it manually. -- Steven
03.03.12 00:18, Guido van Rossum написав(ла):
range([start,] stop[, step]) slice([start,] stop[, step]) itertools.islice(iterable, [start,] stop [, step]) random.randrange([start,] stop[, step]) syslog.syslog([priority,] message) curses.newwin([nlines, ncols,] begin_y, begin_x) curses.window.addch([y, x,] ch[, attr]) curses.window.addnstr([y, x,] str, n[, attr]) curses.window.addstr([y, x,] str[, attr]) curses.window.chgat([y, x, ] [num,] attr) curses.window.derwin([nlines, ncols,] begin_y, begin_x) curses.window.hline([y, x,] ch, n) curses.window.insch([y, x,] ch[, attr]) curses.window.insnstr([y, x,] str, n [, attr]) curses.window.subpad([nlines, ncols,] begin_y, begin_x) curses.window.subwin([nlines, ncols,] begin_y, begin_x) curses.window.vline([y, x,] ch, n)
There are number of range-like functions in third-party modules.
On Mar 2, 2012, at 2:01 PM, Gregory P. Smith wrote:
speaking of range... I think start and stop are plenty obvious, but I'd like to allow step to be specified as a keyword.
range() has been around 20+ years and this has never been requested. In my teaching of Python, it is never arisen as an issue. AFAICT, there isn't any code that would be better if step were written as a keyword. The only expressed motivation for the change is "I'd like" it. There should be a higher bar for changing builtins. Many of the proposals in this thread are gratuitous and will create unnecessary work for other people who have to change anything that purports to have a range-like interface, people who have to change the other Python implementations, folks who who have to remember which version of Python supports it and which other slice-like functions would also take the argument etc. ISTM that having a ton of tiny nit changes to the language doesn't make it better. Instead, effort should be directed as substantive changes (better https support, completing xmlrpc, etc). Micro rearrangements of the language and a real PITA for folks who have to go back-and-forth between different versions of Python. So, we should raise the bar to something higher than "I'd like feature X" and ask for examples of code that would be better or for user requests or some actual demonstration of need. ISTM that 20 years of history with range() suggests that no one needs this (nor have I seen a need in any other language with functions that take a start/stop/step). Raymond
Raymond Hettinger wrote:
I have frequently felt that range(start=5, stop=27, step=2) reads better than range(5, 27, 2), but not better *enough* to bother requesting a change, particular given the conservative nature of the Python devs to such minor interface changes (and rightly so). If it came to a vote, I'd vote 0 on the status quo, +0 to allow all three arguments to be optionally given by keyword, and -1 for singling step out as the only one that can be a keyword. That's just silly (sorry Gregory!) -- can you imagine explaining to a beginner why they can write range(0, 50, 2) or range(0, 50, step=2) but not range(0, stop=50, step=2)? -- Steven
On Fri, Mar 2, 2012 at 7:20 PM, Steven D'Aprano <steve@pearwood.info> wrote:
There's no need to explain anything to beginners, they just accept whatever rules you give them. It's the people who are no longer beginners but not quite experts you have to deal with. But a true zen master, even a zen-of-Python master, would just hit them over the head with a wooden plank. (Seriously, there are plenty of APIs that have some positional parameters and some keyword parameters, and I see nothing wrong with that. You seem to be too in love with keyword parameters to see clearly. :-) Still, I can't think of a reason why we should upset Raymond over such a minor thing, so let's forget about "fixing" range. And that's final. -- --Guido van Rossum (python.org/~guido)
On Fri, 2012-03-02 at 18:22 -0800, Raymond Hettinger wrote:
+1
On a more general note... It seems to me that sometimes the writer of functions wish to have more control of how the function is called, but I think it is better that the user of a function can select the calling form that perhaps matches the data and/or style they are using more closely. I hope in the future that we find ways to simplify function signatures in a way that make them both easier to use and more efficient for the function user, rather than continue adding specific little tweaks that give the function designer more control over how the function user calls it. My 2cents, Ron
Le 03/03/2012 16:20, Ron Adam a écrit :
I agree with that, but it can still make sense to have positional-only arguments. For example, we want d.update(self=4) to update the 'self' key on any Mapping, so the default implementation update on the ABC has to accept *args, **kwargs and have some code to extract self: http://hg.python.org/cpython/file/e67b3a9bd2dc/Lib/collections/abc.py#l511 Without this "hack", passing self=4 would give TypeError: got multiple values for keyword argument 'self'. It would be so much nicer to be able to declare self and other positional-only in "def update(self, other=(), **kwargs):" Regards, -- Simon Sapin
On Sat, Mar 3, 2012 at 7:20 AM, Ron Adam <ron3200@gmail.com> wrote:
Um, the function author chooses the signature. If you disagree with that signature, tough luck.
You seem to forget that API design is an art and that it is the function author's prerogative to design an API that minimizes mistakes for all users of the function. Sometimes that includes requiring that everyone uses positional arguments for a certain situation. Anyway, I now think that adding a built-in @positional(N) decorator makes the most sense since it doesn't require changes to the parser. The built-in can be implemented efficiently. This should be an easy patch for someone who wants to contribute some C code. -- --Guido van Rossum (python.org/~guido)
On 3/3/2012 12:54 PM, Guido van Rossum wrote:
Would you then be okay with using that in documentation? @positional(1) ord(char) Return the integer code for char If you prefer that to ord(char, /) Return the integer code for char fine with me. I care more about being able to document existing apis for C-implemented functions than about being able to limit Python functions I write. (Of course, being able to make C and Python versions of stdlib modules match would also be great!) Currently, one may need to experiment before using name-passing to be sure it will work, which tends to discourage name-passing of args even when it would be more readable. -- Terry Jan Reedy
On Sat, Mar 3, 2012 at 12:11 PM, Terry Reedy <tjreedy@udel.edu> wrote:
The @positional(1) form looks like it would be easier to understand if you aren't familiar with it than the / form.
Yeah, so it does make sense to standardize on a solution for this. Let it be @positional(N). Can you file an issue? -- --Guido van Rossum (python.org/~guido)
Le 03/03/2012 21:39, Guido van Rossum a écrit :
Yeah, so it does make sense to standardize on a solution for this. Let it be @positional(N).
Is the N in positional(N) positional-only itself? ;) More seriously, is N required or can we omit it when all arguments are to be positional? Regards, -- Simon Sapin
On Sun, Mar 4, 2012 at 6:39 AM, Guido van Rossum <guido@python.org> wrote:
Yeah, so it does make sense to standardize on a solution for this. Let it be @positional(N). Can you file an issue?
How could that even work? Consider the following subset of the Mapping API: class C: @positional(2): def __init__(self, data=None, **kwds): self._stored_data = stored = {} if data is not None: stored.update(data) stored.update(kwds) @positional(2): def update(self, data=None, **kwds): stored = self._stored_data if data is not None: stored.update(data) stored.update(kwds) Without gross hacking of the function internals, there's no way for a decorator to make the following two calls work properly: x = C(self=5, data=10) x.update(self=10, data=5) Both will complain about duplicate "self" and "data" arguments, unless the "positional" decorator truly rips the function definition apart and creates a new one that alters how the interpreter maps arguments to parameters. As Simon Sapin pointed out, the most correct way to write such code currently is to accept *args and unpack it manually, which is indeed exactly how the Mapping ABC implementation currently works [1]. While the Mapping implementation doesn't currently use it, one simple way to write such code is to use a *second* parameter binding step like this: class C: def _unpack_args(self, data=None): return self, data def __init__(*args, **kwds): self, data = C._unpack_args(*args) self._stored_data = stored = {} if data: stored.update(data) stored.update(kwds) def update(*args, **kwds): self, data = C._unpack_args(*args) stored = self._stored_data if data is not None: stored.update(data) stored.update(kwds) The downside, of course, is that the error messages that come out of such a binding operation may be rather cryptic (which is why the Mapping ABC instead uses manual unpacking - so it can generate nice error messages) The difficulty of implementing the Mapping ABC correctly in pure Python is the poster child for why the lack of positional-only argument syntax is a language wart - we define APIs (in C) that work that way, which people then have to reconstruct manually in Python. My proposal is that we simply added a *third* alternative for "*args": a full function parameter specification to be used to bind the positional-only arguments. That is: 1. '*args' collects the additional positional arguments and places them in a tuple 2. '*' disallows any further positional arguments. 3. '*(SPEC)' binds the additional positional arguments according to the parameter specification. In all 3 cases, any subsequent parameter defintions are keyword only. The one restriction placed on the nested SPEC is that it would only allow "*args" at the end. The keyword only argument and positional only argument forms would not be allowed, since they would make no sense (as all arguments to the inner parameter binding operation are positional by design). Then the "_unpack_args" hack above would be unnecessary, and you could just write: class C: def __init__(*(self, data=None), **kwds): self._stored_data = stored = {} if data: stored.update(data) stored.update(kwds) def update(*(self, data=None), **kwds): stored = self._stored_data if data is not None: stored.update(data) stored.update(kwds) The objection was raised that this runs counter to the philosophy behind PEP 3113 (which removed tuple unpacking from function signatures). I disagree: - this is not tuple unpacking, it is parameter binding - it does not apply to arbitrary arguments, solely to the "extra arguments" parameter, which is guaranteed to be a tuple - it allows positional-only arguments to be clearly expressed in the function signature, allowing the *interpreter* to deal with the creation of nice error messages - it *improves* introspection, since the binding of positional only arguments is now expressed clearly in the function header (and associated additional metadata on the function object), rather than being hidden inside the function implementation Regards, Nick. [1] http://hg.python.org/cpython/file/e67b3a9bd2dc/Lib/collections/abc.py#l511 -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sat, Mar 3, 2012 at 7:46 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I am very well aware of this (it occurs in two different places in the NDB library that I've been developing for Google App Engine). But either I missed some messages in the thread (quite possible) or you're bringing this up for the first time now -- the @positional decorator wasn't meant to solve this case (which only occurs when **kwds is used in this particular way). *If* you want to solve this I agree that some actual new syntax is probably needed.
Nice; that idiom should be more widely known.
Still, a naming convention for the helper function can probably make this fairly painless -- perhaps you'll need a separate helper function for each API function, named in a systematic fashion.
Nobody else seems to have seen the importance of solving *this* particular issue directly in the function signature -- but I personally support trying!
+1. This is certainly the most thorough solution for both problems at hand (simply requiring some parameters to be positional, and the specific issue when combining this with **kwds).
-- --Guido van Rossum (python.org/~guido)
Guido van Rossum wrote:
I don't understand -- example of what's allowed and not allowed?
So a (more or less) complete rundown would look like this: def foo(*(a, b)): ... # all positional def foo(*(a, b=1)): ... # all positional, b optional def foo(*(a, b), c, d): ... # a, b positional; c, d required and keyword def foo(*(a, b), c, d=1): ... # a, b positional; c required, d optional; c & d keyword def foo(*(a, b=1), c=1, d=1): ... # same, but b, c, d optional If I understand correctly, there is no way to have positional-only, position-or-keyword, and keyword-only in the same signature?
On Sun, Mar 4, 2012 at 9:20 AM, Ethan Furman <ethan@stoneleaf.us> wrote:
If I understand correctly, there is no way to have positional-only, position-or-keyword, and keyword-only in the same signature?
Heh. If that's true, my '/' proposal wins: def foo(pos_only, /, pos_or_kw, *, kw_only): ... Defaults can be added to taste. The restrictions on args-without-defaults being unable to follow args-with-defaults may need to be revisited so we can combine optional positional arguments with required keyword arguments, if we want to support that. Nevertheless all this is pretty esoteric and I wouldn't cry if it wasn't added. There exist solutions for the Mapping-API problem, and a @positional decorator would cover most other cases. -- --Guido van Rossum (python.org/~guido)
Guido van Rossum wrote:
Now that I understand that / will only appear in at most one place, like * (and not following each and every positional-only arg) this is the nicest syntax I've seen yet. If we have to have this feature, +1 on this syntax. I'm still only +0 on the feature itself. -- Steven
On 4 March 2012 22:32, Steven D'Aprano <steve@pearwood.info> wrote:
Agreed. However I've *never* wanted to create a "positional args only" parameter to an api that wasn't covered by *args. I also think that in *general* allowing keyword args is an improvement to APIs. So I guess I'm -0 on the feature, but the "/" syntax seems the best of the ones suggested so far. Michael
-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
Steven D'Aprano wrote:
It's reasonably nice, but I'm not sure about giving the '/' its own slot with commas either side. This works for * and ** because they (optionally now in the case of *) take a name after them, but the '/' never will. So how about treating '/' as a separator instead: def foo(pos1, pos2 / arg3, arg4, *args, **kwds): -- Greg
Greg Ewing wrote:
Looks a lot like division to me. Plus you then have the signature different from the call (as it *would* be division if you tried to use it as a separator when calling it). Unless we have a good reason to treat it differently from a lone '*', I think we should be consistent and treat it the same. (I obviously don't think the lack of a name is a good enough reason to be inconsistent. ;) ~Ethan~
On Mon, Mar 5, 2012 at 4:15 AM, Guido van Rossum <guido@python.org> wrote:
Yes, I only realised after Ethan's reply that my approach puts the "positional only" parameters in the wrong place relative to normal parameters (I didn't notice because I'm mainly interested in the Mapping use case and that doesn't accept any normal parameters - just positional only and arbitrary keywords). So, *if* syntactic support for positional-only arguments were added, I think Guido's syntax would be the way to do it. However, now that I've realised the "arbitrary keyword arguments" problem can be solved fairly cleanly by a helper function that binds the positional arguments, I'm more inclined to just leave it alone and tell people to just accept *args and process it that way. OTOH, having a docs-friendly syntax, and better programmatic introspection for the cases where it does come up would be nice, too...
Already done:
Yep. While I do think it's a slight language wart that we can't cleanly express all the function and method signatures that are used by our own builtins and ABC definitions, it's a *very* minor concern overall. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Mon, 2012-03-05 at 10:31 +1000, Nick Coghlan wrote:
Yeah, I think I agree with that for now. I feel signatures are already pretty complex. If we found a solution that worked while simplifying or merging some of that complexity in a nice way, I'd be +1. Your suggested syntax was leaning in that direction I think. I liked that there was a possibly a clearer separation between positional arguments and keywords arguments. If you look at the C code that parses signatures, it's not simple or straight forward. It's not easy to write a python function that maps (*args, **kwds) to a signature in the same way. I tried it to test out some ideas a while back. What makes it difficult is some of the "*args" values can be keyword arguments assigned by position. Or some values in "**kwds" values may be positional arguments assigned by name. I think the order not being preserved in kwds was also a factor. While in the simple cases, it's fairly easy to mentally parse a signature, the mental slope gets steeper as you start to combine the different concepts into one signature. Maybe it's easy for those who do it every day, but not as easy for those doing it less often, or for those who are just beginning to learn python.
When I was looking at your syntax I was thinking of it like this... def foo(*(a, b=2)->args, **(c=3)->kwds): ... return args, kwds Which would map the positional only arguments to args, and the rest to kwds and include the default values as well. But that's a different thing. That wouldn't be friendly to duck typing because functions in the front of the chain should be not care what the signature of the function at the end of the chain will be. It would be limited to functions (ducks) who's signatures are compatible with each other. The (*args, **kwds) signature only corresponds to positional and keywords arguments in the simple cases where no positional (only) argument has a default value, and no keyword arguments are assigned by position. As far as better docs-friendly syntax, and introspection are concerned, I think we will need a signature object that can be introspected. It also might be helpful in evaluating ideas like these. Cheers, Ron
Antoine Pitrou wrote:
Then please consider also re-introducing parameter tuple unpacking, since that was genuinely useful.
It may have been useful, but my understanding is that it was removed because the complications in implementing it were greater, particularly where introspection was concerned. ~Ethan~
On Sun, Mar 4, 2012 at 8:57 AM, Antoine Pitrou <solipsis@pitrou.net> wrote:
That's debatable - reread PEP 3113. I added my +1 to Nick's proposal a little hastily, it should have been +0. I think that *if* we want to solve this, my '/' solution should also be on the table. It has the downside of not being obvious, but I don't think that Nick's proposal is all that obvious either to people who encounter it for the first time -- you have to combine a bunch of powerful ideas to "get" it. And the () inside () just *begs* for arbitrary nesting, which we don't want to reintroduce. We don't want this: def foo(*(*(*(a, b), c), d), e): ... :-) -- --Guido van Rossum (python.org/~guido)
On 3/3/2012 3:39 PM, Guido van Rossum wrote:
Yeah, so it does make sense to standardize on a solution for this
Agreed. There are actually two issues. Doc: indicate intent, regardless of how enforced in code. Code: indicate intent to interpreter so it enforces intent rather than programmer doing do with *args, defaults if any, and error messages.
Let it be @positional(N).
You seem to have backed off on that. I would like a solution for the docs that Georg can tolerate.
Can you file an issue?
When you have settled on one thing for at least a day ;-). Until then, I think it is better to keep discussion in one place, which is here. --- The pos(n) idea does not work because position-only args may still have defaults. For instance, range() takes 1 to 3 args. That proposal did give me this idea: tag positional names with their index. In a sense, the index *is* the internal name while apparent alphabetic name is suggestive for human understanding. For doc purposes, the tag could be either a prefix or suffix. Either way, it would be a convention that does not conflict with any stdlib names that I know of. range(start_0 = 0, stop_1, step_2 = 1) Retern ... range(0_start = 0, 1_stop, 2_step = 1) Return ... For Python code, I presume the prefix form would be rejected by the lexer. A possibility would be 'mangled' dunder names in the signature, such as __0_start__, which would be stripped to 'start' for use in the code. If this idea makes '/' look better, fine with me. -- Terry Jan Reedy
04.03.12 22:59, Terry Reedy написав(ла):
Extend this for function arguments: """ However, there is a convention that is followed by most Python code: a name prefixed with an underscore (e.g. _spam) should be treated as a non-public part of the API (whether it is a function, a method or a data member). It should be considered an implementation detail and subject to change without notice. """
On 03.03.2012 21:11, Terry Reedy wrote:
I don't think that is a good idea. We currently put argument default values in the function signatures in Python syntax, but only because that also makes sense from a documentation PoV. We also wouldn't write @property name(self) just because that's (one) way for creating properties from Python. Georg (I am -0 on @positional myself: IMO such a completely different way of declaring positional-only and keyword-only arguments lacks grace.)
On Sat, 2012-03-03 at 09:54 -0800, Guido van Rossum wrote:
Yes, except the caller does have the option to use the *args and/or **kwds and other variations of packing and unpacking when it's convenient. And yes, I realize it doesn't really change the signature it self, but it is a different way to spell a function call and sometimes is very helpful when the data can be matched to the signature. For example, by not specifying any arguments as keyword only, or position only, both of the following examples work, and the function caller has these options.
But by specify an argument as keyword only, then it removes the *args option. And also if an argument is specified as position only, then the **kwds spelling wont work. I'm not suggesting there isn't sometimes a need for being more specific, but that quite often it's nicer to let the caller have those options rather than limit the API too narrowly.
I was trying to make a more general point which is why I preceded my comments with, "On a more general note...", which was left out of the reply. Yes, it most definitely is an ART to create a good API. And also yes, sometimes minimizing errors take priority, especially when those errors can be very costly. It seems to me, binding an object by name is less likely to be wrong than binding a object by it's position. The advantage of 'by position' is that many objects are stored in ordered lists. ie.. [start, stop, step], [x, y, z]. So it's both easier and more efficient to use the position rather than a name especially if the object can be *unpacked directly into the signature. I just can't think of a good case where I would want to prohibit setting an argument by name on on purpose. But I suppose if I encountered a certain error that may have been caught by doing so, I may think about doing that. <shrug> Cheers, Ron
On Sat, 2012-03-03 at 16:46 -0800, Guido van Rossum wrote:
Yep, I missed Nicks message where he points out...
def dct(a, **kwds): ... return a, kwds
Would the positional decorator fix this particular case? It seems like it would work for forcing an error, but not for multiple values with the same name. The way to currently get around this is to use *args along with **kwds.
The names used with '*' and '**' are already anonymous as far as the foo signature is concerned, so you can use args or kwds as keywords without a problem. I'm not sure what the positional decorator would gains over this. The other use case mentioned is the one where you point out overriding an undocumented variable name. Seems like this is a question of weather or not it is better to make python behavior match the C builtins behavior, vs making the C builtins behavior match python behavior. Cheers, Ron
On Sun, Mar 4, 2012 at 6:16 AM, Ron Adam <ron3200@gmail.com> wrote:
There are two issues being discussed here: 1. A new syntax for positional-only arguments. I don't really see any good use case for this which can't already be dealt with quite easily using *args. True, it means a bit more work on the documentation, but is it really worth adding new syntax (or even a built-in decorator) just for that? 2. How to avoid the problems with name-binding to an intended positional only argument. Once again, this can be dealt with using *args. In both cases it would be nice to be able to avoid having to manually parse *args and **kwargs, but I haven't really seen anything better that the status quo for dealing with them. The only way I see this really working is to somehow bind positional-only arguments without binding each them to a specific name, and the only way I can think of to do that is to store them in a tuple. Perhaps, then, the syntax should reflect a C-style array: # pos(2) indicates 2 positional arguments def f(pos(2), arg1, *args, **kwargs): print(pos) print(arg1) print(args) print(kwargs)
I'm +0 on the whole idea, but only if it deals with both issues. David
David Townshend wrote:
The problem with *args is that it allows 0-N arguments, when we want, say, 2.
2. How to avoid the problems with name-binding to an intended positional only argument. Once again, this can be dealt with using *args.
Again, the problem is *args accepts a range of possible arguments.
Not good. The issue is not restricting the author from binding the positional arguments to names, the issue is restricting the user from binding the arguments to names -- but even then, the user (and the author!) need to have those names apparent. For example: str.split(pos(2)) Quick, what should be supplied for the two positional arguments? We want the arguments bound to names *in the function* -- we don't want the arguments bound to names *in the function call*. ~Ethan~
On Sun, Mar 4, 2012 at 5:34 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
Agreed, but why not this? def f(*args, **kwargs): assert len(args) == 2 The exception raised could easily be a TypeError too, so it would appear, to a user, the same as defining it in the function signature. There are of course, other limitations (e.g. introspection), but without a specific use case it is difficult to know how important those limitations are.
Why does the user need the names? If the user cannot use them as keywords, then it doesn't matter what the names are, so anything can be used in the documentation (which is the only place they would appear). The author doesn't need the names either, just the data they refer to, in this case a tuple.
Is that a function call or definition? My suggestion was to use the parenthesis in the definition, not the call. Since str.split only has optional arguments its not a good example, but if you were to redefine str.replace (in python) it would look like this: def replace(pos(3), count=None): self, old, new = pos ... It would still be called in the same way though:
'some string'.replace('s', 't', 1) 'tome string'
We want the arguments bound to names *in the function* -- we don't want the arguments bound to names *in the function call*.
That is what I proposed, I just suggested binding all of the positinoal-only arguments to a single name. Having given it a bit more thought, though, maybe it would be easier to optionally apply the parenthesis to *args: def replace(self, *args(2), count=None); old, new = args This looks much closer to current situation, and I suppose could be extended to **kwargs, limiting the number of keyword arguments (although I have no idea why this would ever be wanted!)
On Sat, 3 Mar 2012 09:54:03 -0800 Guido van Rossum <guido@python.org> wrote:
Those situations are probably very rare. AFAIK we haven't seen anyone mention a serious use case. I think concerns of built-in functions shadowed by Python functions or the reverse are mostly academic, since we don't see anyone complaining about dict-alikes accepting keyword args. (besides, what happened to "consenting adults"? :-))
Anyway, I now think that adding a built-in @positional(N) decorator makes the most sense since it doesn't require changes to the parser.
-1 on a built-in for that. The functools module would probably be a good recipient (assuming the decorator is useful at all, of course). Regards Antoine.
On Sat, Mar 3, 2012 at 4:55 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
No, because the base class's insistence on positional args makes it a non-starter to use keyword args. But APIs that are implemented in Python don't have this nudge. Given that some folks here have expressed a desire to use keyword args *everywhere*, which I consider going way overboard, as a readability and consistency advocate I want to be able to remind them strongly in some cases not to do that.
(besides, what happened to "consenting adults"? :-))
We used to say that about the lone star feature too. But it's turned out quite useful, both for readability (require callers to name the options they're passing in) and for allowing evolution of a signature by leaving the option to add another positional argument in the future. Some folks seem to believe that keywords are always better. I want to send a strong message that I disagree.
TBH, I've never gotten the hang of functools. It seems mostly a refuge for things I don't like; so @positional() doesn't belong there. :-) -- --Guido van Rossum (python.org/~guido)
Guido van Rossum wrote:
I think you're reading too much into what has been a pretty luke-warm response to Gregory's suggestion. As far as I can see, I've been the least negative about the idea, and that was a half-hearted +0. I described it as "nice to have" specifically on the basis of consistency, to minimize the differences between pure-Python functions and built-ins. On reflection, your argument about subclassing built-ins has convinced me to drop that to a -1: if we were designing Python from scratch, it would be a nice-to-have for built-ins to take named arguments, but since they don't, it would be too disruptive to add them en-mass. I'm still +1 on adding named arguments to built-ins where needed, e.g.
but I hope that would be uncontroversial!
Positional-only arguments should be considered a new feature that requires justification, not just to send a message against mass re-engineering of built-ins. Some arguments in favour: * Consistency with built-ins. * Functions that take a single argument don't need to be called by keyword. (But is that a reason to prohibit it?) * It gives the API developer another choice when designing their function API. Maybe some people just don't like keyword args. (But is "I don't like them" a good reason to prohibit them? I'd feel more positive about this argument if I knew a good use-case for designing a new function with positional-only arguments.) Arguments against: * YAGNI. The subclass issue is hardly new, and as far as I know, has never been an actual problem in practice. Since people aren't calling subclass methods using keywords, why try to enforce it? * It's another thing to learn about functions. "Does this function take keyword arguments or not?" So far, I'm +0 on this.
What, you don't like @wraps? Astonishing! :-) -- Steven
Guido van Rossum wrote:
I think that's a genuine problem in theory. But is it a problem in practice? Since find('a', end=1) doesn't currently work, there won't be any code using it in practice. Even if a subclass looks like this: class MyString(str): def find(self, substring, beginning=0, last=None): ... internally MyString.find must be using positional arguments if it calls str.find, because keyword arguments don't currently work. So this suggested change will not break existing code. I can see one other objection to the change: if str.find accepts keywords, and MyString.find accepts *different* keywords, that is a violation of the Liskov Substitution Principle. Those who care about this would feel obliged to fix their code to match the argument names used by str.find, so if you call that mismatch "breaking backwards compatibility", I accept that. [Aside: in my experience, most programmers are unaware of Liskov, and accidentally or deliberately violate it frequently.] But given that str.find has been documented as "S.find(sub[, start[, end]])" forever, I don't have a lot of sympathy for anyone choosing different argument names. (I'm one of them. I'm sure I've written string subclasses that used s instead of sub.) I think that the practical benefit in clarity and readability in being able to write s.find('a', end=42) instead of s.find('a', 0, 42) outweighs the theoretical harm, but I will accept that there is a downside. -- Steven
On 02Mar2012 14:01, Gregory P. Smith <greg@krypto.org> wrote: | On Fri, Mar 2, 2012 at 12:00 PM, Guido van Rossum <guido@python.org> wrote: | > I've written such decorators too, but they've got quite a bit of | > overhead... | > | yeah those fall into the gross hacks I alluded to in my original post. ;) | | I intentionally decided to leave out discussion of "should we allow | positional-only arguments to be declared in Python" but it is a valid | discussion and thing to consider... | | if we go that route, could it be possible to implement range([start=0, ] | stop[, step=1]) such that they are positional only but mutliple arguments | are treated different than strictly sequential without writing conditional | code in Python to figure out each one's meaning at runtime. More excitiingly, one could embed a slice in the hypothetical positional-only syntax to say the 0th, 2nd and 4th parameters are positional-only. Or an arbitrary sequence! Hmm, a callable that returns an iterable at function call time! Sorry, too much coffee... -- Cameron Simpson <cs@zip.com.au> DoD#743 http://www.cskk.ezoshosting.com/cs/ Give me the luxuries of life and I will willingly do without the necessities. - Frank Lloyd Wright
On 3/2/2012 3:32 PM, Ethan Furman wrote:
I think this is what we need. I see the problem as being that a) C and Python functions work differently, and b) the doc does not -- and should not -- specify the implementation. One solution is to make all C functions work like Python functions. The other is to allow Python functions to work like C functions. Given the reasonable opposition to the first, we need the second.
That is probably better than using '$' or directly tagging the names. -- Terry Jan Reedy
Terry Reedy wrote:
I chose '?' because it has some similarity to an incompletely-drawn 'p', and also because it suggests a sort of vagueness, as in not being able to specify the name of the argument. I do not know if it is the best possible way, and am looking forward to other ideas. ~Ethan~
On Fri, Mar 2, 2012 at 2:49 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
I'd rather not start using a new punctuation character for this one very limited purpose; it might prevent us from using ? for some other more generic purpose in the future. Alternative proposal: how about using '/' ? It's kind of the opposite of '*' which means "keyword argument", and '/' is not a new character. -- --Guido van Rossum (python.org/~guido)
On 2012-03-02, at 5:46 PM, Guido van Rossum wrote:
Alternative proposal: how about using '/' ? It's kind of the opposite of '*' which means "keyword argument", and '/' is not a new character.
How about ';'? Is it possible to re-use it in this context? def (a; b, *, c) def (; b) - Yury P.S. Sometimes I feel nostalgic for the moratorium...
Guido van Rossum wrote:
So it would look like: def ord(char, /): def split(self, char, /, count) def canary(breed, /, color, wingspan, *, name) I think I like that better -- it stands out, and it looks like a barrier between the positional-only and the positional-keyword arguments. ~Ethan~
Ethan Furman wrote:
Urrggg, ugly and hard to read. Imagine, if you will: def spam(x, /, y, /, z, /, a=2/3, /): ... Placing the tag after the argument as an extra parameter is not the right approach in my opinion. It's excessively verbose, and it puts the tag in the wrong place: as you read from left-to-right, you see "normal argument, no, wait, it's positional only". The tag should prefix the name. With keyworld-only arguments, the * parameter is special because it flags a point in the parameter list, not an individual parameter: you read "normal arg, normal arg, start keyword-only args, keyword-only arg, ...". I believe that the right place to tag the parameter is in the parameter itself, not by adding an extra parameter after it. Hence, something like this: def spam(~x, ~y, ~z, ~a=2/3): ... where ~name means that name cannot be specified by keyword. I read it as "not name", as in, the caller can't use the name. Or if you prefer Guido's pun: def spam(/x, /y, /z, /a=2/3): ... Much less line-noise than spam(x, /, y, /, z, /, a=2/3, /). Personally, I think this is somewhat of an anti-feature. Keyword arguments are a Good Thing, and while I don't believe it is good enough to *force* all C functions to support them, I certainly don't want to discourage Python functions from supporting them. -- Steven
On Fri, Mar 2, 2012 at 6:57 PM, Steven D'Aprano <steve@pearwood.info> wrote:
That can't be right -- if a parameter is positional, surely all parameters before it are also positional, so it would be redundant to have to mark all of them up. Also ~name looks too much like an expression and /name looks just weird (I think DOS command line flags used to look like this).
And yet people invent decorators and other hacks to insist on positional parameters all the time. You *can* have Too Much of a Good Thing, and for readability it's better if calls are consistent. If most calls to a function use positional arguments (at least for the first N positions), it's better to force *all* calls to use positional arguments 1-N: the reader may be unfamiliar with the parameter names. Also remember the subclassing issue I brought up before. That said, I can't come up with a syntax that I really like. Here's my best attempt, but I'm at most -0 on it: Have a stand-alone '/' indicate "all parameters to my left must be positional", just like a stand-alone '*' means "all parameters to my right must be keywords". If there's no stand-alone '*' it is assumed to be all the way on the right; so if there's no '/' it is assumed to be all the way on the left. The following should all be valid: def foo(/, a, b): ... # edge case, same as def foo(a, b): ... def foo(a, b, /): ... # all positional def foo(a, b=1, /): ... # all positional, b optional def foo(a, b, /, c, d): ... # a, b positional; c, d required and either positional or keyword def foo(a, b, /, c, d=1): ... # a, b positional; c required, d optional; c, d either positional or keyword def foo(a, b, /, c, *, d): ... # a, b positional; c required and either positional or keyword; d required keyword def foo(a, b=1, /, c=1, *, d=1): ... # same, but b, c, d optional That about covers it. But agreed it's no thing of beauty, so let's abandon it. -- --Guido van Rossum (python.org/~guido)
Guido van Rossum wrote:
And I was just starting to like it, too. :( Personally, I don't see it as any uglier than having the lone '*' in the signature; although, I don't see lone '*'s all that much, whereas the '/' could be quite prevalent. Is this something we want? We could have a built-in decorator, like property or staticmethod, to make the changes for us (each implementation would have to supply their own, of course): @positional(2) def foo(a, b) Or we could use brackets or more parentheses: def foo([a, b]) def foo((a, b)) That doesn't seem too bad... def foo((a, b=1), c=2, *, d=3) Since tuple-unpacking no longer happens in the definition signature, we don't need the leading * before the parentheses. Just my last two cents (unless the idea garners encouragement, of course ;). ~Ethan~
On Fri, Mar 2, 2012 at 6:57 PM, Steven D'Aprano <steve@pearwood.info> wrote:
I don't believe that's what the proposal is anyway. Note Ethan's "barrier" comment.
That's in the same vein as what I understand the proposal to be. "/" would flag "end of positional-only args"; there's effectively an implicit leading "/" if you don't use one in a function's definition. <snip>
+1 I can see not modifying every single C implementation as it's for little gain, and in a bunch of cases concerns 1-or-2-argument functions which are arguably worth special-casing. But making the function definition syntax even more complicated (we have annotations now, remember?) just to allow forcing calls to be (slightly) more restrictive/opaque? Hard to get on board with that. Cheers, Chris
On 3/2/2012 5:46 PM, Guido van Rossum wrote:
Alternative proposal: how about using '/' ? It's kind of the opposite of '*' which means "keyword argument", and '/' is not a new character.
It took me a moment to get the pun on div / being the inverse of mul *. I like it. Very clever -- and memorable! -- Terry Jan Reedy
On 03Mar2012 11:53, Serhiy Storchaka <storchaka@gmail.com> wrote: | 03.03.12 00:46, Guido van Rossum написав(ла): | > Alternative proposal: how about using '/' ? It's kind of the opposite | > of '*' which means "keyword argument", and '/' is not a new character. | | How about using '**' (and left '/' for some purpose in the future)? -1 from me; too much overlap with **kwargs keyword argument insertion in calls. We'd have ** in calls for keyword arguments and ** in definitions for not keyword arguments. -- Cameron Simpson <cs@zip.com.au> DoD#743 http://www.cskk.ezoshosting.com/cs/ What if there were no hypothetical situations? - Jeff Sauder
03.03.12 23:40, Cameron Simpson написав(ла):
"*identifier" is a tuple receiving any excess positional parameters. "**identifier" is a dictionary receiving any excess keyword arguments. Parameters after "*" are keyword-only parameters. ? Parameters before "**" are positional-only parameters.
Another bikeshed idea on positional-only parameters: def foo([self], a, b, *args, **kwds): ... The square brackets are meant to suggest that the name is something only of interest to the implementation of the function, and not to be taken as part of the API. -- Greg
Serhiy Storchaka wrote:
Please do not give syntactic meaning to [parameter], unless it matches the existing convention for optional parameters. Besides, positional-only arguments are not only of interest to the implementation, they are part of the API.
Or _name, as for "private" class and module members.
In my own functions, I use _name for private implementation arguments, and usually explicitly document that callers should not rely on them. In the implementation, sometimes I need to use that private argument, and I always do so by name so that it stands out that I'm using a special argument, e.g. something like: def func(spam, ham, cheese, _name=eggs): if condition: x = func(spam, ham, cheese, _name=beans) ... If you also overload _name to also mean "positional only", I would have to write x = func(spam, ham, cheese, beans) which looks like a "normal" argument. And as already mentioned, the use of _name to mean positional-only and private would clash with functions which want public positional-only. -- Steven
05.03.12 00:28, Steven D'Aprano написав(ла):
There is no clash. Both the application means the same thing -- you advise a client not to use this name as keyword argument. Of course, he may, but need not, if not aware. 'spam'.find('a', _end=1) looks terrible and no one will inadvertently use it. Or use a double underscore for the reinforcement to prevent the use of this name. def find(self, sub, __start=0, __end=None)
Steven D'Aprano wrote:
Please do not give syntactic meaning to [parameter], unless it matches the existing convention for optional parameters.
Why should it have to do that? We already have a syntax for optional parameters, and there is no reason for a reader to think that a new piece of syntax is simply duplicating existing functionality.
Besides, positional-only arguments are not only of interest to the implementation, they are part of the API.
The fact that a parameter exists in that slot is part of the API, but the *name* of it is not. This is reflected in the fact that the comma is outside the brackets and the name is inside. -- Greg
Greg Ewing wrote:
I see your later comment about metasyntax, but to clarify in case there is still any lingering doubt what I mean: When reading function signatures in *documentation*, we often see func([parameter]) used to indicate that parameter is an optional argument. If your proposal is enacted, when reading function signatures in *code*, we will see func([parameter]) used to indicate that you can't use parameter as a keyword argument. The two uses clash, which means that every time we see a [parameter] in a function signature, there will be a moment of disorientation where we have to decide whether it should be interpreted using the convention for code or the convention for documentation. Certainly there are ways of distinguishing the two from context, particularly if the default value is shown in the signature, or from subtleties of whether the comma is inside the brackets or not, or perhaps from the order ([] early in the signature probably means positional, [] at the end probably means optional). My point is not that it is impossible to distinguish optional from positional arguments, but that the similarity of syntax makes it difficult to distinguish the two *at a glance* and comprehensibility will be reduced. And heaven help us if we have a function like range with positional-only optional parameters: range([[start]=0,] [end] [, [step]=1]) --> iterable For the avoidance of doubt, I am *not* suggesting that we introduce [parameter] as syntax for optional arguments. I don't want to see [] used syntactically inside def func(...) at all, except for the VERY rare use of lists as mutable default arguments. -- Steven
On Sat, Mar 3, 2012 at 5:28 AM, Guido van Rossum <guido@python.org> wrote:
Yeah, on reflection, I'm actually -0 on adding keyword arg support to 1-arg functions.
I currently write such code as: def f(*args): arg1, arg2, arg3 = args This gives rubbish error messages when the caller makes a mistake, but it works. The obvious syntactic alternative is allowing tuple expansion specifically for *args: def f(*(arg1, arg2, arg3)): pass Then the interpreter would have enough info to still generate nice error messages, and we don't have to invent much in the way of new syntax.
Good point. The other use case is APIs like the dict constructor and dict.update which are designed to accept arbitrary keyword arguments, so you don't want to reserve particular names in the calling argument namespace for your positional arguments. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Fri, Mar 2, 2012 at 6:56 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
It's not `def herd(*args, *, breed)`, so I don't see why it would be `def herd(*(size, location), *, breed)`. I think Nick's syntax is the right one, although adding the feature to Python is probably not a good idea. Mike
I'm -1 on this issue after some thought: I think we need to look at this from the function user's perspective. For example let's take this hypothetical declaration: def func(a, b, /, x, y, *, name, color): This function may be called like this: func(v1, v2) func(v1, v2, v3, v4) func(v1, v2, y=v4, x=v3) func(v1, v2, x=v3, y=v4) func(v1, v2, v3, v4, name='westley', color='0xffffff') func(v1, v2, name='westley', color='0xffffff', x=v3, y=v4) func(v1, name='westley', color='0xffffff', x=v3, y=v4, v2) # ERROR! To me, this just feels a little too ... mutable. In C we have one way to call functions that is equal to it's function declaration. I'd be +1 for functions that have ONLY non-keyword arguments which would be declared via decorator: @positional # This name is a bit ambiguous I guess.... def func(a, b)
Gregory P. Smith wrote:
+1 on adding keyword arguments to built-in methods and functions where they would help readability, e.g str.find(c, start=23), even if this happens in a ad-hoc fashion. +0 on forcing *all* built-in methods and functions to be updated to take keyword arguments out of a sense of purity, e.g. ord(char='c'). I think that "all built-ins should take keywords, so as to minimise the difference between them and pure-Python functions" is an admirable ideal. But it is an ideal, a nice-to-have rather than a must-have. -- Steven
On Fri, Mar 2, 2012 at 6:43 PM, Steven D'Aprano <steve@pearwood.info> wrote:
Indeed, this is the approach we have taken to date. For example, str.split() recently gained keyword support for 3.3 because "text.split(maxsplit=1)" is less cryptic than "text.split(None, 1)". It makes the most sense when at least one of the following holds: - the second argument accepts a number that is unclear if you're not familiar with the full function signature - the earlier arguments have sensible default values that you'd prefer not to override So +1 on declaring "make X support keyword arguments" non-controversial for multi-argument functions, +0 on also doing so for single argument functions, but -0 on attempting to boil the ocean and fix them wholesale. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Fri, Mar 2, 2012 at 4:13 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Hm. I think for many (most?) 1-arg and selected 2-arg functions (and rarely 3+-arg functions) this would reduce readability, as the example of ord(char=x) showed. I would actually like to see a syntactic feature to state that an argument *cannot* be given as a keyword argument (just as we already added syntax to state that it *must* be a keyword). One area where I think adding keyword args is outright wrong: Methods of built-in types or ABCs and that are overridable. E.g. consider the pop() method on dict. Since the argument name is currently undocumented, if someone subclasses dict and overrides this method, or if they create another mutable mapping class that tries to emulate dict using duck typing, it doesn't matter what the argument name is -- all the callers (expecting a dict, a dict subclass, or a dict-like duck) will be using positional arguments in the call. But if we were to document the argument names for pop(), and users started to use these, then most dict sublcasses and ducks would suddenly be broken (except if by luck they happened to pick the same name). -- --Guido van Rossum (python.org/~guido)
On 2 March 2012 19:28, Guido van Rossum <guido@python.org> wrote:
There was a discussion about this on this list in 2007. I wrote some decorators to implement it this functionality. Here's one at http://code.activestate.com/recipes/521874-functions-with-positional-only-ar... (note that it didn't attract a lot of attention !). The recipe also refers to the original discussion. -- Arnaud
On 2 March 2012 20:00, Guido van Rossum <guido@python.org> wrote:
On Mar 2, 2012 11:43 AM, "Arnaud Delobelle" <arnodel@gmail.com> wrote:
On 2 March 2012 19:28, Guido van Rossum <guido@python.org> wrote:
The one in the above recipe (which is for 2.X) doesn't incur any runtime overhead - although it is a bit hackish as it changes the 'co_varnames' attribute of the function's code object. -- Arnaud
On 2 March 2012 21:48, Guido van Rossum <guido@python.org> wrote:
True. OTOH if you decided to put such a decorator in CPython's standard library (and I'm not talking about this specific implementation of the decorator), then implementors of other Pythons would have to provide the same functionality. We would then get the ability to have positional only arguments free of overhead without having to make the syntax of function signatures even more complex. Also, a decorator would be a signal to users that positional only argument are not often necessary, whereas offering syntactical support for them may encourage over use of the feature. -- Arnaud
On Fri, Mar 2, 2012 at 12:00 PM, Guido van Rossum <guido@python.org> wrote:
I've written such decorators too, but they've got quite a bit of overhead...
yeah those fall into the gross hacks I alluded to in my original post. ;) I intentionally decided to leave out discussion of "should we allow positional-only arguments to be declared in Python" but it is a valid discussion and thing to consider... if we go that route, could it be possible to implement range([start=0, ] stop[, step=1]) such that they are positional only but mutliple arguments are treated different than strictly sequential without writing conditional code in Python to figure out each one's meaning at runtime. speaking of range... I think start and stop are plenty obvious, but I'd like to allow step to be specified as a keyword. -gps
On Fri, Mar 2, 2012 at 2:01 PM, Gregory P. Smith <greg@krypto.org> wrote:
I just want to remain realistic and acknowledge that positional arguments have their place.
Eew, I don't think this pattern is useful enough to support in syntax, even if one of the most popular builtins (but only one!) uses it.
speaking of range... I think start and stop are plenty obvious, but I'd like to allow step to be specified as a keyword.
That's fine, range() is not overloadable anyway. -- --Guido van Rossum (python.org/~guido)
On Fri, Mar 2, 2012 at 2:18 PM, Guido van Rossum <guido@python.org> wrote:
+1
Technically more than one, if you consider slice() separate from range()... but they are related so I'm willing to consider them "one" ;) anyways, agreed. keeping it simple makes sense. Though the syntax proposals so far aren't looking great to me. I need to stare at them longer. -gps
Guido van Rossum wrote:
I read this at first that you didn't approve of the range API. I agree that the API is too specialized to take syntactical support, but I'd just like to put my hand up and say I like range's API and have very occasionally used it for my own functions. I see no point in having special syntax for it: this is a specialized enough case that I don't mind handling it manually. -- Steven
03.03.12 00:18, Guido van Rossum написав(ла):
range([start,] stop[, step]) slice([start,] stop[, step]) itertools.islice(iterable, [start,] stop [, step]) random.randrange([start,] stop[, step]) syslog.syslog([priority,] message) curses.newwin([nlines, ncols,] begin_y, begin_x) curses.window.addch([y, x,] ch[, attr]) curses.window.addnstr([y, x,] str, n[, attr]) curses.window.addstr([y, x,] str[, attr]) curses.window.chgat([y, x, ] [num,] attr) curses.window.derwin([nlines, ncols,] begin_y, begin_x) curses.window.hline([y, x,] ch, n) curses.window.insch([y, x,] ch[, attr]) curses.window.insnstr([y, x,] str, n [, attr]) curses.window.subpad([nlines, ncols,] begin_y, begin_x) curses.window.subwin([nlines, ncols,] begin_y, begin_x) curses.window.vline([y, x,] ch, n)
There are number of range-like functions in third-party modules.
On Mar 2, 2012, at 2:01 PM, Gregory P. Smith wrote:
speaking of range... I think start and stop are plenty obvious, but I'd like to allow step to be specified as a keyword.
range() has been around 20+ years and this has never been requested. In my teaching of Python, it is never arisen as an issue. AFAICT, there isn't any code that would be better if step were written as a keyword. The only expressed motivation for the change is "I'd like" it. There should be a higher bar for changing builtins. Many of the proposals in this thread are gratuitous and will create unnecessary work for other people who have to change anything that purports to have a range-like interface, people who have to change the other Python implementations, folks who who have to remember which version of Python supports it and which other slice-like functions would also take the argument etc. ISTM that having a ton of tiny nit changes to the language doesn't make it better. Instead, effort should be directed as substantive changes (better https support, completing xmlrpc, etc). Micro rearrangements of the language and a real PITA for folks who have to go back-and-forth between different versions of Python. So, we should raise the bar to something higher than "I'd like feature X" and ask for examples of code that would be better or for user requests or some actual demonstration of need. ISTM that 20 years of history with range() suggests that no one needs this (nor have I seen a need in any other language with functions that take a start/stop/step). Raymond
Raymond Hettinger wrote:
I have frequently felt that range(start=5, stop=27, step=2) reads better than range(5, 27, 2), but not better *enough* to bother requesting a change, particular given the conservative nature of the Python devs to such minor interface changes (and rightly so). If it came to a vote, I'd vote 0 on the status quo, +0 to allow all three arguments to be optionally given by keyword, and -1 for singling step out as the only one that can be a keyword. That's just silly (sorry Gregory!) -- can you imagine explaining to a beginner why they can write range(0, 50, 2) or range(0, 50, step=2) but not range(0, stop=50, step=2)? -- Steven
On Fri, Mar 2, 2012 at 7:20 PM, Steven D'Aprano <steve@pearwood.info> wrote:
There's no need to explain anything to beginners, they just accept whatever rules you give them. It's the people who are no longer beginners but not quite experts you have to deal with. But a true zen master, even a zen-of-Python master, would just hit them over the head with a wooden plank. (Seriously, there are plenty of APIs that have some positional parameters and some keyword parameters, and I see nothing wrong with that. You seem to be too in love with keyword parameters to see clearly. :-) Still, I can't think of a reason why we should upset Raymond over such a minor thing, so let's forget about "fixing" range. And that's final. -- --Guido van Rossum (python.org/~guido)
On Fri, 2012-03-02 at 18:22 -0800, Raymond Hettinger wrote:
+1
On a more general note... It seems to me that sometimes the writer of functions wish to have more control of how the function is called, but I think it is better that the user of a function can select the calling form that perhaps matches the data and/or style they are using more closely. I hope in the future that we find ways to simplify function signatures in a way that make them both easier to use and more efficient for the function user, rather than continue adding specific little tweaks that give the function designer more control over how the function user calls it. My 2cents, Ron
Le 03/03/2012 16:20, Ron Adam a écrit :
I agree with that, but it can still make sense to have positional-only arguments. For example, we want d.update(self=4) to update the 'self' key on any Mapping, so the default implementation update on the ABC has to accept *args, **kwargs and have some code to extract self: http://hg.python.org/cpython/file/e67b3a9bd2dc/Lib/collections/abc.py#l511 Without this "hack", passing self=4 would give TypeError: got multiple values for keyword argument 'self'. It would be so much nicer to be able to declare self and other positional-only in "def update(self, other=(), **kwargs):" Regards, -- Simon Sapin
On Sat, Mar 3, 2012 at 7:20 AM, Ron Adam <ron3200@gmail.com> wrote:
Um, the function author chooses the signature. If you disagree with that signature, tough luck.
You seem to forget that API design is an art and that it is the function author's prerogative to design an API that minimizes mistakes for all users of the function. Sometimes that includes requiring that everyone uses positional arguments for a certain situation. Anyway, I now think that adding a built-in @positional(N) decorator makes the most sense since it doesn't require changes to the parser. The built-in can be implemented efficiently. This should be an easy patch for someone who wants to contribute some C code. -- --Guido van Rossum (python.org/~guido)
On 3/3/2012 12:54 PM, Guido van Rossum wrote:
Would you then be okay with using that in documentation? @positional(1) ord(char) Return the integer code for char If you prefer that to ord(char, /) Return the integer code for char fine with me. I care more about being able to document existing apis for C-implemented functions than about being able to limit Python functions I write. (Of course, being able to make C and Python versions of stdlib modules match would also be great!) Currently, one may need to experiment before using name-passing to be sure it will work, which tends to discourage name-passing of args even when it would be more readable. -- Terry Jan Reedy
On Sat, Mar 3, 2012 at 12:11 PM, Terry Reedy <tjreedy@udel.edu> wrote:
The @positional(1) form looks like it would be easier to understand if you aren't familiar with it than the / form.
Yeah, so it does make sense to standardize on a solution for this. Let it be @positional(N). Can you file an issue? -- --Guido van Rossum (python.org/~guido)
Le 03/03/2012 21:39, Guido van Rossum a écrit :
Yeah, so it does make sense to standardize on a solution for this. Let it be @positional(N).
Is the N in positional(N) positional-only itself? ;) More seriously, is N required or can we omit it when all arguments are to be positional? Regards, -- Simon Sapin
On Sun, Mar 4, 2012 at 6:39 AM, Guido van Rossum <guido@python.org> wrote:
Yeah, so it does make sense to standardize on a solution for this. Let it be @positional(N). Can you file an issue?
How could that even work? Consider the following subset of the Mapping API: class C: @positional(2): def __init__(self, data=None, **kwds): self._stored_data = stored = {} if data is not None: stored.update(data) stored.update(kwds) @positional(2): def update(self, data=None, **kwds): stored = self._stored_data if data is not None: stored.update(data) stored.update(kwds) Without gross hacking of the function internals, there's no way for a decorator to make the following two calls work properly: x = C(self=5, data=10) x.update(self=10, data=5) Both will complain about duplicate "self" and "data" arguments, unless the "positional" decorator truly rips the function definition apart and creates a new one that alters how the interpreter maps arguments to parameters. As Simon Sapin pointed out, the most correct way to write such code currently is to accept *args and unpack it manually, which is indeed exactly how the Mapping ABC implementation currently works [1]. While the Mapping implementation doesn't currently use it, one simple way to write such code is to use a *second* parameter binding step like this: class C: def _unpack_args(self, data=None): return self, data def __init__(*args, **kwds): self, data = C._unpack_args(*args) self._stored_data = stored = {} if data: stored.update(data) stored.update(kwds) def update(*args, **kwds): self, data = C._unpack_args(*args) stored = self._stored_data if data is not None: stored.update(data) stored.update(kwds) The downside, of course, is that the error messages that come out of such a binding operation may be rather cryptic (which is why the Mapping ABC instead uses manual unpacking - so it can generate nice error messages) The difficulty of implementing the Mapping ABC correctly in pure Python is the poster child for why the lack of positional-only argument syntax is a language wart - we define APIs (in C) that work that way, which people then have to reconstruct manually in Python. My proposal is that we simply added a *third* alternative for "*args": a full function parameter specification to be used to bind the positional-only arguments. That is: 1. '*args' collects the additional positional arguments and places them in a tuple 2. '*' disallows any further positional arguments. 3. '*(SPEC)' binds the additional positional arguments according to the parameter specification. In all 3 cases, any subsequent parameter defintions are keyword only. The one restriction placed on the nested SPEC is that it would only allow "*args" at the end. The keyword only argument and positional only argument forms would not be allowed, since they would make no sense (as all arguments to the inner parameter binding operation are positional by design). Then the "_unpack_args" hack above would be unnecessary, and you could just write: class C: def __init__(*(self, data=None), **kwds): self._stored_data = stored = {} if data: stored.update(data) stored.update(kwds) def update(*(self, data=None), **kwds): stored = self._stored_data if data is not None: stored.update(data) stored.update(kwds) The objection was raised that this runs counter to the philosophy behind PEP 3113 (which removed tuple unpacking from function signatures). I disagree: - this is not tuple unpacking, it is parameter binding - it does not apply to arbitrary arguments, solely to the "extra arguments" parameter, which is guaranteed to be a tuple - it allows positional-only arguments to be clearly expressed in the function signature, allowing the *interpreter* to deal with the creation of nice error messages - it *improves* introspection, since the binding of positional only arguments is now expressed clearly in the function header (and associated additional metadata on the function object), rather than being hidden inside the function implementation Regards, Nick. [1] http://hg.python.org/cpython/file/e67b3a9bd2dc/Lib/collections/abc.py#l511 -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sat, Mar 3, 2012 at 7:46 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I am very well aware of this (it occurs in two different places in the NDB library that I've been developing for Google App Engine). But either I missed some messages in the thread (quite possible) or you're bringing this up for the first time now -- the @positional decorator wasn't meant to solve this case (which only occurs when **kwds is used in this particular way). *If* you want to solve this I agree that some actual new syntax is probably needed.
Nice; that idiom should be more widely known.
Still, a naming convention for the helper function can probably make this fairly painless -- perhaps you'll need a separate helper function for each API function, named in a systematic fashion.
Nobody else seems to have seen the importance of solving *this* particular issue directly in the function signature -- but I personally support trying!
+1. This is certainly the most thorough solution for both problems at hand (simply requiring some parameters to be positional, and the specific issue when combining this with **kwds).
-- --Guido van Rossum (python.org/~guido)
Guido van Rossum wrote:
I don't understand -- example of what's allowed and not allowed?
So a (more or less) complete rundown would look like this: def foo(*(a, b)): ... # all positional def foo(*(a, b=1)): ... # all positional, b optional def foo(*(a, b), c, d): ... # a, b positional; c, d required and keyword def foo(*(a, b), c, d=1): ... # a, b positional; c required, d optional; c & d keyword def foo(*(a, b=1), c=1, d=1): ... # same, but b, c, d optional If I understand correctly, there is no way to have positional-only, position-or-keyword, and keyword-only in the same signature?
On Sun, Mar 4, 2012 at 9:20 AM, Ethan Furman <ethan@stoneleaf.us> wrote:
If I understand correctly, there is no way to have positional-only, position-or-keyword, and keyword-only in the same signature?
Heh. If that's true, my '/' proposal wins: def foo(pos_only, /, pos_or_kw, *, kw_only): ... Defaults can be added to taste. The restrictions on args-without-defaults being unable to follow args-with-defaults may need to be revisited so we can combine optional positional arguments with required keyword arguments, if we want to support that. Nevertheless all this is pretty esoteric and I wouldn't cry if it wasn't added. There exist solutions for the Mapping-API problem, and a @positional decorator would cover most other cases. -- --Guido van Rossum (python.org/~guido)
Guido van Rossum wrote:
Now that I understand that / will only appear in at most one place, like * (and not following each and every positional-only arg) this is the nicest syntax I've seen yet. If we have to have this feature, +1 on this syntax. I'm still only +0 on the feature itself. -- Steven
On 4 March 2012 22:32, Steven D'Aprano <steve@pearwood.info> wrote:
Agreed. However I've *never* wanted to create a "positional args only" parameter to an api that wasn't covered by *args. I also think that in *general* allowing keyword args is an improvement to APIs. So I guess I'm -0 on the feature, but the "/" syntax seems the best of the ones suggested so far. Michael
-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
Steven D'Aprano wrote:
It's reasonably nice, but I'm not sure about giving the '/' its own slot with commas either side. This works for * and ** because they (optionally now in the case of *) take a name after them, but the '/' never will. So how about treating '/' as a separator instead: def foo(pos1, pos2 / arg3, arg4, *args, **kwds): -- Greg
Greg Ewing wrote:
Looks a lot like division to me. Plus you then have the signature different from the call (as it *would* be division if you tried to use it as a separator when calling it). Unless we have a good reason to treat it differently from a lone '*', I think we should be consistent and treat it the same. (I obviously don't think the lack of a name is a good enough reason to be inconsistent. ;) ~Ethan~
On Mon, Mar 5, 2012 at 4:15 AM, Guido van Rossum <guido@python.org> wrote:
Yes, I only realised after Ethan's reply that my approach puts the "positional only" parameters in the wrong place relative to normal parameters (I didn't notice because I'm mainly interested in the Mapping use case and that doesn't accept any normal parameters - just positional only and arbitrary keywords). So, *if* syntactic support for positional-only arguments were added, I think Guido's syntax would be the way to do it. However, now that I've realised the "arbitrary keyword arguments" problem can be solved fairly cleanly by a helper function that binds the positional arguments, I'm more inclined to just leave it alone and tell people to just accept *args and process it that way. OTOH, having a docs-friendly syntax, and better programmatic introspection for the cases where it does come up would be nice, too...
Already done:
Yep. While I do think it's a slight language wart that we can't cleanly express all the function and method signatures that are used by our own builtins and ABC definitions, it's a *very* minor concern overall. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Mon, 2012-03-05 at 10:31 +1000, Nick Coghlan wrote:
Yeah, I think I agree with that for now. I feel signatures are already pretty complex. If we found a solution that worked while simplifying or merging some of that complexity in a nice way, I'd be +1. Your suggested syntax was leaning in that direction I think. I liked that there was a possibly a clearer separation between positional arguments and keywords arguments. If you look at the C code that parses signatures, it's not simple or straight forward. It's not easy to write a python function that maps (*args, **kwds) to a signature in the same way. I tried it to test out some ideas a while back. What makes it difficult is some of the "*args" values can be keyword arguments assigned by position. Or some values in "**kwds" values may be positional arguments assigned by name. I think the order not being preserved in kwds was also a factor. While in the simple cases, it's fairly easy to mentally parse a signature, the mental slope gets steeper as you start to combine the different concepts into one signature. Maybe it's easy for those who do it every day, but not as easy for those doing it less often, or for those who are just beginning to learn python.
When I was looking at your syntax I was thinking of it like this... def foo(*(a, b=2)->args, **(c=3)->kwds): ... return args, kwds Which would map the positional only arguments to args, and the rest to kwds and include the default values as well. But that's a different thing. That wouldn't be friendly to duck typing because functions in the front of the chain should be not care what the signature of the function at the end of the chain will be. It would be limited to functions (ducks) who's signatures are compatible with each other. The (*args, **kwds) signature only corresponds to positional and keywords arguments in the simple cases where no positional (only) argument has a default value, and no keyword arguments are assigned by position. As far as better docs-friendly syntax, and introspection are concerned, I think we will need a signature object that can be introspected. It also might be helpful in evaluating ideas like these. Cheers, Ron
Antoine Pitrou wrote:
Then please consider also re-introducing parameter tuple unpacking, since that was genuinely useful.
It may have been useful, but my understanding is that it was removed because the complications in implementing it were greater, particularly where introspection was concerned. ~Ethan~
On Sun, Mar 4, 2012 at 8:57 AM, Antoine Pitrou <solipsis@pitrou.net> wrote:
That's debatable - reread PEP 3113. I added my +1 to Nick's proposal a little hastily, it should have been +0. I think that *if* we want to solve this, my '/' solution should also be on the table. It has the downside of not being obvious, but I don't think that Nick's proposal is all that obvious either to people who encounter it for the first time -- you have to combine a bunch of powerful ideas to "get" it. And the () inside () just *begs* for arbitrary nesting, which we don't want to reintroduce. We don't want this: def foo(*(*(*(a, b), c), d), e): ... :-) -- --Guido van Rossum (python.org/~guido)
On 3/3/2012 3:39 PM, Guido van Rossum wrote:
Yeah, so it does make sense to standardize on a solution for this
Agreed. There are actually two issues. Doc: indicate intent, regardless of how enforced in code. Code: indicate intent to interpreter so it enforces intent rather than programmer doing do with *args, defaults if any, and error messages.
Let it be @positional(N).
You seem to have backed off on that. I would like a solution for the docs that Georg can tolerate.
Can you file an issue?
When you have settled on one thing for at least a day ;-). Until then, I think it is better to keep discussion in one place, which is here. --- The pos(n) idea does not work because position-only args may still have defaults. For instance, range() takes 1 to 3 args. That proposal did give me this idea: tag positional names with their index. In a sense, the index *is* the internal name while apparent alphabetic name is suggestive for human understanding. For doc purposes, the tag could be either a prefix or suffix. Either way, it would be a convention that does not conflict with any stdlib names that I know of. range(start_0 = 0, stop_1, step_2 = 1) Retern ... range(0_start = 0, 1_stop, 2_step = 1) Return ... For Python code, I presume the prefix form would be rejected by the lexer. A possibility would be 'mangled' dunder names in the signature, such as __0_start__, which would be stripped to 'start' for use in the code. If this idea makes '/' look better, fine with me. -- Terry Jan Reedy
04.03.12 22:59, Terry Reedy написав(ла):
Extend this for function arguments: """ However, there is a convention that is followed by most Python code: a name prefixed with an underscore (e.g. _spam) should be treated as a non-public part of the API (whether it is a function, a method or a data member). It should be considered an implementation detail and subject to change without notice. """
On 03.03.2012 21:11, Terry Reedy wrote:
I don't think that is a good idea. We currently put argument default values in the function signatures in Python syntax, but only because that also makes sense from a documentation PoV. We also wouldn't write @property name(self) just because that's (one) way for creating properties from Python. Georg (I am -0 on @positional myself: IMO such a completely different way of declaring positional-only and keyword-only arguments lacks grace.)
On Sat, 2012-03-03 at 09:54 -0800, Guido van Rossum wrote:
Yes, except the caller does have the option to use the *args and/or **kwds and other variations of packing and unpacking when it's convenient. And yes, I realize it doesn't really change the signature it self, but it is a different way to spell a function call and sometimes is very helpful when the data can be matched to the signature. For example, by not specifying any arguments as keyword only, or position only, both of the following examples work, and the function caller has these options.
But by specify an argument as keyword only, then it removes the *args option. And also if an argument is specified as position only, then the **kwds spelling wont work. I'm not suggesting there isn't sometimes a need for being more specific, but that quite often it's nicer to let the caller have those options rather than limit the API too narrowly.
I was trying to make a more general point which is why I preceded my comments with, "On a more general note...", which was left out of the reply. Yes, it most definitely is an ART to create a good API. And also yes, sometimes minimizing errors take priority, especially when those errors can be very costly. It seems to me, binding an object by name is less likely to be wrong than binding a object by it's position. The advantage of 'by position' is that many objects are stored in ordered lists. ie.. [start, stop, step], [x, y, z]. So it's both easier and more efficient to use the position rather than a name especially if the object can be *unpacked directly into the signature. I just can't think of a good case where I would want to prohibit setting an argument by name on on purpose. But I suppose if I encountered a certain error that may have been caught by doing so, I may think about doing that. <shrug> Cheers, Ron
On Sat, 2012-03-03 at 16:46 -0800, Guido van Rossum wrote:
Yep, I missed Nicks message where he points out...
def dct(a, **kwds): ... return a, kwds
Would the positional decorator fix this particular case? It seems like it would work for forcing an error, but not for multiple values with the same name. The way to currently get around this is to use *args along with **kwds.
The names used with '*' and '**' are already anonymous as far as the foo signature is concerned, so you can use args or kwds as keywords without a problem. I'm not sure what the positional decorator would gains over this. The other use case mentioned is the one where you point out overriding an undocumented variable name. Seems like this is a question of weather or not it is better to make python behavior match the C builtins behavior, vs making the C builtins behavior match python behavior. Cheers, Ron
On Sun, Mar 4, 2012 at 6:16 AM, Ron Adam <ron3200@gmail.com> wrote:
There are two issues being discussed here: 1. A new syntax for positional-only arguments. I don't really see any good use case for this which can't already be dealt with quite easily using *args. True, it means a bit more work on the documentation, but is it really worth adding new syntax (or even a built-in decorator) just for that? 2. How to avoid the problems with name-binding to an intended positional only argument. Once again, this can be dealt with using *args. In both cases it would be nice to be able to avoid having to manually parse *args and **kwargs, but I haven't really seen anything better that the status quo for dealing with them. The only way I see this really working is to somehow bind positional-only arguments without binding each them to a specific name, and the only way I can think of to do that is to store them in a tuple. Perhaps, then, the syntax should reflect a C-style array: # pos(2) indicates 2 positional arguments def f(pos(2), arg1, *args, **kwargs): print(pos) print(arg1) print(args) print(kwargs)
I'm +0 on the whole idea, but only if it deals with both issues. David
David Townshend wrote:
The problem with *args is that it allows 0-N arguments, when we want, say, 2.
2. How to avoid the problems with name-binding to an intended positional only argument. Once again, this can be dealt with using *args.
Again, the problem is *args accepts a range of possible arguments.
Not good. The issue is not restricting the author from binding the positional arguments to names, the issue is restricting the user from binding the arguments to names -- but even then, the user (and the author!) need to have those names apparent. For example: str.split(pos(2)) Quick, what should be supplied for the two positional arguments? We want the arguments bound to names *in the function* -- we don't want the arguments bound to names *in the function call*. ~Ethan~
On Sun, Mar 4, 2012 at 5:34 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
Agreed, but why not this? def f(*args, **kwargs): assert len(args) == 2 The exception raised could easily be a TypeError too, so it would appear, to a user, the same as defining it in the function signature. There are of course, other limitations (e.g. introspection), but without a specific use case it is difficult to know how important those limitations are.
Why does the user need the names? If the user cannot use them as keywords, then it doesn't matter what the names are, so anything can be used in the documentation (which is the only place they would appear). The author doesn't need the names either, just the data they refer to, in this case a tuple.
Is that a function call or definition? My suggestion was to use the parenthesis in the definition, not the call. Since str.split only has optional arguments its not a good example, but if you were to redefine str.replace (in python) it would look like this: def replace(pos(3), count=None): self, old, new = pos ... It would still be called in the same way though:
'some string'.replace('s', 't', 1) 'tome string'
We want the arguments bound to names *in the function* -- we don't want the arguments bound to names *in the function call*.
That is what I proposed, I just suggested binding all of the positinoal-only arguments to a single name. Having given it a bit more thought, though, maybe it would be easier to optionally apply the parenthesis to *args: def replace(self, *args(2), count=None); old, new = args This looks much closer to current situation, and I suppose could be extended to **kwargs, limiting the number of keyword arguments (although I have no idea why this would ever be wanted!)
On Sat, 3 Mar 2012 09:54:03 -0800 Guido van Rossum <guido@python.org> wrote:
Those situations are probably very rare. AFAIK we haven't seen anyone mention a serious use case. I think concerns of built-in functions shadowed by Python functions or the reverse are mostly academic, since we don't see anyone complaining about dict-alikes accepting keyword args. (besides, what happened to "consenting adults"? :-))
Anyway, I now think that adding a built-in @positional(N) decorator makes the most sense since it doesn't require changes to the parser.
-1 on a built-in for that. The functools module would probably be a good recipient (assuming the decorator is useful at all, of course). Regards Antoine.
On Sat, Mar 3, 2012 at 4:55 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
No, because the base class's insistence on positional args makes it a non-starter to use keyword args. But APIs that are implemented in Python don't have this nudge. Given that some folks here have expressed a desire to use keyword args *everywhere*, which I consider going way overboard, as a readability and consistency advocate I want to be able to remind them strongly in some cases not to do that.
(besides, what happened to "consenting adults"? :-))
We used to say that about the lone star feature too. But it's turned out quite useful, both for readability (require callers to name the options they're passing in) and for allowing evolution of a signature by leaving the option to add another positional argument in the future. Some folks seem to believe that keywords are always better. I want to send a strong message that I disagree.
TBH, I've never gotten the hang of functools. It seems mostly a refuge for things I don't like; so @positional() doesn't belong there. :-) -- --Guido van Rossum (python.org/~guido)
Guido van Rossum wrote:
I think you're reading too much into what has been a pretty luke-warm response to Gregory's suggestion. As far as I can see, I've been the least negative about the idea, and that was a half-hearted +0. I described it as "nice to have" specifically on the basis of consistency, to minimize the differences between pure-Python functions and built-ins. On reflection, your argument about subclassing built-ins has convinced me to drop that to a -1: if we were designing Python from scratch, it would be a nice-to-have for built-ins to take named arguments, but since they don't, it would be too disruptive to add them en-mass. I'm still +1 on adding named arguments to built-ins where needed, e.g.
but I hope that would be uncontroversial!
Positional-only arguments should be considered a new feature that requires justification, not just to send a message against mass re-engineering of built-ins. Some arguments in favour: * Consistency with built-ins. * Functions that take a single argument don't need to be called by keyword. (But is that a reason to prohibit it?) * It gives the API developer another choice when designing their function API. Maybe some people just don't like keyword args. (But is "I don't like them" a good reason to prohibit them? I'd feel more positive about this argument if I knew a good use-case for designing a new function with positional-only arguments.) Arguments against: * YAGNI. The subclass issue is hardly new, and as far as I know, has never been an actual problem in practice. Since people aren't calling subclass methods using keywords, why try to enforce it? * It's another thing to learn about functions. "Does this function take keyword arguments or not?" So far, I'm +0 on this.
What, you don't like @wraps? Astonishing! :-) -- Steven
Guido van Rossum wrote:
I think that's a genuine problem in theory. But is it a problem in practice? Since find('a', end=1) doesn't currently work, there won't be any code using it in practice. Even if a subclass looks like this: class MyString(str): def find(self, substring, beginning=0, last=None): ... internally MyString.find must be using positional arguments if it calls str.find, because keyword arguments don't currently work. So this suggested change will not break existing code. I can see one other objection to the change: if str.find accepts keywords, and MyString.find accepts *different* keywords, that is a violation of the Liskov Substitution Principle. Those who care about this would feel obliged to fix their code to match the argument names used by str.find, so if you call that mismatch "breaking backwards compatibility", I accept that. [Aside: in my experience, most programmers are unaware of Liskov, and accidentally or deliberately violate it frequently.] But given that str.find has been documented as "S.find(sub[, start[, end]])" forever, I don't have a lot of sympathy for anyone choosing different argument names. (I'm one of them. I'm sure I've written string subclasses that used s instead of sub.) I think that the practical benefit in clarity and readability in being able to write s.find('a', end=42) instead of s.find('a', 0, 42) outweighs the theoretical harm, but I will accept that there is a downside. -- Steven
On 02Mar2012 14:01, Gregory P. Smith <greg@krypto.org> wrote: | On Fri, Mar 2, 2012 at 12:00 PM, Guido van Rossum <guido@python.org> wrote: | > I've written such decorators too, but they've got quite a bit of | > overhead... | > | yeah those fall into the gross hacks I alluded to in my original post. ;) | | I intentionally decided to leave out discussion of "should we allow | positional-only arguments to be declared in Python" but it is a valid | discussion and thing to consider... | | if we go that route, could it be possible to implement range([start=0, ] | stop[, step=1]) such that they are positional only but mutliple arguments | are treated different than strictly sequential without writing conditional | code in Python to figure out each one's meaning at runtime. More excitiingly, one could embed a slice in the hypothetical positional-only syntax to say the 0th, 2nd and 4th parameters are positional-only. Or an arbitrary sequence! Hmm, a callable that returns an iterable at function call time! Sorry, too much coffee... -- Cameron Simpson <cs@zip.com.au> DoD#743 http://www.cskk.ezoshosting.com/cs/ Give me the luxuries of life and I will willingly do without the necessities. - Frank Lloyd Wright
On 3/2/2012 3:32 PM, Ethan Furman wrote:
I think this is what we need. I see the problem as being that a) C and Python functions work differently, and b) the doc does not -- and should not -- specify the implementation. One solution is to make all C functions work like Python functions. The other is to allow Python functions to work like C functions. Given the reasonable opposition to the first, we need the second.
That is probably better than using '$' or directly tagging the names. -- Terry Jan Reedy
Terry Reedy wrote:
I chose '?' because it has some similarity to an incompletely-drawn 'p', and also because it suggests a sort of vagueness, as in not being able to specify the name of the argument. I do not know if it is the best possible way, and am looking forward to other ideas. ~Ethan~
On Fri, Mar 2, 2012 at 2:49 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
I'd rather not start using a new punctuation character for this one very limited purpose; it might prevent us from using ? for some other more generic purpose in the future. Alternative proposal: how about using '/' ? It's kind of the opposite of '*' which means "keyword argument", and '/' is not a new character. -- --Guido van Rossum (python.org/~guido)
On 2012-03-02, at 5:46 PM, Guido van Rossum wrote:
Alternative proposal: how about using '/' ? It's kind of the opposite of '*' which means "keyword argument", and '/' is not a new character.
How about ';'? Is it possible to re-use it in this context? def (a; b, *, c) def (; b) - Yury P.S. Sometimes I feel nostalgic for the moratorium...
Guido van Rossum wrote:
So it would look like: def ord(char, /): def split(self, char, /, count) def canary(breed, /, color, wingspan, *, name) I think I like that better -- it stands out, and it looks like a barrier between the positional-only and the positional-keyword arguments. ~Ethan~
Ethan Furman wrote:
Urrggg, ugly and hard to read. Imagine, if you will: def spam(x, /, y, /, z, /, a=2/3, /): ... Placing the tag after the argument as an extra parameter is not the right approach in my opinion. It's excessively verbose, and it puts the tag in the wrong place: as you read from left-to-right, you see "normal argument, no, wait, it's positional only". The tag should prefix the name. With keyworld-only arguments, the * parameter is special because it flags a point in the parameter list, not an individual parameter: you read "normal arg, normal arg, start keyword-only args, keyword-only arg, ...". I believe that the right place to tag the parameter is in the parameter itself, not by adding an extra parameter after it. Hence, something like this: def spam(~x, ~y, ~z, ~a=2/3): ... where ~name means that name cannot be specified by keyword. I read it as "not name", as in, the caller can't use the name. Or if you prefer Guido's pun: def spam(/x, /y, /z, /a=2/3): ... Much less line-noise than spam(x, /, y, /, z, /, a=2/3, /). Personally, I think this is somewhat of an anti-feature. Keyword arguments are a Good Thing, and while I don't believe it is good enough to *force* all C functions to support them, I certainly don't want to discourage Python functions from supporting them. -- Steven
On Fri, Mar 2, 2012 at 6:57 PM, Steven D'Aprano <steve@pearwood.info> wrote:
That can't be right -- if a parameter is positional, surely all parameters before it are also positional, so it would be redundant to have to mark all of them up. Also ~name looks too much like an expression and /name looks just weird (I think DOS command line flags used to look like this).
And yet people invent decorators and other hacks to insist on positional parameters all the time. You *can* have Too Much of a Good Thing, and for readability it's better if calls are consistent. If most calls to a function use positional arguments (at least for the first N positions), it's better to force *all* calls to use positional arguments 1-N: the reader may be unfamiliar with the parameter names. Also remember the subclassing issue I brought up before. That said, I can't come up with a syntax that I really like. Here's my best attempt, but I'm at most -0 on it: Have a stand-alone '/' indicate "all parameters to my left must be positional", just like a stand-alone '*' means "all parameters to my right must be keywords". If there's no stand-alone '*' it is assumed to be all the way on the right; so if there's no '/' it is assumed to be all the way on the left. The following should all be valid: def foo(/, a, b): ... # edge case, same as def foo(a, b): ... def foo(a, b, /): ... # all positional def foo(a, b=1, /): ... # all positional, b optional def foo(a, b, /, c, d): ... # a, b positional; c, d required and either positional or keyword def foo(a, b, /, c, d=1): ... # a, b positional; c required, d optional; c, d either positional or keyword def foo(a, b, /, c, *, d): ... # a, b positional; c required and either positional or keyword; d required keyword def foo(a, b=1, /, c=1, *, d=1): ... # same, but b, c, d optional That about covers it. But agreed it's no thing of beauty, so let's abandon it. -- --Guido van Rossum (python.org/~guido)
Guido van Rossum wrote:
And I was just starting to like it, too. :( Personally, I don't see it as any uglier than having the lone '*' in the signature; although, I don't see lone '*'s all that much, whereas the '/' could be quite prevalent. Is this something we want? We could have a built-in decorator, like property or staticmethod, to make the changes for us (each implementation would have to supply their own, of course): @positional(2) def foo(a, b) Or we could use brackets or more parentheses: def foo([a, b]) def foo((a, b)) That doesn't seem too bad... def foo((a, b=1), c=2, *, d=3) Since tuple-unpacking no longer happens in the definition signature, we don't need the leading * before the parentheses. Just my last two cents (unless the idea garners encouragement, of course ;). ~Ethan~
On Fri, Mar 2, 2012 at 6:57 PM, Steven D'Aprano <steve@pearwood.info> wrote:
I don't believe that's what the proposal is anyway. Note Ethan's "barrier" comment.
That's in the same vein as what I understand the proposal to be. "/" would flag "end of positional-only args"; there's effectively an implicit leading "/" if you don't use one in a function's definition. <snip>
+1 I can see not modifying every single C implementation as it's for little gain, and in a bunch of cases concerns 1-or-2-argument functions which are arguably worth special-casing. But making the function definition syntax even more complicated (we have annotations now, remember?) just to allow forcing calls to be (slightly) more restrictive/opaque? Hard to get on board with that. Cheers, Chris
On 3/2/2012 5:46 PM, Guido van Rossum wrote:
Alternative proposal: how about using '/' ? It's kind of the opposite of '*' which means "keyword argument", and '/' is not a new character.
It took me a moment to get the pun on div / being the inverse of mul *. I like it. Very clever -- and memorable! -- Terry Jan Reedy
On 03Mar2012 11:53, Serhiy Storchaka <storchaka@gmail.com> wrote: | 03.03.12 00:46, Guido van Rossum написав(ла): | > Alternative proposal: how about using '/' ? It's kind of the opposite | > of '*' which means "keyword argument", and '/' is not a new character. | | How about using '**' (and left '/' for some purpose in the future)? -1 from me; too much overlap with **kwargs keyword argument insertion in calls. We'd have ** in calls for keyword arguments and ** in definitions for not keyword arguments. -- Cameron Simpson <cs@zip.com.au> DoD#743 http://www.cskk.ezoshosting.com/cs/ What if there were no hypothetical situations? - Jeff Sauder
03.03.12 23:40, Cameron Simpson написав(ла):
"*identifier" is a tuple receiving any excess positional parameters. "**identifier" is a dictionary receiving any excess keyword arguments. Parameters after "*" are keyword-only parameters. ? Parameters before "**" are positional-only parameters.
Another bikeshed idea on positional-only parameters: def foo([self], a, b, *args, **kwds): ... The square brackets are meant to suggest that the name is something only of interest to the implementation of the function, and not to be taken as part of the API. -- Greg
Serhiy Storchaka wrote:
Please do not give syntactic meaning to [parameter], unless it matches the existing convention for optional parameters. Besides, positional-only arguments are not only of interest to the implementation, they are part of the API.
Or _name, as for "private" class and module members.
In my own functions, I use _name for private implementation arguments, and usually explicitly document that callers should not rely on them. In the implementation, sometimes I need to use that private argument, and I always do so by name so that it stands out that I'm using a special argument, e.g. something like: def func(spam, ham, cheese, _name=eggs): if condition: x = func(spam, ham, cheese, _name=beans) ... If you also overload _name to also mean "positional only", I would have to write x = func(spam, ham, cheese, beans) which looks like a "normal" argument. And as already mentioned, the use of _name to mean positional-only and private would clash with functions which want public positional-only. -- Steven
05.03.12 00:28, Steven D'Aprano написав(ла):
There is no clash. Both the application means the same thing -- you advise a client not to use this name as keyword argument. Of course, he may, but need not, if not aware. 'spam'.find('a', _end=1) looks terrible and no one will inadvertently use it. Or use a double underscore for the reinforcement to prevent the use of this name. def find(self, sub, __start=0, __end=None)
Steven D'Aprano wrote:
Please do not give syntactic meaning to [parameter], unless it matches the existing convention for optional parameters.
Why should it have to do that? We already have a syntax for optional parameters, and there is no reason for a reader to think that a new piece of syntax is simply duplicating existing functionality.
Besides, positional-only arguments are not only of interest to the implementation, they are part of the API.
The fact that a parameter exists in that slot is part of the API, but the *name* of it is not. This is reflected in the fact that the comma is outside the brackets and the name is inside. -- Greg
Greg Ewing wrote:
I see your later comment about metasyntax, but to clarify in case there is still any lingering doubt what I mean: When reading function signatures in *documentation*, we often see func([parameter]) used to indicate that parameter is an optional argument. If your proposal is enacted, when reading function signatures in *code*, we will see func([parameter]) used to indicate that you can't use parameter as a keyword argument. The two uses clash, which means that every time we see a [parameter] in a function signature, there will be a moment of disorientation where we have to decide whether it should be interpreted using the convention for code or the convention for documentation. Certainly there are ways of distinguishing the two from context, particularly if the default value is shown in the signature, or from subtleties of whether the comma is inside the brackets or not, or perhaps from the order ([] early in the signature probably means positional, [] at the end probably means optional). My point is not that it is impossible to distinguish optional from positional arguments, but that the similarity of syntax makes it difficult to distinguish the two *at a glance* and comprehensibility will be reduced. And heaven help us if we have a function like range with positional-only optional parameters: range([[start]=0,] [end] [, [step]=1]) --> iterable For the avoidance of doubt, I am *not* suggesting that we introduce [parameter] as syntax for optional arguments. I don't want to see [] used syntactically inside def func(...) at all, except for the VERY rare use of lists as mutable default arguments. -- Steven
On Sat, Mar 3, 2012 at 5:28 AM, Guido van Rossum <guido@python.org> wrote:
Yeah, on reflection, I'm actually -0 on adding keyword arg support to 1-arg functions.
I currently write such code as: def f(*args): arg1, arg2, arg3 = args This gives rubbish error messages when the caller makes a mistake, but it works. The obvious syntactic alternative is allowing tuple expansion specifically for *args: def f(*(arg1, arg2, arg3)): pass Then the interpreter would have enough info to still generate nice error messages, and we don't have to invent much in the way of new syntax.
Good point. The other use case is APIs like the dict constructor and dict.update which are designed to accept arbitrary keyword arguments, so you don't want to reserve particular names in the calling argument namespace for your positional arguments. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Fri, Mar 2, 2012 at 6:56 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
It's not `def herd(*args, *, breed)`, so I don't see why it would be `def herd(*(size, location), *, breed)`. I think Nick's syntax is the right one, although adding the feature to Python is probably not a good idea. Mike
I'm -1 on this issue after some thought: I think we need to look at this from the function user's perspective. For example let's take this hypothetical declaration: def func(a, b, /, x, y, *, name, color): This function may be called like this: func(v1, v2) func(v1, v2, v3, v4) func(v1, v2, y=v4, x=v3) func(v1, v2, x=v3, y=v4) func(v1, v2, v3, v4, name='westley', color='0xffffff') func(v1, v2, name='westley', color='0xffffff', x=v3, y=v4) func(v1, name='westley', color='0xffffff', x=v3, y=v4, v2) # ERROR! To me, this just feels a little too ... mutable. In C we have one way to call functions that is equal to it's function declaration. I'd be +1 for functions that have ONLY non-keyword arguments which would be declared via decorator: @positional # This name is a bit ambiguous I guess.... def func(a, b)
participants (22)
-
Antoine Pitrou -
Arnaud Delobelle -
Cameron Simpson -
Carl M. Johnson -
Chris Rebert -
David Townshend -
Ethan Furman -
Georg Brandl -
Greg Ewing -
Gregory P. Smith -
Guido van Rossum -
Michael Foord -
Mike Graham -
Nick Coghlan -
Raymond Hettinger -
Ron Adam -
Serhiy Storchaka -
Simon Sapin -
Steven D'Aprano -
Terry Reedy -
Westley Martínez -
Yury Selivanov