Positional only arguments
This was posted on python-dev two weeks ago [1] but didn't get very far; probably python-ideas is a more appropriate list if this is to be taken any further. In short, PEP 3102 enables the declaration of keyword-only arguments, making it easier for an API to grow in a backwards compatible way. As Benji York, I see the utility of positional-only arguments, not just for the sake of symmetry but for pragmatic reasons; in fact the same reasons that serve as rationale to the keywords-only PEP. I don't have any concrete proposal (syntactic or otherwise) at this point, but I'd like to see this feature in Py3K even if I dislike the specific syntax or other machinery that implements it (just like decorators). For now it's worth just getting an idea of whether the feature itself is deemed useful enough or the current status quo is adequate. George [1] http://mail.python.org/pipermail/python-dev/2006-May/064790.html -- "If I have been able to see further, it was only because I stood on the shoulders of million monkeys." Unknown
+1 for the concept. Personally, when I really wanted this, I've given the arguments names like __foo. --Guido On 5/16/07, George Sakkis <george.sakkis@gmail.com> wrote:
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
Personally, when I really wanted this, I've given the arguments names like __foo.
That could be enshrined as the syntax, much like "private" attributes. -- Benji York http://benjiyork.com
Guido van Rossum wrote:
Personally, when I really wanted this, I've given the arguments names like __foo.
But that doesn't address all the reasons for wanting keyword-only args. An important reason is when you want to accept an arbitrary number of positional args, plus some keyword args. You can only do that now by manually unpacking things from **kwds. -- Greg
On 5/16/07, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
In Py3k, the syntax is extended to support this: def f(*args, foo=1, bar=2): pass -- --Guido van Rossum (home page: http://www.python.org/~guido/)
I wrote:
Sorry, I got mixed up between the positional-only and keyword-only threads going on at the same time. Please disregard that comment. Personally I don't care much about positional-only args other than through *args. If an elegant way of accommodating them can be found, that's fine, but I wouldn't go out of my way to find one. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing@canterbury.ac.nz +--------------------------------------+
On 5/16/07, George Sakkis <george.sakkis@gmail.com> wrote:
Well, if the syntax currently suggested in PEP 3102 [1] is adopted: def compare(a, b, *, key=None): Then it could be decided that arguments before the asterisk are positional-only, and those after the asterisk are keyword-only. This would be backwards compatible, since functions defined without the asterisk could work as they currently do. Alternatively, we could decide that we don't like positional-or-keyword-arguments at all any more, and do away with them altogether. All arguments would be positional only by default, unless they are declared keyword-only (by whatever syntax is adopted). Now waiting for the big guys to knock this down :) - Tal [1] http://www.python.org/dev/peps/pep-3102/
On 16/05/2007 16.50, George Sakkis wrote:
I've needed this only a few times, and I have used something like: def foo(*args): a,b,c = args as a workaround. Doesn't look too clumsy after all. For Py4k, the only thing that occurred to me is to reuse the parenthesis: def foo((a, b, c, d), e, f, *, g, h): pass - a,b,c,d are positional-only - e,f can be either positional or keyword - g,h are only keyword. -- Giovanni Bajo
On 5/17/07, Giovanni Bajo <rasky@develer.com> wrote:
Well, it's better than nothing but it's an abuse of the *varargs syntax and not friendly to signature inspection tools.
Not too bad, but the pair of parentheses is redundant; positional-only args (if any) must start from index 0. Also this doesn't address pos-only args with defaults like Guido's example foo(abc, xyz=42). A single "delimiter" symbol (e.g. '%') between pos-only and pos-or-kw would kill both birds with one stone, i.e. something like: def foo(a, b=42, # pos-only % , # delimiter c=0, d=None, # pos-or-kw *args, # varargs e=-1,z='', # kw-only **kwds # keyword dictionary ) Pros: - Backwards compatible: args are pos-or-kw by default as they are now; only those before the new delimiter are positional. - Easy to change the status of a parameter; just move it to the appropriate section (at least if it has a default value). Cons: - Yet another magic symbol, smells like perl (reusing the single star would be ambiguous). - The presumably common cases of functions without pos-or-kw args is clumsy: def foo(a, b, c=31, %): # all pos-only def bar(a, b=1, %, *, c=31, d=None): # pos-only or kw-only Overall, I still prefer the double underscores. George -- "If I have been able to see further, it was only because I stood on the shoulders of million monkeys."
On Thursday 17 May 2007, George Sakkis wrote:
single "delimiter" symbol (e.g. '%') between pos-only and pos-or-kw would kill both birds with one stone, i.e. something like:
A delimeter that would be needed anyway would make this a little nicer: def myfunc(a, b, c=24; *, kw=42): pass This example would have 3 positional-only arguments (a, b, c) and one keyword-only argument (kw). Replacing the comma with a semicolon avoids the need for a syntax-only position (good, IMO), avoids introducing a new special character, and re-uses one that's rarely used anyway. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>
Fred L. Drake, Jr. wrote:
[A few thoughts] def foo([positonal_only_args][; [kwd_args][; kwds_only]]): ... def myfunct(a, b, c=42; e=99; kw=42): ... args = (1, 2, 3, 4) # 'e' can be in either args or kwds, kwds = {'kw': 42} # but not both. bar = foo(*args, **kwds) How about signature objects that are similar to slice objects? sig = signature(*args, **kwds) # Pre package signature. sig = signature(1, 2, 3, 4, kw=42) Could something like this have performance benefit if the same exact signature is used over and over? Possibly just passing a pre parsed name space to the function? bar = foo(sig) # No need to unpack with * or **. bar = foo(@sig) # Be lenient, extra args and kwds # are consumed or ignored. # But always error if something is missing. The '@' sign is like a vortex which sucks up unused args. ;-) Ron
One of the things I like about Python is we've got some extra room in the syntax to complexify. Careful how we spend it. The extra is on a budget, not unlimited as some things are. def foo(a, b=42; # pos-only c=0, d=None, # pos-or-kw *args; # varargs e=-1,z='', # kw-only **kwds # keyword dictionary ) def foo(a, b, c=31; ): # all pos-only def bar(a, b=1;; c=31, d=None): # pos-only or kw-only
- Yet another magic symbol, smells like perl (reusing the single star would be ambiguous).
True dat.
George Sakkis wrote:
It seems there is a growing menu of arguments options. I suppose that's a good thing. As long as it doesn't get too confusing. Another thing I've wished I had recently is a way to over specify an argument list by giving it a larger list of arguments (or keywords) than is needed. For example if I'm dispatching a collection of functions that each have different set or number of arguments, I'd like to say on the calling end for the function to take what it needs from a list and/or dictionary, but ignore what it doesn't need. In the case of positional only arguments, it could take them in order, and in the case of keywords only, by name. The point is to be able to specify it from the calling end in a generic way instead of having to do it inside the function or by adding a decorator to each function. It's kind of like an simplified or generic adapter in a way. Cheers, Ron
On 5/16/07, George Sakkis <george.sakkis@gmail.com> wrote:
In the hopes of keeping this discussion enough on track that we can eventually come to a conclusion, I thought I'd just take a couple minutes to summarize the proposals so far. I'm looking at the following function as an example:: def f(a, b=None, *, c=42, **kwargs) where ``a`` and ``b`` should be positional-only and ``c`` should be keyword-only. Here are the alternatives I've seen so far: * Do nothing, e.g.:: def (*args, c=42, **kwargs): if len(args) == 1: a = args[0] b = None elif len(args) == 2: a, b = args else: raise TypeError() Pro: backwards compatible, requires no new syntax Con: unfriendly to introspection, requires extra boiler-plate * Enforce double-underscore names as being positional-only syntactically, e.g.:: def f(__a, __b=None, *, c=42, **kwargs) Pro: requires no new syntax Con: slightly backwards incompatible (but who calls f(__a) anyway?) * Use a decorator, e.g.:: @posonly(2) def f(a, b=None, *, c=42, **kwargs) Pro: backwards compatible, requires no new syntax, could work with Python 2.5 Con: moves some signature info out of the argument list * Make the lone '*' force everything to the left to be positional-only and everything to the right be keyword-only, e.g.:: def f(a, b=None, *, c=42, **kwargs): Pro: backwards compatible (since lone '*' is not yet introduced), requires no new syntax Con: disallows named arguments with keyword-only arguments, disallows positional-only arguments with *args. * Reuse syntax like what was removed for tuple arguments, e.g.:: def f((a, b=None), *, c=42, **kwargs): Pro: ? Con: changes meaning of tuple argument syntax * Add a new delimiter, e.g.:: def f(a, b=None, %, *, c=42, **kwargs) def f(a, b=None; *, c=42, **kwargs) Pro: backwards compatible Con: requires new syntax, in second version semi-colon vs. comma is hard to spot So at the moment, it looks to me like the double-underscores and the decorator are the top competitors. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
On 5/16/07, George Sakkis <george.sakkis@gmail.com> wrote:
uhh... could you spell those out, please? keywords-only is justified by functions that take both *args and a keyword that meets meets at least one of (1) Wasn't there before, so you can't stick it in front for backwards-compatibility reasons. (think "cmp") (2) Doesn't have a default (3) Should always be called by name, for readability. The following signature is awful def max(cmp=mysort, *args): And if you've already released max, it isn't even legal. How could there ever be an argument that *needs* to be positional? There are some that don't work if you call them by keyword, because they happen to be implemented in C without keyword support, but ... is there any reason to postively forbid using the argument name? The closest I can come to an example is def lappend(object): # This mimics list.append, and the *real* function wouldn't # take a keyword, so neither will I, just to prevent bad habits. Even that can already be written if you really want to... def lappend(*object): if len(object) != 1: raise TypeError("lappend takes ...") object = object[0] ... -jJ
On 5/18/07, Jim Jewett <jimjjewett@gmail.com> wrote:
How could there ever be an argument that *needs* to be positional?
As I've mentioned before, a great example of this is the dict() signature. It takes an optional "container" argument, followed by arbitrary keywords. If you write this in the naive way:: def __init__(self, container=None, **kwargs) ... for key, value in kwargs.items(): self[key] = value then you get TypeErrors for the following code:: d.update(sequence=1, container=2) d.update(other=1, self=2) That is, the **kwargs can never include anything named 'self' or 'container'. If you could declare these two arguments as positional-only, you'd never run into this problem. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
On 5/18/07, Steven Bethard <steven.bethard@gmail.com> wrote:
Sorry, those should have been dict(sequence=1, container=2) dict(other=1, self=2) Of course, the same argument is valid for dict.update().
STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
Might you use d.update(container=2, sequence=1)? 'Can never' caught me up.
How could there ever be an argument that *needs* to be positional?
To deliberately poise, evaluate extremes. Couple continuums going on here so don't contradict one with another's counterexample. Annotate every call parameter? Double equals sign? Not advocating, but illustrating this one. Balance is trade-off between flexibility to call, and convenience to define. Function decoration is a fledgling addition to syntax. Can we use this to kindle, as opposed to snuff? Shows great ease of use promise.
On 5/18/07, Steven Bethard <steven.bethard@gmail.com> wrote:
* Use ** for keyword-only and * for positional-only: def (a, b=None, *, **, c=42, **kwargs): Pro: backwards compatible, reuses existing meanings Con: requires new syntax, changes keyword-only PEP -- Adam Olsen, aka Rhamphoryncus
On 5/18/07, Adam Olsen <rhamph@gmail.com> wrote:
I like the idea of matching the * with positional-only arguments and the ** with keyword-only arguments. In the example above though, I don't really like how we look forward for ** but backward for *. I feel like I the semantics should instead be:: - Everything after a * is a positional-only argument - Everything after a ** is a keyword-only argument These interpretations are consistent with the current meanings of *args and **kwargs. So to write the dict() or dict.update() signatures, we'd write:: def update(*, self, container=None, **kwargs) And to write the signature from my example, we'd write:: def f(*, a, b=None, **, c=42, **kwargs) My only use cases are for functions with '*' at the beginning, because the only time I care about having positional only arguments is when I also have a **kwargs, and in that situation, I want *all* other arguments to be positional only. Does anyone have a use case that requires both positional-only and positional-or-keyword arguments? STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
On 5/20/07, Steven Bethard <steven.bethard@gmail.com> wrote:
I don't have a hard use case, but more of a good API design argument, similar to those used to support the second subproposal of PEP 3102 (e.g. see [1]). Maybe it's just me but I often cannot find a good name for a parameter (at least not when I first write it) or I think of a more appropriate name later. I'd like to be able to go back and change it, knowing it won't break anything as it is always passed positionally. IOW, it's a way to say "these names are not part of the function's signature" and enforce this rather than just document it. For instance, think of a function that initially was designed to work for sequences and was defined as "def f(sequence, **kwds)". Later the implementor realizes that the same function works (or its implementation can easily change so that it works) for arbitrary iterables. It would be nice to go back and change the signature to "def f(iterable, **kwds)", knowing that nobody is calling it as f(sequence=range(10)). George [1] http://mail.python.org/pipermail/python-dev/2006-May/064695.html -- "If I have been able to see further, it was only because I stood on the shoulders of million monkeys."
On 5/20/07, George Sakkis <george.sakkis@gmail.com> wrote:
This is a good example for why you might want positional-only arguments. But I guess I wasn't clear -- I'm looking for an example where you need both a positional-only argument and a positional-or-keyword argument *in the same signature*. The reason I ask is that I'd like to propose that the lone * can only appear at the beginning of the signature. That would cover the dict() and dict.update() use cases as well as your use case above:: def f(*, sequence, **kwds) Are there any use cases that it wouldn't cover? Seems like the cautious programmer from your example above would want all of his positional arguments to be positional-only. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
On 5/20/07, Steven Bethard <steven.bethard@gmail.com> wrote:
Are you proposing the removal of positional-or-keyword arguments altogether or only in the presence of positional-only ? In the former case, perhaps this needs a separate thread on its own as it's too big of a change. In the latter case, whatever reasons one has for using positional-or-keyword arguments, chances are that they will hold also in the presence of positional-only. Honestly, I can't think of a compelling reason for positional-or-keyword arguments (with or without the presence of positional-only in the same signature) other than that by definition offer two ways to the caller to spell the same thing. Even that could be considered a misfeature since it goes against TOOWTDI. In any case, that's more of a language design and philosophy argument ("should the caller of a function have the freedom on how to pass the arguments or that should be decided only by the function writer ?") rather than a hard use case. George -- "If I have been able to see further, it was only because I stood on the shoulders of million monkeys."
On 5/20/07, George Sakkis <george.sakkis@gmail.com> wrote:
Are you proposing the removal of positional-or-keyword arguments
No. My current proposal is: - Everything after a * is a positional-only argument, up to the first ** - Everything after a ** is a keyword-only argument - A lone * may only appear at the beginning of the signature In essence, this is changing the lone * of `PEP 3102`_ into a lone **, and allowing a lone * at the beginning of the signature to indicate that all positional arguments are positional-only. .. _PEP 3102: http://www.python.org/dev/peps/pep-3102/ That means that in the following signature:: def f(*, a, b=None, *args, **, c=42, **kwargs) - 'a' and 'b' are positional-only - 'args' is positional-only (as usual) - 'c' is keyword-only - 'kwargs' is keyword-only (as usual) If you want positional-or-keyword arguments, simply omit the initial *:: def f(a, b=None, *args, **, c=42, **kwargs) - 'a' and 'b' are positional-or-keyword (as usual) - 'args' is positional-only (as usual) - 'c' is keyword-only - 'kwargs' is keyword-only (as usual) If you want keyword-only arguments with no *args, simply omit it:: def f(a, b=None, **, c=42, **kwargs) - 'a' and 'b' are positional-or-keyword (as usual) - 'c' is keyword-only - 'kwargs' is keyword-only (as usual) Does anyone have use cases that this doesn't cover? STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
On 5/20/07, Steven Bethard <steven.bethard@gmail.com> wrote:
** is redundant here. You get the same behaviour without it: def f(a, b=None, *args, c=42, **kwargs) See below though.
I initially had the same reservations you do about my proposal, but now I think there's a bigger problem: it undermines the obviousness of PEP 3102's behaviour. Consider this example: def compare(a, b, *args, key=None): Although not legal today, the behaviour is obvious. All positional arguments get sucked up in *args so key *must* be keyword-only. That leads to this variant: def compare(a, b, *, key=None): You continue sucking up extra positional args, but since it's nameless there must be none of them. Again, obvious. Changing the token required from * to ** or giving * extra behaviour when nameless just isn't obvious. Most of the proposed syntax changes also have a question of whether or not to include *args: "def f(a, *, *args)" or "def(a, *args, *)"? "def f((a), *args)" or "def f((a, *args))"? "def f(__a, *args)" or "def(__a, *__args)"? Finally (I hope) is the problem of motivation. Where as PEP 3102 is motivated by "explicit is better than implicit", positional-only arguments actually *want* implicit! It seems to be more for completeness and "C functions already support it". -1 on any syntax changes. A decorator may be okay, but I'm unsure the use cases are significant enough to warrant adding it. Go figure. -- Adam Olsen, aka Rhamphoryncus
On 5/21/07, Adam Olsen <rhamph@gmail.com> wrote:
I note that below you argue for explicit is better than implicit. Requiring the ** even when it could be inferred from the *args is a good example of EIBTI. I don't have strong feelings on this issue though.
I have to disagree. The first thing I think when I see a lone * is multiplication, not "keyword-only arguments start after me". I would say it's just as obvious to see:: def g(*, a, b) and say, "oh, it's after a * so it must be positional-only just like *args is". That is, I don't think the PEP 3102 meaning for lone * is any more intuitive than the meaning I'm proposing here.
Where as PEP 3102 is motivated by "explicit is better than implicit", positional-only arguments actually *want* implicit!
I don't follow this logic. How is a positional-only argument more implicit? You still have to supply it on every function call. It's not like you can omit it. Or are you saying that using positional arguments instead of keyword arguments is implicit?
A decorator may be okay, but I'm unsure the use cases are significant enough to warrant adding it.
Just for reference, here are the current motivating use cases: * User mapping types that need to match the dict() and dict.update() method signatures * Code where positional argument names might change, e.g. George's def f(sequence) which later becomes def f(iterable) Particularly since user mappings are relatively common, I do think this issue deserves at least a decorator. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
Steven Bethard wrote:
I don't think the PEP 3102 meaning for lone * is any more intuitive than the meaning I'm proposing here.
Although I wouldn't go so far as to call the PEP 3102 behaviour "obvious", it is a fairly *logical* extension of the existing semantics. On the other hand, a rule like "arguments after a lone * are positional-only" is something new and arbitrary. It doesn't follow logically from anything existing. Also, your proposal assigns two quite different meanings to *. If it's followed by a name, it sucks up all remaining positional args, but if it's not, it can't suck up anything, since it's followed by more positional args! -- Greg
Steven Bethard wrote:
I agree with Greg, it seems arbitrary. The number of meanings for the '*' is also growing too large I think. * multiply * pack in function defs # but at call time. * pack in left hand expressions # left of '=' * unpack in function calls * positional only indicator in function defs ** power ** pack dictionary in functions defs ** unpack dictionary in function calls ** keyword only indicator in function defs I'm sure this will get confusing to new users. Yes, each one is clear by context, but there's a lot of subtle behaviors by context going on here. What I would like is for unpack to be '^' instead of '*', then you would have... Packing operations: * multiply * pack in function defs * pack in left hand expressions ** power ** pack dictionary in functions defs Unpacking operations: ^ unpack in function calls ^ unpack in right hand expressions (possible) ^ positional only in function defs (See why I think so below) ^^ unpack dictionary in function calls ^^ keyword only in function defs (also see below) I think these become much easier to parse visually and mentally, so it's more of a human interface issue than a computer parsing issue. When using the "name=value" form in function definitions, it can be a positional or keyword argument. So all you need is a way to say this is either one or the other. (Using * here to show similarity, introducing the '^' further down.) *(b=2, c=3) positional only args *args rest of args **(b=2, c=3) keywords only kwds **kwds rest of keywords def f(a, *(b=2, c=3), *args) # b and c are positional only def f(a, **(b=2, c=3), **kwds) # b and c are keyword only Now the problem with this is when it's use with the *() and **() form, it resembles unpacking behavior, but in the *args, and **kwds form, it's packing behavior. Normally you would have only packing in a function def and unpacking in a function call. Now if we use ^ for unpacking, and * for packing, it reduces the number of meanings for the '*' and fixes the context of positional and keyword only arguments. def f(a, ^(b=2, c=3), *rest): b and c are positional only here. def f(a, d=5, ^^(b=2, c=3), **rest_kwds): # b and c are keywords only here. I feel this code better expresses it's intent. Think of the ^() as unpack these as args, and **() as unpack these as keywords. If '^' is used outside of function defs like the '*', it can retain it's meaning. a, b, *c = 1, 2, 3, 4, 5 # pack remainder z = (x, y, z) a, b, c, d = (a, ^z) # unpack z There is the possibility of extending this for use in other ways as well. xlist = [[1, 2], [3, 4, 5], [6, 7, 8, 9]] alist = [^x for x in xlist] This would have list "extend" behavior instead of list "append" behavior. Ok, just some possible ideas of how to organize all this. Cheers, Ron
On 5/21/07, Ron Adam <rrr@ronadam.com> wrote:
Yes, both my proposal and the PEP 3102 use of * are pretty arbitrary, and I agree that we're probably getting too many uses of *. I'm going to try to post a decorator that I think will cover the use cases so far.
What I would like is for unpack to be '^' instead of '*'
-1. This is at least as arbitrary as any of the '*', '**' or ';' suggestions, so I see no reason to prefer it. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
What I would like is for unpack to be '^' instead of '*', then you would have...
My first reaction was negative -- but then I remembered that I was caught by this trap myself more than once, because of the asymmetry between pack and unpack, with * magically doing the normally-right thing. So now I wonder if I wouldn't like if after all, and I'm instead picking at the presentation, so you can make a stronger case. On 5/21/07, Ron Adam <rrr@ronadam.com> wrote:
def f(a, ^(b=2, c=3), *rest): b and c are positional only here.
I don't think there should ever be positional-only arguments *after* a keywordable argument. Perhaps # a is a mandatory positional argument # b is an optional positional argument # c can be passed as a keyword, or as the third positional argument def f(^a, ^b=2, c=3, *rest): Or were you suggesting a tuple of the positional-only arguments, like # a is a mandatory positional argument # b is an optional positional argument # c can be passed as a keyword, or as the third positional argument def f(^(a, b=2), c=3, *rest):
So alist == [1, 2, 3, 4, 5, 6, 7, 8, 9]? -jJ
Jim Jewett wrote:
Yes, it's gotten me a few times as well. Usually when I have a class and some of the methods are accepting *args, **kwds, and some aren't. Then I find I'm packing arguments in some cases and unpacking them in others in the same subclass, and the methods definitions are not on the same page so it isn't immediately obvious I got them mixed up. As I said it isn't a computer issue, it's a human interface issue.
Ah, ok, I see what I missed. I think I got optional args mixed up with positional args in my mind. Not quite the same thing. So yes, the second example fits what I was trying to think of better. # a is mandatory, positional # b is optional, positional # c is optional, positional, or by keyword # d is mandatory, keyword only # e is optional, keyword only def f(^(a, b=2), c=3, ^^(d, e=5)): Of course it could be interpreted a bit differently given the unpacking context. # x is a two tuple (a, b) # y is a single value, c # z is a dict, {'d':4, 'e':5} result = f(x, y, z) Which solves a different problem entirely. ;-) It allows you to specify unpacking behavior in the definition instead of requiring it on the calling side. So I guess I'd have to fall back to the semi-colon as the positional only and keyword only designators. def f(a, b=2; c=3; d, e=5): Or alternately use the colon as it is used in slices. def f(a, b=2: c=3: d, e=5): I'd almost rather change the slices to semicolons and keep the colon unique and with only one purpose. And ... def f(a, b=2, c=3) would be equivalent to... def f(; a, b=2, c=3;): No positional only, no keyword only Anyway, I'm not sure there is a simple way to do this. Sigh...
Exactly. Cheers, Ron
Do you prohibit defaults in type 1, the pos-only? Not necc. Does this look right?
Also, does this look right?
Any more groups? So far I have pos-only, pos-or-kw, and kw-only. If not, a ascii character works fine to delimit, like * and **. Do we want to annotate each parameter individually? This could get out of hand. But decorators won't let you do @f( k ) def j( k ): ... where @f( 'k' ) def j( k ): ... gets unwieldy and could show promise. Cf. Steven's positional only arguments decorator.
Cf. PEP 3102 Talin http://www.python.org/dev/peps/pep-3102/ Cf. PEP 3107 Winder and Lownds http://www.python.org/dev/peps/pep-3107/ Cf. PEP 3117 Brandl http://www.python.org/dev/peps/pep-3117/ def f( @k, @j ) Half-way compromise to Brandl, only in declaration not in use, and always optional.
Don't forget. Any the things you want can be composed in hand.
def f(*, a, b=None, *args, **, c=42, **kwargs)
def f( *args, **kwargs ): if len( args ) < 2: raise error if 'b' in kwargs: raise error if 'c' not in kwargs: raise error a, b, args = args[:2], args[2:] c = kwargs['c'] It's an approximation; use case to the contrary?
Aaron Brady wrote:
I'm not sure why else you would post this reply other than a lack of understanding. The whole point of this discussion is to avoid burying this information inside the function body and have a complete function signature. f(*args, **kwargs) is the least useful function signature possible in python. -- Scott Dial scott@scottdial.com scodial@cs.indiana.edu
You all seemed to me to be wanting things to apply in every case. This does, but you scream at me here too. Generality or speciality, take your pick. I would say perl is highly specialized, very useful, more than python for a certain subset of tasks. What subset? Are you here for popularity, money, productivity, aesthetics, deep aesthetics, raw lines of code per day per time zone around the world, or none of the above? 'Cause ML is a beautiful idea; doesn't take a grad student to appreciate that. Just to keep your job is money. Love it.
On 5/21/07, Steven Bethard <steven.bethard@gmail.com> wrote:
I wasn't aware the proposals given would meet this use case! PEP 3102 has no effect on correct code, instead it just adds some error detection for incorrect code. In contrast, the dict()/dict.update() signature use case needs semantics to change for an example like this: def x((a=None), **kwargs): ... x(a=3) This can't be wrapped properly without very special care. For instance, this would behave wrong: def method(self, *args, **kwargs): x(*args, **kwargs) I'm strongly against adding a feature to split **kwargs into a separate namespace when we have such a common convention of NOT doing it for self[1][2]. The decorator proposed in the other thread is a much better solution unless we want to overturn that convention. [1] Google codesearch for "self[^()]*\*\* lang:python" returned 28,800 results.. make that 200 results.. or 186 results.. numbers aren't too accurate it seems. [2] "\([a-z][^()]*\*\* lang:python" claims 29,500, but won't let me go past 200 results. Maybe there's an internal limiter? -- Adam Olsen, aka Rhamphoryncus
On 5/21/07, Adam Olsen <rhamph@gmail.com> wrote:
This may not be so obvious anymore now that we have extended iterable unpacking from `PEP 3132`_. Given an iterable unpacking like:: a, b, *c, d = iterable I may assume that names after a *args work just like names before one:: def f(a, b, *c, d) So in this case I'd assume that the 'd' is also positional, not keyword-only. Again, I'm not saying people can't learn the appropriate meaning, just that it's not inherently obvious. .. _PEP 3132: http://www.python.org/dev/peps/pep-3132/ STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
On 5/21/07, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
I wasn't either, but I believe the PEP has already been accepted and implemented. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
Steven Bethard wrote:
That means that in the following signature::
def f(*, a, b=None, *args, **, c=42, **kwargs)
I don't like this. Having more than one * or ** in the same signature seems like it would be very confusing. I think the desire for positional-only args would be addressed well enough using a naming convention. Even without further explanation, if I saw something like def f(_a, _b, _c): ... it would be pretty clear to me that the author intended the parameter names to be an implementation detail. This could even be enforced if relying on a convention were not considered strong enough, although it would be more consistent to insist on a double underscore to get automatic enforcement. -- Greg
On 5/21/07, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
So what about the signature for dict() and dict.update() where you want to write:: def update(self, container=None, **kwargs) Note that I shouldn't want to write '__container' because it's not an implementation detail. It's a valid positional argument that my clients can use. But if I write 'container', then I'm limiting what they can send as **kwargs and I can't reproduce the dict behavior:: >>> d = {} >>> d.update(container=1) >>> d {'container': 1} STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
On 5/21/07, Steven Bethard <steven.bethard@gmail.com> wrote:
def update(self, container=None, **kwargs)
Note that I shouldn't want to write '__container' because it's not an implementation detail.
Yes, it is. The fact that there *is* a parameter there isn't an implementation detail, but the *choice of name* for that parameter is explicitly an implementation detail -- if it weren't, then you wouldn't need to insist on positional-only calling. -jJ
+1 for the concept. Personally, when I really wanted this, I've given the arguments names like __foo. --Guido On 5/16/07, George Sakkis <george.sakkis@gmail.com> wrote:
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
Personally, when I really wanted this, I've given the arguments names like __foo.
That could be enshrined as the syntax, much like "private" attributes. -- Benji York http://benjiyork.com
Guido van Rossum wrote:
Personally, when I really wanted this, I've given the arguments names like __foo.
But that doesn't address all the reasons for wanting keyword-only args. An important reason is when you want to accept an arbitrary number of positional args, plus some keyword args. You can only do that now by manually unpacking things from **kwds. -- Greg
On 5/16/07, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
In Py3k, the syntax is extended to support this: def f(*args, foo=1, bar=2): pass -- --Guido van Rossum (home page: http://www.python.org/~guido/)
I wrote:
Sorry, I got mixed up between the positional-only and keyword-only threads going on at the same time. Please disregard that comment. Personally I don't care much about positional-only args other than through *args. If an elegant way of accommodating them can be found, that's fine, but I wouldn't go out of my way to find one. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing@canterbury.ac.nz +--------------------------------------+
On 5/16/07, George Sakkis <george.sakkis@gmail.com> wrote:
Well, if the syntax currently suggested in PEP 3102 [1] is adopted: def compare(a, b, *, key=None): Then it could be decided that arguments before the asterisk are positional-only, and those after the asterisk are keyword-only. This would be backwards compatible, since functions defined without the asterisk could work as they currently do. Alternatively, we could decide that we don't like positional-or-keyword-arguments at all any more, and do away with them altogether. All arguments would be positional only by default, unless they are declared keyword-only (by whatever syntax is adopted). Now waiting for the big guys to knock this down :) - Tal [1] http://www.python.org/dev/peps/pep-3102/
On 16/05/2007 16.50, George Sakkis wrote:
I've needed this only a few times, and I have used something like: def foo(*args): a,b,c = args as a workaround. Doesn't look too clumsy after all. For Py4k, the only thing that occurred to me is to reuse the parenthesis: def foo((a, b, c, d), e, f, *, g, h): pass - a,b,c,d are positional-only - e,f can be either positional or keyword - g,h are only keyword. -- Giovanni Bajo
On 5/17/07, Giovanni Bajo <rasky@develer.com> wrote:
Well, it's better than nothing but it's an abuse of the *varargs syntax and not friendly to signature inspection tools.
Not too bad, but the pair of parentheses is redundant; positional-only args (if any) must start from index 0. Also this doesn't address pos-only args with defaults like Guido's example foo(abc, xyz=42). A single "delimiter" symbol (e.g. '%') between pos-only and pos-or-kw would kill both birds with one stone, i.e. something like: def foo(a, b=42, # pos-only % , # delimiter c=0, d=None, # pos-or-kw *args, # varargs e=-1,z='', # kw-only **kwds # keyword dictionary ) Pros: - Backwards compatible: args are pos-or-kw by default as they are now; only those before the new delimiter are positional. - Easy to change the status of a parameter; just move it to the appropriate section (at least if it has a default value). Cons: - Yet another magic symbol, smells like perl (reusing the single star would be ambiguous). - The presumably common cases of functions without pos-or-kw args is clumsy: def foo(a, b, c=31, %): # all pos-only def bar(a, b=1, %, *, c=31, d=None): # pos-only or kw-only Overall, I still prefer the double underscores. George -- "If I have been able to see further, it was only because I stood on the shoulders of million monkeys."
On Thursday 17 May 2007, George Sakkis wrote:
single "delimiter" symbol (e.g. '%') between pos-only and pos-or-kw would kill both birds with one stone, i.e. something like:
A delimeter that would be needed anyway would make this a little nicer: def myfunc(a, b, c=24; *, kw=42): pass This example would have 3 positional-only arguments (a, b, c) and one keyword-only argument (kw). Replacing the comma with a semicolon avoids the need for a syntax-only position (good, IMO), avoids introducing a new special character, and re-uses one that's rarely used anyway. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>
Fred L. Drake, Jr. wrote:
[A few thoughts] def foo([positonal_only_args][; [kwd_args][; kwds_only]]): ... def myfunct(a, b, c=42; e=99; kw=42): ... args = (1, 2, 3, 4) # 'e' can be in either args or kwds, kwds = {'kw': 42} # but not both. bar = foo(*args, **kwds) How about signature objects that are similar to slice objects? sig = signature(*args, **kwds) # Pre package signature. sig = signature(1, 2, 3, 4, kw=42) Could something like this have performance benefit if the same exact signature is used over and over? Possibly just passing a pre parsed name space to the function? bar = foo(sig) # No need to unpack with * or **. bar = foo(@sig) # Be lenient, extra args and kwds # are consumed or ignored. # But always error if something is missing. The '@' sign is like a vortex which sucks up unused args. ;-) Ron
One of the things I like about Python is we've got some extra room in the syntax to complexify. Careful how we spend it. The extra is on a budget, not unlimited as some things are. def foo(a, b=42; # pos-only c=0, d=None, # pos-or-kw *args; # varargs e=-1,z='', # kw-only **kwds # keyword dictionary ) def foo(a, b, c=31; ): # all pos-only def bar(a, b=1;; c=31, d=None): # pos-only or kw-only
- Yet another magic symbol, smells like perl (reusing the single star would be ambiguous).
True dat.
George Sakkis wrote:
It seems there is a growing menu of arguments options. I suppose that's a good thing. As long as it doesn't get too confusing. Another thing I've wished I had recently is a way to over specify an argument list by giving it a larger list of arguments (or keywords) than is needed. For example if I'm dispatching a collection of functions that each have different set or number of arguments, I'd like to say on the calling end for the function to take what it needs from a list and/or dictionary, but ignore what it doesn't need. In the case of positional only arguments, it could take them in order, and in the case of keywords only, by name. The point is to be able to specify it from the calling end in a generic way instead of having to do it inside the function or by adding a decorator to each function. It's kind of like an simplified or generic adapter in a way. Cheers, Ron
On 5/16/07, George Sakkis <george.sakkis@gmail.com> wrote:
In the hopes of keeping this discussion enough on track that we can eventually come to a conclusion, I thought I'd just take a couple minutes to summarize the proposals so far. I'm looking at the following function as an example:: def f(a, b=None, *, c=42, **kwargs) where ``a`` and ``b`` should be positional-only and ``c`` should be keyword-only. Here are the alternatives I've seen so far: * Do nothing, e.g.:: def (*args, c=42, **kwargs): if len(args) == 1: a = args[0] b = None elif len(args) == 2: a, b = args else: raise TypeError() Pro: backwards compatible, requires no new syntax Con: unfriendly to introspection, requires extra boiler-plate * Enforce double-underscore names as being positional-only syntactically, e.g.:: def f(__a, __b=None, *, c=42, **kwargs) Pro: requires no new syntax Con: slightly backwards incompatible (but who calls f(__a) anyway?) * Use a decorator, e.g.:: @posonly(2) def f(a, b=None, *, c=42, **kwargs) Pro: backwards compatible, requires no new syntax, could work with Python 2.5 Con: moves some signature info out of the argument list * Make the lone '*' force everything to the left to be positional-only and everything to the right be keyword-only, e.g.:: def f(a, b=None, *, c=42, **kwargs): Pro: backwards compatible (since lone '*' is not yet introduced), requires no new syntax Con: disallows named arguments with keyword-only arguments, disallows positional-only arguments with *args. * Reuse syntax like what was removed for tuple arguments, e.g.:: def f((a, b=None), *, c=42, **kwargs): Pro: ? Con: changes meaning of tuple argument syntax * Add a new delimiter, e.g.:: def f(a, b=None, %, *, c=42, **kwargs) def f(a, b=None; *, c=42, **kwargs) Pro: backwards compatible Con: requires new syntax, in second version semi-colon vs. comma is hard to spot So at the moment, it looks to me like the double-underscores and the decorator are the top competitors. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
On 5/16/07, George Sakkis <george.sakkis@gmail.com> wrote:
uhh... could you spell those out, please? keywords-only is justified by functions that take both *args and a keyword that meets meets at least one of (1) Wasn't there before, so you can't stick it in front for backwards-compatibility reasons. (think "cmp") (2) Doesn't have a default (3) Should always be called by name, for readability. The following signature is awful def max(cmp=mysort, *args): And if you've already released max, it isn't even legal. How could there ever be an argument that *needs* to be positional? There are some that don't work if you call them by keyword, because they happen to be implemented in C without keyword support, but ... is there any reason to postively forbid using the argument name? The closest I can come to an example is def lappend(object): # This mimics list.append, and the *real* function wouldn't # take a keyword, so neither will I, just to prevent bad habits. Even that can already be written if you really want to... def lappend(*object): if len(object) != 1: raise TypeError("lappend takes ...") object = object[0] ... -jJ
On 5/18/07, Jim Jewett <jimjjewett@gmail.com> wrote:
How could there ever be an argument that *needs* to be positional?
As I've mentioned before, a great example of this is the dict() signature. It takes an optional "container" argument, followed by arbitrary keywords. If you write this in the naive way:: def __init__(self, container=None, **kwargs) ... for key, value in kwargs.items(): self[key] = value then you get TypeErrors for the following code:: d.update(sequence=1, container=2) d.update(other=1, self=2) That is, the **kwargs can never include anything named 'self' or 'container'. If you could declare these two arguments as positional-only, you'd never run into this problem. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
On 5/18/07, Steven Bethard <steven.bethard@gmail.com> wrote:
Sorry, those should have been dict(sequence=1, container=2) dict(other=1, self=2) Of course, the same argument is valid for dict.update().
STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
Might you use d.update(container=2, sequence=1)? 'Can never' caught me up.
How could there ever be an argument that *needs* to be positional?
To deliberately poise, evaluate extremes. Couple continuums going on here so don't contradict one with another's counterexample. Annotate every call parameter? Double equals sign? Not advocating, but illustrating this one. Balance is trade-off between flexibility to call, and convenience to define. Function decoration is a fledgling addition to syntax. Can we use this to kindle, as opposed to snuff? Shows great ease of use promise.
On 5/18/07, Steven Bethard <steven.bethard@gmail.com> wrote:
* Use ** for keyword-only and * for positional-only: def (a, b=None, *, **, c=42, **kwargs): Pro: backwards compatible, reuses existing meanings Con: requires new syntax, changes keyword-only PEP -- Adam Olsen, aka Rhamphoryncus
On 5/18/07, Adam Olsen <rhamph@gmail.com> wrote:
I like the idea of matching the * with positional-only arguments and the ** with keyword-only arguments. In the example above though, I don't really like how we look forward for ** but backward for *. I feel like I the semantics should instead be:: - Everything after a * is a positional-only argument - Everything after a ** is a keyword-only argument These interpretations are consistent with the current meanings of *args and **kwargs. So to write the dict() or dict.update() signatures, we'd write:: def update(*, self, container=None, **kwargs) And to write the signature from my example, we'd write:: def f(*, a, b=None, **, c=42, **kwargs) My only use cases are for functions with '*' at the beginning, because the only time I care about having positional only arguments is when I also have a **kwargs, and in that situation, I want *all* other arguments to be positional only. Does anyone have a use case that requires both positional-only and positional-or-keyword arguments? STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
On 5/20/07, Steven Bethard <steven.bethard@gmail.com> wrote:
I don't have a hard use case, but more of a good API design argument, similar to those used to support the second subproposal of PEP 3102 (e.g. see [1]). Maybe it's just me but I often cannot find a good name for a parameter (at least not when I first write it) or I think of a more appropriate name later. I'd like to be able to go back and change it, knowing it won't break anything as it is always passed positionally. IOW, it's a way to say "these names are not part of the function's signature" and enforce this rather than just document it. For instance, think of a function that initially was designed to work for sequences and was defined as "def f(sequence, **kwds)". Later the implementor realizes that the same function works (or its implementation can easily change so that it works) for arbitrary iterables. It would be nice to go back and change the signature to "def f(iterable, **kwds)", knowing that nobody is calling it as f(sequence=range(10)). George [1] http://mail.python.org/pipermail/python-dev/2006-May/064695.html -- "If I have been able to see further, it was only because I stood on the shoulders of million monkeys."
On 5/20/07, George Sakkis <george.sakkis@gmail.com> wrote:
This is a good example for why you might want positional-only arguments. But I guess I wasn't clear -- I'm looking for an example where you need both a positional-only argument and a positional-or-keyword argument *in the same signature*. The reason I ask is that I'd like to propose that the lone * can only appear at the beginning of the signature. That would cover the dict() and dict.update() use cases as well as your use case above:: def f(*, sequence, **kwds) Are there any use cases that it wouldn't cover? Seems like the cautious programmer from your example above would want all of his positional arguments to be positional-only. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
On 5/20/07, Steven Bethard <steven.bethard@gmail.com> wrote:
Are you proposing the removal of positional-or-keyword arguments altogether or only in the presence of positional-only ? In the former case, perhaps this needs a separate thread on its own as it's too big of a change. In the latter case, whatever reasons one has for using positional-or-keyword arguments, chances are that they will hold also in the presence of positional-only. Honestly, I can't think of a compelling reason for positional-or-keyword arguments (with or without the presence of positional-only in the same signature) other than that by definition offer two ways to the caller to spell the same thing. Even that could be considered a misfeature since it goes against TOOWTDI. In any case, that's more of a language design and philosophy argument ("should the caller of a function have the freedom on how to pass the arguments or that should be decided only by the function writer ?") rather than a hard use case. George -- "If I have been able to see further, it was only because I stood on the shoulders of million monkeys."
On 5/20/07, George Sakkis <george.sakkis@gmail.com> wrote:
Are you proposing the removal of positional-or-keyword arguments
No. My current proposal is: - Everything after a * is a positional-only argument, up to the first ** - Everything after a ** is a keyword-only argument - A lone * may only appear at the beginning of the signature In essence, this is changing the lone * of `PEP 3102`_ into a lone **, and allowing a lone * at the beginning of the signature to indicate that all positional arguments are positional-only. .. _PEP 3102: http://www.python.org/dev/peps/pep-3102/ That means that in the following signature:: def f(*, a, b=None, *args, **, c=42, **kwargs) - 'a' and 'b' are positional-only - 'args' is positional-only (as usual) - 'c' is keyword-only - 'kwargs' is keyword-only (as usual) If you want positional-or-keyword arguments, simply omit the initial *:: def f(a, b=None, *args, **, c=42, **kwargs) - 'a' and 'b' are positional-or-keyword (as usual) - 'args' is positional-only (as usual) - 'c' is keyword-only - 'kwargs' is keyword-only (as usual) If you want keyword-only arguments with no *args, simply omit it:: def f(a, b=None, **, c=42, **kwargs) - 'a' and 'b' are positional-or-keyword (as usual) - 'c' is keyword-only - 'kwargs' is keyword-only (as usual) Does anyone have use cases that this doesn't cover? STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
On 5/20/07, Steven Bethard <steven.bethard@gmail.com> wrote:
** is redundant here. You get the same behaviour without it: def f(a, b=None, *args, c=42, **kwargs) See below though.
I initially had the same reservations you do about my proposal, but now I think there's a bigger problem: it undermines the obviousness of PEP 3102's behaviour. Consider this example: def compare(a, b, *args, key=None): Although not legal today, the behaviour is obvious. All positional arguments get sucked up in *args so key *must* be keyword-only. That leads to this variant: def compare(a, b, *, key=None): You continue sucking up extra positional args, but since it's nameless there must be none of them. Again, obvious. Changing the token required from * to ** or giving * extra behaviour when nameless just isn't obvious. Most of the proposed syntax changes also have a question of whether or not to include *args: "def f(a, *, *args)" or "def(a, *args, *)"? "def f((a), *args)" or "def f((a, *args))"? "def f(__a, *args)" or "def(__a, *__args)"? Finally (I hope) is the problem of motivation. Where as PEP 3102 is motivated by "explicit is better than implicit", positional-only arguments actually *want* implicit! It seems to be more for completeness and "C functions already support it". -1 on any syntax changes. A decorator may be okay, but I'm unsure the use cases are significant enough to warrant adding it. Go figure. -- Adam Olsen, aka Rhamphoryncus
On 5/21/07, Adam Olsen <rhamph@gmail.com> wrote:
I note that below you argue for explicit is better than implicit. Requiring the ** even when it could be inferred from the *args is a good example of EIBTI. I don't have strong feelings on this issue though.
I have to disagree. The first thing I think when I see a lone * is multiplication, not "keyword-only arguments start after me". I would say it's just as obvious to see:: def g(*, a, b) and say, "oh, it's after a * so it must be positional-only just like *args is". That is, I don't think the PEP 3102 meaning for lone * is any more intuitive than the meaning I'm proposing here.
Where as PEP 3102 is motivated by "explicit is better than implicit", positional-only arguments actually *want* implicit!
I don't follow this logic. How is a positional-only argument more implicit? You still have to supply it on every function call. It's not like you can omit it. Or are you saying that using positional arguments instead of keyword arguments is implicit?
A decorator may be okay, but I'm unsure the use cases are significant enough to warrant adding it.
Just for reference, here are the current motivating use cases: * User mapping types that need to match the dict() and dict.update() method signatures * Code where positional argument names might change, e.g. George's def f(sequence) which later becomes def f(iterable) Particularly since user mappings are relatively common, I do think this issue deserves at least a decorator. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
Steven Bethard wrote:
I don't think the PEP 3102 meaning for lone * is any more intuitive than the meaning I'm proposing here.
Although I wouldn't go so far as to call the PEP 3102 behaviour "obvious", it is a fairly *logical* extension of the existing semantics. On the other hand, a rule like "arguments after a lone * are positional-only" is something new and arbitrary. It doesn't follow logically from anything existing. Also, your proposal assigns two quite different meanings to *. If it's followed by a name, it sucks up all remaining positional args, but if it's not, it can't suck up anything, since it's followed by more positional args! -- Greg
Steven Bethard wrote:
I agree with Greg, it seems arbitrary. The number of meanings for the '*' is also growing too large I think. * multiply * pack in function defs # but at call time. * pack in left hand expressions # left of '=' * unpack in function calls * positional only indicator in function defs ** power ** pack dictionary in functions defs ** unpack dictionary in function calls ** keyword only indicator in function defs I'm sure this will get confusing to new users. Yes, each one is clear by context, but there's a lot of subtle behaviors by context going on here. What I would like is for unpack to be '^' instead of '*', then you would have... Packing operations: * multiply * pack in function defs * pack in left hand expressions ** power ** pack dictionary in functions defs Unpacking operations: ^ unpack in function calls ^ unpack in right hand expressions (possible) ^ positional only in function defs (See why I think so below) ^^ unpack dictionary in function calls ^^ keyword only in function defs (also see below) I think these become much easier to parse visually and mentally, so it's more of a human interface issue than a computer parsing issue. When using the "name=value" form in function definitions, it can be a positional or keyword argument. So all you need is a way to say this is either one or the other. (Using * here to show similarity, introducing the '^' further down.) *(b=2, c=3) positional only args *args rest of args **(b=2, c=3) keywords only kwds **kwds rest of keywords def f(a, *(b=2, c=3), *args) # b and c are positional only def f(a, **(b=2, c=3), **kwds) # b and c are keyword only Now the problem with this is when it's use with the *() and **() form, it resembles unpacking behavior, but in the *args, and **kwds form, it's packing behavior. Normally you would have only packing in a function def and unpacking in a function call. Now if we use ^ for unpacking, and * for packing, it reduces the number of meanings for the '*' and fixes the context of positional and keyword only arguments. def f(a, ^(b=2, c=3), *rest): b and c are positional only here. def f(a, d=5, ^^(b=2, c=3), **rest_kwds): # b and c are keywords only here. I feel this code better expresses it's intent. Think of the ^() as unpack these as args, and **() as unpack these as keywords. If '^' is used outside of function defs like the '*', it can retain it's meaning. a, b, *c = 1, 2, 3, 4, 5 # pack remainder z = (x, y, z) a, b, c, d = (a, ^z) # unpack z There is the possibility of extending this for use in other ways as well. xlist = [[1, 2], [3, 4, 5], [6, 7, 8, 9]] alist = [^x for x in xlist] This would have list "extend" behavior instead of list "append" behavior. Ok, just some possible ideas of how to organize all this. Cheers, Ron
On 5/21/07, Ron Adam <rrr@ronadam.com> wrote:
Yes, both my proposal and the PEP 3102 use of * are pretty arbitrary, and I agree that we're probably getting too many uses of *. I'm going to try to post a decorator that I think will cover the use cases so far.
What I would like is for unpack to be '^' instead of '*'
-1. This is at least as arbitrary as any of the '*', '**' or ';' suggestions, so I see no reason to prefer it. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
What I would like is for unpack to be '^' instead of '*', then you would have...
My first reaction was negative -- but then I remembered that I was caught by this trap myself more than once, because of the asymmetry between pack and unpack, with * magically doing the normally-right thing. So now I wonder if I wouldn't like if after all, and I'm instead picking at the presentation, so you can make a stronger case. On 5/21/07, Ron Adam <rrr@ronadam.com> wrote:
def f(a, ^(b=2, c=3), *rest): b and c are positional only here.
I don't think there should ever be positional-only arguments *after* a keywordable argument. Perhaps # a is a mandatory positional argument # b is an optional positional argument # c can be passed as a keyword, or as the third positional argument def f(^a, ^b=2, c=3, *rest): Or were you suggesting a tuple of the positional-only arguments, like # a is a mandatory positional argument # b is an optional positional argument # c can be passed as a keyword, or as the third positional argument def f(^(a, b=2), c=3, *rest):
So alist == [1, 2, 3, 4, 5, 6, 7, 8, 9]? -jJ
Jim Jewett wrote:
Yes, it's gotten me a few times as well. Usually when I have a class and some of the methods are accepting *args, **kwds, and some aren't. Then I find I'm packing arguments in some cases and unpacking them in others in the same subclass, and the methods definitions are not on the same page so it isn't immediately obvious I got them mixed up. As I said it isn't a computer issue, it's a human interface issue.
Ah, ok, I see what I missed. I think I got optional args mixed up with positional args in my mind. Not quite the same thing. So yes, the second example fits what I was trying to think of better. # a is mandatory, positional # b is optional, positional # c is optional, positional, or by keyword # d is mandatory, keyword only # e is optional, keyword only def f(^(a, b=2), c=3, ^^(d, e=5)): Of course it could be interpreted a bit differently given the unpacking context. # x is a two tuple (a, b) # y is a single value, c # z is a dict, {'d':4, 'e':5} result = f(x, y, z) Which solves a different problem entirely. ;-) It allows you to specify unpacking behavior in the definition instead of requiring it on the calling side. So I guess I'd have to fall back to the semi-colon as the positional only and keyword only designators. def f(a, b=2; c=3; d, e=5): Or alternately use the colon as it is used in slices. def f(a, b=2: c=3: d, e=5): I'd almost rather change the slices to semicolons and keep the colon unique and with only one purpose. And ... def f(a, b=2, c=3) would be equivalent to... def f(; a, b=2, c=3;): No positional only, no keyword only Anyway, I'm not sure there is a simple way to do this. Sigh...
Exactly. Cheers, Ron
Do you prohibit defaults in type 1, the pos-only? Not necc. Does this look right?
Also, does this look right?
Any more groups? So far I have pos-only, pos-or-kw, and kw-only. If not, a ascii character works fine to delimit, like * and **. Do we want to annotate each parameter individually? This could get out of hand. But decorators won't let you do @f( k ) def j( k ): ... where @f( 'k' ) def j( k ): ... gets unwieldy and could show promise. Cf. Steven's positional only arguments decorator.
Cf. PEP 3102 Talin http://www.python.org/dev/peps/pep-3102/ Cf. PEP 3107 Winder and Lownds http://www.python.org/dev/peps/pep-3107/ Cf. PEP 3117 Brandl http://www.python.org/dev/peps/pep-3117/ def f( @k, @j ) Half-way compromise to Brandl, only in declaration not in use, and always optional.
Don't forget. Any the things you want can be composed in hand.
def f(*, a, b=None, *args, **, c=42, **kwargs)
def f( *args, **kwargs ): if len( args ) < 2: raise error if 'b' in kwargs: raise error if 'c' not in kwargs: raise error a, b, args = args[:2], args[2:] c = kwargs['c'] It's an approximation; use case to the contrary?
Aaron Brady wrote:
I'm not sure why else you would post this reply other than a lack of understanding. The whole point of this discussion is to avoid burying this information inside the function body and have a complete function signature. f(*args, **kwargs) is the least useful function signature possible in python. -- Scott Dial scott@scottdial.com scodial@cs.indiana.edu
You all seemed to me to be wanting things to apply in every case. This does, but you scream at me here too. Generality or speciality, take your pick. I would say perl is highly specialized, very useful, more than python for a certain subset of tasks. What subset? Are you here for popularity, money, productivity, aesthetics, deep aesthetics, raw lines of code per day per time zone around the world, or none of the above? 'Cause ML is a beautiful idea; doesn't take a grad student to appreciate that. Just to keep your job is money. Love it.
On 5/21/07, Steven Bethard <steven.bethard@gmail.com> wrote:
I wasn't aware the proposals given would meet this use case! PEP 3102 has no effect on correct code, instead it just adds some error detection for incorrect code. In contrast, the dict()/dict.update() signature use case needs semantics to change for an example like this: def x((a=None), **kwargs): ... x(a=3) This can't be wrapped properly without very special care. For instance, this would behave wrong: def method(self, *args, **kwargs): x(*args, **kwargs) I'm strongly against adding a feature to split **kwargs into a separate namespace when we have such a common convention of NOT doing it for self[1][2]. The decorator proposed in the other thread is a much better solution unless we want to overturn that convention. [1] Google codesearch for "self[^()]*\*\* lang:python" returned 28,800 results.. make that 200 results.. or 186 results.. numbers aren't too accurate it seems. [2] "\([a-z][^()]*\*\* lang:python" claims 29,500, but won't let me go past 200 results. Maybe there's an internal limiter? -- Adam Olsen, aka Rhamphoryncus
On 5/21/07, Adam Olsen <rhamph@gmail.com> wrote:
This may not be so obvious anymore now that we have extended iterable unpacking from `PEP 3132`_. Given an iterable unpacking like:: a, b, *c, d = iterable I may assume that names after a *args work just like names before one:: def f(a, b, *c, d) So in this case I'd assume that the 'd' is also positional, not keyword-only. Again, I'm not saying people can't learn the appropriate meaning, just that it's not inherently obvious. .. _PEP 3132: http://www.python.org/dev/peps/pep-3132/ STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
On 5/21/07, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
I wasn't either, but I believe the PEP has already been accepted and implemented. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
Steven Bethard wrote:
That means that in the following signature::
def f(*, a, b=None, *args, **, c=42, **kwargs)
I don't like this. Having more than one * or ** in the same signature seems like it would be very confusing. I think the desire for positional-only args would be addressed well enough using a naming convention. Even without further explanation, if I saw something like def f(_a, _b, _c): ... it would be pretty clear to me that the author intended the parameter names to be an implementation detail. This could even be enforced if relying on a convention were not considered strong enough, although it would be more consistent to insist on a double underscore to get automatic enforcement. -- Greg
On 5/21/07, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
So what about the signature for dict() and dict.update() where you want to write:: def update(self, container=None, **kwargs) Note that I shouldn't want to write '__container' because it's not an implementation detail. It's a valid positional argument that my clients can use. But if I write 'container', then I'm limiting what they can send as **kwargs and I can't reproduce the dict behavior:: >>> d = {} >>> d.update(container=1) >>> d {'container': 1} STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
On 5/21/07, Steven Bethard <steven.bethard@gmail.com> wrote:
def update(self, container=None, **kwargs)
Note that I shouldn't want to write '__container' because it's not an implementation detail.
Yes, it is. The fact that there *is* a parameter there isn't an implementation detail, but the *choice of name* for that parameter is explicitly an implementation detail -- if it weren't, then you wouldn't need to insist on positional-only calling. -jJ
participants (15)
-
Aaron Brady -
Adam Olsen -
Benji York -
Fred L. Drake, Jr. -
Georg Brandl -
George Sakkis -
George Sakkis -
Giovanni Bajo -
Greg Ewing -
Guido van Rossum -
Jim Jewett -
Ron Adam -
Scott Dial -
Steven Bethard -
Tal Einat