Optional parameters without default value
Function implemented in Python can have optional parameters with default value. It also can accept arbitrary number of positional and keyword arguments if use var-positional or var-keyword parameters (*args and **kwargs). But there is no way to declare an optional parameter that don't have default value. Currently you need to use the sentinel idiom for implementing this: _sentinel = object() def get(store, key, default=_sentinel): if store.exists(key): return store.retrieve(key) if default is _sentinel: raise LookupError else: return default There are drawback of this: * Module's namespace is polluted with sentinel's variables. * You need to check for the sentinel before passing it to other function by accident. * Possible name conflicts between sentinels for different functions of the same module. * Since the sentinel is accessible outside of the function, it possible to pass it to the function. * help() of the function shows reprs of default values. "foo(bar=<object object at 0xb713c698>)" looks ugly. I propose to add a new syntax for optional parameters. If the argument corresponding to the optional parameter without default value is not specified, the parameter takes no value. As well as the "*" prefix means "arbitrary number of positional parameters", the prefix "?" can mean "single optional parameter". Example: def get(store, key, ?default): if store.exists(key): return store.retrieve(key) try: return default except NameError: raise LookupError Alternative syntaxes: * "=" not followed by an expression: "def get(store, key, default=)". * The "del" keyword: "def get(store, key, del default)". This feature is orthogonal to supporting positional-only parameters. Optional parameters without default value can be positional-or-keyword, keyword-only or positional-only (if the latter is implemented).
On 02.03.2017 09:03, Serhiy Storchaka wrote:
Why a new syntax ? Can't we just have a pre-defined sentinel singleton NoDefault and use that throughout the code (and also special case it in argument parsing/handling)? def get(store, key, default=NoDefault): if store.exists(key): return store.retrieve(key) ... I added a special singleton NotGiven to our mxTools long ago for this purpose. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Mar 02 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
On 2 March 2017 at 09:36, M.-A. Lemburg <mal@egenix.com> wrote:
Why a new syntax ? Can't we just have a pre-defined sentinel
singleton NoDefault and use that throughout the code (and also special case it in argument parsing/handling)?
I think for the sane reason that we didn't add Undefined to PEP 484 and PEP 526: Having "another kind of None" will cause code everywhere to expect it. (Plus Guido didn't like it) -- Ivan
On 02.03.2017 09:45, Ivan Levkivskyi wrote:
But that's exactly the point :-) Code should be made aware of such a special value and act accordingly. I had introduced NotGiven in our code to be able to differentiate between having a parameter provided to a method/function or not, and I needed a new singleton, because None was in fact a permitted value for the parameters, but I still had to detect whether this parameter was passed in or not. Example:
Help on function f in module __main__: f(x=NotGiven) Because it's a singleton, you can use "is" for test which is very fast. BTW: NotGiven was named after NotImplemented, another singleton we have in Python. I had introduced this a long time ago to implement better coercion logic: http://web.archive.org/web/20011222024710/http://www.lemburg.com/files/pytho... -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Mar 02 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
On 02.03.2017 10:06, Serhiy Storchaka wrote:
This is not new syntax, nor is it a keyword. It's only a new singleton and it is well usable outside of function declarations as well, e.g. for class attributes which are not yet initialized (and which can accept None as value). The only special casing would be in function call parameter parsing to signal errors when the parameter is used as keyword parameter. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Mar 02 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
In cases like this I would recommend creating the sentinel yourself: NoDefault = object() def get(store, key, default=NoDefault): if default is NoDefault: # do something You can arrange to not export NoDefault so that the client code cannot even access the sentinel value. This is strictly preferable over having yet another global value meaning "no value", since that just moves the goal posts: clients will complain they cannot pass in a default=NoDefault and get back NoDefault. Stephan 2017-03-02 11:04 GMT+01:00 M.-A. Lemburg <mal@egenix.com>:
On 02.03.2017 11:22, Stephan Houben wrote:
Yes, I know... I've been using the mxTools NotGiven since 1998.
Not really. NoDefault would mean: no value provided, not that you don't want a value. As a result, passing NoDefault would not be allowed, since then you'd be providing a value :-)
-- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Mar 02 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
I am not sure if I fully understand the proposal then. NoDefault would be special syntax so that this would be disallowed: f(NoDefault) but this would be allowed: def f(x=NoDefault): ... and also this: x is NoDefault So this would seem to require an exhaustive list of syntactic contexts in which NoDefault is allowed. I mean, can I do: x = NoDefault ? I observe that I can always get to the underlying NoDefault object in this way: (lambda x=NoDefault:x)() So what happens if I do: f((lambda x=NoDefault:x)()) ? Stephan 2017-03-02 12:15 GMT+01:00 M.-A. Lemburg <mal@egenix.com>:
On 02.03.2017 12:31, Stephan Houben wrote:
Sorry for the confusion. NoDefault would be usable just like any other singleton. There would only be one case where it would cause an exception, namely when you declare a parameter as having NoDefault as value. This would trigger special logic in the argument parsing code to disallow using that parameter as keyword parameter. Example: def f(x=NoDefault): # x is an optional positional parameter if x is NoDefault: # x was not passed in as parameter ... else: # x was provided as parameter ... These would all work fine: f() f(1) f(None) This would trigger an exception in the argument parsing code: f(x=NoDefault) e.g. TypeError('x is a positional only parameter') This would not trigger an exception: f(NoDefault) since x is not being used as keyword parameter and the function f may want to pass the optional positional parameter down to other functions with optional positional paramters as well. Is this clearer now ? Note: The name of the singleton could be something else as well, e.g. NoKeywordParameter :-)
-- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Mar 02 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
OK, I get it, I think. I presume it is really the object identity which matters, not the syntax, so: y = NoDefault f(x=y) would be equally an error. Would this also apply if we provide or capture the keyword arguments using ** ? I.e. f(**{"x": NoDict}) (lambda **kw: kw)(x=NoDict) In that case I see a problem with this idiom: newdict = dict(**olddict) This would now start throwing errors in case any of the values of olddict was NoDefault. Stephan 2017-03-02 13:08 GMT+01:00 M.-A. Lemburg <mal@egenix.com>:
On 02.03.2017 13:22, Stephan Houben wrote:
Yes.
I think you meant NoDefault here.
Continuing the example, this case would throw an error as well: kwargs = {'x': NoDefault) f(**kwargs) e.g. TypeError('x is a positional only parameter') However, only because f "declared" x as optional positional parameter. If you'd pass the same dict to a function g as in: def g(x): pass g(**kwargs) it would not raise an exception, since Python functions always allow passing in keyword parameters for positional parameters (unlike C functions, which only allow this if configured that way).
-- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Mar 02 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
On Thu, Mar 2, 2017 at 11:22 PM, Stephan Houben <stephanh42@gmail.com> wrote:
You shouldn't be returning NoDefault anywhere, though, so the only problem is that the error is being reported in the wrong place. If this were to become syntax, enhanced linters could track down exactly where NoDefault came from, and report the error, because that's really where the bug is. ChrisA
Hi Chris, I do not think such a magic linter can be written. It seems an obvious instance of the Halting Problem to me. If it is possible, then the problem is moot anyway since we can just write it and teach it to treat some arbitrary sentinel NoDefault = object() as the magic value and there will be no need to patch the Python core. Regarding your remark that "you shouldn't return NoDefault anyway": well, *I* will obviously not do so ;-) , but some substantial fraction of programmers will use it as a handy extra sentinel when they can get away with it. And then libraries which deal with processing arbitrary parameters, return values or member values are going to have to deal with the issue if they are going to become "NoDefault-clean" or if they are going to document their NoDefault-uncleanliness. For examples from the standard library, would these work: multiprocessing.Process(target=foo, args=(NoDefault,)) asyncio.AbstractEventLoop.call_later(delay, callback, NoDefault) functools.partial(f, NoDefault) shelve.open("spam")["x"] = NoDefault They should, shouldn't they? I never passed in NoDefault by keyword argument. This generator: def bad_idea(): yield NoDefault can we guarantee it works with all stuff in itertools? And that is just the standard library. Now for all the stuff on Pypy.. Stephan 2017-03-02 13:35 GMT+01:00 Chris Angelico <rosuav@gmail.com>:
On Fri, Mar 3, 2017 at 1:15 AM, Stephan Houben <stephanh42@gmail.com> wrote:
I do not think such a magic linter can be written. It seems an obvious instance of the Halting Problem to me.
Yeah it can :) Static analysis is pretty impressive these days. Check out tools like Coverity, which can analyse your source code and tell you that, at this point in the code, it's possible for x to be >100 and y to have only 100 bytes of buffer, and then you index past a buffer. You could do the same to track down the origin of an object in Python. However, I think this is far from an ideal solution to the problem. ChrisA
So let's turn the question around: Since Coverity is user-extensible (and supports Python), can you write a Coverity rule which detects wrong use of some given NoDefault sentinel with a useful level of reliability? Actually I feel this should be feasible. (And if so, mission accomplished?) Stephan 2017-03-02 15:18 GMT+01:00 Chris Angelico <rosuav@gmail.com>:
On Thu, Mar 02, 2017 at 01:08:42PM +0100, M.-A. Lemburg wrote:
Sorry for the confusion. NoDefault would be usable just like any other singleton.
But that is exactly the trouble! We already have a singleton to indicate "no default" -- that is spelled None. Occasionally, we need to allow None as a legitimate value, not just as a sentinel indicating a missing value. So the current practice is to create your own sentinel. That just pushes the problem back one more level: what happens when you have a function where the new NoDefault singleton is a legitimate value? You need a *third* sentinel value. And a fourth, and so on...
Did you miss Serhiy's comment? Optional parameters without default value can be positional-or-keyword, keyword-only or positional-only (if the latter is implemented). It doesn't matter whether the parameter is positional or keyword, or how you call the function: f(NoDefault) f(x=NoDefault) the effect should be the same. But I think that's the wrong solution. If it were a good solution, we should just say that None is the "No Default" value and prohibit passing None as an argument. But of course we can't do that, even if we were willing to break backwards compatibility. There are good reasons for passing None as a legitimate value, and there will be good reasons for passing NoDefault as a legitimate value too. The problem with having a special value that means "no value" is that it actually is a value. Serhiy has a good idea here: cut the gordian knot by *avoiding having a value at all*. The parameter name remains unbound. If you don't pass a value for the optional parameter, and then try to access the parameter, you get a NameError exception! That's really clever and I like it a lot. -- Steve
On 2 March 2017 at 11:31, Stephan Houben <stephanh42@gmail.com> wrote:
NoDefault would be special syntax so that this would be disallowed:
f(NoDefault)
I think the key point of confusion here is whether the language needs to enforce this or it's just convention. MAL is saying that f(NoDefault) is disallowed - but not that the language will somehow notice what you've done and refuse to let you, just that you mustn't do it or your program will be wrong. Stephan seems to be saying that you'd get a SyntaxError (or a RuntimeError? I'm not sure when you'd expect this to be detected - consider f(*[NoDefault])) if you wrote that. Philosophically, Python has always tended in the past towards a "consenting adults" rule - so we don't reject code like this but expect people to use the constructs given in the way they were intended. The OP's proposal is about making it more convenient to specify that parameters are "positional only", by avoiding the need to create custom sentinels (or agree on a common conventional value). That seems to me to be a reasonable proposal - typical sentinel handling code is fairly verbose. OTOH, creating a language mandated sentinel does nothing to improve readability (all we gain is omission of the creation of a custom sentinel) and has the downside that we add yet another special value to the language, that provides a subtly different meaning of "not present" than the ones we have. So I guess I'm +0.5 on the proposed "positional only parameters" syntax, and -1 on any form of new language-defined sentinel value. Paul
On 2 March 2017 at 13:11, Serhiy Storchaka <storchaka@gmail.com> wrote:
Bah, sorry. I'm getting muddled between two different threads. I'm not having a good day, it seems :-( On the proposed feature, I don't like any of the proposed syntaxes (I'd rate "default=" with no value as the least bad, but I don't like it much; "default?" as opposed to "?default" is a possible option). I'm not convinced that the version using the new syntax is any easier to read or understand - the sentinel pattern is pretty well-understood by now, and a built-in replacement would need to result in noticeably simpler code (which this proposal doesn't seem to). Agreed that the help() output is ugly. It would of course be possible to give the sentinel a nicer repr, if you wanted:
get(store, key, default=<Raise an exception>) Whether it's worth doing this depends on the application, of course (just like it's possible to hide the name of the sentinel if it matters sufficiently). Paul
On 02.03.2017 14:08, Serhiy Storchaka wrote:
Yes, sure. My proposal was just to address the problems of changing Python syntax and making it possible to define positional only arguments in Python functions/methods in a backwards compatible way. The same could be had by adding a C function proxy to Python which then takes care of the error handling, since we already have the logic for C functions via PyArg_ParseTuple*(). A decorator could then apply the proxy as needed or ignore this for older Python versions without breaking compatibility (or a PyPI extension could provide the same proxy logic for older versions). FWIW: I don't think the use case for positional only arguments to Python functions is strong enough to warrant breaking backwards compatibility by introducing new syntax. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Mar 02 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
Is it just me that find that having the un-assigned parameter raise NameError (or other exception) much more cumbersome than havign a sentinel-value? I definitely don't find it clever - for one, a common default parameter - sentinel or not, can be replaced in a single line of code by an expression using "if" or "or", while the Exception raising variant require a whole try-except block. So, while I like the idea of simplifying the "sentinel idiom", I don't find any suggestion here useful so far. Maybe something to the stlib that would allow something along: from paramlib import NoDefault, passed def myfunc(a, b, count=NoDefault): if not passed(count): ... else: ... That would simplify a bit the sentinel pattern, do not pollute the namespace, and don't need any new syntax, (and still allow "if" expressions without a full try/except block) On 2 March 2017 at 12:04, M.-A. Lemburg <mal@egenix.com> wrote:
On Thu, 2 Mar 2017 at 08:58 Ethan Furman <ethan@stoneleaf.us> wrote:
I don't like the NameError solution either. What I would like to know is how common is this problem? That will help frame whether this warrants syntax or just providing a sentinel in some module in the stdlib that people can use (e.g. functools.NotGiven; I do prefer MAL's naming of the sentinel). Sticking it into a module would help minimize people from using it in places where None is entirely acceptable and not confusing the whole community when people suddenly start peppering their code with NotGiven instead of None for default values. And if this is really common enough to warrant syntax, then I would want: def foo(a, b, opt?): ... to represent that 'opt' is optional and if not provided by the user then it is given the value of NotGiven (or None if we are just after a syntactic shortcut to say "this argument is optional"). So to me, there's actually two things being discussed. Do we need another sentinel to handle the "None is valid" case, and do we want syntax to more clearly delineate optional arguments?
Agreed, I've rarely found a need for a "second None" or sentinel either, but once every few years I do. So, this use case doesn't seem to be common enough to devote special syntax or a keyword to from my perspective. But, I'll let you know my secret. I don't make my own sentinel, but rather use another singleton that is built-in already. And if you squint just right, it even makes sense. It is a built-in singleton so rarely known that you will almost never encounter code with it, so you'll have it all to yourself. Even on a python mailing list, in a thread about sentinels/singletons, it will not be mentioned. Some may "consider it unnatural." It is… … … (hint) … (wait for it) …
Ellipsis Ellipsis
Don't think I've ever needed a "third None" but if I did I'd probably try an enum instead. -Mike On 2017-03-02 15:02, Barry Warsaw wrote:
On Thu, Mar 2, 2017 at 9:13 PM, Mike Miller <python-ideas@mgmiller.net> wrote:
<3 the suspens ! I got it before the end though ! I'll tell you my secret as well. I have my own (https://pypi.python.org/pypi/undefined) I've set it up to raise on __eq__ or __bool__ to enforce checking for identity. -- M
2017-03-03 6:13 GMT+01:00 Mike Miller <python-ideas@mgmiller.net>:
The question here is how to have an official support of this feature in inspect.signature(). If we go to the special value (singleton) way, Ellispis doesn't work neither since a few modules use Ellipsis for legit use case. Recent user: the typing module for "Callable[[arg, ...], result]". Victor
On 03.03.2017 14:06, Victor Stinner wrote:
Exactly. So, it should be obvious to you, that introducing "official" support leads yet to an endless chain of "but I would need yet another None" because we already use [None, Undefined, NotUsed, Ellipsis, <syntax>, pypi project] in our projects. Having every project rolling their own "NotDefined" makes it completely incompatible to each other. So, it's not possible to plug in return values to other functions and gets bitten. Not providing an "official" solution, solves the matter. Best, Sven
On Thu, Mar 02, 2017 at 10:03:29AM +0200, Serhiy Storchaka wrote:
I like this! If the caller doesn't provide a value, the parameter remains unbound and any attempt to look it up will give a NameError or UnboundLocalError. The only question is, how often do we need a function with optional parameter that don't have a default? I've done it a few times, and used the sentinel trick, but I'm not sure this is common enough to need support from the compiler. It is a really clever solution though.
Alternative syntaxes:
* "=" not followed by an expression: "def get(store, key, default=)".
Too easy to happen by accident if you accidently forget to add the default, or delete it.
* The "del" keyword: "def get(store, key, del default)".
This feature has nothing to do with del. -- Steve
On 2 March 2017 at 14:24, Steven D'Aprano <steve@pearwood.info> wrote:
Hmm. But those exceptions currently indicate with almost 100% certainty, a programming error (usually a mis-spelled name or a control flow error). The proposal makes them a normal runtime behaviour, in certain circumstances. What would happen if you mis-spelled the name of the optional parameter? You'd get a NameError from using the wrong name, rather than from the user not supplying a value. I don't think re-using NameError is a good idea here. Paul
I honestly don't understand the reasoning behind using anything more complex than a built-in sentinel value. Just plop "NotGiven" or whatever in the built-ins and say "it's like None, but for the specific case of optional parameters with no default value". Why prohibit people from passing it to functions? That would just be an explicit way of saying: "I'm not giving you a value for this parameter". Anything more than that is just paranoia that people won't know how to use it in an expected manner. I'm -0.5 on this proposal. It seems like it would add more confusion for dubious benefit. On Thu, Mar 2, 2017 at 2:07 PM, MRAB <python@mrabarnett.plus.com> wrote:
On Mar 02, 2017, at 10:03 AM, Serhiy Storchaka wrote:
The reason for using a special sentinel here is to ensure that there's no way (other than deliberate subterfuge) for code using this API to pass that sentinel in. Normally, None is just fine, but for some cases None is a possible legitimate value, so it won't do as a sentinel. Thus you make one up that you know will mean "wasn't given". A classic example is dict.get(): missing = object() if d.get('key', missing) is missing: its_definitely_not_in_the_dictionary() i.e. because it's possible for d['key'] == None. I don't think this use case is common enough for special syntax, or a keyword. I'm -0 for adding a new built-in because while it might serve your purposes, it's easier to commit subterfuge.
get(store, key, default=NoDefault) # Whoop!
or if d.get('key', NoDefault) is NoDefault: hopefully_its_not_in_the_dictionary() Cheers, -Barry
Since yet another sentinel singleton sounds like a dead end, I suggest to use [arg=value] syntax and require a default value in the prototype, as currently required for *optional keyword* arguments. "[...]" syntax for optional parameter is commonly used in Python documentation (but the exact syntax for multiple optional arguments is different than what I propose, see below). I already saw this syntax for optional parameters in C and PHP languages at least. Python prototype of the standard library and their generated signature: def bool([x=False]) => bool([x]) def bytearray([source=None], [encoding=None], errors="strict") => bytearray([source], [encoding], [errors]) # in practice, default value of 'default' parameter (and maybe also 'key'?) # will more likely be a custom sentinel def sum(iterable, *args, [key=None], [default=None]) => sum(iterable, *args, [key], [default]) # "/" is an hypothetical marker for positional-only arguments def str.replace(old, new, [count=-1], /) => str.replace(old, new, [count], /) def pickle.dump(obj, file, [protocol=3], *, fix_imports=True) => pickle.dump(obj, file, [protocol], *, fix_imports=True) An alternative for generated signature of multiple optional arguments is "bytearray([source[, encoding[, errors]]])", but I'm not a big fan of nested [...], IMHO it's harder to read. And I like the idea of having a signature closer to the actual Python code. Invalid syntaxes raising SyntaxError: * no default value: "def func([x]): ..." * optional arguments before *args: "def func(arg, [opt=None], *args):" In practice, calling a function without an optional argument or pass the (private?) sentinel as the optional argument should have the same behaviour. Each module is free to decide how the sentinel is exposed or not. For example, the inspect module has 2 sentinels: _empty is exposed as Signature.empty and Parameter.empty, whereas _void is private. If I understood correctly, Argument Clinic already supports optional positional arguments, and so doesn't need to be changed. I'm not sure that it's useful for optional keyword-only arguments: def func(*, [arg=None]) => func(*, [arg]) The only difference with optional keyword-only arguments with a default value is the signature: def func(*, arg=None) => func(*, arg=None) See also the discussion on converting the bisect functions to Argument Clinic and issues with the generated signature: http://bugs.python.org/issue28754 Victor
TBH I think that optional parameters isn't a problem requiring new syntax. We probably do need syntax for positional-only arguments (since we already have them in a way), but optional parameters can be solved easily without a new syntax. Syntax like: 1. def a(?foo), 2. def a(foo=pass), 3. def a([foo]), will complicate the language too much IMO. Yury On 2017-03-03 9:29 AM, Victor Stinner wrote:
On 03.03.2017 16:24, Yury Selivanov wrote:
I never really encountered a real-world use where I would have needed this kind of parameter declaration ability. It's like the ++ operator of C which comes in pre- and postfix notation. It's really cool to teach the nuances of it. And to create exam questions using it. And to confuse students. But completely unnecessary in real-life code. Sven
On 03/03/2017 06:29 AM, Victor Stinner wrote:
But that's not the same thing. bytearry([source,] [encoding,] [errors]) says that each argument can be passed without passing any others. bytearray([source [, encoding [,errors]]]) says that in order to pass encoding, source must also be specified. At least, that's what it says to me. -- ~Ethan~
On 3/2/2017 3:03 AM, Serhiy Storchaka wrote:
In other words, Python signature possibilities are already unusually complex.
-1 Being able to do this would violate what I believe is the fundamental precondition for python-coded function bodies: all parameters are bound to an object (so that using a parameter name is never a NameError); all arguments are used exactly once in the binding process; the binding is done without ambiguity (or resort to disambiguation rules). Calls that prevent establishment of this precondition result in an exception. This precondition is normal in computing languages. I believe that all of the ~20 languages I have used over decades have had it. In any case, I believe it is important in understanding Python signatures and calls, and that there would need to be a strong reason to alter this precondition. (Stronger than I judge the one given here to be.)
Currently you need to use the sentinel idiom
which binds a special object to a parameter, thus fulfilling the precondition.
If one cares, one can change the internal reference to 'get._sentinel' and add get._sentinel = _sentinel; del _sentinel after the def (or package this in a decorator.
* You need to check for the sentinel before passing it to other function by accident.
This might be a feature.
* Possible name conflicts between sentinels for different functions of the same module.
Since None can be used as a sentinel for multiple functions, I don't understand the problem you are pointing to.
* Since the sentinel is accessible outside of the function, it possible to pass it to the function.
1. Give it a more private name (___private___?) similar to a reserved name. 2. Hide it better (as a object attribute, for instance).
* help() of the function shows reprs of default values. "foo(bar=<object object at 0xb713c698>)" looks ugly.
Someone suggested a subclass of object with str = repr that prints something like 'bar = <undefined>'. I think functools would be the appropriate place for the class, predefined instance, and possibly a decorator. Make the instance an attribute of the class so it a) would have the same name both in the header and body, and b) would not be an attribute of user module or user functions. __ Terry Jan Reedy
On Mar 10, 2017 02:42, "Terry Reedy" <tjreedy@udel.edu> wrote: On 3/2/2017 3:03 AM, Serhiy Storchaka wrote:
In other words, Python signature possibilities are already unusually complex. But there is no way to declare an optional parameter that
don't have default value.
... [moving the following up]
-1 Being able to do this would violate what I believe is the fundamental precondition for python-coded function bodies: all parameters are bound to an object (so that using a parameter name is never a NameError); all arguments are used exactly once in the binding process; the binding is done without ambiguity (or resort to disambiguation rules). Calls that prevent establishment of this precondition result in an exception. This precondition is normal in computing languages. I believe that all of the ~20 languages I have used over decades have had it. In any case, I believe it is important in understanding Python signatures and calls, and that there would need to be a strong reason to alter this precondition. (Stronger than I judge the one given here to be.) -1 also Having used a language extensively that does not enforce this precondition (MATLAB), I agree that not being able count on arguments existing makes handing function arguments much more difficult.
On 02.03.2017 09:03, Serhiy Storchaka wrote:
Why a new syntax ? Can't we just have a pre-defined sentinel singleton NoDefault and use that throughout the code (and also special case it in argument parsing/handling)? def get(store, key, default=NoDefault): if store.exists(key): return store.retrieve(key) ... I added a special singleton NotGiven to our mxTools long ago for this purpose. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Mar 02 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
On 2 March 2017 at 09:36, M.-A. Lemburg <mal@egenix.com> wrote:
Why a new syntax ? Can't we just have a pre-defined sentinel
singleton NoDefault and use that throughout the code (and also special case it in argument parsing/handling)?
I think for the sane reason that we didn't add Undefined to PEP 484 and PEP 526: Having "another kind of None" will cause code everywhere to expect it. (Plus Guido didn't like it) -- Ivan
On 02.03.2017 09:45, Ivan Levkivskyi wrote:
But that's exactly the point :-) Code should be made aware of such a special value and act accordingly. I had introduced NotGiven in our code to be able to differentiate between having a parameter provided to a method/function or not, and I needed a new singleton, because None was in fact a permitted value for the parameters, but I still had to detect whether this parameter was passed in or not. Example:
Help on function f in module __main__: f(x=NotGiven) Because it's a singleton, you can use "is" for test which is very fast. BTW: NotGiven was named after NotImplemented, another singleton we have in Python. I had introduced this a long time ago to implement better coercion logic: http://web.archive.org/web/20011222024710/http://www.lemburg.com/files/pytho... -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Mar 02 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
On 02.03.2017 10:06, Serhiy Storchaka wrote:
This is not new syntax, nor is it a keyword. It's only a new singleton and it is well usable outside of function declarations as well, e.g. for class attributes which are not yet initialized (and which can accept None as value). The only special casing would be in function call parameter parsing to signal errors when the parameter is used as keyword parameter. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Mar 02 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
In cases like this I would recommend creating the sentinel yourself: NoDefault = object() def get(store, key, default=NoDefault): if default is NoDefault: # do something You can arrange to not export NoDefault so that the client code cannot even access the sentinel value. This is strictly preferable over having yet another global value meaning "no value", since that just moves the goal posts: clients will complain they cannot pass in a default=NoDefault and get back NoDefault. Stephan 2017-03-02 11:04 GMT+01:00 M.-A. Lemburg <mal@egenix.com>:
On 02.03.2017 11:22, Stephan Houben wrote:
Yes, I know... I've been using the mxTools NotGiven since 1998.
Not really. NoDefault would mean: no value provided, not that you don't want a value. As a result, passing NoDefault would not be allowed, since then you'd be providing a value :-)
-- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Mar 02 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
I am not sure if I fully understand the proposal then. NoDefault would be special syntax so that this would be disallowed: f(NoDefault) but this would be allowed: def f(x=NoDefault): ... and also this: x is NoDefault So this would seem to require an exhaustive list of syntactic contexts in which NoDefault is allowed. I mean, can I do: x = NoDefault ? I observe that I can always get to the underlying NoDefault object in this way: (lambda x=NoDefault:x)() So what happens if I do: f((lambda x=NoDefault:x)()) ? Stephan 2017-03-02 12:15 GMT+01:00 M.-A. Lemburg <mal@egenix.com>:
On 02.03.2017 12:31, Stephan Houben wrote:
Sorry for the confusion. NoDefault would be usable just like any other singleton. There would only be one case where it would cause an exception, namely when you declare a parameter as having NoDefault as value. This would trigger special logic in the argument parsing code to disallow using that parameter as keyword parameter. Example: def f(x=NoDefault): # x is an optional positional parameter if x is NoDefault: # x was not passed in as parameter ... else: # x was provided as parameter ... These would all work fine: f() f(1) f(None) This would trigger an exception in the argument parsing code: f(x=NoDefault) e.g. TypeError('x is a positional only parameter') This would not trigger an exception: f(NoDefault) since x is not being used as keyword parameter and the function f may want to pass the optional positional parameter down to other functions with optional positional paramters as well. Is this clearer now ? Note: The name of the singleton could be something else as well, e.g. NoKeywordParameter :-)
-- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Mar 02 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
OK, I get it, I think. I presume it is really the object identity which matters, not the syntax, so: y = NoDefault f(x=y) would be equally an error. Would this also apply if we provide or capture the keyword arguments using ** ? I.e. f(**{"x": NoDict}) (lambda **kw: kw)(x=NoDict) In that case I see a problem with this idiom: newdict = dict(**olddict) This would now start throwing errors in case any of the values of olddict was NoDefault. Stephan 2017-03-02 13:08 GMT+01:00 M.-A. Lemburg <mal@egenix.com>:
On 02.03.2017 13:22, Stephan Houben wrote:
Yes.
I think you meant NoDefault here.
Continuing the example, this case would throw an error as well: kwargs = {'x': NoDefault) f(**kwargs) e.g. TypeError('x is a positional only parameter') However, only because f "declared" x as optional positional parameter. If you'd pass the same dict to a function g as in: def g(x): pass g(**kwargs) it would not raise an exception, since Python functions always allow passing in keyword parameters for positional parameters (unlike C functions, which only allow this if configured that way).
-- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Mar 02 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
On Thu, Mar 2, 2017 at 11:22 PM, Stephan Houben <stephanh42@gmail.com> wrote:
You shouldn't be returning NoDefault anywhere, though, so the only problem is that the error is being reported in the wrong place. If this were to become syntax, enhanced linters could track down exactly where NoDefault came from, and report the error, because that's really where the bug is. ChrisA
Hi Chris, I do not think such a magic linter can be written. It seems an obvious instance of the Halting Problem to me. If it is possible, then the problem is moot anyway since we can just write it and teach it to treat some arbitrary sentinel NoDefault = object() as the magic value and there will be no need to patch the Python core. Regarding your remark that "you shouldn't return NoDefault anyway": well, *I* will obviously not do so ;-) , but some substantial fraction of programmers will use it as a handy extra sentinel when they can get away with it. And then libraries which deal with processing arbitrary parameters, return values or member values are going to have to deal with the issue if they are going to become "NoDefault-clean" or if they are going to document their NoDefault-uncleanliness. For examples from the standard library, would these work: multiprocessing.Process(target=foo, args=(NoDefault,)) asyncio.AbstractEventLoop.call_later(delay, callback, NoDefault) functools.partial(f, NoDefault) shelve.open("spam")["x"] = NoDefault They should, shouldn't they? I never passed in NoDefault by keyword argument. This generator: def bad_idea(): yield NoDefault can we guarantee it works with all stuff in itertools? And that is just the standard library. Now for all the stuff on Pypy.. Stephan 2017-03-02 13:35 GMT+01:00 Chris Angelico <rosuav@gmail.com>:
On Fri, Mar 3, 2017 at 1:15 AM, Stephan Houben <stephanh42@gmail.com> wrote:
I do not think such a magic linter can be written. It seems an obvious instance of the Halting Problem to me.
Yeah it can :) Static analysis is pretty impressive these days. Check out tools like Coverity, which can analyse your source code and tell you that, at this point in the code, it's possible for x to be >100 and y to have only 100 bytes of buffer, and then you index past a buffer. You could do the same to track down the origin of an object in Python. However, I think this is far from an ideal solution to the problem. ChrisA
So let's turn the question around: Since Coverity is user-extensible (and supports Python), can you write a Coverity rule which detects wrong use of some given NoDefault sentinel with a useful level of reliability? Actually I feel this should be feasible. (And if so, mission accomplished?) Stephan 2017-03-02 15:18 GMT+01:00 Chris Angelico <rosuav@gmail.com>:
On Thu, Mar 02, 2017 at 01:08:42PM +0100, M.-A. Lemburg wrote:
Sorry for the confusion. NoDefault would be usable just like any other singleton.
But that is exactly the trouble! We already have a singleton to indicate "no default" -- that is spelled None. Occasionally, we need to allow None as a legitimate value, not just as a sentinel indicating a missing value. So the current practice is to create your own sentinel. That just pushes the problem back one more level: what happens when you have a function where the new NoDefault singleton is a legitimate value? You need a *third* sentinel value. And a fourth, and so on...
Did you miss Serhiy's comment? Optional parameters without default value can be positional-or-keyword, keyword-only or positional-only (if the latter is implemented). It doesn't matter whether the parameter is positional or keyword, or how you call the function: f(NoDefault) f(x=NoDefault) the effect should be the same. But I think that's the wrong solution. If it were a good solution, we should just say that None is the "No Default" value and prohibit passing None as an argument. But of course we can't do that, even if we were willing to break backwards compatibility. There are good reasons for passing None as a legitimate value, and there will be good reasons for passing NoDefault as a legitimate value too. The problem with having a special value that means "no value" is that it actually is a value. Serhiy has a good idea here: cut the gordian knot by *avoiding having a value at all*. The parameter name remains unbound. If you don't pass a value for the optional parameter, and then try to access the parameter, you get a NameError exception! That's really clever and I like it a lot. -- Steve
On 2 March 2017 at 11:31, Stephan Houben <stephanh42@gmail.com> wrote:
NoDefault would be special syntax so that this would be disallowed:
f(NoDefault)
I think the key point of confusion here is whether the language needs to enforce this or it's just convention. MAL is saying that f(NoDefault) is disallowed - but not that the language will somehow notice what you've done and refuse to let you, just that you mustn't do it or your program will be wrong. Stephan seems to be saying that you'd get a SyntaxError (or a RuntimeError? I'm not sure when you'd expect this to be detected - consider f(*[NoDefault])) if you wrote that. Philosophically, Python has always tended in the past towards a "consenting adults" rule - so we don't reject code like this but expect people to use the constructs given in the way they were intended. The OP's proposal is about making it more convenient to specify that parameters are "positional only", by avoiding the need to create custom sentinels (or agree on a common conventional value). That seems to me to be a reasonable proposal - typical sentinel handling code is fairly verbose. OTOH, creating a language mandated sentinel does nothing to improve readability (all we gain is omission of the creation of a custom sentinel) and has the downside that we add yet another special value to the language, that provides a subtly different meaning of "not present" than the ones we have. So I guess I'm +0.5 on the proposed "positional only parameters" syntax, and -1 on any form of new language-defined sentinel value. Paul
On 2 March 2017 at 13:11, Serhiy Storchaka <storchaka@gmail.com> wrote:
Bah, sorry. I'm getting muddled between two different threads. I'm not having a good day, it seems :-( On the proposed feature, I don't like any of the proposed syntaxes (I'd rate "default=" with no value as the least bad, but I don't like it much; "default?" as opposed to "?default" is a possible option). I'm not convinced that the version using the new syntax is any easier to read or understand - the sentinel pattern is pretty well-understood by now, and a built-in replacement would need to result in noticeably simpler code (which this proposal doesn't seem to). Agreed that the help() output is ugly. It would of course be possible to give the sentinel a nicer repr, if you wanted:
get(store, key, default=<Raise an exception>) Whether it's worth doing this depends on the application, of course (just like it's possible to hide the name of the sentinel if it matters sufficiently). Paul
On 02.03.2017 14:08, Serhiy Storchaka wrote:
Yes, sure. My proposal was just to address the problems of changing Python syntax and making it possible to define positional only arguments in Python functions/methods in a backwards compatible way. The same could be had by adding a C function proxy to Python which then takes care of the error handling, since we already have the logic for C functions via PyArg_ParseTuple*(). A decorator could then apply the proxy as needed or ignore this for older Python versions without breaking compatibility (or a PyPI extension could provide the same proxy logic for older versions). FWIW: I don't think the use case for positional only arguments to Python functions is strong enough to warrant breaking backwards compatibility by introducing new syntax. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Mar 02 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
Is it just me that find that having the un-assigned parameter raise NameError (or other exception) much more cumbersome than havign a sentinel-value? I definitely don't find it clever - for one, a common default parameter - sentinel or not, can be replaced in a single line of code by an expression using "if" or "or", while the Exception raising variant require a whole try-except block. So, while I like the idea of simplifying the "sentinel idiom", I don't find any suggestion here useful so far. Maybe something to the stlib that would allow something along: from paramlib import NoDefault, passed def myfunc(a, b, count=NoDefault): if not passed(count): ... else: ... That would simplify a bit the sentinel pattern, do not pollute the namespace, and don't need any new syntax, (and still allow "if" expressions without a full try/except block) On 2 March 2017 at 12:04, M.-A. Lemburg <mal@egenix.com> wrote:
On Thu, 2 Mar 2017 at 08:58 Ethan Furman <ethan@stoneleaf.us> wrote:
I don't like the NameError solution either. What I would like to know is how common is this problem? That will help frame whether this warrants syntax or just providing a sentinel in some module in the stdlib that people can use (e.g. functools.NotGiven; I do prefer MAL's naming of the sentinel). Sticking it into a module would help minimize people from using it in places where None is entirely acceptable and not confusing the whole community when people suddenly start peppering their code with NotGiven instead of None for default values. And if this is really common enough to warrant syntax, then I would want: def foo(a, b, opt?): ... to represent that 'opt' is optional and if not provided by the user then it is given the value of NotGiven (or None if we are just after a syntactic shortcut to say "this argument is optional"). So to me, there's actually two things being discussed. Do we need another sentinel to handle the "None is valid" case, and do we want syntax to more clearly delineate optional arguments?
Agreed, I've rarely found a need for a "second None" or sentinel either, but once every few years I do. So, this use case doesn't seem to be common enough to devote special syntax or a keyword to from my perspective. But, I'll let you know my secret. I don't make my own sentinel, but rather use another singleton that is built-in already. And if you squint just right, it even makes sense. It is a built-in singleton so rarely known that you will almost never encounter code with it, so you'll have it all to yourself. Even on a python mailing list, in a thread about sentinels/singletons, it will not be mentioned. Some may "consider it unnatural." It is… … … (hint) … (wait for it) …
Ellipsis Ellipsis
Don't think I've ever needed a "third None" but if I did I'd probably try an enum instead. -Mike On 2017-03-02 15:02, Barry Warsaw wrote:
On Thu, Mar 2, 2017 at 9:13 PM, Mike Miller <python-ideas@mgmiller.net> wrote:
<3 the suspens ! I got it before the end though ! I'll tell you my secret as well. I have my own (https://pypi.python.org/pypi/undefined) I've set it up to raise on __eq__ or __bool__ to enforce checking for identity. -- M
2017-03-03 6:13 GMT+01:00 Mike Miller <python-ideas@mgmiller.net>:
The question here is how to have an official support of this feature in inspect.signature(). If we go to the special value (singleton) way, Ellispis doesn't work neither since a few modules use Ellipsis for legit use case. Recent user: the typing module for "Callable[[arg, ...], result]". Victor
On 03.03.2017 14:06, Victor Stinner wrote:
Exactly. So, it should be obvious to you, that introducing "official" support leads yet to an endless chain of "but I would need yet another None" because we already use [None, Undefined, NotUsed, Ellipsis, <syntax>, pypi project] in our projects. Having every project rolling their own "NotDefined" makes it completely incompatible to each other. So, it's not possible to plug in return values to other functions and gets bitten. Not providing an "official" solution, solves the matter. Best, Sven
On Thu, Mar 02, 2017 at 10:03:29AM +0200, Serhiy Storchaka wrote:
I like this! If the caller doesn't provide a value, the parameter remains unbound and any attempt to look it up will give a NameError or UnboundLocalError. The only question is, how often do we need a function with optional parameter that don't have a default? I've done it a few times, and used the sentinel trick, but I'm not sure this is common enough to need support from the compiler. It is a really clever solution though.
Alternative syntaxes:
* "=" not followed by an expression: "def get(store, key, default=)".
Too easy to happen by accident if you accidently forget to add the default, or delete it.
* The "del" keyword: "def get(store, key, del default)".
This feature has nothing to do with del. -- Steve
On 2 March 2017 at 14:24, Steven D'Aprano <steve@pearwood.info> wrote:
Hmm. But those exceptions currently indicate with almost 100% certainty, a programming error (usually a mis-spelled name or a control flow error). The proposal makes them a normal runtime behaviour, in certain circumstances. What would happen if you mis-spelled the name of the optional parameter? You'd get a NameError from using the wrong name, rather than from the user not supplying a value. I don't think re-using NameError is a good idea here. Paul
I honestly don't understand the reasoning behind using anything more complex than a built-in sentinel value. Just plop "NotGiven" or whatever in the built-ins and say "it's like None, but for the specific case of optional parameters with no default value". Why prohibit people from passing it to functions? That would just be an explicit way of saying: "I'm not giving you a value for this parameter". Anything more than that is just paranoia that people won't know how to use it in an expected manner. I'm -0.5 on this proposal. It seems like it would add more confusion for dubious benefit. On Thu, Mar 2, 2017 at 2:07 PM, MRAB <python@mrabarnett.plus.com> wrote:
On Mar 02, 2017, at 10:03 AM, Serhiy Storchaka wrote:
The reason for using a special sentinel here is to ensure that there's no way (other than deliberate subterfuge) for code using this API to pass that sentinel in. Normally, None is just fine, but for some cases None is a possible legitimate value, so it won't do as a sentinel. Thus you make one up that you know will mean "wasn't given". A classic example is dict.get(): missing = object() if d.get('key', missing) is missing: its_definitely_not_in_the_dictionary() i.e. because it's possible for d['key'] == None. I don't think this use case is common enough for special syntax, or a keyword. I'm -0 for adding a new built-in because while it might serve your purposes, it's easier to commit subterfuge.
get(store, key, default=NoDefault) # Whoop!
or if d.get('key', NoDefault) is NoDefault: hopefully_its_not_in_the_dictionary() Cheers, -Barry
Since yet another sentinel singleton sounds like a dead end, I suggest to use [arg=value] syntax and require a default value in the prototype, as currently required for *optional keyword* arguments. "[...]" syntax for optional parameter is commonly used in Python documentation (but the exact syntax for multiple optional arguments is different than what I propose, see below). I already saw this syntax for optional parameters in C and PHP languages at least. Python prototype of the standard library and their generated signature: def bool([x=False]) => bool([x]) def bytearray([source=None], [encoding=None], errors="strict") => bytearray([source], [encoding], [errors]) # in practice, default value of 'default' parameter (and maybe also 'key'?) # will more likely be a custom sentinel def sum(iterable, *args, [key=None], [default=None]) => sum(iterable, *args, [key], [default]) # "/" is an hypothetical marker for positional-only arguments def str.replace(old, new, [count=-1], /) => str.replace(old, new, [count], /) def pickle.dump(obj, file, [protocol=3], *, fix_imports=True) => pickle.dump(obj, file, [protocol], *, fix_imports=True) An alternative for generated signature of multiple optional arguments is "bytearray([source[, encoding[, errors]]])", but I'm not a big fan of nested [...], IMHO it's harder to read. And I like the idea of having a signature closer to the actual Python code. Invalid syntaxes raising SyntaxError: * no default value: "def func([x]): ..." * optional arguments before *args: "def func(arg, [opt=None], *args):" In practice, calling a function without an optional argument or pass the (private?) sentinel as the optional argument should have the same behaviour. Each module is free to decide how the sentinel is exposed or not. For example, the inspect module has 2 sentinels: _empty is exposed as Signature.empty and Parameter.empty, whereas _void is private. If I understood correctly, Argument Clinic already supports optional positional arguments, and so doesn't need to be changed. I'm not sure that it's useful for optional keyword-only arguments: def func(*, [arg=None]) => func(*, [arg]) The only difference with optional keyword-only arguments with a default value is the signature: def func(*, arg=None) => func(*, arg=None) See also the discussion on converting the bisect functions to Argument Clinic and issues with the generated signature: http://bugs.python.org/issue28754 Victor
TBH I think that optional parameters isn't a problem requiring new syntax. We probably do need syntax for positional-only arguments (since we already have them in a way), but optional parameters can be solved easily without a new syntax. Syntax like: 1. def a(?foo), 2. def a(foo=pass), 3. def a([foo]), will complicate the language too much IMO. Yury On 2017-03-03 9:29 AM, Victor Stinner wrote:
On 03.03.2017 16:24, Yury Selivanov wrote:
I never really encountered a real-world use where I would have needed this kind of parameter declaration ability. It's like the ++ operator of C which comes in pre- and postfix notation. It's really cool to teach the nuances of it. And to create exam questions using it. And to confuse students. But completely unnecessary in real-life code. Sven
On 03/03/2017 06:29 AM, Victor Stinner wrote:
But that's not the same thing. bytearry([source,] [encoding,] [errors]) says that each argument can be passed without passing any others. bytearray([source [, encoding [,errors]]]) says that in order to pass encoding, source must also be specified. At least, that's what it says to me. -- ~Ethan~
On 3/2/2017 3:03 AM, Serhiy Storchaka wrote:
In other words, Python signature possibilities are already unusually complex.
-1 Being able to do this would violate what I believe is the fundamental precondition for python-coded function bodies: all parameters are bound to an object (so that using a parameter name is never a NameError); all arguments are used exactly once in the binding process; the binding is done without ambiguity (or resort to disambiguation rules). Calls that prevent establishment of this precondition result in an exception. This precondition is normal in computing languages. I believe that all of the ~20 languages I have used over decades have had it. In any case, I believe it is important in understanding Python signatures and calls, and that there would need to be a strong reason to alter this precondition. (Stronger than I judge the one given here to be.)
Currently you need to use the sentinel idiom
which binds a special object to a parameter, thus fulfilling the precondition.
If one cares, one can change the internal reference to 'get._sentinel' and add get._sentinel = _sentinel; del _sentinel after the def (or package this in a decorator.
* You need to check for the sentinel before passing it to other function by accident.
This might be a feature.
* Possible name conflicts between sentinels for different functions of the same module.
Since None can be used as a sentinel for multiple functions, I don't understand the problem you are pointing to.
* Since the sentinel is accessible outside of the function, it possible to pass it to the function.
1. Give it a more private name (___private___?) similar to a reserved name. 2. Hide it better (as a object attribute, for instance).
* help() of the function shows reprs of default values. "foo(bar=<object object at 0xb713c698>)" looks ugly.
Someone suggested a subclass of object with str = repr that prints something like 'bar = <undefined>'. I think functools would be the appropriate place for the class, predefined instance, and possibly a decorator. Make the instance an attribute of the class so it a) would have the same name both in the header and body, and b) would not be an attribute of user module or user functions. __ Terry Jan Reedy
On Mar 10, 2017 02:42, "Terry Reedy" <tjreedy@udel.edu> wrote: On 3/2/2017 3:03 AM, Serhiy Storchaka wrote:
In other words, Python signature possibilities are already unusually complex. But there is no way to declare an optional parameter that
don't have default value.
... [moving the following up]
-1 Being able to do this would violate what I believe is the fundamental precondition for python-coded function bodies: all parameters are bound to an object (so that using a parameter name is never a NameError); all arguments are used exactly once in the binding process; the binding is done without ambiguity (or resort to disambiguation rules). Calls that prevent establishment of this precondition result in an exception. This precondition is normal in computing languages. I believe that all of the ~20 languages I have used over decades have had it. In any case, I believe it is important in understanding Python signatures and calls, and that there would need to be a strong reason to alter this precondition. (Stronger than I judge the one given here to be.) -1 also Having used a language extensively that does not enforce this precondition (MATLAB), I agree that not being able count on arguments existing makes handing function arguments much more difficult.
participants (20)
-
Abe Dillon -
Barry Warsaw -
Brett Cannon -
Chris Angelico -
Ethan Furman -
Ivan Levkivskyi -
Joao S. O. Bueno -
M.-A. Lemburg -
Matthias Bussonnier -
Mike Miller -
MRAB -
Paul Moore -
Serhiy Storchaka -
Stephan Houben -
Steven D'Aprano -
Sven R. Kunze -
Terry Reedy -
Todd -
Victor Stinner -
Yury Selivanov