Eric V. Smith <eric@trueblade.com> added the comment: To your last point, I think it's important to specify the default value placeholder (basically a sentinel) in the documentation. For example, if a function takes -1 to mean "all occurrences", then the caller needs to know how what value to pass in in order to let the function compute the value. This is especially true if it's cheaper for the function to compute the value instead of the caller. I've run into this problem before, where I wanted to pass in some sentinel value and I had to read the source to figure out what it was. I think the function was in the standard library, but now I can't recall what it was. ---------- nosy: +eric.smith _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13386> _______________________________________

Eli Bendersky <eliben@gmail.com> added the comment: You should also explicitly specify what happens in several optional but not keyword args are needed. AFAIU the convention is: func(arg1, arg2[, opt1, opt2]) ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13386> _______________________________________

Ezio Melotti <ezio.melotti@gmail.com> added the comment:

You should also explicitly specify what happens in several optional but not keyword args are needed. AFAIU the convention is: func(arg1, arg2[, opt1, opt2])

IIUC that would mean that either you pass only arg1 and arg2, or you also pass both opt1 and opt2. I think the correct notation for that is e.g.: str.startswith(prefix[, start[, end]]) I also saw "func(foo[, bar][, baz])" for cases where either bar or baz can be passed, but since this requires keyword arguments, the "func(foo, bar=x, baz=y)" notation should be used instead, and the documentation should then explain that either one can be passed. I also agree with what you said in your last message. What can't be expressed with a notation can always be described with words. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13386> _______________________________________

Changes by Petri Lehtinen <petri@digip.org>: ---------- nosy: +petri.lehtinen _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13386> _______________________________________

Baptiste Carvello <devel@baptiste-carvello.net> added the comment: Hi all, here is a relevant user story. I'm afraid it won't help you much, but it highlights the importance of consistent conventions in doc. My girlfriend is learning Python with no prior programing experience. She quite naturally got used to calling help(function), and noted the following: 1) she naturally understood the meaning of the [opt] notation 2) she did not understand the opt=default notation, as she didn't have a sufficient experience with Python to recognize the syntax 3) even after learning what it meant, she still found that notation obscure and unappealing 4) she got annoyed that two completely different notations where used for two very close concepts 5) she got annoyed that there was no user-discoverable and user-understandable document introducing those notations (if there is one, my mistake :-) I have no ovious solutions to the annoyances. Regarding 4), maybe the [opt=default] notation has something good after all: that it reminds of the [opt] one. And regarding 5), if there is a canonical document about documentation conventions, I could try to summarize it in a language aimed at beginners. ---------- nosy: +baptiste.carvello _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13386> _______________________________________

Eric Snow <ericsnowcurrently@gmail.com> added the comment:

...

4) she got annoyed that two completely different notations where used for two very close concepts

This is a good point, and we are trying to move to the arg=default notation. Unfortunately there are still places that use the old notation. C functions that have optional arguments but don't accept keyword arguments are a bit unusual, and IIUC in most of the cases that's an implementation detail that could be removed.

So would it be worth the effort to identify each such place in the built-ins/stdlib and eventually change them all? I've seen support for doing so in other tracker issues and think it's a good idea personally. ---------- nosy: +eric.snow _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13386> _______________________________________

Eli Bendersky <eliben@gmail.com> added the comment: ""So would it be worth the effort to identify each such place in the built-ins/stdlib and eventually change them all? I've seen support for doing so in other tracker issues and think it's a good idea personally."" Probably, if this will bring some added value in addition to being easier to document. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13386> _______________________________________

Ezio Melotti <ezio.melotti@gmail.com> added the comment:

Me too. (Can you give the #ids of these other issues?)

See for example #13012.

I think we should fix C functions to accept kwargs for the sake of Python programmers, not merely to ease documentation (that would just be a nice side-effect :)

And also for compatibility for other implementations like PyPy. I'm still not sure that is a good idea to do a mass conversion of all the functions though.

Sphinx lets us give multiple signatures. I’ve just checked that this markup is valid and does not create duplicate index entries

This is something I was considering, but I'm afraid it might get too verbose (and introduce yet another convention). Sometimes this feature is also (mis?)used to group similar functions. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13386> _______________________________________

Eric Snow <ericsnowcurrently@gmail.com> added the comment:

Me too. (Can you give the #ids of these other issues?)

#13012 is the one that I was thinking of (msg144328 specifically). However, I'm sure there was one more recently (which I can't find now). ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13386> _______________________________________

Eric V. Smith <eric@trueblade.com> added the comment: I just ran across the other reason that having the actual default values documented is important. Sometimes I want to do this: some_func(param if some_condition else <use the default value>) If some_condition is False, I want the default behavior, if not, I want to pass in a parameter. If I don't know the real default value, I have to write: if some_condition: some_func(param) else: some_func() ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13386> _______________________________________

Terry J. Reedy <tjreedy@udel.edu> added the comment:

From Ezio's original post: ''' If a function has optional arguments but it doesn't accept keyword arguments, the "func([arg1])" notation is used instead. ... The notation "func([arg=default])" should never be used, and "func([arg])" should be used only when keyword args are not accepted. '''

In the following, I give objections to this PO (position only) rule and suggest an alternative ND (no default) rule: use 'name=default' when there is a default and '[name]' when there is not'. The issue of whether an argument is required or optional is orthogonal to whether it can be passed by both position and name, only by name, or only by position. All combinations are possible. Optional arguments may or may not have a definition-time (or even run-time) default value, regardless of how passed. (In Python, use of *args and **kwds allows args to be optional without default.) In the CPython stdlib, I think position-only arguments only occur with some, but only some, C functions. One can emulate such C functions in Python by doing the equivalent of what is going on with such C functions. Use a collective *varargs in the definition while naming the required and optional components of varargs in the doc as if they were the actual parameters. But I think we agree that emulating a limitation of C in Python is a bad idea. So by using [] to mean both 'argument is optional' and 'function only take parameters by position' (or at least 'this parameter can only be passed to this function by position'), we are simultaneously documenting an intended and permanent feature of the Python function and a possibly temporary and unwanted side-effect of the current CPython implementation of that function. I think a separate PO indication might be better. 1: The PO rule goes against the effort to separate the Python language from the CPython implementation. With it, the doc for a function does not apply to other implementations that do not have the PO limitation for that function. 2: The PO rule is incomplete. It only marks an arg as position-only if it is optional, but not if it is required. And even if marking one arg as PO means that other args of the function might be, so 'watch out', there is still no special marking for a function with only required PO args. A separate sentence like "For CPython, all args must be passed by position." would solve both of the above problems. 3: The PO rule does not account for the possibility that an argument can be passed by keyword, perhaps only by keyword, but have no default. This is possibly in Python with **kwds in the def and recognized optional names in the doc. With 'name=default' and '[name]' not allowed, how should such an argument be documented is a signature? 4: The PO rule omits useful information on defaults from the place of prominence - the signature header for the entry. Sometimes the information, needed by some users and all implementers, gets omitted altogether. For example, the doc string and manual entry give the signature for str.startwith as str.startswith(prefix[, start[, end]]) The unmentioned defaults are None, None. In summary, the PO rule primarily indicates, but only for optional args, whether the arg can be passed by keyword or not. It secondarily indicates, but only if it can be passed by keyword, what its default is. But if fails if the arg can be passed by keyword but does not have a default. It also fails, in its primary role, for required args. To me, this is all mixed up. Method of passing is not related to optionality. What is special about optional args, regardless of how passed, is the default value, if it has one. The ND rule is to give exactly this information. With an implementation-independent signature and a separate note on passing method, when needed, it solves all the problems listed above. For .startswith, I would like to see something like str.startswith(prefix, start=None, end=None) ... CPython: pass args by position only. --- #13355 illustrates Eric's point with a twist. "random.triangular(low, high, mode) Return a random floating point number N such that low <= N <= high and with the specified mode between those bounds. The low and high bounds default to zero and one. The mode argument defaults to the midpoint between the bounds, giving a symmetric distribution." The *actual* default for mode is None. The function *usually* acts if the default were as described. Twist 1 is that it does not actually calculate the midpoint, as it is not actually needed. Twist 2 is that there is currently a bug (easily fixed) such that triangular does not work if low>=high and mode is not specified, whereas it does work if the true default None is passed ;-). So one needs to know the real default to avoid the bug. Of course, as I said on the issue, all defaults should be given in the signature (by either PO or ND rule): "random.triangular(low=0.0, high=1.0, mode=None) ..." And yes, +1 to documenting visible document conventions both in the documenting howto *and* in the docs themselves. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13386> _______________________________________

Ezio Melotti <ezio.melotti@gmail.com> added the comment:

From Ezio's original post: ''' If a function has optional arguments but it doesn't accept keyword arguments, the "func([arg1])" notation is used instead. ... The notation "func([arg=default])" should never be used, and "func([arg])" should be used only when keyword args are not accepted. '''

In the following, I give objections to this PO (position only) rule and > suggest an alternative ND (no default) rule: use 'name=default' when there is a default and '[name]' when there is not'.

Maybe we should try to keep it simple and just document the signature of the function. Everything that can not be described in the signature can be explained by words. I tried to write down all the combinations of optional/non optional, with/without default, works/doesn't work with keywords to see how to represent them, but it started being a bit messy. The "problematic" combinations (for example a function that accepts an optional arguments with no default but that doesn't work with keywords) seem quite rare, and for them we could just write down what's special about them. There are two more cases that could be solved with a specific notation though: 1) optional arg, with default, doesn't work with keywords (e.g. range, startswith): func(arg1) func(arg1, arg2) *arg2* defaults to <default>. 2) optional arg, with no default, that works only with keywords: func(arg1, *, arg2) The keyword-only *, and the multiple signatures "tricks" can also be used for other similar cases. func(arg1, **kwargs) can be used for functions that accept kwargs without expecting any specific value; if the values are known and have defaults they could be included in the signature (even if the default is like foo = kwargs.get('foo', default)). This should cover most of the cases, it only uses valid Python syntax and avoids potentially confusing []. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13386> _______________________________________

Changes by Ezio Melotti <ezio.melotti@gmail.com>: ---------- nosy: +chris.jerdonek type: -> enhancement _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13386> _______________________________________

Raymond Hettinger added the comment: Please don't add a new notation that makes the docs less readable than they are now. For the most part, the existing docs have done a great job communicating how to use our functions. Please don't undo 20 years of tradition because it bugs you. ---------- nosy: +rhettinger _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13386> _______________________________________