AC Derby and accepting None for optional positional arguments
One of the downsides of converting positional-only functions to Argument Clinic is that it can result in misleading docstring signatures. Example: socket.getservbyname(servicename[, protocolname]) -> socket.getservbyname(servicename, protocolname=None) The problem with the new signature is that it indicates passing None for protocolname is the same as omitting it (the other, much larger problem is that it falsely indicates keyword compatibility, but that's a separate indoor elephant). My question: Is it OK to change a longstanding function to treat None like an absent parameter, where previously it was an error? (This also entails a docs update and maybe a changelog entry)
On Wed, Jan 15, 2014 at 7:57 PM, Ryan Smith-Roberts <rmsr@lab.net> wrote:
socket.getservbyname(servicename[, protocolname]) -> socket.getservbyname(servicename, protocolname=None)
Here is a more complicated example, since the above does technically have an alternative fix: sockobj.sendmsg(buffers[, ancdata[, flags[, address]]]) -> sockobj.sendmsg(buffers, ancdata=None, flags=0, address=None)
On 01/15/2014 08:35 PM, Ryan Smith-Roberts wrote:
On Wed, Jan 15, 2014 at 7:57 PM, Ryan Smith-Roberts <rmsr@lab.net <mailto:rmsr@lab.net>> wrote:
socket.getservbyname(servicename[, protocolname]) -> socket.getservbyname(servicename, protocolname=None)
Here is a more complicated example, since the above does technically have an alternative fix:
sockobj.sendmsg(buffers[, ancdata[, flags[, address]]]) -> sockobj.sendmsg(buffers, ancdata=None, flags=0, address=None)
I feel like Ryan didn't sufficiently explain the problem. I'll elaborate. For functions implemented in Python, it's always true that: * a parameter that is optional always has a default value, and * this default value is visible to Python and is a Python value. The default value of every parameter is part of the function's signature--you can see them with inspect.signature() or inspect.getfullargspec(). A corollary of this: for every function implemented in Python, you can construct a call to it where you fill in every optional value with its published default value, and this is exactly equivalent to calling it without specifying those arguments: def foo(a=any_python_value): ... sig = inspect.signature(foo) defaults = [p.default for p in sig.parameters.values()] foo(*defaults) == foo() Assuming neither foo nor "any_python_value" have side effects, those two calls to foo() will be exactly the same in every way. But! Builtin functions implemented in C commonly have optional parameters whose default value is not only opaque to Python, it's not renderable as a Python value. That default value is NULL. Consider the first two paragraphs of SHA1_new() in Modules/sha1module.c: static PyObject * SHA1_new(PyObject *self, PyObject *args, PyObject *kwdict) { static char *kwlist[] = {"string", NULL}; SHA1object *new; PyObject *data_obj = NULL; Py_buffer buf; if (!PyArg_ParseTupleAndKeywords(args, kwdict, "|O:new", kwlist, &data_obj)) { return NULL; } ... The "string" parameter is optional. Its value, if specified, is written to "data_obj". "data_obj" has a default value of NULL. There is no Python value you could pass in to this function that would result in "data_obj" being (redundantly) set to NULL. In Python SHA1_new is known as "_sha1.sha1". And: sig = inspect.signature(_sha1.sha1) defaults = [p.default for p in sig.parameters.values()] _sha1.sha1(*defaults) == _sha1.sha1() There's no value we could put in the signature for _sha1.sha1 that would behave exactly the same as that NULL. The ultimate goal of Argument Clinic is to provide introspection information for builtins. But we can't provide a default value to Python for the "string" parameter to _sha1.sha1() without changing the semantics of the function. We're stuck. Ryan's question, then, is "can we change a function like this so it accepts None?" My initial reaction is "no". That might be okay for _sha1.sha1(), but it'd be best to avoid. In the specific case of SHA1_new's "string" parameter, we could lie and claim that the default value is b''. Internally we could still use NULL as a default and get away with it. But this is only a happy coincidence. Many (most?) functions like this won't have a clever Python value we can trick you with. What else could we do? We could add a special value, let's call it sys.NULL, whose specific semantics are "turns into NULL when passed into builtins". This would solve the problem but it's really, really awful. The only other option I can see: don't convert SHA1_new() to use Argument Clinic, and don't provide introspection information for it either. Can you, gentle reader, suggest a better option? //arry/ p.s. Ryan's function signatures above suggest that he's converting code from using PyArg_ParseTuple into using PyArg_ParseTupleAndKeywords. I don't think he's *actually* doing that, and if I saw that in patches submitted to me I would ask that it be fixed.
Well, I think these are mostly artifacts from old times, and usually passing None *should* be the same as omitting the argument. But check each case! On Wednesday, January 15, 2014, Larry Hastings <larry@hastings.org> wrote:
On 01/15/2014 08:35 PM, Ryan Smith-Roberts wrote:
On Wed, Jan 15, 2014 at 7:57 PM, Ryan Smith-Roberts <rmsr@lab.net<javascript:_e({}, 'cvml', 'rmsr@lab.net');>
wrote:
socket.getservbyname(servicename[, protocolname]) -> socket.getservbyname(servicename, protocolname=None)
Here is a more complicated example, since the above does technically have an alternative fix:
sockobj.sendmsg(buffers[, ancdata[, flags[, address]]]) -> sockobj.sendmsg(buffers, ancdata=None, flags=0, address=None)
I feel like Ryan didn't sufficiently explain the problem. I'll elaborate.
For functions implemented in Python, it's always true that:
- a parameter that is optional always has a default value, and - this default value is visible to Python and is a Python value.
The default value of every parameter is part of the function's signature--you can see them with inspect.signature() or inspect.getfullargspec().
A corollary of this: for every function implemented in Python, you can construct a call to it where you fill in every optional value with its published default value, and this is exactly equivalent to calling it without specifying those arguments:
def foo(a=any_python_value): ...
sig = inspect.signature(foo) defaults = [p.default for p in sig.parameters.values()] foo(*defaults) == foo()
Assuming neither foo nor "any_python_value" have side effects, those two calls to foo() will be exactly the same in every way.
But! Builtin functions implemented in C commonly have optional parameters whose default value is not only opaque to Python, it's not renderable as a Python value. That default value is NULL. Consider the first two paragraphs of SHA1_new() in Modules/sha1module.c:
static PyObject * SHA1_new(PyObject *self, PyObject *args, PyObject *kwdict) { static char *kwlist[] = {"string", NULL}; SHA1object *new; PyObject *data_obj = NULL; Py_buffer buf;
if (!PyArg_ParseTupleAndKeywords(args, kwdict, "|O:new", kwlist, &data_obj)) { return NULL; } ...
The "string" parameter is optional. Its value, if specified, is written to "data_obj". "data_obj" has a default value of NULL. There is no Python value you could pass in to this function that would result in "data_obj" being (redundantly) set to NULL. In Python SHA1_new is known as "_sha1.sha1". And:
sig = inspect.signature(_sha1.sha1) defaults = [p.default for p in sig.parameters.values()] _sha1.sha1(*defaults) == _sha1.sha1()
There's no value we could put in the signature for _sha1.sha1 that would behave exactly the same as that NULL.
The ultimate goal of Argument Clinic is to provide introspection information for builtins. But we can't provide a default value to Python for the "string" parameter to _sha1.sha1() without changing the semantics of the function. We're stuck.
Ryan's question, then, is "can we change a function like this so it accepts None?" My initial reaction is "no". That might be okay for _sha1.sha1(), but it'd be best to avoid.
In the specific case of SHA1_new's "string" parameter, we could lie and claim that the default value is b''. Internally we could still use NULL as a default and get away with it. But this is only a happy coincidence. Many (most?) functions like this won't have a clever Python value we can trick you with.
What else could we do? We could add a special value, let's call it sys.NULL, whose specific semantics are "turns into NULL when passed into builtins". This would solve the problem but it's really, really awful.
The only other option I can see: don't convert SHA1_new() to use Argument Clinic, and don't provide introspection information for it either.
Can you, gentle reader, suggest a better option?
*/arry*
p.s. Ryan's function signatures above suggest that he's converting code from using PyArg_ParseTuple into using PyArg_ParseTupleAndKeywords. I don't think he's *actually* doing that, and if I saw that in patches submitted to me I would ask that it be fixed.
-- --Guido van Rossum (on iPad)
On 01/15/2014 09:37 PM, Guido van Rossum wrote:
Well, I think these are mostly artifacts from old times, and usually passing None *should* be the same as omitting the argument. But check each case!
Vajrasky Kok's recent posting to python-dev discusses the same problem. His example is itertools.repeat's second parameter, which is slightly nastier. Consider the implementation: static PyObject * repeat_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { repeatobject *ro; PyObject *element; Py_ssize_t cnt = -1; static char *kwargs[] = {"object", "times", NULL}; if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|n:repeat", kwargs, &element, &cnt)) return NULL; if (PyTuple_Size(args) == 2 && cnt < 0) cnt = 0; I draw your attention to the last two lines. And remember, Argument Clinic doesn't provide the "args" and "kwargs" parameters to the "impl" function. That means two things: * itertools.repeat deliberately makes it impossible to provide an argument for "times" that behaves the same as not providing the "times" argument, and * there is currently no way to implement this behavior using Argument Clinic. (I'd have to add a hack where impl functions also get args and kwargs.) Passing in "None" here is inconvenient as it's an integer argument. -1 actually seems like a pretty sane default to mean "repeat forever", but the author has gone to some effort to prevent this. I therefore assume they had a very good reason. So again we seem stuck. Are you suggesting that, when converting builtins to Argument Clinic with unrepresentable default values, we're permitted to tweak the defaults to something representable? //arry/
On Wed, Jan 15, 2014 at 9:55 PM, Larry Hastings <larry@hastings.org> wrote:
On 01/15/2014 09:37 PM, Guido van Rossum wrote:
Well, I think these are mostly artifacts from old times, and usually passing None *should* be the same as omitting the argument. But check each case!
Vajrasky Kok's recent posting to python-dev discusses the same problem. His example is itertools.repeat's second parameter, which is slightly nastier. Consider the implementation:
static PyObject * repeat_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { repeatobject *ro; PyObject *element; Py_ssize_t cnt = -1; static char *kwargs[] = {"object", "times", NULL};
if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|n:repeat", kwargs, &element, &cnt)) return NULL;
if (PyTuple_Size(args) == 2 && cnt < 0) cnt = 0;
I draw your attention to the last two lines. And remember, Argument Clinic doesn't provide the "args" and "kwargs" parameters to the "impl" function. That means two things:
itertools.repeat deliberately makes it impossible to provide an argument for "times" that behaves the same as not providing the "times" argument, and there is currently no way to implement this behavior using Argument Clinic. (I'd have to add a hack where impl functions also get args and kwargs.)
Passing in "None" here is inconvenient as it's an integer argument. -1 actually seems like a pretty sane default to mean "repeat forever", but the author has gone to some effort to prevent this. I therefore assume they had a very good reason. So again we seem stuck.
Are you suggesting that, when converting builtins to Argument Clinic with unrepresentable default values, we're permitted to tweak the defaults to something representable?
In this specific case it's clear to me that the special-casing of negative count is intentional -- presumably it emulates sequence repetition, where e.g. 'a'*-1 == ''. But not accepting None is laziness -- accepting either a number or None requires much more effort, if you need to have the number as a C integer. I'm not sure how AC could make this any easier, unless you want to special-case maxint or -maxint-1. In the sha1 example, however, accepting None and converting it to NULL (without a reference leak, please :-) seems fine though. -- --Guido van Rossum (python.org/~guido)
16.01.14 08:05, Guido van Rossum написав(ла):
In this specific case it's clear to me that the special-casing of negative count is intentional -- presumably it emulates sequence repetition, where e.g. 'a'*-1 == ''.
In this specific case it's contrary to sequence repetition. Because repeat('a', -1) repeats 'a' forever. This is a point of Vajrasky's issue [1].
But not accepting None is laziness -- accepting either a number or None requires much more effort, if you need to have the number as a C integer. I'm not sure how AC could make this any easier, unless you want to special-case maxint or -maxint-1.
getattr(foo, 'bar', None) is not the same as getattr(foo, 'bar'). So None can't be used as universal default value. [1] http://bugs.python.org/issue19145
On 1/16/2014 3:31 AM, Serhiy Storchaka wrote:
16.01.14 08:05, Guido van Rossum написав(ла):
In this specific case it's clear to me that the special-casing of negative count is intentional -- presumably it emulates sequence repetition, where e.g. 'a'*-1 == ''.
In this specific case it's contrary to sequence repetition. Because repeat('a', -1) repeats 'a' forever.
'Forever' only when the keyword is used and the value is -1. In 3.4b2
itertools.repeat('a', -1) repeat('a', 0) itertools.repeat('a', times=-1) repeat('a') itertools.repeat('a', times=-2) repeat('a', -2)
This is a point of Vajrasky's issue [1].
The first line is correct in both behavior and representation. The second line behavior (and corresponding repr) are wrong. The third line repr is wrong but the behavior is like the first.
-- Terry Jan Reedy
On Thu, 16 Jan 2014 04:42:43 -0500 Terry Reedy <tjreedy@udel.edu> wrote:
On 1/16/2014 3:31 AM, Serhiy Storchaka wrote:
16.01.14 08:05, Guido van Rossum написав(ла):
In this specific case it's clear to me that the special-casing of negative count is intentional -- presumably it emulates sequence repetition, where e.g. 'a'*-1 == ''.
In this specific case it's contrary to sequence repetition. Because repeat('a', -1) repeats 'a' forever.
'Forever' only when the keyword is used and the value is -1. In 3.4b2
itertools.repeat('a', -1) repeat('a', 0) itertools.repeat('a', times=-1) repeat('a') itertools.repeat('a', times=-2) repeat('a', -2)
Looks like a horrible bug to me. Passing an argument by position should mean the same as passing it by keyword! Regards Antoine.
Am 16.01.2014 12:39, schrieb Antoine Pitrou:
On Thu, 16 Jan 2014 04:42:43 -0500 Terry Reedy <tjreedy@udel.edu> wrote:
On 1/16/2014 3:31 AM, Serhiy Storchaka wrote:
16.01.14 08:05, Guido van Rossum написав(ла):
In this specific case it's clear to me that the special-casing of negative count is intentional -- presumably it emulates sequence repetition, where e.g. 'a'*-1 == ''.
In this specific case it's contrary to sequence repetition. Because repeat('a', -1) repeats 'a' forever.
'Forever' only when the keyword is used and the value is -1. In 3.4b2
itertools.repeat('a', -1) repeat('a', 0) itertools.repeat('a', times=-1) repeat('a') itertools.repeat('a', times=-2) repeat('a', -2)
Looks like a horrible bug to me. Passing an argument by position should mean the same as passing it by keyword!
Indeed, that should be fixed regardless of AC. Georg
On Thu, Jan 16, 2014 at 1:42 AM, Terry Reedy <tjreedy@udel.edu> wrote:
On 1/16/2014 3:31 AM, Serhiy Storchaka wrote:
16.01.14 08:05, Guido van Rossum написав(ла):
In this specific case it's clear to me that the special-casing of negative count is intentional -- presumably it emulates sequence repetition, where e.g. 'a'*-1 == ''.
In this specific case it's contrary to sequence repetition. Because repeat('a', -1) repeats 'a' forever.
'Forever' only when the keyword is used and the value is -1. In 3.4b2
itertools.repeat('a', -1) repeat('a', 0) itertools.repeat('a', times=-1) repeat('a') itertools.repeat('a', times=-2) repeat('a', -2)
This is a point of Vajrasky's issue [1].
The first line is correct in both behavior and representation. The second line behavior (and corresponding repr) are wrong. The third line repr is wrong but the behavior is like the first.
Eew. This is much more wacko than I thought. (Serves me right for basically not caring about itertools :-(. ) It also mostly sounds like unintended -- I can't imagine the intention was to treat the keyword argument different than the positional argument, but I can easily imagine getting the logic slightly wrong. If I had complete freedom in redefining the spec I would treat positional and keyword the same, interpret absent or None to mean "forever" and explicit negative integers to mean the same as zero, and make repr show a positional integer >= 0 if the repeat isn't None. But I don't know if that's too much of a change. @Antoine:
Passing in "None" here is inconvenient as it's an integer argument.
Inconvenient for whom? The callee or the caller?
I meant for the callee -- it's slightly complex to code up right. But IMO worth it. -- --Guido van Rossum (python.org/~guido)
On Thu, Jan 16, 2014 at 1:42 AM, Terry Reedy <tjreedy@udel.edu> wrote:
itertools.repeat('a', -1) repeat('a', 0) itertools.repeat('a', times=-1) repeat('a') itertools.repeat('a', times=-2) repeat('a', -2)
The first line is correct in both behavior and representation. The second line behavior (and corresponding repr) are wrong. The third line repr is wrong but the behavior is like the first.
On 1/16/2014 1:42 PM, Guido van Rossum wrote:
If I had complete freedom in redefining the spec I would treat positional and keyword the same, interpret absent or None to mean "forever" and explicit negative integers to mean the same as zero, and make repr show a positional integer >= 0 if the repeat isn't None.
But I don't know if that's too much of a change.
I copied the unsnipped stuff above to a tracker message. http://bugs.python.org/issue19145 -- Terry Jan Reedy
On Thu, Jan 16, 2014 at 12:31 AM, Serhiy Storchaka <storchaka@gmail.com> wrote:
getattr(foo, 'bar', None) is not the same as getattr(foo, 'bar'). So None can't be used as universal default value.
Not universal, but I still think that most functions don't need to have such a subtle distinction. E.g. in the case of sha1() I still believe that it's totally fine to switch the default to b''. In that particular case I don't see the need to also accept None as a way to specify the default. Basically, my philosophy about this is that anytime you can't easily reimplement the same signature in Python (without reverting to manually parsing the args using *args and **kwds) it is a pain, and you should think twice before canonizing such a signature. Also, there are two somewhat different cases: (a) The default can easily be expressed as a value of the same type that the argument normally has. This is the sha1() case. In this case I see no need to also accept None as an argument (unless it is currently accepted, which it isn't for sha1()). Another example is <stream>.read() -- here, passing in a negative integer means the same as not passing an argument. (b) The default has a special meaning that does something different than any valid value. A good example is getattr(), which must forever be special. To me, most functions should fall in (a) even if there is currently ambiguity, and it feels like repeat() was *meant* to be in (a). I'm not sure how AC should deal with (b), but I still hope that true examples are rare enough that we can keep hand-coding them. -- --Guido van Rossum (python.org/~guido)
16.01.14 07:55, Larry Hastings написав(ла):
* itertools.repeat deliberately makes it impossible to provide an argument for "times" that behaves the same as not providing the "times" argument, and * there is currently no way to implement this behavior using Argument Clinic. (I'd have to add a hack where impl functions also get args and kwargs.)
/*[clinic input] itertools.times object: object [ times: int ] [clinic start generated code]*/
Are you suggesting that, when converting builtins to Argument Clinic with unrepresentable default values, we're permitted to tweak the defaults to something representable?
I think we need some standard placeholder for unrepresentable default value. May be "..." or "?"?
On Wed, 15 Jan 2014 21:55:46 -0800 Larry Hastings <larry@hastings.org> wrote:
Passing in "None" here is inconvenient as it's an integer argument.
Inconvenient for whom? The callee or the caller? Regards Antoine.
On 01/16/2014 03:38 AM, Antoine Pitrou wrote:
On Wed, 15 Jan 2014 21:55:46 -0800 Larry Hastings <larry@hastings.org> wrote:
Passing in "None" here is inconvenient as it's an integer argument. Inconvenient for whom? The callee or the caller?
The callee, specifically the C argument parsing code. (Even more specifically: the Argument Clinic argument parsing code generator.) //arry/
On 2014-01-16 05:32, Larry Hastings wrote: [snip]
In the specific case of SHA1_new's "string" parameter, we could lie and claim that the default value is b''. Internally we could still use NULL as a default and get away with it. But this is only a happy coincidence. Many (most?) functions like this won't have a clever Python value we can trick you with.
What else could we do? We could add a special value, let's call it sys.NULL, whose specific semantics are "turns into NULL when passed into builtins". This would solve the problem but it's really, really awful.
Would it be better if it were called "__null__"?
The only other option I can see: don't convert SHA1_new() to use Argument Clinic, and don't provide introspection information for it either.
Can you, gentle reader, suggest a better option?
//arry/
p.s. Ryan's function signatures above suggest that he's converting code from using PyArg_ParseTuple into using PyArg_ParseTupleAndKeywords. I don't think he's *actually* doing that, and if I saw that in patches submitted to me I would ask that it be fixed.
On 01/16/2014 04:21 AM, MRAB wrote:
On 2014-01-16 05:32, Larry Hastings wrote: [snip]
We could add a special value, let's call it sys.NULL, whose specific semantics are "turns into NULL when passed into builtins". This would solve the problem but it's really, really awful.
Would it be better if it were called "__null__"?
No. The problem is not the name, the problem is in the semantics. This would mean a permanent special case in Python's argument parsing (and "special cases aren't special enough to break the rules"), and would inflict these same awful semantics on alternate implementations like PyPy, Jython, and IronPython. //arry/
On 16/01/14 19:43, Larry Hastings wrote:
On 01/16/2014 04:21 AM, MRAB wrote:
On 2014-01-16 05:32, Larry Hastings wrote: [snip]
We could add a special value, let's call it sys.NULL, whose specific semantics are "turns into NULL when passed into builtins". This would solve the problem but it's really, really awful.
Would it be better if it were called "__null__"?
No. The problem is not the name, the problem is in the semantics. This would mean a permanent special case in Python's argument parsing (and "special cases aren't special enough to break the rules"), and would inflict these same awful semantics on alternate implementations like PyPy, Jython, and IronPython.
Indeed. Why not just change the clinic spec a bit, from 'The "default" is a Python literal value.' to 'The "default" is a Python literal value or NULL.'? A NULL default would imply the parameter is optional with no default. Cheers, Mark.
Let me expand on the issue, and address some of the replies. The goal of Argument Clinic is to create new docstring signatures for builtins, with the following properties: 1) Useful. While one can create a signature of func(*args) and then document complex and arbitrary restrictions on what args contains, that isn't helpful to the end-user examining the docstring, or to automated tools. 2) Inspectable. For a signature to be compatible with inspect.signature(), it *must be a valid native Python declaration*. This means no optional positional arguments of the form func(foo[, bar]), and no non-Python default values. 3) Correct. The semantics of the builtin's signature should match the expectations users have about pure Python declarations. There are two classes of builtins whose signatures do not have these properties. The first is those with very weird signatures, like curses.window.addstr(). It's fine that those don't get converted, they're hopeless. A second class is builtins with "almost but not quite" usable signatures, mostly the ones with optional positional parameters. It would be nice to "rescue" those builtins. So, let us return to my original example, getservbyname(). Its current signature: socket.getservbyname(servicename[, protocolname]) This is not an inspectable signature, since pure Python does not support bracketed arguments. To make it inspectable, we must give protocolname a (valid Python) default value: socket.getservbyname(servicename, protocolname=None) Unfortunately, while useful and inspectable, this signature is not correct. For a pure Python function, passing None for protocolname is the same as omitting it. However, if you pass None to getservbyname(), it raises a TypeError. So, we have these three options: 1) Don't give getservbyname() an inspectable signature. 2) Lie to the user about the acceptability of None. 3) Alter the semantics of getservbyname() to treat None as equivalent to omitting protocolname. Obviously #2 is out. My question: is #3 ever acceptable? It's a real change, as it breaks any code that relies on the TypeError exception.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 01/16/2014 04:08 PM, Ryan Smith-Roberts wrote:
[L]et us return to my original example, getservbyname(). Its current signature:
socket.getservbyname(servicename[, protocolname])
This is not an inspectable signature, since pure Python does not support bracketed arguments. To make it inspectable, we must give protocolname a (valid Python) default value:
socket.getservbyname(servicename, protocolname=None)
Unfortunately, while useful and inspectable, this signature is not correct. For a pure Python function, passing None for protocolname is the same as omitting it. However, if you pass None to getservbyname(), it raises a TypeError. So, we have these three options:
1) Don't give getservbyname() an inspectable signature. 2) Lie to the user about the acceptability of None. 3) Alter the semantics of getservbyname() to treat None as equivalent to omitting protocolname.
Obviously #2 is out. My question: is #3 ever acceptable? It's a real change, as it breaks any code that relies on the TypeError exception.
+1 for #3, especially in a new "major" release (w/ sufficient documentation of the change). Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlLYWU8ACgkQ+gerLs4ltQ6obQCglHmIM4kcNOQte7jj9NjL6Xia KQwAn2ircAlSR6iwFIAt6PDz0bs6iIDt =G+GC -----END PGP SIGNATURE-----
On 01/16/2014 01:08 PM, Ryan Smith-Roberts wrote:
There are two classes of builtins whose signatures do not have these properties. The first is those with very weird signatures, like curses.window.addstr(). It's fine that those don't get converted, they're hopeless.
Speaking as the father of Argument Clinic, I disagree. My goal with Clinic is to convert every function in CPython whose semantics can be expressed with a PyArg_ parsing function. For example, curses.window.addstr could be converted just fine. Its signature is exactly the same as curses.window.addch which has already been converted. socket.sendto eludes me for now--but I haven't given up yet. Don't give up hope, //arry/
On Thu, Jan 16, 2014 at 01:08:47PM -0800, Ryan Smith-Roberts wrote:
socket.getservbyname(servicename[, protocolname])
This is not an inspectable signature, since pure Python does not support bracketed arguments. To make it inspectable, we must give protocolname a (valid Python) default value:
socket.getservbyname(servicename, protocolname=None)
Unfortunately, while useful and inspectable, this signature is not correct. For a pure Python function, passing None for protocolname is the same as omitting it. However, if you pass None to getservbyname(), it raises a TypeError. So, we have these three options:
1) Don't give getservbyname() an inspectable signature. 2) Lie to the user about the acceptability of None. 3) Alter the semantics of getservbyname() to treat None as equivalent to omitting protocolname.
Obviously #2 is out. My question: is #3 ever acceptable? It's a real change, as it breaks any code that relies on the TypeError exception.
The answer seems straightforward to me: it should be treated as any other change of behaviour, and judged on a case-by-case basis. I think the bug tracker is the right place to ask. Since it's not a bug fix, it may be able to be changed, but not lightly, and not in a bug-fix release. The fact that the motivation for the behaviour change is Argument Clinic should not change the decision, as far as I can see. Would a feature request "Allow None as default protocolname" be accepted? -- Steven
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 01/16/2014 12:32 AM, Larry Hastings wrote:
We could add a special value, let's call it sys.NULL, whose specific semantics are "turns into NULL when passed into builtins". This would solve the problem but it's really, really awful.
That doesn't smell too bad too me -- I would prefer to be able to build up all such calls programmatically for testing purposes (e.g., to ensure identical semantics for all code paths between a Python reference implementation and a C extension). Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlLYNVMACgkQ+gerLs4ltQ79NwCgy3231to9rnw/8I+52hFJE+2w Z9QAnR0pAMfkofhT82K1yQctm0E8TF7j =QaC4 -----END PGP SIGNATURE-----
Am 16.01.2014 05:35, schrieb Ryan Smith-Roberts:
On Wed, Jan 15, 2014 at 7:57 PM, Ryan Smith-Roberts <rmsr@lab.net <mailto:rmsr@lab.net>> wrote:
socket.getservbyname(servicename[, protocolname]) -> socket.getservbyname(servicename, protocolname=None)
Here is a more complicated example, since the above does technically have an alternative fix:
sockobj.sendmsg(buffers[, ancdata[, flags[, address]]]) -> sockobj.sendmsg(buffers, ancdata=None, flags=0, address=None)
As far as I understand you should convert these with the "optional group" syntax (i.e. brackets). Georg
On 01/15/2014 10:21 PM, Georg Brandl wrote:
Am 16.01.2014 05:35, schrieb Ryan Smith-Roberts:
On Wed, Jan 15, 2014 at 7:57 PM, Ryan Smith-Roberts <rmsr@lab.net <mailto:rmsr@lab.net>> wrote:
socket.getservbyname(servicename[, protocolname]) -> socket.getservbyname(servicename, protocolname=None)
Here is a more complicated example, since the above does technically have an alternative fix:
sockobj.sendmsg(buffers[, ancdata[, flags[, address]]]) -> sockobj.sendmsg(buffers, ancdata=None, flags=0, address=None) As far as I understand you should convert these with the "optional group" syntax (i.e. brackets).
That's correct. Functions that use PyArg_ParseTuple should continue to use PyArg_ParseTuple. Ryan: add a "/" on a line by itself after the last parameter of each of these functions, and that will let you use the [ ] syntax too. Please see the Argument Clinic howto for more. //arry/
participants (11)
-
Antoine Pitrou -
Georg Brandl -
Guido van Rossum -
Larry Hastings -
Mark Shannon -
MRAB -
Ryan Smith-Roberts -
Serhiy Storchaka -
Steven D'Aprano -
Terry Reedy -
Tres Seaver