[Python-Dev] signature.object, argument clinic and grouped parameters

[Larry Hastings]

On 01/20/2014 04:59 AM, Nick Coghlan wrote:
> When I wrote that, I was thinking we had made
> inspect.Signature.__repr__ produce a nice string format, but then I
> noticed in the REPL today that we never got around to doing that - I
> think because we didn't know how to handle positional-only arguments,
> which already can't be expressed as Python syntax. (I haven't checked
> if we have an RFE filed anywhere)

I don't know what you had intended to do, but right now 
inspect.Signature inherits the standard repl from object. 
inspect.Signature.__str__ produces something that looks like a Python 
function signature, starting and ending with parentheses. (For those of 
you unfamiliar with inspect.Signature: A signature is agnostic about the 
name of the function.  So it doesn't include the name.)


> However, while I know you're keen to finally make introspection work
> for all C level callables in 3.4, even the ones with signatures that
> can't be expressed as Python function signatures, I'd like to strongly
> encourage you to hold off on that last part until Python 3.5.

If we hold off on all of this until 3.5, the signatures for most 
builtins will be wrong in 3.4, because most builtins take 
positional-only parameters.  I had higher hopes for Python 3.4 than 
that.  To be honest I'd rather not have the feature at all than have it 
be wrong most of the time.

I think it's fair to summarize your argument as "there could be monsters 
lurking in CPython with signatures that can't be expressed in PEP 457 
syntax".  To me this smacks of FUD.  Let me open my kimono and tell you 
all the counter-examples we know of so far.

  * socket.sendto() has an optional group in the middle of required
    parameters.  (This signature is from 1993.)  PEP 457 could support
    this just by relaxing one requirement.  I know what's needed here,
    but given that PEP 457 was such a dud I haven't bothered to update
    it.  Regardless, Argument Clinic, and the syntax used for text
    signatures, could (and I expect will soon) support this.  The
    inspect.Parameter.group proposal from my email last night supports
    this just fine.
  * itertools.repeat() has a parameter that behaves differently if it's
    passed by keyword vs passed by position.  Guido already ruled that
    this signature must be changed so it is representable with Python
    syntax--this behavior is a "bug".
  * Many functions have default values that are not representable in
    Python, chiefly a NULL pointer.  Guido has already ruled that these
    signatures should be changed so that they're representable in
    Python.  The best approach is often accepting None, which is
    inconvenient for non-pointer arguments like integers.  Right now
    Argument Clinic gives you no assistance in this area, but I plan to
    add explicit support making it easy (via "nullable ints").

In short, there's a clear trend: functions must have signatures 
representable in Python syntax, with the exception of optional groups 
which are a legacy feature we can't get rid of but won't support in 
Python syntax.  Any functions whose signatures are not representable in 
Python syntax shall be tweaked until they are.

Any new monsters we discover lurking in CPython will be slain, not 
supported.

-----

We could split the difference, and not add a feature to the inspect 
module to support optional groups.  We could still support marking 
positional-only parameters, as inspect currently supports that. That 
would mean nearly all signatures for builtins would be correct.

Personally I'd rather go the extra distance and support optional groups 
too.  There are important callables that can only be expressed with 
optional groups (range, type).  Given the trend above, Parameter 
arguments with optional groups should be sufficient to express every 
signature available in Python.  We've come this far... or, as the 
British say, in for a penny, in for a pound. Let's hash it out right now 
and get it done.


> While the text string used to communicate between Argument Clinic and
> inspect.signature will be private, the representation on
> inspect.Signature objects will be a new *public* API. As the
> discussions between you, me and Yury show, I don't think there's an
> immediately obvious best answer of how to do that. Your suggestion of
> just adding the group numbers to the Parameter objects would *work*,
> but it's not very Pythonic - we have container types that support
> nesting,

Apparently you didn't read my proposal in the email you replied to. I 
didn't propose that "group" contain a number, I proposed it contain a 
ParameterGroup object that supports nesting.

We could take another approach, one you seem to be suggesting, where the 
nesting is outside the Parameter objects.  In this alternate approach, 
the Signature.parameters array can contain either Parameter objects or 
OrderedDicts.  The nested OrderedDicts themselves can contain either 
Parameter objects or more nested OrderedDicts.  The API would specify 
that the nested OrderedDicts of parameters are optional en masse.  This 
works fine too.

The chief difference between these proposals: if you ignore the 
complexity of optional groups, the failure mode with ".group" is that it 
kind of works except when it doesn't, whereas with having OrderedDicts 
in .parameters the failure mode is that your code blows up with missing 
attributes (like "couldn't find an attribute called name on this 
OrderedDict object").  That's probably a vote in favor of the nested 
OrderedDicts.


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140120/630e5cd8/attachment.html>