[Python-Dev] Updated PEP 362 (Function Signature Object)

[Yury Selivanov]

Steven,

On 2012-06-06, at 11:38 AM, Steven D'Aprano wrote:
> Brett Cannon wrote:
>> Python has always supported powerful introspection capabilities,
>> including introspecting functions and methods.  (For the rest of
>> this PEP, "function" refers to both functions and methods).  By
>> examining a function object you can fully reconstruct the function's
>> signature.  Unfortunately this information is stored in an inconvenient
>> manner, and is spread across a half-dozen deeply nested attributes.
>> This PEP proposes a new representation for function signatures.
>> The new representation contains all necessary information about a function
>> and its parameters, and makes introspection easy and straightforward.
> 
> It's already easy and straightforward, thanks to the existing inspect.getfullargspec function. If this existing inspect function is lacking something, the PEP should explain what, and why the inspect function can't be fixed.

Well, the PEP addresses this question by saying: "Unfortunately this 
information is stored in an inconvenient manner, and is spread across 
a half-dozen deeply nested attributes."

And that's true.  As of now 'inspect.getfullargspec' returns a named
tuple, where all parameters are spread between different tuple items.
Essentially, what 'inspect.getfullargspec' really does, is simply
packing all the function and its code object attributes into a tuple.
Compare it with the elegance 'Signature' and 'Parameter' class
provides.

Example:

  def foo(a, bar:int=1):
     pass

With signature:
  
  sig = signature(foo)
  print('bar: annotation', sig.parameters['bar'].annotation)
  print('bar: default', sig.parameters['bar'].default)

end with 'inspect':

  args_spec = inspect.getfullargspec(foo)
  print('bar: default', args_spec.annotations['bar'])
  print('bar: default', args_spec.defaults[0]) # <- '0'?

Signature object is a much nicer API.  It becomes especially obvious
when you need to write more complicated stuff (you can see the PEP's
examples.)

>> However, this object does not replace the existing function
>> metadata, which is used by Python itself to execute those
>> functions.  The new metadata object is intended solely to make
>> function introspection easier for Python programmers.
> 
> What happens when the existing function metadata and the __signature__ object disagree?

That's an issue to think about.  We may want to make 
'Signature.parameters' immutable by hiding it behind the newly added 
'types.MappingProxyType'.

But I don't foresee modifications of Signature objects to be somewhat
frequent.  It may happen for a very specific reason in a very specific
user code, and I see no point in denying this.

>> Signature Object
>> ================
>> A Signature object represents the overall signature of a function.
>> It stores a `Parameter object`_ for each parameter accepted by the
>> function, as well as information specific to the function itself.
> 
> There's a considerable amount of data recorded, including a number of mappings (dicts?). This potentially increase the size of functions, and the overhead of creating them. Since most functions are never introspected, or only rarely introspected, it seems rather wasteful to record all this data "just in case", particularly since it's already recorded once in the function metadata and/or code object.

The object is now created in a lazy manner, once requested.

>> A Signature object has the following public attributes and methods:
>> * name : str
>>    Name of the function.
> 
> Functions already record their name (twice!), and it is simple enough to query func.__name__. What reason is there for recording it a third time, in the Signature object?

Signature object holds function's information and presents it in a
convenient manner.  It makes sense to store the function's name,
together with the information about its parameters and return 
annotation.

> Besides, I don't consider the name of the function part of the function's signature. Functions can have multiple names, or no name at all, and the calling signature remains the same.

It always have _one_ name it was defined with, unless it's
a lambda function.

> Even if we limit the discussion to distinct functions (rather than a single function with multiple names), I consider spam(x, y, z) ham(x, y, z) and eggs(x, y, z) to have the same signature. Otherwise, it makes it difficult to talk about one function having the same signature as another function, unless they also have the same name. Which would be unfortunate.

I see the point ;)  Let's see what other devs think.

>> * qualname : str
>>    Fully qualified name of the function.
> 
> What's the fully qualified name of the function, and why is it needed?

See PEP 3155.

> [...]
>> The structure of the Parameter object is:
> 
>> * is_args : bool
>>    True if the parameter accepts variable number of arguments
>>    (``\*args``-like), else False.
> 
> 
> "args" is just a common name for the parameter, not for the kind of parameter. *args (or *data, *whatever) is a varargs parameter, and so the attribute should be called "is_varargs".

I had a discussion regarding that with Brett and Larry.  They've convinced
me that the '*args', and '**kwargs' is The Ultimate Convention to name
these parameters and work with them.  This convention is immediately
recognized by most of the developers, hence, it'd be easier to understand
from the first glance what 'is_args' means, compared to 'is_varargs'.

>> * is_implemented : bool
>>    True if the parameter is implemented for use.  Some platforms
>>    implement functions but can't support specific parameters
>>    (e.g. "mode" for os.mkdir).  Passing in an unimplemented
>>    parameter may result in the parameter being ignored,
>>    or in NotImplementedError being raised.  It is intended that
>>    all conditions where ``is_implemented`` may be False be
>>    thoroughly documented.
> 
> What to do about parameters which are partly implemented? E.g. mode='spam' is implemented but mode='ham' is not.
> 
> Is there a use-case for is_implemented?

That's a question to Larry.

> [...]
>> Annotation Checker
> 
>>        def check_type(sig, arg_name, arg_type, arg_value):
>>            # Internal function that incapsulates arguments type checking
> 
> /s/incapsulates/encapsulates

Thanks!

>> Open Issues
>> ===========
> 
> inspect.getfullargspec is currently unable to introspect builtin functions and methods. Should builtins gain a __signature__ so they can be introspected?

Again, a question to Larry.  He's building a new API to work with function 
arguments on a C level, and basically, it can construct Signature objects
automatically.

Brett, on the other hand, has the idea of parsing builtin functions
doc strings, as they almost always have their signature described on
the first line of it:

  >>> any.__doc__
  'any(iterable) -> bool\n\nReturn True if bool(x) is True for any x 
  in the iterable.'

We can surely implement it (and in a short time), but I doubt that
3.3 is a good target for this particular feature.

>> Deprecate ``inspect.getfullargspec()`` and ``inspect.getcallargs()``?
>> ---------------------------------------------------------------------
> 
> 
> -1
> 
>> Since the Signature object replicates the use of ``getfullargspec()``
>> and ``getcallargs()`` from the ``inspect`` module it might make sense
>> to begin deprecating them in 3.3.
> 
> I think it is way to soon to deprecate anything. I don't think we should even consider PendingDeprecation until at least 3.4.

+0.

> Actually, I would go further: leave getfullargspec to extract the *actual* argument spec from the code object, and __signature__ to be the claimed argument spec. Earlier, you state:
> 
> "Changes to the Signature object, or to any of its data members,
> do not affect the function itself."
> 
> which leaves the possibility that __signature__ may no longer match the actual argument spec, for some reason. If you remove getfullargspec, people will have to reinvent it to deal with such cases.

Again, as I explained above - we can make Signatures immutable, but
the cases of modifying it should be extremely rare.

Thank you,
-
Yury