[Python-Dev] Rough idea for adding introspection information for builtins

[Stefan Behnel]

Larry Hastings, 19.03.2013 05:45:
> The original impetus for Argument Clinic was adding introspection
> information for builtins [...]
> On to the representation.  Consider the function
> 
>    def foo(arg, b=3, *, kwonly='a'):
>         pass
> [...]
> 4. Store a string that looks like the Python declaration of the signature,
> and parse it (Nick's suggestion).  For foo above, this would be
> "(arg,b=3,*,kwonly='a')".

I had already noted that this would be generally useful, specifically for
Cython, so I'm all for going this route. No need to invent something new here.


>  Length: 23 bytes.

I can't see why the size would matter in any way.


> Of those, Nick's suggestion seems best.  It's slightly bigger than the
> specialized bytecode format, but it's human-readable (and human-writable!),
> and it'd be the easiest to implement.

Plus, if it becomes the format how C level signatures are expressed anyway,
it wouldn't require any additional build time preprocessing.


> My first idea for implementation: add a "def x" to the front and ": pass"
> to the end

Why not require it to be there already? Maybe more like

    def foo(arg, b=3, *, kwonly='a'):
         ...

(i.e. using Ellipsis instead of pass, so that it's clear that it's not an
empty function but one the implementation of which is hidden)


> then run it through ast.parse.  Iterate over the tree,
> converting parameters into inspect.Parameters and handling the return
> annotation if present.  Default values and annotations would be turned into
> values by ast.eval_literal.  (It wouldn't surprise me if there's a cleaner
> way to do it than the fake function definition; I'm not familiar with the
> ast module.)

IMHO, if there is no straight forward way currently to convert a function
header from a code blob into a Signature object in Python code, preferably
using the ast module (either explicitly or implicitly through inspect.py),
then that's a bug.


> We'd want one more mild hack: the DSL will support positional parameters,
> and inspect.Signature supports positional parameters, so it'd be nice to
> render that information.  But we can't represent that in Python syntax (or
> at least not yet!), so we can't let ast.parse see it.  My suggestion: run
> it through ast.parse, and if it throws a SyntaxError see if the problem was
> a slash.  If it was, remove the slash, reprocess through ast.parse, and
> remember that all parameters are positional-only (and barf if there are
> kwonly, args, or kwargs).

Is sounds simpler to me to just make it a Python syntax feature. Or at
least an optional one, supported by the ast module with a dedicated
compiler flag.

Stefan