[Python-Dev] Rough idea for adding introspection information for builtins

[Nick Coghlan]

On Mon, Mar 18, 2013 at 11:08 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> I can't see why the size would matter in any way.

We're mildly concerned about the possible impact on the size of the
ever-growing CPython binaries. However, it turns out that this is a
case where readability and brevity are allies rather than enemies, so
we don't need to choose one or the other.

>
>
>> Of those, Nick's suggestion seems best.  It's slightly bigger than the
>> specialized bytecode format, but it's human-readable (and human-writable!),
>> and it'd be the easiest to implement.
>
> Plus, if it becomes the format how C level signatures are expressed anyway,
> it wouldn't require any additional build time preprocessing.
>
>
>> My first idea for implementation: add a "def x" to the front and ": pass"
>> to the end
>
> Why not require it to be there already? Maybe more like
>
>     def foo(arg, b=3, *, kwonly='a'):
>          ...
>
> (i.e. using Ellipsis instead of pass, so that it's clear that it's not an
> empty function but one the implementation of which is hidden)

I like this notion. The groups notation and '/' will still cause the
parser to choke and require special handling, but OTOH, they have
deliberately been chosen as potentially acceptable notations for
providing the same features in actual Python function declarations.

>
>
>> then run it through ast.parse.  Iterate over the tree,
>> converting parameters into inspect.Parameters and handling the return
>> annotation if present.  Default values and annotations would be turned into
>> values by ast.eval_literal.  (It wouldn't surprise me if there's a cleaner
>> way to do it than the fake function definition; I'm not familiar with the
>> ast module.)
>
> IMHO, if there is no straight forward way currently to convert a function
> header from a code blob into a Signature object in Python code, preferably
> using the ast module (either explicitly or implicitly through inspect.py),
> then that's a bug.

The complexity here is that Larry would like to limit the annotations
to compatibility with ast.literal_eval. If we drop that restriction,
then the inspect module could handle the task directly. Given the
complexity of implementing it, I believe the restriction needs more
justification than is currently included in the PEP.

>
>
>> We'd want one more mild hack: the DSL will support positional parameters,
>> and inspect.Signature supports positional parameters, so it'd be nice to
>> render that information.  But we can't represent that in Python syntax (or
>> at least not yet!), so we can't let ast.parse see it.  My suggestion: run
>> it through ast.parse, and if it throws a SyntaxError see if the problem was
>> a slash.  If it was, remove the slash, reprocess through ast.parse, and
>> remember that all parameters are positional-only (and barf if there are
>> kwonly, args, or kwargs).
>
> Is sounds simpler to me to just make it a Python syntax feature. Or at
> least an optional one, supported by the ast module with a dedicated
> compiler flag.

Agreed. Guido had previously decided "not worth the hassle", but this
may be enough to make him change his mind. Also, Larry's "simple"
solution here isn't enough, since it doesn't handle optional groups
correctly.

While the support still has some odd limitations under the covers, I
think an explicit compiler flag is a good compromise between a lot of
custom hacks and exposing an unfinished implementation of a new
language feature.

Cheers,
Nick.

>
> Stefan
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com



-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia