[Python-3000] Cleaning up argument list parsing (was Re: More wishful thinking)

[Nick Coghlan]

Talin wrote:
> Guido van Rossum <guido <at> python.org> writes:
> 
>> On 4/15/06, Talin <talin <at> acm.org> wrote:
>>> Another grab-bag of language ideas / wishes. Some of these are
>>> items that have been raised before on other lists - I want to get them
>>> written down so that they can be rejected quickly :)
>>>
>>> 1) I don't know if this is already on the table, but I sure would like
>>> to be able to have more options as far as mixing positional and
>>> keyword arguments.
>> I agree. Please produce a patch that implements what you want.

A question for Guido: Is this flexibility for argument passing only, or does 
it apply to parameter declarations as well?

In this approach, parameters declared after *args would be "keyword-only" 
parameters, so should a single "*" be permitted in the parameter list to allow 
functions with a fixed number of positional arguments to also declare 
keyword-only arguments?

Currently, the only way to do this is to declare **kwds and go poking into the 
dictionary manually (the same way C code does it), which means important 
information is missing from the function signature.

> All right, in that case, I'd like to post a general implementation
> strategy for criticism before I end up going too far down the
> wrong path.
> 
> As I see it, the problem breaks down into 3 parts:
> 
> -- parsing
> -- representing the argument list
> -- executing a call
> 
> The parsing part  is fairly straightforward. The new syntax would
> allow keyword arguments to appear anywhere in the argument
> list, intermixed with positional arguments, and after the varargs
> argument.

I think that's making things a bit *too* flexible - I believe its worthwhile 
keeping the positional argument at the beginning of the call. The only thing 
we're trying to fix here is that the current syntax requires the varargs 
argument to be separated from the normal positional arguments by any specified 
keyword arguments.

> (It would be possible to have non-keyword arguments after the
> varargs argument as well, but that's harder to implement, and I
> don't really see the value in it.)

I think keeping the positional arguments before the varargs argument is the 
right thing to do.

> The kwargs argument would remain unchanged - there's no real
> reason to allow it to be moved around in the argument list.

This also sounds right.

> So the argument list syntax would be (in psuedo-grammar):
> 
>    (keyword | positional )* [*varargs] keyword* [**kwargs]

This seems unnecessarily flexible to me. I'd prefer:

  positional* keyword* ["*"varargs] keyword* ["**"kwargs]

That is, the only difference from the status quo is the ability to put the 
keyword arguments after the varargs argument instead of before it.

> (This is of course omitting all of the complexities of generator
> expressions for the moment.)

Genexp's pretty much result in a giant "or" with this whole section of the 
grammar anyway.

> Representing the argument list. The current _arguments struct
> looks like this:
> 
>   struct _arguments {
>           asdl_seq *args;
>           identifier vararg;
>           identifier kwarg;
>           asdl_seq *defaults;
>   };

Calling this the argument list is a mistake in the current AST definition - 
this structure is only used for the parameter list in a function definition. 
It is *NOT* used for call arguments. (I'd personally be happy with a patch 
that fixed this discrepancy by changing the name to "parameters").

The actual call definition wouldn't require any changes to handle this:

     Call(expr func, expr* args, keyword* keywords,
			 expr? starargs, expr? kwargs)

Changes to the 'arguments' structure would only be needed if Guido extended 
this idea to cover declaration of keyword-only arguments.

> The approach that I would take would be to remove the 'vararg',
> and instead put the vararg argument into args. A special sentinel
> value in the *defaults array (say, (void*)-1) would be used to
> indicate that this argument was a varargs argument.
> 
> Thus, for each argument (other than kwargs), you would have
> both an argument name and a default, where a NULL value indicated
> a positional argument, a non-NULL value indicated a keyword argument,
> and a sentinel value indicated a varargs argument.

Since this is only needed if Guido decides to permit keyword-only arguments in 
function definitions, I'd suggest simply adding a second sequence for keyword 
arguments after the varargs argument:

    struct _arguments {
            asdl_seq *args;
            identifier vararg;
            asdl_seq *kwargs;
            identifier kwarg;
            asdl_seq *defaults;
    };


> Executing a call: This is the tricky part. Specifically, we want to insure
> that the mapping of values to formal parameters remains the same
> for existing code.

In the simple case, this shouldn't have to change at all, since the AST for it 
won't change.

In the case where keyword-only arguments are permitted, it's simply a matter 
of checking the called function's "kwargs" list in addition to its normal 
"args" list before dumping an unregognised keyword parameter in the 
double-star dictionary.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org