Variable arguments (*args, **kwargs): seeking elegance

[Steven D'Aprano]

On Sat, 05 Oct 2013 21:04:25 -0700, John Ladasky wrote:

> Hi folks,
> 
> I'm trying to make some of Python class definitions behave like the ones
> I find in professional packages, such as Matplotlib.  A Matplotlib class
> can often have a very large number of arguments -- some of which may be
> optional, some of which will assume default values if the user does not
> override them, etc.

What makes Matplotlib so professional?

Assuming that "professional" packages necessarily do the right thing is 
an unsafe assumption. Many packages have *lousy* interfaces. They often 
get away with it because of the "sunk cost" fallacy -- if you've spent 
$3000 on a licence for CrapLib, you're likely to stick through the pain 
of learning its crap interface rather than admit you wasted $3000. Or the 
package is twenty years old, and remains compatible with interfaces that 
wouldn't be accepted now, but that's what the user community have learned 
and they don't want to learn anything new. Or backwards compatibility 
requires them to keep the old interface.

I have not used mathplotlib enough to judge its interface, but see below.


> I have working code which does this kind of thing.  I define required
> arguments and their default values as a class attribute, in an
> OrderedDict, so that I can match up defaults, in order, with *args.  I'm
> using set.issuperset() to see if an argument passed in **kwargs
> conflicts with one which was passed in *args.  I use  set.isdisjoint()
> to look for arguments in **kwargs which are not expected by the class
> definition, raising an error if such arguments are found.

The cleanest way is:

class Spam:
    def __init__(
            self, arg, required_arg, 
            another_required_arg,
            arg_with_default=None, 
            another_optional_arg=42,
            and_a_third="this is the default",
            ):


and so on, for however many arguments your class wants. Then, when you 
call it:

s = Spam(23, "something", another_optional_arg="oops, missed one")


Python will automatically:

    [quote]
    match up defaults, in order, ... 
    see if an argument conflicts with one ...
    look for arguments ... which are not expected...
    raising an error if such arguments are found
    [end quote]


Why re-invent the wheel? Python already checks all these things for you, 
and probably much more efficiently than you do. What benefit are you 
getting from manually managing the arguments?

When you have a big, complex set of arguments, you should have a single 
point of truth, one class or function or method that knows what args are 
expected and uses Python's argument-handling to handle them. Other 
classes and functions which are thin (or even not-so-thin) wrappers 
around that class shouldn't concern themselves with the details of what's 
in *args and **kwargs, they should just pass them on untouched.


There are two main uses for *args:

1) Thin wrappers, where you just collect all the args and pass them on, 
without caring what name they eventually get assigned to:

class MySubclass(MyClass):
    def spam(self, *args):
        print("calling MySubclass")
        super(MySubclass, self).spam(*args)


2) Collecting arbitrary, homogeneous arguments for processing, where the 
arguments don't get assigned to names, e.g.:

def mysort(*args):
    return sorted(args)

mysort(2, 5, 4, 7, 1)
=> [1, 2, 4, 5, 7]


Using *args and then manually matching up each argument with a name just 
duplicates what Python already does.



> Even though my code works, I'm finding it to be a bit clunky.  And now,
> I'm writing a new class which has subclasses, and so actually keeps the
> "extra" kwargs instead of raising an error... This is causing me to
> re-evaluate my original code.
> 
> It also leads me to ask: is there a CLEAN and BROADLY-APPLICABLE way for
> handling the *args/**kwargs/default values shuffle that I can study?

Yes. Don't do it :-)

It is sometimes useful to collect extra keyword arguments, handle them in 
the subclass, then throw them away before passing them on:

class MySubclass(MyClass):
    def spam(self, *args, **kwargs):
        reverse = kwargs.pop('reverse', False)
        msg = "calling MySubclass"
        if reverse:
            msg = msg[::-1]
        print(msg)
        super(MySubclass, self).spam(*args, **kwargs)

kwargs is also handy for implementing keyword-only arguments in Python 2 
(in Python 3 it isn't needed). But in that case, you don't have to worry 
about matching up keyword args by position, since position is normally 
irrelevant. Python's basic named argument handling should cover nearly 
all the code you want to write, in my opinion. 


-- 
Steven