[Python-ideas] proto-PEP: Fixing Non-constant Default Arguments

[Jim Jewett]

On 1/30/07, Chris Rebert <cvrebert at gmail.com> wrote:
> Jim Jewett wrote:
> > On 1/28/07, Chris Rebert <cvrebert at gmail.com> wrote:

> >> Naive programmers desiring mutable default arguments
> >> often make the mistake of writing the following:

> >>      def foo(mutable=some_expr_producing_mutable):
> >>          #rest of function

> > Yes, it is an ugly gotcha, but so is the alternative.

> > If it is changed, then just as many naive programmers (though perhaps
> > not exactly the same ones) will make the opposite mistake -- and so
> > will some experienced programmers who are used to current semantics.

> Anyone (ab)using the current semantics with the above construct is or
> ought to be aware that such usage is unpythonic.

Are you worried about naive programmers or not?

> Also, due to the
> hairiness/obscurity of the construct, hardly anyone uses it, so the
> impact of removing it should be relatively minor.

I just did found 181 such uses in my own installation.  29 were either
3rd-party or test code, but that still leaves over 150 uses in the
standard library.  It is possible that some of them would also work
with your semantics, or may even be bugs today -- but you would have
to go through them one-by-one as part of the PEP process.

> Additionally, I see
> making the opposite mistake as usually only maybe causing a performance
> problem, as opposed to causing the program to work incorrectly.

That is true only when the argument is a cache.

Generally, resetting the storage will make the program work
incorrectly, but the program won't fail on the first data tested --
which means the bug is less likely to be caught.

> >>      There are currently few, if any, known good uses of the current
> >> behavior of mutable default arguments.  The most common one is to
> >> preserve function state between calls.  However, as one of the lists [2]
> >> comments, this purpose is much better served by decorators, classes, or
> >> (though less preferred) global variables.

> > I disagree.  This is particularly wrong for someone coming from a
> > functional background.

> I assume you disagree with the "purpose is much better served by
> decorators, classes, or local variables" part as opposed to the "default
> mutable arguments with current semantics have few good uses" part.
> Please correct me if I'm in error.

I disagree with both.  There are many uses for preserving function
state between calls.  C uses static local variables.   Object Oriented
languages often use objects.

> > A class plus an instantiation seems far too heavyweight for what ought
> > to be a simple function.  I'm not talking (only) about the runtime;
> > the number of methods and lines of code is the real barrier for me.
> > I'll sometimes do it (or use a global) anyhow, but it feels wrong.  If
> > I had felt that need more often when I was first learning python (or
> > before I knew about the  __call__ workaround), I might have just
> > written the language off as no less bloated than java.

> I'm sorry, but when you have a function sharing state between calls,
> that just screams to me that you should make it a method of an object

That's because you drank the OO koolaid.  Python tries not to enforce
any particular programming style.  Its object model happens to be
pretty good, but there are still times when OO isn't the answer.

> so
> you don't have to store the state in some roundabout way, such as in
> mutable default arguments. If performance is your concern, the decorator
> version might perform better (I don't really know), or in the extreme
> case, you could use a global variable, which is definitely faster.

I think there is something wrong with your benchmark.

The default argument is accessible straight from the function object
itself; even a cell variable shouldn't be that fast, let alone global
variables.

That said, speed is *not* my concern -- code size is.  If I have to
add several lines of boilerplate, the code becomes much less readable.
 That is roughly the same justification you have for not liking the
"if arg is None:  arg=foo()" idiom.  The difference is that you're
suggesting adding far more than a line or two.

> > You see the problem with globals, but decorators are in some sense
> > worse -- a function cannot see its own decorations.  At best, it can
> > *assume* that repeating its own name (despite Don't Repeat Yourself)
> > will get another reference to self, but this isn't always true.

> > Programmers used to creating functions outside of toplevel (or
> > class-level) will be more aware of this, and see the suggestion as an
> > indication that python is inherently buggy.

> I really don't see how the decorator in the PEP is any worse than other
> decorators in this regard. The problem you describe currently applies to
> all decorated functions, though functools.wraps might help mitigate this
> situation.

Either you effectively hide the real function so as to create a
lexical closure (plenty of extra work, both mentally and
computationally) or you accept buggy code.  The bug possibility isn't
inherent to decorations; only to functions reading their *own*
decorations.

> Alternatively, we could make the new
> semantics the default and have syntax to use the old semantics via
> 'once' or some other keyword. This does nicely emphasize that such
> semantics would be purely for optimization reasons.

Using a mutable default is almost never for optimization reasons.

-jJ