[Python-ideas] Default arguments in Python - the return

[Steven D'Aprano]

On Sun, 10 May 2009 10:19:01 am Tennessee Leeuwenburg wrote:
> Hi Pascal,
> Taking the example of
>
> def foo(bar = []):
>   bar.append(4)
>   print(bar)
>
> I'm totally with you in thinking that what is 'natural' is to expect
> to get a new, empty, list every time. 

That's not natural to me. I would be really, really surprised by the 
behaviour you claim is "natural":

>>> DEFAULT = 3
>>> def func(a=DEFAULT):
...     return a+1
... 
>>> func()
4
>>> DEFAULT = 7
>>> func()
8

For deterministic functions, the same argument list should return the 
same result each time. By having default arguments be evaluated every 
time they are required, any function with a default argument becomes 
non-deterministic. Late evaluation of defaults is, essentially, 
equivalent to making the default value a global variable. Global 
variables are rightly Considered Harmful: they should be used with 
care, if at all.


> However this isn't want 
> happens. As far as I'm concerned, that should more or less be the end
> of the discussion in terms of what should ideally happen.

As far as I'm concerned, what Python does now is the idea behaviour. 
Default arguments are part of the function *definition*, not part of 
the body of the function. The definition of the function happens 
*once* -- the function isn't recreated each time you call it, so 
default values shouldn't be recreated either.


> The responses to the change in behaviour which I see as more natural
> are, to summarise, as follows:
>   -- For all sorts of technical reasons, it's too hard
>   -- It changes the semantics of the function definition being
> evaluated at compile time
>   -- It's not what people are used to

And it's not what many people want.

You only see the people who complain about this feature. For the 
multitude of people who expect it or like it, they have no reason to 
say anything (except in response to complaints). When was the last time 
you saw somebody write to the list to say "Gosh, I really love that 
Python uses + for addition"? Features that *just work* never or rarely 
get mentioned.


> With regards to the second point, it's not like the value of
> arguments is set at compile time, so I don't really see that this
> stands up.

I don't see what relevance that has. If the arguments are provided at 
runtime, then the default value doesn't get used.


> I don't think it's intuitive, 

Why do you think that intuitiveness is more valuable than performance 
and consistency?

Besides, intuitiveness is a fickle thing. Given this pair of functions:

def expensive_calculation():
    time.sleep(60)
    return 1

def useful_function(x=expensive_calculation()):
    return x + 1

I think people would be VERY surprised that calling useful_function() 
with no arguments would take a minute *every time*, and would complain 
that this slowness was "unintuitive".


> it's just that people become  
> accustomed to it. There is indeed, *some sense* in understanding that
> the evaluation occurs at compile-time, but there is also a lot of
> sense (and in my opinion, more sense) in understanding the evaluation
> as happening dynamically when the function is called.

No. The body of the function is executed each time the function is 
called. The definition of the function is executed *once*, at compile 
time. Default arguments are part of the definition, not the body, so 
they too should only be executed once. If you want them executed every 
time, put them in the body:

def useful_function(x=SENTINEL):
    if x is SENTINEL:
        x = expensive_calculation()
    return x+1



> With regards to the first point, I'm not sure that this is as
> significant as all of that, although of course I defer to the
> language authors here. However, it seems as though it could be no
> more costly than the lines of code which most frequently follow to
> initialise these variables.
>
> On the final point, that's only true for some people. For a whole lot
> of people, they stumble over it and get it wrong. It's one of the
> most un-Pythonic things which I have to remember about Python when
> programming -- a real gotcha.

I accept that it is a Gotcha. The trouble is, the alternative behaviour 
you propose is *also* a Gotcha, but it's a worse Gotcha, because it 
leads to degraded performance, surprising introduction of global 
variables where no global variables were expected, and a breakdown of 
the neat distinction between creating a function and executing a 
function.

But as for it being un-Pythonic, I'm afraid that if you really think 
that, your understanding of Pythonic is weak. From the Zen:

The Zen of Python, by Tim Peters

Special cases aren't special enough to break the rules.
Although practicality beats purity.
If the implementation is hard to explain, it's a bad idea.

(1) Assignments outside of the body of a function happen once, at 
compile time. Default values are outside the body of the function. You 
want a special case for default values so that they too happen at 
runtime. That's not special enough to warrant breaking the rules.

(2) The potential performance degradation of re-evaluating default 
arguments at runtime is great. For practical reasons, it's best to 
evaluate them once only.

(3) In order to get the behaviour you want, the Python compiler would 
need a more complicated implementation which would be hard to explain.


> I don't see it as changing one way of 
> doing things for another equally valid way of doing things, but
> changing something that's confusing and unexpected for something
> which is far more natural and, to me, Pythonic.

I'm sorry, while re-evaluation of default arguments is sometimes useful, 
it's more often NOT useful. Most default arguments are simple objects 
like small ints or None. What benefit do you gain from re-evaluating 
them every single time? Zero benefit. (Not much cost either, for simple 
cases, but no benefit.)

But for more complex cases, there is great benefit to evaluating default 
arguments once only, and an easy work-around for those rare cases that 
you do want re-evaluation.


> For me, Python 3k appears to be a natural place to do this. Python 3
> still appears to be regarded as a work-in-progress by most people,
> and I don't think that it's 'too late' to change for Python 3k.

Fortunately you're not Guido, and fortunately this isn't going to 
happen. I recommend you either accept that this behaviour is here to 
stay, or if you're *particularly* enamoured of late evaluation 
behaviour of defaults, that you work on some sort of syntax to make it 
optional.



-- 
Steven D'Aprano