[Python-ideas] function defaults and an empty() builtin

[Steven D'Aprano]

On Sat, 21 May 2011 05:10:25 am Masklinn wrote:
> On 2011-05-20, at 21:12 , Ethan Furman wrote:
> > In this scenario:
> >
> > def func(mylist=empty()):
> >   do_some_stuff_with_mylist
> >
> > mylist is the empty() object, you *know* func() does not modify
> > mylist, you are wrong (heh) and it does... but your program always
> > calls func() with an actual list -- how is empty() going to save
> > you then?
>
> It will not until one day func() is called without an actual list,
> and then you get a clear and immediate error instead of silent data
> corruption, or a memory leak, 

I wouldn't call it a memory leak. As I understand it, a memory leak is 
normally understood to mean that your program is assigning memory in 
such a way that neither you, nor the compiler, can free it, not that 
you merely haven't noticed that you're assigning memory. Since the 
default value is exposed, either the function or the caller can free 
that memory.


> which are generally the result of an 
> improperly mutated default argument collection and much harder to
> spot.

You are simply wrong there. There is no reason to imagine that the 
caller will *immediately* attempt to modify the result:

y = func()  # returns immutable empty list
y.append(None)

The attempt to mutate y might not happen until much later, in some 
distant part of the code, in another function, or module, or thread, or 
even another process. There is no limit to how distant in time or space 
the exception could be.

It's a landmine waiting to blow up, not an assertion.

y = func()
# ... much later
data = {'key': y}
# ... much later still
params.update(data)
response = connect('something', params)

def connect(x, params):
    a = params.get('key', [])
    a.append('something')


When connect fails, there's nothing to associate the error with the 
mistake of calling func() without supplying an argument.

But note that calling func() without an argument is supposed to be 
legal. Why is it a mistake? It's only a mistake because func exposes 
internal data to the caller, and then compounds that bug by punishing 
the caller for inadvertently modifying that internal data rather than 
not exposing it in the first place.

This is *astonishingly* awful design.


[...]
> That's it. That's a pretty common bug in Python, and it solves it. No
> more, and no less.

This doesn't solve the problem, it just creates a new one. If people 
can't remember to use the "if default is None" idiom, what makes you 
think they will remember to use empty()? And if they do remember, 
they're just disguising their bug as the caller's mistake.



-- 
Steven D'Aprano