Apologies in advance for the lateness of my reply. I missed a few emails in all the shuffle.
Jim Jewett wrote:
On 1/30/07, Chris Rebert cvrebert@gmail.com wrote:
Jim Jewett wrote:
On 1/28/07, Chris Rebert cvrebert@gmail.com wrote:
Naive programmers desiring mutable default arguments often make the mistake of writing the following:
def foo(mutable=some_expr_producing_mutable): #rest of functionYes, it is an ugly gotcha, but so is the alternative. If it is changed, then just as many naive programmers (though perhaps not exactly the same ones) will make the opposite mistake -- and so will some experienced programmers who are used to current semantics.
Anyone (ab)using the current semantics with the above construct is or ought to be aware that such usage is unpythonic.
Are you worried about naive programmers or not?
[snip]
Additionally, I see making the opposite mistake as usually only maybe causing a performance problem, as opposed to causing the program to work incorrectly.
That is true only when the argument is a cache. Generally, resetting the storage will make the program work incorrectly, but the program won't fail on the first data tested -- which means the bug is less likely to be caught.
I am worried about naive programmers. It seems that the opposite mistake (thinking arguments will be evaluated only once, at def-time, rather than at each call) won't have non-performance related effects a good bit of the time. For many of the remaining cases, the person probably is/should be using a constant instead of a default argument. Yes, there will be some cases where badness could happen. However, (1) this is intended for Py3k, so there will already be a bunch of changes for newbies to learn, (2) the chance of weirdness happening is a good bit lower compared to the current state of mutable default arguments.
Also, due to the hairiness/obscurity of the construct, hardly anyone uses it, so the impact of removing it should be relatively minor.
I just did found 181 such uses in my own installation. 29 were either 3rd-party or test code, but that still leaves over 150 uses in the standard library. It is possible that some of them would also work with your semantics, or may even be bugs today -- but you would have to go through them one-by-one as part of the PEP process.
I have a script to gather statistics on default argument usage in the standard library 90% done. Its results will be included in the next PEP draft.
There are currently few, if any, known good uses of the currentbehavior of mutable default arguments. The most common one is to preserve function state between calls. However, as one of the lists [2] comments, this purpose is much better served by decorators, classes, or(though less preferred) global variables.
I disagree. This is particularly wrong for someone coming from a functional background.
I assume you disagree with the "purpose is much better served by decorators, classes, or local variables" part as opposed to the "default mutable arguments with current semantics have few good uses" part. Please correct me if I'm in error.
I disagree with both. There are many uses for preserving function state between calls. C uses static local variables. Object Oriented languages often use objects.
First, I was referring to using default args for caching being evil, not mutable default arguments in general. Just wanted to clarify in case you misinterpreted me. True, there are many uses for preserving function state between calls, but how much better/clearer is doing so using default arguments as opposed to objects or other approaches? It's an aesthetic decision. IMHO, the other ways seem better. YMMV.
A class plus an instantiation seems far too heavyweight for what ought to be a simple function. I'm not talking (only) about the runtime; the number of methods and lines of code is the real barrier for me. I'll sometimes do it (or use a global) anyhow, but it feels wrong. If I had felt that need more often when I was first learning python (or before I knew about the __call__ workaround), I might have just written the language off as no less bloated than java.
I'm sorry, but when you have a function sharing state between calls, that just screams to me that you should make it a method of an object
That's because you drank the OO koolaid. Python tries not to enforce any particular programming style. Its object model happens to be pretty good, but there are still times when OO isn't the answer.
Well, it's a trade-off: support procedural/functional programming a bit more, or make default arguments a bit more intuitive and obsolete the 'foo=None; if foo is None' idiom.
If performance is your concern, the decorator version might perform better (I don't really know), or in the extreme case, you could use a global variable, which is definitely faster.
I think there is something wrong with your benchmark.
Sorry, yeah, you're right. I was being a little fast and loose. But the point was that there are (at the price of some elegance) ways to optimize such situations even more.
That said, speed is *not* my concern -- code size is. If I have to add several lines of boilerplate, the code becomes much less readable. That is roughly the same justification you have for not liking the "if arg is None: arg=foo()" idiom. The difference is that you're suggesting adding far more than a line or two.
True, however I'm willing to bet that the boilerplate for evaluating default arguments only once won't be needed nearly as much, as such situations requiring such behavior is not too common. The statistics should help show exactly how much mutable default arguments are exploited in the wild.
Alternatively, we could make the new semantics the default and have syntax to use the old semantics via 'once' or some other keyword. This does nicely emphasize that such semantics would be purely for optimization reasons.
Using a mutable default is almost never for optimization reasons.
Actually, I was more thinking it might be used to stop immutable default arguments from being re-evaluated multiple times, if it turns out that re-evaluations really do have major effects on performance. Now that I think about it, I guess it could be used to get the old behavior for mutable default arguments.
- Chris Rebert