A crazy idea, didn't think much about it yet: def subfunc(c,a=0,b=1): Blabla def function(c,d=3, from args(subfunc) import a,b): Blabla return anotherfunc(a+b+c+d,subfunc(c,a,b))

On Fri, Jul 20, 2018 at 1:30 PM, Jonathan Fine wrote:

I sort of think we've now got a reasonable answer for Peter's problem. What do you think, Peter? And Brice, are you happy with my interpretation of your deferred keyword?

I think the problem with the "None" approach (second pattern) is that it forces the writer of the subfunction to write their defaults in a more awkward way in anticipation of other functions which defer their defaults to it. Translated to the original example, it would become: def subfunction_1(a=None, b=None, c=None): if a is None: a=1 if b is None: b=2 if c is None: c=3 return a+b*c def subfunction_2(d=None, e=None, f=None): if d is None: d=5 if e is None: e=6 if f is None: f=7 return d*e+f def main_function(a=None, b=None, c=None, d=None, e=None, f=None): return subfunction_1(a=a, b=b, c=c) + subfunction_2(d=d, e=e, f=f) subfunction_1 may be written by someone totally different from the author of main_function, and may even be in a different codebase. For the author of subfunction_1, it makes no sense to use the "None" approach instead of python's normal default mechanism (since all arguments here are immutables). On Fri, Jul 20, 2018 at 1:30 PM, Jonathan Fine wrote:

Excellent contributions. I'm going to try to (partially) consolidate what we've got.

REVIEW ======= I'll start by reviewing the situation regarding default arguments. There are two basic patterns for default arguments.

The first is --- def fn(a=EXP): # body of function ---

The second is --- def fn(a=None): if a is None: a = EXP # body of function ---

Here, EXP is any Python expression. A fairly gotcha is to use a list, or some other mutable object, as EXP. This happens when you write --- def fn(a=[]): # body of function --- because then EXP = '[]' which will be evaluated just once, and every call fn() will be using the same list! To avoid this you should use the second pattern. I think there may be an example of this in the standard Python tutorial.

(An aside. You probably need the second pattern if you EXP is, say, ([], []). Although technically immutable, this value has mutable members. And can't be hashed, or use as a dictionary key or element of a set.)

WHEN TO USE None =================

If you want something mutable as the 'default argument' you have to use the second pattern. If your default argument is immutable then you can if you wish use the second pattern.

But you don't have to use the second pattern. When you use the second pattern, the expression f(None) means 'create for me the default argument' (and raise an exception if there isn't one).

Think about it. For immutable EXP, fn() is the same, whether fn is coded using the first pattern or the second. But the value of fn(None) depends very much on which pattern is used to code fn().

So here's the big conclusion (drum roll): === fn should be coded using the second pattern if we wish to pass None as a sentinel argument to fn. ===

SUMMARY ========= My suggestion was to use the second pattern to solve Peter O'Connor's original problem. It can be done now, but is a bit verbose, and looses useful information in help(fn).

Brice Parent's suggestion was to introduce a keyword deferred, like so --- def fn(deferred a=EXP): # body of function --- which I like to think of as a syntactic shorthand for the second pattern.

I sort of think we've now got a reasonable answer for Peter's problem. What do you think, Peter? And Brice, are you happy with my interpretation of your deferred keyword?

--- Jonathan

Ah, right, the fix_it(fcn) is a nice idea. It might also be a good idea, if we're making an external library anyway, to have a "deferred" object to avoid overloading "None" (which may mean something else than "differ argument"). I implemented the decorator here https://github.com/petered/peters_example_code/blob/master/peters_example_co..., and it can be used as: from deferral import deferrable_args, deferred @deferrable_args def subfunction_1(a=2, b=3, c=4): return a+b*c @deferrable_args def subfunction_2(d=5, e=6, f=7): return d*e+f def main_function(a=deferred, b=deferred, c=deferred, d=deferred, e=deferred, f=deferred): return subfunction_1(a=a, b=b, c=c) + subfunction_2(d=d, e=e, f=f) assert main_function() == (2+3*4)+(5*6+7) assert main_function(a=8) == (8+3*4)+(5*6+7) I still think it would be nice to have this as a built-in python feature, for a few reasons: - When using non-differable functions (say in other codebases), we have to do a bunch of "func = deferrable_args(func)" at the top of the module (or we can just do them at runtime, but then we're doing inspection every time, which is probably slow). - It adds a layer to the call stack for every deferrable function you're in. - To avoid annoying errors where you've defined an arg as deferred but forgot to wrap the function in question. On Fri, Jul 20, 2018 at 3:39 PM, Jonathan Fine wrote:

Hi Peter

You make the very good point, that

...

subfunction_1 may be written by someone totally different from the author of main_function, and may even be in a different codebase. For the author of subfunction_1, it makes no sense to use the "None" approach instead of python's normal default mechanism (since all arguments here are immutables).

Good point. To rephrase, what should we do if we want to use a third party or legacy function, which begins === def fn(a=1, b=2, c=3): # function body ===

We can solve this by defining a function decorator. Suppose we have a function fix_it, whose argument and return value are both functions. The basic specification of fix_it is that --- fixed_fn = fix_it(fn) --- is in practice equivalent to --- def fixed_fn(a=None, b=None, c=None): if a is None: a = 1 if b is None: b = 2 if c is None: c = 3 # function body for fn # or if you prefer return fn(a, b, c) ---

An aside. We can code fix_it by using https://docs.python.org/3/library/inspect.html ===

...

...

...

import inspect def fn(a=1, b=2, c=3): pass ... str(inspect.signature(fn)) '(a=1, b=2, c=3)' ===

You could also use with new code, like so: --- @fix_it def fn(a=1, b=2, c=3): # function body ---

I think this helps solve your problem. Is there, Peter, anything else that would be left to do (except, of course, write the fix_it function).

Thank you again for your problem and comments.

-- Jonathan

On Fri, Jul 20, 2018 at 04:43:56PM +0200, Peter O'Connor wrote:

I still think it would be nice to have this as a built-in python feature, for a few reasons: - When using non-differable functions (say in other codebases), we have to do a bunch of "func = deferrable_args(func)" at the top of the module (or we can just do them at runtime, but then we're doing inspection every time, which is probably slow). - It adds a layer to the call stack for every deferrable function you're in. - To avoid annoying errors where you've defined an arg as deferred but forgot to wrap the function in question.

What makes you think that a built-in deferred feature won't have exactly the same issues? Do you have an implementation that doesn't need to do intraspection? -- Steve

On Fri, Jul 20, 2018 at 05:54:44PM +0200, Peter O'Connor wrote:

On Fri, Jul 20, 2018 at 5:41 PM, Steven D'Aprano wrote:

...

What makes you think that a built-in deferred feature won't have exactly the same issues? Do you have an implementation that doesn't need to do intraspection?

I don't know about these low level things, but I assume it'd be implemented in C and wouldn't have the same cost as entering a new function in Python. I imagine it just being a small modification of the mechanism that Python already uses to assign values to arguments when a function is called. Is that not the case?

I don't know. That's why I'm asking why you think it wouldn't. -- Steve

On 20 July 2018 at 22:45, Steven D'Aprano wrote:

Perhaps you mean duplicate, or repeat, or copy. But surely they're not redefined -- then they would have different values. Being able to redefine the defaults in a wrapper function is a feature.

Putting aside the terminology, I think this is a minor annoyance: DRY violations when setting default values.

FWIW, I tend to handle this problem the same way I handle other DRY problems with magic constants: give the default value a name and either export it directly, or export an API for retrieving it. If that results in name sprawl ("But now I have 15 defaults to export!"), then I take it as a hint that I may not be modeling my data correctly, and am missing a class definition or two somewhere. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia