[Python-ideas] func attribute & default arg

[spir]

Hello,

I think there is some confusion on the topic of static variables in relation to default arguments and function attributes. Or, maybe, my point of view is different from the one of the majority on python-ideas. (a rather long post)

To sum up, I think the core of the issue is not about static/dymamic evaluation, or compile time/run time; rather it lies in the fact that python internally does not separate a default *value* from the corresponding local *variable*.

What may be, for the programmer, the meaning and purpose of a default argument? I think there is nothing misleading in the fact that it is evaluated at definition time. Instead, this is precisely the way we see them, otherwise there would be no gotcha and we would not discuss. This semantics is properly denoted, I guess, by a default beeing part of the function definition's headline:
   def f(arg, default=value):
      .......

Once set, a default becomes a kind of constant attribute of the function. So that the gotcha happens precisely when this expectation is deceived because the local variable called 'default' may change back the previously defined value.

To address this issue, having a default value recomputed on every call is not only overkill, but also semantically aside the point. What we need is only *caching* a default value while the function def is first evaluated.
A possible implementation (probably requiring few changes) may be to add a kind of __defaults__ dict attribute to functions. This would be filled at definition time and used at call time. Even if probably not needed, for python provides better ways, having it as an available attribute also lets programmers tweak a func's guts if they like it -- e.g. for runtime updatable defaults:
   func.__defaults__["default_arg"] = newDefaultValue()

With this semantics, we are free to use and update local variables safely, without worrying of changing the default value.



Here is a short and probably non-exhaustive review of common uses of defaults.

-1- real default arg which value is immutable
No issue.

-2- real default arg which value is mutable but not updated in func body
No issue. 
With the above proposal, we do not need to worry anyway; and we can change the code later safely to case -3-. I find this important.

-3- real default arg which value is mutable and updated in func body
Present implementation: gotcha.
Proposal: default argument and local var point to distinct values. No more issue.
Possibly use a __defaults__ dict attribute.

-4- default used as static var
A common use of mutable defaults is a workaround for python seemingly not having static variables. Actually, python programmers have them in the form of function attributes. Since this feature seems +/- unknown (undocumented?), some programmers write for instance:

   def f(arg, l=[]):
      <update l>

while the following better expresses the semantics, imho, because l is not a real argument:

   def f(arg):
      <update f.l>
   f.l = []

Generators can also be used in cases where the purpose of the func is (mainly) to return successive values. 
   def f(arg):
      l = []
      while True:
          <update l>
          yield something


-5- func factory
In the case of a factory, default arguments are used as function definition *parameters*. Similarly to the previous case, they are rather function attributes, with the difference that this time they are constant ones.
Anyway, the proposal does not change the semantics here because each generated function has its own parameter.
Still, a common idiom using a dummy default, such as:

    def paramPower(exponent):
        def power(number, exponent=exponent):
            return number ** exponent
        return power
        
would be better expressed, without any trick:

    def paramPower(exponent):
        def power(number):
            return number ** power.exponent
        power.exponent = exponent
        return power

-6- variable defaults
If we want a runtime evaluation of a default argument, we usually express it in the function body and use a sentinel. This would not change, and anyway it seems to make the intent clearer than in the case where the default would be silently recomputed (read: "explicit better than implicit"):

    from random import random   # provides runtime default
    number_count = 3            # may also be variable
    UNDEF = object()
    def f(i, numbers=UNDEF):
        # case numbers not provided: must be evaluated at runtime
        if numbers is UNDEF:
            numbers = [int(random()*10) for i in range(number_count)]
        # change pointed number
        numbers[i] = int(random()*10)
        print numbers
    numbers = [1,2,3]
    f(0, numbers) ; f(2, numbers) ; f(0) ; f(2)
==>
    [8, 2, 3]
    [8, 2, 2]
    [5, 3, 5]
    [8, 4, 1]

Using the proposal of a default attribute, an alternative would be to re-compute the value in calling code -- only when needed:
    def f(i, numbers=[random() for i in range(number_count)]):
        # change pointed number
        numbers[i] = random()
        print numbers
    .......
    if cond:
        f.__defaults__['numbers'] = [random() for i in range(number_count)]
    f(i)



A transition phase could be planned toward cached defaults, possibly with proper warnings:

"Warning: use of a mutable default value updated in the function body.
From python 3.x, default values will be cached so as not to change between calls. If your intention is instead to have a kind of static variable, rather set an attribute on the function."

Now, why are func attributes rarely used; while, as shown above, they provide a clear and handy way to express several semantic patterns -- and defaults are (mis)used instead? I wonder whether we could make them more obvious using a dedicated syntax.
In the thread about default arguments, a proposed syntax, to have defaults dynamically reevaluated at each call (which I find *wrong*), was to reuse the keyword "yield". The meaning of this idiom is rather for me the creation of some data at definition time. As a consequence, I found it a rather good possibility to denote the creation of a func attribute. :

    def f(arglist) yield attr=val:

for more attributes, reuse import-like syntax:

    def f(arglist) yield attr1=val1, attr2=val2...:

Having them defined outside the parens properly differenciate arguments from attributes (and the content of parens is already complicated enough with defaults and '*' and '**' thingies).
Also, the "yield keyword not only suggests the creation of data, but also is used in contexts (generators) where state is kept between function calls.

For instance, the following code using a func with a custom attribute:

    startup_numbers = [4, 5]
    def f(number, sorting=False):
        f.numbers.append(number)
        if sorting:
            f.numbers.sort()
        print f.numbers
    f.numbers = startup_numbers

could be expressed with a func def header as follows:

    def f(number, sorting=False) yield numbers=startup_numbers:

without the need to separately set the attribute on the func.
Sure, we do not need that at all. Moreover, when the func happens to be a method instance, then the attribute better fits on the instance object. I'm just looking for a way to let programmers be (more) aware of func attributes. 
To have cached default solving the common gotcha, we need python programmers to use defaults only as defaults, and use other solutions for semantics such as static vars. So that the transition be easier.

Denis
------
la vita e estrany