[Python-ideas] 'Injecting' objects as function-local constants

[Jan Kaliszewski]

Terry Reedy dixit (2011-06-11, 16:09):

> On 6/11/2011 9:30 AM, Jan Kaliszewski wrote:
[...]
> >In all that cases (and probably some other too) that technique appears
> >to be quite useful.
> >
> >
> >== The problem ==
> >
> >...is that it is not very elegant. We add arguments which:
> >a) mess up function signatures (both in the code and in auto-generated docs);
> >b) can be incidentally overriden (especially when a function has an "open"
> >    signature with **kwargs).
> 
> One problem with trying to 'fix' this is that there can be defaulted args
> which are not intended to be overwritten by users but which are intended to
> be replaced in recursive calls.

I think this is another case... Although I can imagine that such 'private'
arguments could be specified when calling -- after **{...}/bare **, e.g.:

    fun(1, b=3, **{'c':3}, my_secret_hidden_arg='xyz')
    fun(1, b=3, **, my_secret_hidden_arg='xyz')

Though at the first sight I don't like this (`after-** args in calls') idea
so much (contrary to `after-** args in definitions' idea).

[...]
> >2. (which personally I would prefer)
> >To add `dummy' (or `hidden') keyword arguments, defined after **kwargs
> >(and after bare ** if kwargs are not needed; we have already have
> >keyword-only arguments after *args or bare *):
> >
> >     def do_and_remember(val, verbose=False, **, mem=collections.Counter()):
> >         ...
> 
> I thought of this while reading 'the problem'. It is at least
> plausible to me.
[...]

Arnaud Delobelle dixit (2011-06-11, 21:47):

> On 11 Jun 2011, at 14:30, Jan Kaliszewski wrote:
[...]
> > 3.
> > To provide a special decorator, e.g. functools.within:
> >    @functools.within(mem=collections.Counter())
> >    def do_and_remember(val, verbose=False):
> >        ...
> 
> That's hard to do as (assuming the function is defined at the global
> scope), mem will be compiled as a global, meaning that you will have

Here mem is a keyword argument, not a variable. Though I understand that
making it local/closure would need some code/closures hacking... Unless
built in to the interpreter.

> to modify the bytecode.  Oh but this makes me think about something I
> wrote a while ago (see below).
> 
> 
> 4. Use closures.
> 
> def factory(mem):
>      def do_and_remember(val, verbose=False)
>          result = do_something(val)
>          mem[val] += 1
>          if verbose:
>              print('Done {} times for {!r}'.format(mem[val], val))         ....
>      return do_and_remember
> do_and_remember = factory(mem=collections.Counter())
> 
> Added bonus: you can create many instances of do_and_remember.

Yes, but this method makes code longer and more complex.
And simple is better :)

Consider my multi-factory example:

    def make_my_callbacks(callback_params):
        my_callbacks = []
        for params in callback_params:
            def fun1(*args, **kwargs, params=params):
                "...do something with args and params..."
            def fun2(*args, **kwargs, params=params):
                "...do something with args and params..."
            def fun3(*args, **kwargs, fun1=fun1, fun2=fun2):
                """...do something with args and with functions fun1, fun2,
                for example pass them as callbacks to other functions..."
            my_callbacks.append((fun1, fun2, fun3))
        return my_callbacks

...compared to:

    def make_fun1(params):
        def fun1(*args, **kwargs):
            "...do something with args and params..."
        return fun1

    def make_fun2(params):
        def fun2(*args, **kwargs):
            "...do something with args and params..."
        return fun2

    def make_fun3(fun1, fun2):
        def fun3(*args, **kwargs):
            """...do something with args and with functions fun1, fun2,
            for example pass them as callbacks to other functions..."
        return fun3

    def make_my_callbacks(callback_params):
        my_callbacks = []
        for params in callback_params:
            fun1 = make_fun1(params)
            fun2 = make_fun2(params)
            fun3 = make_fun3(fun1, fun2)
            my_callbacks.append((fun1, fun2, fun3))
        return my_callbacks

Though, maybe it'a a matter of individual taste...

> Related to this, here's a "localize" decorator that I wrote some time
> ago for fun (I think it was from a discussion on this list).  It was
> for python 2.x (could easily be modified for 3.x I think, it's a
> matter of adapting the attribute names of the function object).  It
> "freezes" all non local variables in the function.  It's a hack! It
> may be possible to adapt it.
> 
> def new_closure(vals):
>     args = ','.join('x%i' % i for i in range(len(vals)))
>     f = eval("lambda %s:lambda:(%s)" % (args, args))
>     return f(*vals).func_closure
> 
> def localize(f):
>     f_globals = dict((n, f.func_globals[n]) for n in f.func_code.co_names)
>     f_closure = ( f.func_closure and
>                   new_closure([c.cell_contents for c in f.func_closure]) )
>     return type(f)(f.func_code, f_globals, f.func_name,
>                    f.func_defaults, f_closure)

Nice :) (and, as far as I understand, it could be used to implement the
decorator I ment).

Best regards.
*j