[Python-ideas] 'Injecting' objects as function-local constants

[Nick Coghlan]

On Mon, Jun 13, 2011 at 5:11 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> Function parameters should be kept for actual arguments, not for optimizing
> name look-ups.

Still, the post-** shared state (Jan's option 2) is likely the most
obvious way to get early binding for *any* purpose without polluting
the externally visible parameter list. Several questions that are
unclear in the general case of definition-time code are resolved in
obvious ways by that approach:

Q. When is the code executed?
A. At definition time, just like default argument values

Q. Where are the results of the calculation stored?
A. On the function object, just like default argument values

Q. How does the compiler know to generate local variable lookups for
those attributes?
A. The names are specified in the function header, just like public parameters

Q. What is the advantage over custom classes with __call__ methods?
A. Aside from the obvious speed disadvantage, moving from a function
with state that is preserved between calls to a stateful class that
happens to be callable is a surprisingly large mental shift that may
not fit well with the conceptual structure of a piece of code. While
*technically* they're the same thing (just expressed in different
ways), in reality the difference in relative emphasis of algorithm vs
shared state can make one mode of expression far more natural than the
other in a given context.

    class DoAndRemember():
        def __init__(self):
            self.mem = collections.Counter()

       def __call__(self, val, verbose=False):
           result = do_something(val)
           self.mem[val] += 1
           if verbose:
               print('Done {} times for {!r}'.format(self.mem[val], val))

    do_and_remember = DoAndRemember()

Custom classes also suffer grievously when it comes to supporting
introspection (e.g. try help() or inspect.getargspec() on the above)
and lack natural support for other features of functions (such as easy
decorator compatibility, descriptor protocol support, standard
annotations, appropriate __name__ assignment).


Q. What is the advantage over using an additional level of closure?
A. This is actually the most viable alternative, since the conceptual
model is quite a close match and it doesn't break introspection the
way a custom class does. The problems with this approach are largely
syntactic:

   def _make_do_and_remember():
       mem=collections.Counter()
       def do_and_remember(val, verbose=False):
           result = do_something(val)
           mem[val] += 1
           if verbose:
               print('Done {} times for {!r}'.format(mem[val], val))
       return do_and_remember
  do_and_remember = _make_do_and_remember()

1. The function signature is buried inside "_make_do_and_remember"
(the class approach and even PEP 3150 have the same problem)
2. The name of the function in the current namespace and its __name__
attribute have been decoupled, require explicit repetition to keep
them the same
3. This is basically an unreadable mess

I'd actually be far happier with the default argument hack equivalent:

       def do_and_remember(val, verbose=False, *, _mem=collections.Counter()):
           result = do_something(val)
           _mem[val] += 1
           if verbose:
               print('Done {} times for {!r}'.format(_mem[val], val))

All a "persistent state" proposal would do is create an alternative to
the default argument hack that doesn't suffer from the same problems:

       def do_and_remember(val, verbose=False, **, mem=collections.Counter()):
           result = do_something(val)
           mem[val] += 1
           if verbose:
               print('Done {} times for {!r}'.format(_mem[val], val))

It seems like the path of least resistance to me - the prevalence of
the default argument hack means there's an existing, widespread
practice that solves real programming issues, but is flawed in some
ways (specifically, messing with the function's signature). Allowing
declarations of shared state after the keyword-only arguments seems
like a fairly obvious answer.

The one potential trap is the classic one with immutable nonlocal
variables that haven't been declared as such (this trap also applies
to any existing use of the default argument hack): reassignment will
*not* modify the shared state, only the name binding in the current
invocation.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia