[Python-ideas] several different needs [Explicit variable capture list]

[Jim J. Jewett]

On Tue, Jan 26, 2016 at 3:59 PM, Andrew Barnert <abarnert at yahoo.com> wrote:
> On Jan 26, 2016, at 11:40, Jim J. Jewett <jimjjewett at gmail.com> wrote:

>> (1)  Auxiliary variables

>>    def f(x, _len=len): ...

>> This is often a micro-optimization;

> When _isn't_ it a micro-optimization?

It can improve readability, usually by providing a useful rename.

I have a vague sense that there might be other cases I'm forgetting,
simply because I haven't had much use for them myself.

> I think if it isn't, it's a very different case, e.g.:
>
>     def len(iterable, _len=len):
>         if something(iterable): special_case()
>         else: return _len(iterable)

I would (perhaps wrongly) still have assumed that was at least
intended for optimization.

> Obviously non-optimization use cases can't be solved
> by an optimizer. I think this is really more a special case
> of your #4 ...

[#4 was current-value capture]

I almost never set out to capture a snapshot of the current
environment's values.  I get around to that solution after being
annoyed that something else didn't work, but it isn't the original
intent.  (That might be one reason I sometimes have to stop and think
about closures created in a loop.)

The "this shouldn't be in the signature" and "why is something being
assigned to itself" problems won't go away even if current-value
capture is resolved.  I suspect current-value capture would even
become an attractive nuisance that led to obscure bugs when the value
was captured too soon.

>> But realistically, that _len isn't ugly *just* because it shouldn't be
>> overridden; it is also inherently ugly.  I would prefer that something
>> like Victor's FAT optimizer just make this idiom obsolete.

> But, like most micro-optimizations, you should use this
> only when you really need it. Which means you probably
> can't count on a general-purpose optimizer that may do it
> for you, on some people's installations.

That still argues for not making any changes to the language; I think
the equivalent of (faster access to unchanged globals or builtins) is
a better portability bet than new language features.


> Also, marking that you're using an intentional
> micro-optimization is useful, even (or maybe especially)
> if it's ugly: it signals to any future maintainer that
> performance is particularly important here, and they
> should be careful with any changes.

Changing the language to formalize that signal takes away
some of the emphasis you get from ugliness.  I also wouldn't
assume that such speed assessments are likely to be valid
across the timescales needed for adoption of new syntax.


>> (2)  immutable bindings

>> once X
>> final Y
>> const Z

> But a default value neither guarantees immutability,
> nor signals such an intent. Parameters can be rebound
> or mutated just like any other variables.

It is difficult to signal "once set, this should not change"
in Python, largely because it is so difficult to enforce.

This case might actually be worth new syntax, or a keyword.

Alternatively, it might be like const contagion, that ends
up being applied too often and just adding visual noise.

>> So again, I think something like Victor's FAT optimizer (plus comments
>> when immutability really is important) is a better long-term solution,
>> but I'm not as sure as I was for case 1.

> How could an optimizer enforce immutability, much less signal it?

Victor's guards can "enforce" immutability by recognizing when it
fails in practice.  It can't signal, but comments can ... and
immutability being semantically important (as opposed to merely useful
for optimization) is rare enough that I think a comment is more likely
to be accurate than a type declaration.

>> (3)  Persistent storage

>>    def f(x, _cached_results={}): ...

>> I still think it might be nice to just have a way of easily opening a
>> new scope ...

> You mean to open a new scope _outside_ the function
> definition, so it can capture the cache in a closure, without
> leaving it accessible from outside the scope? But then f won't
> be accessible either, unless you have some way to "return"
> the value to the parent scope. And a scope that returns
> something--that's just a function, isn't it?

It is a function plus a function call, rather than just a function.
Getting that name (possible several names) bound properly in the outer
scope is also beyond the abilities of a call.  But "opening a new
scope" can start to look a lot like creating a new class instance,
yes.


> Meanwhile, a C-style function-static variable isn't really
> the same thing. Statics are just globals with names nobody
> else can see. So, for a nested function (or a method) that
> had a "static cache", any copies of the function would all
> share the same cache, while one with a closure over a
> cache defined in a new scope  (or a default parameter value,
> or a class instance) would get a new cache for each copy.
> So, if you give people an easier way to write statics, they'd
> still have to use something else when they want the other.

And explaining when they want one instead of the other will still be
so difficult that whichever is easier to write will become an
attractive nuisance, that would only cause problems under load.

-jJ