[Python-ideas] Fixing class scope brainstorm

[Guido van Rossum]

On Mon, Mar 26, 2018 at 6:33 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 26 March 2018 at 01:50, Guido van Rossum <guido at python.org> wrote:
> > In the PEP 572 threads there's some grumbling about class scopes.
> >
> > Here's a random brainstorm. How about we change the scoping rules so that
> > only functions defined with 'def' (or inside one of those) cannot see the
> > class scope, and comprehensions and lambdas treat the class scope as an
> > outer scope so they can close over class variables?
> >
> > Not sure what should happen to nested classes, they should probably be
> > lumped in with the 'def' scopes.
>
> I think it's mainly in comprehensions that folks may find the current
> behaviour surprising, as that's the case where we define a function
> and immediately call it, so it mostly looks like regular inline code
> execution.
>
> Once explicitly delayed invocation is involved (lambdas, def
> statements), folks seem to be more comfortable with the idea "Oh, of
> course that's going to behave like a method at class scope". Generator
> expressions also mostly get a pass, since the *iteration* is delayed,
> even though the generator-iterator itself is created immediately.
>
> One possibility that has occurred to me (but I've never investigated
> to check the technical feasibility) is to see whether we might be able
> to define a notion of "implied arguments" for the implicit
> comprehension and generator expression function definitions.
>
> The gist of that idea would be:
>
> - when walking the symbol table for a comprehension or genexp, keep
> track of every variable name mentioned in that subtree which is not
> resolved in that subtree (which we already have the machinery to do,
> as it's needed for compiling functions in general)
> - treat all those names as *implied arguments* to the implicit
> function, and call it with the right references in the right positions
>
> While we'd likely still want to keep evaluation of the outermost
> iterator outside the nested scope for the sake of better genexp
> tracebacks, that would still be enough to enable cases like:
>
>     class C:
>         y = 1
>         values = [x+y for x in range(10)]
>
> The trick would be to make it so that that comprehension gets expanded as:
>
>     def _listcomp(_outermost_iter, y):
>         result = []
>         for x in _outermost_iter:
>             result.append(x+y)
>         return result
>
>     _listcomp_result = _listcomp(range(10), y)
>
> One of the more attractive aspects of this possibility (assuming it
> can be made to work) is that it isn't even a special case in the
> runtime name lookup rules - it's just changing the definition of the
> implicit functions that we create for comprehensions and generator
> expressions to make sure that they can refer to *any* name that
> resolves in the namespace where they're called (even
> invisible-to-the-compiler names like those injected via __prepare__
> methods, or assignments via locals() at class scope).
>
> Cheers,
> Nick.
>
> P.S. I suspect we'd also find that this served as a performance
> optimisation, since it wouldn't only prebind class locals, it would
> prebind references to builtins and module globals as well.
>

Honestly that sounds way too complex. In my ideal world, only functions
defined with 'def' are methods whose __get__ auto-binds the first argument
(unless @classmethod or @staticmethod is present), and do not get to "see"
the class scope. This is how you define methods after all. Lambdas used at
class scope do not get this benefit, and they see the class scope as just
another closure; ditto for comprehensions and genexprs.

I don't think that the "is the call delayed" idea is the proper way to
distinguish the two cases here. IMO the key concept is "is it used to
define a method". I betcha if you see a lambda here it's because there's
some calculation going on whose result will be stored as a class variable.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20180326/189b4523/attachment-0001.html>