Circular refs

[Tim Peters]

[Greg Ewing and Tim tangle over lexical closures vs immortal cyclic
 trash. Tim quotes
     http://www.cosc.canterbury.ac.nz/~greg/python/lexscope.html
 thusly:

     It is still possible to create cycles if you're not careful. For
     instance, storing a local function or the value of a lambda
     expression in a local variable of a scope enclosing its definition
     will create a cycle.

 and sez that's "precisely the flavor of cycle that killed lexical closures
 3 years ago, as that's precisely the kind of thing the Scheme camp wants
 to do routinely"
]

Which isn't true!  I finally studied the patch instead of reading the docs
<wink>, and Greg's clever approach *does* stop the majority of "Scheme-like"
closure-slinging from creating cycles.  In the quote above, the module's
global scope needs to be exempted (it's also the module's local scope, but
"doesn't count" as being an enclosing scope in the above).

It appears difficult to give a precise user-friendly statement of when
cycles will and won't get created; e.g., this does create a cycle:

    def f():
        x = 1
        g = lambda: x
        return g
    k = f()

but this doesn't:

    def f():
        x = 1
        def g(): return x
        return g
    k = f()

provided they're both at top level.  OTOH, this variant of the first way
does create a cycle:

    def h():
        def f():
            x = 1
            def g(): return x
            return g
        a = f()
        return a
    k = h()

as does also:

    def h():
        def f():
            x = 1
            def g(): return x
            return g
        a = [f()]
        return a[0]
    k = h()

and, in general, code "that's safe" at top level may end up creating cycles
if it gets nested.

I expect most people would be safe most of the time, though.  At least at
first <0.5 wink> -- once it's there, people will push it beyond its original
goals, and soon enough any case of a cycle will get moaned over.

Upside:  code using this for modest little lambdas certainly looks prettier
than code abusing the default-argument mechanism.

Downside:  pystone took a 10% hit on my machine.  It may be due to the
notorious touchiness of ceval.c, but that seems a lot touchier on Guido's
favored platforms.  I expect it's more that the patch inserts a new
load-test-and-branch in the second-most (most, under -O) frequently executed
opcode; and, unfortunately, pystone has an atypically low dynamic incidence
of LOAD_FAST.  A fellow at work suggested an optimization I poo-poo'ed at
first, but one hack begets another <wink>:

1. Leave LOAD_FAST alone.

2. Add a new MAYBE_CREATE_CLOSURE opcode, whose body does what the patched
LOAD_FAST did.

3. Add smarts to compile.c, and particularly to its "optimize" function, to
analyze the targets of LOAD_FAST opcodes.  If a LOAD_FAST target name did
not appear in a "def" statement, the new check can't possibly succeed, so
there's no need to check it at runtime.  Else without flow analysis we can't
be sure, so change the LOAD_FAST to a MAYBE_CREATE_CLOSURE opcode.

In the absence of an unqualified win, how much is a partial solution to
lexical closures worth?

certain-the-python-community-will-be-in-100%-agreement<wink>-ly y'rs  - tim