[Python-Dev] Scoping vs augmented assignment vs sets (Re: 'fast locals' in Python 2.5)

[Josiah Carlson]

Boris Borcic <bborcic at gmail.com> wrote:
> Josiah Carlson wrote:
> 
> > You seem to not realize that these different use-cases.  Your new
> > example involves a global variable that is *shared* among everyone that
> > knows  about this particular module.  It also is repaired by a simple
> > insertion of 'global freebits' at the beginning of the search function. 
> 
> My point here : a simple repair, and by a statement that amounts to a compiler 
> directive with no other effect than obtaining my intent in bytecodes, so that 
> "all other things being equal" comparisons of code versions remain possible 
> (recall that I was studying the impact on working code of adopting sets - 
> including performance).

Time to access and/or modify a particular variable is only important if
you are performing the operation often enough to matter.

Have you profiled your code to show that the fetch/assignment portion of
x ^= y is slowing you down? Have you tested to see if x[0] ^= y slows
you down significantly (embed your set in a list of size 1)? Have you
tested to see if x.a ^= y slows you down significantly?

If not, then you haven't really profiled your code.  If you have, I'm
curious as to your results; as my results have shown that for the two
functions...

    def foo1(x):
        a = 0
        for i in xrange(x):
            a += 1

    def foo2(x)
        a = [0]
        def bar(x):
            for i in xrange(x):
                a[0] += 1
        return bar(x)

... and an x of 1000000, foo1(x) runs in .15 seconds, while foo2(x) runs
in .30 seconds.  So the overhead of 1 million fetch/assignment to the
first item in a list is ~.15 seconds, or 150 ns each.

Further, I can almost guarantee that your set manipulations are going to
dwarf the assignment/fetch times in either of these cases, so it seems
to me that your desired Python change, for the sake of speed as you
claim, is arbitrarily insignificant to your application, and likely so
in many applications (being roughly equivalent to the cost of an
iterator.next(), a name binding, and an int.__iadd__).


> > The closure/class example is merely a method of encapsulating state,
> > which I find easier to define, describe, and document than the closure
> > version.
> 
> Your priviledge of course (and I am not saying it is misguided, although I would 
> in particular argue that the matter gets debatable in the low LOC-count limit). 
> (I am also wondering about performance comparisons).

Unless your code profiling has shown that such assignments are a
limiting factor (if 150 ns per pass is a limiting factor, please let us
know), then performance comparisons are moot.  Further, code length is
almost immaterial to this discussion, because even if you didn't want to
bite the bullet and convert to a class-based approach, using a list of
length 1, just for the sake of set modification, would solve your
problem.

def solve(problem) :
     freebits = set(range(N))
     fb = [freebits]
     def search(data) :
         ....
         fb[0] ^= swap
     ...
     search(initial_data)
     ...


> > Back in February, there was a discussion about allowing people to
> > 'easily' access and modify variables defined in lexically nested scopes,
> > but I believed then, as I believe now, that such attempted uses of
> > closures are foolish when given classes.   Given the *trivial* conversion
> > of your closure example to a class, and my previous comments on closures
> > "I find their use rarely, if ever, truely elegant, [...] more like
> > kicking a puppy for barking: [...] there are usually better ways of
> > dealing with the problem (don't kick puppies for barking and don't use
> > closures).", you are not likely to find me agreeing with you about
> > augmented assignment and/or lexically nested scopes.
> 
> I see. Thanks for the background. Background for backround, let me just say that 
> python hadn't yet grown a lambda when I first played with it. May I read your 
> last statement as acknowledging that I am not so much asking for a door to be 
> created, than asking for a naturally builtin door, not to be locked by special 
> efforts ?

No, you may not.  There are two options for the Python compiler/runtime
behavior when faced with an augmented assignment for a name not yet
known in this particular lexical scope but known in a parent scope:
assume you meant the name in the parent scope, or assume you made a
mistake. The current state of affairs (for all released Pythons with
augmented assignment) is to assume that you meant the local scope. Why?
Because while allowing the augmented assignment would make your life
easier in this case, the error would "pass silently" when it was an
actual error (a not uncommon case), which is a bit of a no-no, and the
case when you meant the parent scope is easily worked around.

 - Josiah