[Tutor] Generator next()

[Steven D'Aprano]

On Sun, Dec 29, 2013 at 01:57:31AM -0500, Keith Winston wrote:

> Also: in the timer function, it has a series of attribute assignments to
> 0/None after the inner function definition... from the behaviour, I assume
> those are applied once, the first time the timer function is called
> wrapping a new method/function, and then not again, but that doesn't really
> make sense. Again, I don't really have a well-formed question, maybe that's
> clear enough?

Here's the code in question, stripped to the absolutely bare minimum, 
and with a change of name:

def decorator(function):
    def inner(*args, **kwargs):
        result = function(*args, **kwargs)
        inner.count += 1
        return result
    inner.count = 0
    return inner


Let's see what happens when you call "decorator" with some function as 
argument:

def spam(n):
    return "spam"*n

f = decorator(spam)

What does this function "f()" do? Let's go through the process step by 
step.

When we call decorator(spam), we enter the decorator() function with the 
local variable "function" set to the argument spam. The next thing that 
happens is that decorator() creates a new function called "inner". 
Inside "inner", stuff happens, but we don't care about that yet. (We'll 
come back to that in a moment.) The "inner" function is created, and 
then decorator() adds a new attribute to it:

    inner.count = 0

Functions, being objects, can have attributes. We can add our own. We 
use this attribute to keep track of how many times the function gets 
called.

Then decorator() returns this newly created function object, complete 
with its attribute count=0. This new function object is assigned to the 
variable "f".

What happens when we call f()?

Here's the code for "f" a.k.a. "inner", again, stripped of the outer 
layer:

    def inner(*args, **kwargs):
        result = function(*args, **kwargs)
        inner.count += 1
        return result

Notice that this is a *closure* (remember them from my previous email?). 
There are two non-local variables inside this inner function, one called 
"function" and one called "inner". "function" is the original spam 
function, the argument passed to the outer "decorator" function. "inner" 
is also a closure: it is the inner function itself. So inside the 
"inner" function, we have five variables:

- local variable "args", which collects positional arguments;

- local variable "kwargs", which collects keyword arguments;

- non-local variable "function", set to spam (the argument to the
  outer "decorator" function);

- local variable "result", set to the result of calling "function"
  with the collected arguments;

- and finally non-local variable "inner", which is set to the 
  inner function itself.


So calling f() collects all the arguments (if any) into parameters args 
and kwargs. Then spam gets called with those arguments, and the result 
saved. Then the count attribute is incremented, and finally the result 
returned.

All this can be a bit tricky to keep track of, because the one function 
can be known by different names depending on where in the code you are 
looking. Outside of the decorator function, we have a function called 
"spam". Inside the decorator, it is known as "function". Inside the 
decorator, there is a nested function called "inner". Outside of the 
decorator, it is known as "f".

We say that f (a.k.a. inner) is a wrapper function around spam. 
Alternatively, we say that f "decorates" spam. The special decorator 
syntax:

@decorator
def spam(n):
    return "spam"*n


is a short-cut for this:

def spam(n):
    return "spam"*n

spam = decorator(spam)


This may be a lot to digest in one sitting. Factory functions, 
decorators, closures, they're all fantastically useful and powerful 
features of Python, but they can be a bit tricky for people to wrap 
their brains around.

[more to come]



-- 
Steven