[Edu-sig] teaching Python

[kirby urner]

I had the good fortune to lead a three day
immersive training in Python for scientist
technicians working with astronomical data
from Hubble.

The focus was basic, core Python, although
the workaday environment is intensively
NumPy and Pyfits related (also matplotlib).

Students found this behavior somewhat
confusing.

>>> def f(arg, y=[]):
	y.append(arg)
	return y

>>> r = f(10)
>>> r
[10]
>>> r = f(11)
>>> r
[10, 11]
>>> r.append(100)
>>> f(12)
[10, 11, 100, 12]
>>>

Here's one of the explanations for this behavior:

http://mail.python.org/pipermail/python-list/2007-March/1116072.html

In the above example, y is bound to a mutable object
at the time the function def is evaluated, not at runtime
each time the function is called.  Once this object
starts filling with data, it doesn't go out of scope but
persists between function calls -- even though "y" is not
in the globals namespace (it "lives" in the function).

It gets worse:

>>> f(11,[])
[11]
>>> f(12,[])
[12]
>>> f(13)
[10, 11, 100, 12, 13]
>>> f(14)
[10, 11, 100, 12, 13, 14]

One would think the empty list passed in as y would
"override" and/or "reinitialize" y to the empty list.
Instead, the passed in argument is bound to y at
runtime, then goes out of scope.  Meanwhile, the
object assigned to y at the time of evaluation is
still there in the background, ready for duty if no
2nd argument is passed.

>>> r = f(12, [])
>>> r
[12]
>>> r = f(r)
>>> r
[10, 11, 100, 12, 13, 14, [12]]

A metaphor I use is the "guard at the castle gate".

At the time of evaluation (when the module is compiled
and the function is defined -- whether or not it gets
called), objects get stored in a closet at the gate
entrance, and those parameters assigned to defaults
will always get those same objects out of the
closet, dust them off, and use them whenever
nothing gets passed for the parameter to "run with"
at the time the function is called.

If nothing is handed over at "call time" then use
whatever you've got in the closet, is the rule.

If the default object is mutable and nothing is passed
in, then the guard at the gate (the parameter in
question) is bound to the "closet object" and does
whatever work with that object, returning it to the
closet when done.

There's no re-evaluation or re-initialization of the
default object at the time the function is called so
if it's mutable and stuff got added or changed, then
it returns to the closet in its changed form.  It
does not "revert" to some evaluation-time value.

>>> del r
>>> def f(arg, y=[]):
	global r
	y.append(arg)
	r = y
	return y

>>> r

Traceback (most recent call last):
  File "<pyshell#68>", line 1, in <module>
    r
NameError: name 'r' is not defined
>>> r = f(10)
>>> r
[10]
>>> r = f(11)
>>> r
[10, 11]
>>> r = []
>>> r = f(11)
>>> r
[10, 11, 11]

In the above case, r has been made a global variable
inside the function.  Assigning r to the empty list
above merely rebinds it externally however, does
not affect y's default object in the closet.  When
the function is run with no argument for y, r is
rebound within the function to our growing default
list object.

>>> f(9)
[10, 11, 11, 9]
>>> r
[10, 11, 11, 9]
>>> f(12)
[10, 11, 11, 9, 12]
>>> r
[10, 11, 11, 9, 12]
>>> r = []
>>> f(12)
[10, 11, 11, 9, 12, 12]

Again, it does no good to set r to the empty list
with the expectation of reaching the y default in
the castle closet.  r is simply being rebound and
is on the receiving end for y, which, in getting
no arguments, simply reverts to using the growing
list.

At the end of the function call, however, r is bound
to the same object as y (because of r = y, with
r declared global), so we do have an opportunity
to affect the y default object...

>>> r[0]=999
>>> r
[999, 11, 11, 9, 12, 12]
>>> f(12)
[999, 11, 11, 9, 12, 12, 12]

Ta dah!

>>> r.pop(0)
999
>>> r.pop(0)
11
>>> r.pop(0)
11
>>> r.pop(0)
9
>>> r.pop(0)
12
>>> r.pop(0)
12
>>> r.pop(0)
12

The closet object has now had its members popped.  We're
back to an empty list to start with, thanks to runtime
operations:

>>> f(9)
[9]
>>> f(9)
[9, 9]

So what's a quick way to empty a list without rebinding, i.e.
we don't want to pop everything or remove one by one.  Nor
do we want to end up with a bunch of None objects.

Here's our global r:

>>> r
[9, 9]

Check it out with a new global:  we're able to delete slices:

>>> test
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> del test[0]
>>> test
[1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> del test[1:5]
>>> test
[1, 6, 7, 8, 9]

... so that must mean we're able to delete all members of a
list, without rebinding to a new list object:

>>> del test[:]
>>> test
[]

Experimenting with our function some more:

>>> f(11)
[9, 9, 11]
>>> f(12)
[9, 9, 11, 12]
>>> f(13)
[9, 9, 11, 12, 13]

So this is r before:

>>> r
[9, 9, 11, 12, 13]

And this is r after:

>>> del r[:]
>>> r
[]

... meaning the y's default object in the
castle gate closet has likewise been emptied
out, given r is bound to it.

>>> f(13)
[13]
>>> f(13)
[13, 13]

... like starting over eh?

This is safe syntax BTW:

del [][:]

In other words, it's safe to delete "all members" from
an empty list without first checking to see if the list
is empty or not.

>>> r = []
>>> del r[:]
>>> r
[]

Note that there's really no need to declare r global in
order to reach in and change y, provided there's binding
going on at the time of return:

>>> del r
>>> def f(arg, y=[]):
	y.append(arg)
	return y

>>> f(10)
[10]
>>> f(11)
[10, 11]
>>> f(12)
[10, 11, 12]
>>> r

Traceback (most recent call last):
  File "<pyshell#148>", line 1, in <module>
    r
NameError: name 'r' is not defined
>>> r = f(13)
>>> r
[10, 11, 12, 13]
>>> del r[:]
>>> r
[]
>>> f(10)
[10]
>>> f(11)
[10, 11]

Kirby