Counterintuitive Python behavior

[Michael Hudson]

dominikush at yahoo.com writes:

> one thing I like very much about Python is that statements
> work like you would expect them to work.

Well, Python works very much as I expect it, but it's not clear if
this says more about me or Python <wink>.

At the end of your email, you say:

> Who is wrong here: my intuition or Python (2.2)? If it's
> my intuition, how can I train my thinking about Python's
> execution model, so that my intuition get's better ;-)

It's you :) As I can't read my email at the moment[1], I have no
better way of wasting my time to hand than drawing you some ascii art.

First, some terminology.  Actually, the very first thing is some
anti-terminology; I find the word "variable" to be particularly
uphelpful in a Python context.  I prefer "names", "bindings" and
"objects".

Names look like this:

    ,-----.
    | foo |
    `-----'

Names live in namespaces, but that's not really important for the
matter at hand as the only namespace in play is the one associated
with the read-eval-print loop of the interpreter.  In fact names are
only minor players in the current drama; bindings and objects are the
real stars.

Bindings look like this:

    ------------>

Bindings' left ends can be attached to names, or other "places" such
as attributes of objects and entries in lists or dictionaries.  Their
right hand ends are always attached to objects[2].

Objects look like this:

    +-------+
    | "bar" |
    +-------+

This is meant to be the string "bar".  Other types of object will be
drawn differently, but I hope you'll work out what I'm up to.

> Take for example the use of dict.values() for dictionaries: If you
> store the result of dict.values(), and change the dictionary after-
> wards, the previously stored result remains untouched.
> 
> >>> dict = {'a':1,'b':2}

After this statement, it would seem appropriate to draw this picture:

    ,------.       +-------+
    | dict |------>|+-----+|     +---+
    `------'       || "a" |+---->| 1 |
                   |+-----+|     +---+
                   |+-----+|     +---+
                   || "b" |+---->| 2 |
                   |+-----+|     +---+
                   +-------+

> >>> list = dict.values()

Now this:

    ,------.       +-------+
    | dict |------>|+-----+|             +---+
    `------'       || "a" |+------------>| 1 |
                   |+-----+|             +---+
                   |+-----+|              /\
                   || "b" |+-----.    ,---'
                   |+-----+|     |    |
                   +-------+     `----+----.
                                      |    |
    ,------.       +-----+            |    \/
    | list |------>| [0]-+------------'   +---+
    `------'       | [1]-+--------------->| 2 |
                   +-----+                +---+

> >>> list
> [1, 2]

Which is of course, no surprise.

> >>> dict['a']=3

Now this:


    ,------.       +-------+
    | dict |------>|+-----+|             +---+
    `------'       || "a" |+-.           | 1 |
                   |+-----+| |           +---+
                   |+-----+| |            /\
                   || "b" |+-+---.    ,---'
                   |+-----+| |   |    |
                   +-------+ |   `----+----.
                             |        |    |
    ,------.       +-----+   |        |    \/
    | list |------>| [0]-+---+--------'   +---+
    `------'       | [1]-+---+----------->| 2 |
                   +-----+   |            +---+
                             |            +---+
                             `----------->| 3 |
                                          +---+


> >>> list
> [1, 2]
> >>> dict
> {'a': 3, 'b': 2}

These should also come as no surprise; just chase the arrows
(bindings) above.

> However, if a dictionary has lists as value entries, I get
> a counterintuitive behavior (which, just recently, broke my
> code): If you change the dict, the list you previously
> created via dict.values() gets automagically updated. A nice
> feature, but nothing I would have expected!

That's because you're not thinking in terms of Names, Objects and
Bindings.

> >>> dict = {'a':[1],'b':[2]}

    ,------.       +-------+
    | dict |------>|+-----+|     +-----+   +---+
    `------'       || "a" |+---->| [0]-+-->| 1 |
                   |+-----+|     +-----+   +---+
                   |+-----+|     +-----+   +---+
                   || "b" |+---->| [0]-+-->| 2 |
                   |+-----+|     +-----+   +---+
                   +-------+

> >>> list = dict.values()

    ,------.       +-------+
    | dict |------>|+-----+|             +-----+   +---+
    `------'       || "a" |+------------>| [0]-+-->| 1 |
                   |+-----+|             +-----+   +---+
                   |+-----+|               /\
                   || "b" |+-----.    ,----'
                   |+-----+|     |    |
                   +-------+     `----+-----.
                                      |     |
    ,------.       +-----+            |     \/
    | list |------>| [0]-+------------'   +-----+   +---+
    `------'       | [1]-+--------------->| [0]-+-->| 2 |
                   +-----+                +-----+   +---+

> >>> list
> [[1], [2]]

Again, no surprises here.

> >>> dict['a'].append(3)

                                                    +---+
    ,------.       +-------+                     ,->| 1 |
    | dict |------>|+-----+|             +-----+ |  +---+
    `------'       || "a" |+------------>| [0]-+-'
                   |+-----+|             | [1]-+-.
                   |+-----+|             +-----+ |  +---+
                   || "b" |+-----.         /\    `->| 3 |
                   |+-----+|     |    ,----'        +---+
                   +-------+     |    |
                                 `----+-----.
    ,------.       +-----+            |     \/
    | list |------>| [0]-+------------'   +-----+   +---+
    `------'       | [1]-+--------------->| [0]-+-->| 2 |
                   +-----+                +-----+   +---+

> >>> dict
> {'a': [1, 3], 'b': [2]}
> >>> list
> [[1, 3], [2]]

And now these should not be surprising either.

> Looks like that in the first case a copy is returned while
> in the latter case list references are returned. Ok, but
> according to Python's philosophy I shouldn't mind if I work
> with lists in the dictionary or anything else. If the
> behavior depends on the knowledge of the type of values I
> put into a dictionary, I find that somehow counterintuitive.

If you haven't realised where you're misconceptions come from from the
above pictures, I'm not sure more prose would help.

Cheers,
M.
[1] Does anyone know where the starship's gone?
[2] Anyone mentioning UnboundLocalError at this point will be shot.

-- 
  A.D. 1517: Martin Luther nails his 95 Theses to the church door and
             is promptly moderated down to (-1, Flamebait).
        -- http://slashdot.org/comments.pl?sid=01/02/09/1815221&cid=52
                                        (although I've seen it before)