[Tutor] A deeper explanation of ability to modify list elements in-place

[Steven D'Aprano]

Serdar Tumgoren wrote:
> Hi folks,
> I'm trying to gain a deeper understanding of why it's possible to modify
> list elements in-place *without* replacing them. For instance, why is the
> below possible?

Why? Because the designer of Python, Guido van Rossum, wanted it to be 
possible, and he designed the language so that it would be.

Do you perhaps mean *how* is it possible?


>>>> class Dummy(object):
> ...     pass
> ...
>>>> a = Dummy()
>>>> b = Dummy()
>>>> x = [a, b]
> # modify list elements in-place
>>>> x[1].name = 'The Dude'
>>>> print x[1].name
> The Dude

I don't understand why you are surprised by this behaviour. Python is 
not a purely functional language like Haskell, where data is always 
immutable. Most Python objects are mutable -- they can be modified in 
place. (Although the exceptions include some of the most commonly used 
built-in objects like ints, tuples, and strings.)

Would you be surprised by this behaviour?

b = Dummy()
b.name = 'The Dude'
print b.name
=> prints 'The Dude'

If not, then why should it make any difference whether you refer to the 
instance via the name "b" or via the list item x[0]? They both refer to 
the same object.

Objects can have many different references to it:

x = [None, None]  # placeholders
y = [None]
d = {}
a = b = c = x[0] = x[1] = y[0] = d['key'] = Dummy()

Now the names "a", "b", "c", the list items x[0], x[1], y[0], and the 
dictionary item d['key'] all refer to the same object. You can create 
new names on the fly:

d = c
e = x[0]
for item in x:
     pass
d['another key'] = y[0]

or delete them:

del a, c, x[1], d['key']


[...]
> I figured the below quote offers the beginning of an explanation on this
> page:
> "The list object stores pointers to objects, not the actual objects
> themselves."
> http://effbot.org/zone/python-list.htm

The use of pointers is an implementation detail for efficiency, and it 
refers to how Python's object model is implemented in C.

It is important not to lose sight of the distinction between the 
semantics of Python objects themselves, and the way that those objects 
are implemented in lower-level languages such as C.

In Python, if you have x[0] = Dummy(), the list object x stores the 
Dummy instance itself. That's what the code *means* -- create an 
instance of the Dummy class and store it in the list x. But of course 
for efficiency reasons, and ease of programming, the most obvious 
implementation will be to place the instance in the heap somewhere, and 
keep a pointer to it in the list. How else could you do it? You could 
store the entire object in the list, but that would be terribly inefficient.

But it could be done -- you could, with difficulty, implement Python in 
a language like Fortran 77 (which has no dynamic memory allocation, and 
therefore no pointers), or in Haskell, which is purely functional. Or 
you could come up with a (slow, inefficient) implementation which moved 
around large instance objects instead of small pointers.

There is no way to get access to the underlying pointers from pure 
Python code. From pure Python, pointers don't exist -- Python gives you 
no way to create or access pointers from Python code. All you have 
access to is objects.


-- 
Steven