[Tutor] A deeper explanation of ability to modify list elements in-place

[Steven D'Aprano]

Serdar Tumgoren wrote:

>> In Python, if you have x[0] = Dummy(), the list object x stores the Dummy
>> instance itself.
> 
> 
> I think the above example gets to the source of my confusion. Clearly the
> instance remains accessible via x[0], but I somehow never thought of a
> specific list index as an obvious "symbol" for -- or reference to -- an
> instance.

Yes, that's it -- objects can be referenced by *any* , er, reference in 
any other object. Unfortunately, I'm not aware of any useful synonym for 
"reference", to distinguish the Python-specific idea of a reference to 
an object from the plain English meaning of reference, which makes it a 
little confusing to talk about these things.

Named variables are not the only references: you can create anonymous 
objects inside lists, dicts or sets, or pass instances directly to a 
function (where of course they get given a name), or by a closure:

class Dummy:
     def __init__(self):
         print("created %s" % self)
     def __del__(self):
         print("deleted %s" % self)


def func():
     x = Dummy()
     def inner():
         return x
     return inner

f = func()
del Dummy, func


Despite both the class and func being deleted, the instance x remains 
alive so long as the inner function remains alive.



 > I suspect that my confusion stems from the day-to-day routine of
> instantiating an object and assigning it to a variable at the same time,
> often inside a loop, and then performing additional processing on this
> instance *before* storing it in a list or some other container.

Of course binding objects to names is a very common use, but it's hardly 
the only one.


> Above, it seems that you're *still* performing simultaneous instantiation
> and assignment (from a code perspective), but the "symbol" that you're
> assigning to is the list index itself.

Not to the index, which would be 0, 1, 2, ... but to the list item.


> And that seems to explain why the Dummy instance remains accessible and is
> not garbage-collected. Is that a correct understanding?


Yes, you've got it!


[...]

>> There is no way to get access to the underlying pointers from pure Python
>> code. From pure Python, pointers don't exist -- Python gives you no way to
>> create or access pointers from Python code. All you have access to is
>> objects.
> 
> 
> So is a reference simply a Python-level symbol (ie some collections of
> characters accessible from Python code) that corresponds to a pointer, which
> in turn points to the location in memory where the object is stored?

Now you're digging deep into the implementation... at *some level*, some 
piece of code must know where in memory the object itself lies. But that 
level is not Python code.

It's hard to talk about these things without confusion. There are 
various levels: symbols like "x[1]" or "name.attribute" exist in source 
code. Another level is the byte-code, which has its own symbols for 
referring to objects. A third level is the implementation ("under the 
hood") of the language. In the CPython implementation, a reference will 
be a pointer to a chunk of memory (the object). But of course other 
implementations, such as PyPy, IronPython, Jython and many others may 
choose different implementations. But at the top level, no, Python has 
no idea about the memory location of objects.

It is true that CPython uses memory locations for the ID of objects, but 
that's just a label, you can't do anything with it -- you can't take an 
address and fetch the object at that address, for instance. And other 
implementations make other choices: Jython and (I think) IronPython 
number objects sequentially 1, 2, 3, 4, ... as they are created, and use 
that as the unique ID, rather than the memory address.


-- 
Steven