[Python-Dev] Alternative implementation of interning

[Guido van Rossum]

>     Guido> Better still, I think we could safely make all interned strings
>     Guido> mortal -- I don't see any use for immortal strings.
> 
> Wasn't this part of the original discussion?  Extension modules are free to
> call PyString_InternInPlace and may well expect immortal strings, so for
> backward compatibility, the functionality probably has to remain for a time,
> yes?

In core Python, there are two common usage patterns for interning.

The most common pattern uses PyString_InternFromString() to intern
some string constant that will be used as a frequent key
(e.g. "__class__") and then stores the resulting object in a static
variable.  Those strings are immortal because the static variable has
a reference that is never released.

The other common pattern uses PyString_InternInPlace() to intern a
string object (usually a function argument) that's being used as a
dictionary key or attribute name, in the hope that the dict lookup
will be faster.  In these cases, the dict will keep the interned
string alive as long as it makes sense, and when it's no longer a key
in the dict, there's no point in having the interned object around.
(It's also fairly pointless since PyObject_SetAttr() already does
this; even that seems questionable and should probably be done by the
setattro handler of individual object types.)

Making such keys mortal might cause some churning, if non-existing
keys are frequently constructed (say, from user data) and then thrown
away -- each time the key is thrown away it is removed from the
interned dict now, and each time it is recreated and used as a key, it
is interned again -- to no avail.  But I think that's pretty rare (a
non-existing key) and it certainly isn't going to cause any breakage.

I expect that the usage patterns in 3rd party extensions are pretty
much the same.

Tim once posted a theoretical example that depended on interned
strings staying alive while no user object references a particular
string object.  But that was highly theoretical.

> OF course, I'm speaking with my fake expert hat on.  I've never even
> considered interning a string, immortal, immoral, or otherwise.

:-)

--Guido van Rossum (home page: http://www.python.org/~guido/)