fork()

Tim Peters tim_one at email.msn.com
Fri Jun 11 12:08:51 EDT 1999


[Tim]
>> ...
>> Guido will also want a doubly-linked list (where the links don't
>> count against the refcount!) so that when a dict goes out of
>> existence via the normal refcount-hits-0 route, the dict can
>> efficiently unlink itself from the list.

[Guido Himself]
> Probably, although everytime this comes up I try to figure out a
> clever scheme to use a singly-linked list, where the zero-refcnt ones
> are recycled at the next mark-and-sweep.

When a class instance (not involved in a cycle) loses its last reference, I
assume __del__ will still get called right away (else we're combining the
worst features of all possible schemes <0.5 wink>).  This is also the time
to clear the instance's __dict__ (provided the instance __del__ doesn't
resurrect the instance), and for the same reason.  Ditto for ordinary dead
objects of other kinds holding dicts.  Provided those are true, it may not
hurt much to keep the stripped dict hanging around (I see that dict.clear()
releases the internal table memory, so all that's left is the dict header).

OTOH, straightforward M&S (without generational etc gimmicks) is quite
expensive, so you won't want to run it often; e.g., I don't think we can
afford to wait until a thread dies to finish cleaning up all the non-cyclic
trash (Perl gets away with waiting that long because it *doesn't* delay
tossing any non-cyclic trash).

>> ...
>> So, ignoring that, you make one pass over the trash list just to run
>> finalizers.

> Bzzt!  We only have a trash list of dictionaries.  The finalizers are
> typically on class instances (or extension type objects) that are somehow
> referenced by those dictionaries!

Ack, yes.  I was regurgitating the (start of the) scheme we (w/ JimF) hashed
out in email a few years ago, which tracked every object.  Ignoring the
wheels means it can't get up enough speed to fly <wink>.

> Could we get away with not calling (user) finalizers on objects in
> trash cycles at all?  Since the finalization order is problematic at
> best, this almost seems acceptable: in the formal semantics, objects
> that are part of cycles would live forever, and as an invisible
> optimization we recycle their memory anyway if we know they are
> unreachable (I've got the feeling that I've seen this rule before
> somewhere.)

The usual dodge is to say that GC is emulating infinite memory, "therefore"
the user has no right to expect finalizers to ever get called.  It's usually
used to justify not adding finalizers to a language, not to castrate the
semantics of languages that already have them <0.7 wink>.

Cycles are irrelevant to that line of argument, and if you introduce a
distinction ("finalizers in cycles don't get called, all others do") then
the "optimization" is not invisible:  cleaning up the cycle can cause
objects that are not *themselves* in a cycle to become trash, so *their*
finalizers will get called (else the distinction is as inexplicable as a
coin toss), but would not have been called had the cycle been left in
unreachable peace.

IOW, Humpty Dumpty sits on a cyclic wall.  If the wall goes away, his poor
little egg butt certainly notices, and especially if the wall doesn't get to
toss out a pillow on its way to reincarnation as an omelet pan <wink>.

> We'd still get complaints "my __del__ doesn't get called" from some
> users, and the answer would still be "it's in a cycle -- use
> an explicit close"; but those who know what they are doing can be
> guaranteed that their memory [gets] recycled.

The last clause should be stronger (if for no other reason than that those
who know what they're doing can already worm around this!):  those who
*don't* know what they're doing get a guarantee that their memory will
(eventually) get recycled.  Certainly value in that.

> In other words it would be no worse than today and for some people
> it would be better.

So far as that goes, yes.  It would, e.g., be better for me -- I like to
create an occasional cycle, and almost never write __del__ methods anyway.
It would be more convenient not to need to write & call dict-clearing
.close() methods.

Hard to weigh that against the probable increases in memory and runtime;
and, philosophically, you gotta admit it's a mess.  OTOH, the combo of
finalizers, cycles and resurrection seems to be a philosophical mess even in
languages that take it all <ahem> seriously.

Don't know, Guido!  I could live happily with it provided it didn't slow
things down much, and especially provided I don't think about the semantics
at all <wink>.  But then I'm not a __del__-lover to begin with.  Anyone
following this who is?

resurrection-should-be-reserved-to-gods-ly y'rs  - tim






More information about the Python-list mailing list