[Python-Dev] Sandboxed Threads in Python

[Phillip J. Eby]

At 07:17 PM 10/7/2005 -0600, Adam Olsen wrote:
>On 10/7/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> > At 06:12 PM 10/7/2005 -0600, Adam Olsen wrote:
> > >Turns out it's quite easy and it doesn't harm performance of existing
> > >code or require modification (but a recompile is necessary).  The idea
> > >is to only use a cyclic garbage collector for cleaning them up,
> >
> > Um, no, actually.  You need a mark-and-sweep GC or something of that
> > ilk.  Python's GC only works with objects that *have refcounts*, and it
> > works by clearing objects that are in cycles.  The clearing causes
> > DECREF-ing, which then causes objects to be freed.  If you have objects
> > without refcounts, they would be immortal and utterly unrecoverable.
>
>Perhaps I wasn't clear enough, I was assuming appropriate changes to
>the GC would be done.  The important thing is it can be done without
>changing the interface that the existing modules use.

No, it can't.  See more below.


> > >That's all it takes.  Modify Py_INCREF and Py_DECREFs to check for a
> > >magic constant.  Ahh, but the performance?  See for yourself.
> >
> > First, you need to implement a garbage collection scheme that can deal with
> > not having refcounts.  Otherwise you're not comparing apples to apples
> > here, and your programs will leak like crazy.
> >
> > Note that implementing a root-based GC for Python is non-trivial, since
> > extension modules can store pointers to PyObjects anywhere they
> > like.  Further, many Python objects don't even support being tracked by the
> > current cycle collector.
> >
> > So, changing this would probably require a lot of C extensions to be
> > rewritten to support the needed API changes for the new garbage collection
> > strategy.
>
>They only need to be rewritten if you want them to provide an
>immutable type that can be transferred between sandboxes.

No.  You're missing my point.  If they are able to *reference* these 
objects, then the garbage collector has to know about it, or else it can't 
know when to reclaim them.  Ergo, these objects will leak, or else 
extensions will crash when they refer to the deallocated memory.

In other words, you can't handwave the whole problem away by assuming "a 
garbage collector".  The garbage collector has to actually be able to work, 
and you haven't specified *how* it can work without changing the C API.


>I was aware that weakrefs needed some special handling (I just forgot
>to mention it), but I didn't know it was used by subclassing.
>Unfortunately I don't know what purpose it serves so I can't
>contemplate how to deal with it.

It allows changes to a supertype's C-level slots to propagate to subclasses.