[Python-Dev] Active Objects in Python

[Phillip J. Eby]

At 11:18 AM 9/27/2005 -0600, Bruce Eckel wrote:
>Yes, defining an class as "active" would:
>1) Install a worker thread and concurrent queue in each object of that
>class.
>2) Automatically turn method calls into tasks and enqueue them
>3) Prevent any other interaction other than enqueued messages

#3 is the sticky bit.  Enforcing such restrictions in Python is 
*hard*.  Each active object would have to have its own sys.modules, for 
example, because modules and classes in Python are mutable and use 
dictionaries, and modifying them from two thread simultaneously would be fatal.

For built-in types it's not so bad, so they could be shared...  except for 
the fact that they have reference counts, which need lock protection as 
well to avoid fouling up GC.  So really, this idea doesn't really help with 
GIL removal at all.

What it *does* help with, is effective use of multiprocessor machines on 
platforms where fork() is available, if the API works across processes as 
well as threads.


>So yes, the best way for this to work might be some kind of enforced
>implementation -- but forcing you to do something is not particularly
>Pythonic,

But keeping the interpreter from dumping core due to program bugs *is* 
Pythonic, which is why the GIL would need to stay.  :)


>so I would think that it would be possible to build an
>active object framework along with some checking tools to warn you if
>you are breaking the rules.

Well, you could pickle and unpickle the objects you send from one function 
to another, and for cross-process communication, you'll need to do 
something like that anyway, or else use that shared-memory objects 
thing.  PySHM?  I don't remember its name, but it's an extension that lets 
you store Python objects in shared memory and use them from multiple 
processes, modulo certain strict limitations.


>  But what's not clear is whether this would
>require knowledge of the innards of the Python interpreter (which I
>don't have) or if it could be built using libraries.

If you're not trying to get rid of the GIL or truly "enforce" things, you 
can do everything you need for CSP in plain old Python.  peak.events has a 
generator-based microthread facility that's CSP-ish, for example, although 
it doesn't have decorators for async methods.  They're not hard to write, 
though; Chandler has some for methods that get called across thread 
barriers now.  That is, methods that get called from Chandler's UI thread 
but are run in the Twisted reactor thread.  I've occasionally also thought 
about implementing async C#'s "chord" features in Python for peak.events, 
but I haven't actually had a use case that needed that much generality yet.