2.6, 3.0, and truly independent intepreters

[Jesse Noller]

On Wed, Oct 22, 2008 at 5:34 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> The *current* developers seem to be more interested in exploiting multiple
> processors with multiprocessing.  Note that Google choose that route for
> Chrome (as I understood their comic introduction). 2.6 and 3.0 come with a
> new multiprocessing module that mimics the threading module api fairly
> closely.  It is now being backported to run with 2.5 and 2.4.

That's not exactly correct. Multiprocessing was added to 2.6 and 3.0
as a *additional* method for parallel/concurrent programming that
allows you to use multiple cores - however, as I noted in the PEP:

"    In the future, the package might not be as relevant should the
    CPython interpreter enable "true" threading, however for some
    applications, forking an OS process may sometimes be more
    desirable than using lightweight threads, especially on those
    platforms where process creation is fast and optimized."

Multiprocessing is not a replacement for a "free threading" future
(ergo my mentioning Adam Olsen's work) - it is a tool in the
"batteries included" box. I don't want my cheerleading and driving of
this to somehow implicate that the rest of Python-Dev thinks this is
the "silver bullet" or final answer in concurrency.

However, a free-threaded python has a lot of implications, and if we
were to do it, it requires we not only "drop" the GIL - it also
requires we consider the ramifications of enabling true threading ala
Java et al - just having "true threads" lying around is great if
you've spent a ton of time learning locking, avoiding shared data/etc,
stepping through and cursing poor debugger support for multiple
threads, etc.

This is why I've been a fan of Adam's approach - enabling free
threading via GIL removal is actually secondary to the project's
stated goal: Enable Safe Threading.

In any case, I've jumped the rails - let's just say there's room in
python for multiprocessing, threading and possible a concurrent
package ala java.util.concurrent - but it really does have to be
thought out and done right.

Speaking of which: If you wanted "real" threads, you could use a
combination of JCC (http://pypi.python.org/pypi/JCC/) and Jython. :)

-jesse