[Python-Dev] GIL, Python 3, and MP vs. UP

[Paul Moore]

On 9/21/05, Josiah Carlson <jcarlson at uci.edu> wrote:
>
> Antoine Pitrou <solipsis at pitrou.net> wrote:
> >
> >
> > > The best way to make people stop complaining about the GIL and start
> > > using
> > > process-based multiprogramming is to provide solid, standardized support
> > > for process-based multiprogramming.
> >
> > 100% agreed. I needed a portable way to know if a program was already
> > running (and then to send it a simple command), the only simple solution
> > I found was to listen on a local TCP socket and store the port number in
> > a file in a known location. Not very elegant IMO.
>
> On *nix, use a unix domain socket (location in the filesystem which acts
> as a listening socket).  On Windows, you can use cTypes, pywin32, etc.,
> to create a global mutex and/or COM interface.

Yes, but that's the point - having a cross-platform solution available
"out of the box" (where the box may be the stdlib, or it may be third
party).

The problems I see are:

1. Encapsulating a suitable core feature set in a nice API
2. Making it cross-platform

With something covering both of these, a case could be made for
including it in the stdlib, to support/encourage multiprocess style
coding in place of multithreading.

I use threading quite a lot, often to spawn multiple workers all doing
"the same thing" on different targets. A simple thread pool plus
threading.Queue is perfect for this. A package which provided a
similar abstraction using worker processes would be fine for me. Of
course, *I* want it for Windows, and most multiprocess stuff tends to
be Unix specific (where multiprocessing is natural and fast). And I'm
happy enough with threads (I'm scaling up the number of blocking calls
I'm waiting on - which already release the GIL - and so I don't care
too much about SMP per se).

This is getting offtopic, though, and probably more suitable for
c.l.p, where design issues of a multiprocessing module should be
thrashed out before coming back to python-dev with a proposal for the
stdlib.

Paul.