Multiple threads

[Michael Hunter]

On Wed, Nov 16, 2011 at 9:27 AM, Dave Angel <d at davea.name> wrote:
> On 11/16/2011 12:00 PM, Jack Keegan wrote:
>[...] Processes [...] and the OS is generally better at scheduling them than it is at
> scheduling threads within a single process.  If you have multiple cores, the
> processes can really run simultaneously, frequently with very small
> overhead.  [...]

Maybe you are trying to simplify things but in a lot of cases this is
just false.  In at least some operating systems these days a thread is
the basic unit that is scheduled.  Processes are thread containers
that provide other things (fds, separate address space, etc.).  The
comment about multiple cores can be extended to multiple threads on a
core (CMT) but applies to threads as well as processes.  Switching
between processes tends to be heavier weight then switching between
threads in a process because of the needs to change the address space.

Just because Python sucks at threads doesn't make them heavier for the OS.

That doesn't mean you shouldn't use multiprocessing.  The problem
asked about seems a good fit to me to a single python process starting
and managing a set of external converter processes.

              Michael