Message queues [Re: Best processor (i386) for Python performance?]

[David Bolen]

Ville Vainio <ville at spammers.com> writes:

> >>>>> "David" == David Bolen <db3l at fitlinxx.com> writes:
> 
>     David> I do think it can be tricky to determine just what case an
>     David> application falls into (and many oscillate between I/O and
>     David> CPU bound modes), and indeed a purely CPU bound Python
>     David> application (if in Python code and not a well-behaving
>     David> extension module) isn't going to be helped at all.
> 
> The sensible thing to do then is to use multiple processes, not just
> multiple threads. Many Python apps use Queue.Queue anyway, and such an
> approach is often easy to convert over to use processes instead of
> threads.

Well, "sensible" may depend on your needs and environment.  I'm far
less a fan of multi-process situations under Windows than I am under
Unix systems for example.  In Windows process creation is far less
efficient, and proper parent/child relationships don't always work
properly (particularly when it comes to killing processes off) and
such.  Threading, on the other hand, just plain works extremely well,
at least on the WinNT/2K/XP variants.  That's almost backwards to the
way I feel about things under Unix, where the various thread
implementations and support for them on different systems can make
separate processes more attractive.

But you're right that multi-process solutions are certainly something
to keep in the toolbox as available options.

-- David