Running Python on SMP machine

Mark Hammond MarkH at ActiveState.com
Mon Oct 9 18:00:38 EDT 2000


"Aahz Maruch" <aahz at panix.com> wrote in message
news:8rsssg$lcs$1 at panix3.panix.com...

> That's the Global Interpreter Lock.  But my experience is that
> frequently the overhead of synchronizing between two separate
processes
> overwhelms the payback you get from actual use of SMP -- that trick
works
> best if the two processes do no synchronization.

Exactly - and this sort of Python program is one that performs _worst_
on SMP machines.  No thread synchronization == more oppurtunities for
global lock conflicts.

> Furthermore, it's not that hard to beat the GIL.  If any significant
> amount of blocking I/O takes place, you'll benefit from threads.

You will.  But you will still _not_ benefit from a SMP machine for
those instants where more than one thread is ready to run in the
Python interpreter.  Even for generally IO bound programs, this can
sometimes be significant.

> Inaddition, it's certainly possible to create C extensions for
> computation that release the GIL.
>
> Overall, it's almost always worth trying threading first.

So, it is your opinion that the best solution in this case is to
re-write any CPU-bound code in his Python program in C???

> [What I sometimes wonder is, how/why did *I* get to be the threading
> champion on c.l.py?  It's not like I use threads very much, nor am I
an
> expert on them...  ;-)]

I must admit I haven't noticed, but I do hope your advice can be a
little more broad in some cases!  Threads are often at risk of the
C++-hammer/thumb syndrome - when your tool of preference is a thread,
every problem becomes threaded :-)

[Original: When the only tool you have is a hammer, everything starts
looking like a nail.  My favourite derivation: When the hammer you
have is C++, everything starts looking like a thumb.]

Mark.





More information about the Python-list mailing list