Parallelization on muli-CPU hardware?
exarkun at divmod.com
exarkun at divmod.com
Tue Oct 12 15:37:35 CEST 2004
On Tue, 12 Oct 2004 11:07:57 +0200, "Nicolas Lehuen" <nicolas.lehuen at thecrmcompany.com> wrote:
> "Neil Hodgson" <nhodgson at bigpond.net.au> a =E9crit dans le message de =
> news:1pDad.22529$5O5.5713 at news-server.bigpond.net.au...
> > Luis P Caamano:
> > > Therefore, fixing the GIL was always in competition with existing =
> > > and libraries, and it always lost because it was (and still is)
> > > considered "not worth the effort."
> > Greg Stein implemented a "free-threaded" version of Python in 1996 =
> > had another look in 2000.
> > http://python.tpnet.pl/contrib-09-Dec-1999/System/threading.README
> > http://mail.python.org/pipermail/python-dev/2000-April/003605.html
> > So, it is not the initial implementation of a GIL-free Python that =
> > the stumbling block but the maintenance of the code. The poor =
> performance of
> > free-threaded Python contributed to the lack of interest.
> Yeah, and the poor performance of the ActiveState Python for .NET =
> implementation led them to tell that the CLR could not be efficient in =
> implementing Python. Turns out that the problem was in the =
> implementation, not the CLR, and that IronPython gives pretty good =
> It's not because an implementation gives poor performance that the whole =
> free-threaded concept is bad.
> Plus, the benchmarks is highly questionable. Compare a single threaded =
> CPU intensive program with a multi-threaded CPU intensive program, the =
> single threaded program wins. Now compare IO intensive (especially =
> network IO) programs, the multi-threaded program wins. Granted, you can =
> built high performing asynchronous frameworks (Twisted or medusa come to =
> mind), but sooner or later you'll have to use threads (see the DB =
> adapter for Twisted).
Uh, no. Threads are useful for supporting libraries with blocking APIs. Nothing about database access requires threads. Twisted uses threads DB-API 2.0 specifies a blocking API, and it is useful to support DB-API 2.0 modules.
Several SQL modules provide an asynchronous API which can be used without threads.
When it comes to network-bound tasks, single-threaded wins the performance contest.
More information about the Python-list