[concurrency] Inside the Python GIL

[Jeremy Hylton]

On Fri, Jun 12, 2009 at 11:45 AM, Jesse Noller<jnoller at gmail.com> wrote:
> Really? Is this the worse thing ever? How many of us building heavily
> threaded I/O bound applications are truly hampered by this? Yes; this
> sucks for CPU bound applications, that's been known since the earth
> cooled.

I'm not sure I understand how to distinguish between I/O bound threads
and CPU bound threads.  If you've got a relatively simple
multi-threaded application like an HTTP fetcher with a thread pool
fetching a lot of urls, you're probably going to end up having more
than one thread  with input to process at any instant.  There's a ton
of Python code that executes when that happens.  You've got a urllib
addinfourl wrapper, a httplib HTTPResponse (with read & _safe_read)
and a socket _fileobject.  Heaven help you if you are using readline.
So I could image even this trivial I/O bound program having lots of
CPU contention.

Jeremy


> I, and many others, have been using threads in python w/o issue, now
> that multiprocessing is in core, when I do run into a limitation, I
> simply swap out the imports in many cases.
>
> On Fri, Jun 12, 2009 at 11:16 AM, Pete<pfein at pobox.com> wrote:
>> I didn't attend last night's UG, but I saw Dave give a version of this talk
>> about a month ago.  I'll second Carl's opinion - this talk is of critical
>> importance to anyone using threads in Python.
>>
>> Begin forwarded message:
>>
>>> From: Carl Karsten <carl at personnelware.com>
>>> Date: June 12, 2009 10:51:33 AM EDT
>>> To: The Chicago Python Users Group <chicago at python.org>
>>> Subject: Re: [Chicago] Posted : Video
>>>
>>> * David Beazley: mind-blowing presentation about how the Python GIL
>>> actually works and why it's even worse than most people even imagine.
>>> http://blip.tv/file/2232410   http://www.dabeaz.com/python/GIL.pdf
>>
>> _______________________________________________
>> concurrency-sig mailing list
>> concurrency-sig at python.org
>> http://mail.python.org/mailman/listinfo/concurrency-sig
>>
> _______________________________________________
> concurrency-sig mailing list
> concurrency-sig at python.org
> http://mail.python.org/mailman/listinfo/concurrency-sig
>