Beginner Threaded file reading access
jcarlson at nospam.uci.edu
Sat Mar 27 20:02:27 CET 2004
>>It is even worse than that. In windows, Python is really a single
>>process, and no amount of threading is going to let one instance of
>>Python use more than 100% of a single CPU. I'm not sure if this is also
>>the case with linux or OS X.
> Go through this with me. I agree with your conclusion, and I
> don't keep up with Windows engineering. Moreover, I think it's
> important that we educate the masses that multithreading in-
> herently *slows* operations on a single CPU. However, I was
> sure that Windows *does* include the ability to distribute
> intraprocess native threads across SMP. Am I misreading you?
> Are you making the point that Python's use of Windows threads
> only provides for concurrency between, in practice, one I/O
> and one computational operation?
I should have given some background. I've got dual processors and run
windows 2000. It works great. I've done threaded development, both in
IO bound and in processor bound applications.
On my dual processor machine, no amount of threads in a single Python
process will ever allow you to use 100% of both processors. Up to 100%
of one processor, which amounts to 50% of both with the automatic
processor migration that Windows likes to do. I believe this is the
result of Python threads needing the GIL, but don't quote me on that, I
don't have an intimate working knowledge of the GIL or the inner
workings of Python to know for sure.
In practice, threads work just fine for IO bound applications; I've done
blocking sockets programming with threads, and they've worked pretty
well. However, the thread overhead is awful for more than a half dozen
threads when you want to do /anything/ other than reading and writing
More information about the Python-list