using threads with for-loops
peter at engcorp.com
Tue Sep 28 22:00:44 CEST 2004
Mathias Waack wrote:
> Peter Hansen wrote:
>>Klaus Neuner wrote:
>>>I wrote a program that does essentially the following:
>>>for rule in rules:
>>>for line in line_list:
>>>line = my_apply(rule, line)
>>>line_list contains the lines of some input text.
>>>To "apply a rule" always means to
>>> 1. do some regex matches on line
>>> 2. substitute something in line
>>>My question is: Given this "architecture", does it make sense
>>>to use threads? And if so, how?
>>The code is (based on what you give above) "CPU bound",
>>which means you will not see any advantage in using
> Have you ever seen machines with more than one CPU?
Why yes, I have! And have *you* seen an implementation
of Python which will effectively use those multiple CPUs
in code like that above which runs in a single process?
And do you think it likely that the OP is dealing with
a multiple CPU situation, but managed to forget to
mention it? I didn't think it likely, which is why when
I considered the multi-CPU situation I discarded the
idea. Perhaps, however, I was too quick to judge...
> The code above is perfectly suited for parallelization.
Yes, it is. Once you take into account implementation
issues (e.g. the GIL), would you still think so?
More information about the Python-list