Threads not Improving Performance in Program

Jean-Paul Calderone exarkun at
Thu Mar 19 18:35:20 CET 2009

On Thu, 19 Mar 2009 09:50:51 -0700, Ryan Rosario <uclamathguy at> wrote:
>I have a parser that needs to process 7 million files. After running
>for 2 days, it had only processed 1.5 million. I want this script to
>parse several files at once by using multiple threads: one for each
>file currently being analyzed.

Threads don't magically make a program faster.  In Python in particular,
threads won't do much, if anything, to speed up a CPU bound task.  Assuming
you have more than one CPU, Python still limits you to one thread executing
Python bytecode at a time.  You could try running multiple processes instead,
if there really is more hardware that's sitting idle with your single
threaded version.


More information about the Python-list mailing list