Threads not Improving Performance in Program

Ryan Rosario uclamathguy at gmail.com
Thu Mar 19 18:36:08 EDT 2009


On Mar 19, 10:35 am, Jean-Paul Calderone <exar... at divmod.com> wrote:
> On Thu, 19 Mar 2009 09:50:51 -0700, Ryan Rosario <uclamath... at gmail.com> wrote:
> >I have a parser that needs to process 7 million files. After running
> >for 2 days, it had only processed 1.5 million. I want this script to
> >parse several files at once by using multiple threads: one for each
> >file currently being analyzed.
>
> Threads don't magically make a program faster.  In Python in particular,
> threads won't do much, if anything, to speed up a CPU bound task.  Assuming
> you have more than one CPU, Python still limits you to one thread executing
> Python bytecode at a time.  You could try running multiple processes instead,
> if there really is more hardware that's sitting idle with your single
> threaded version.
>
> Jean-Paul

Thank you for your response. I did not realize that. That seems like a
huge limitation for such a great language.
I will look into forking off processes instead of using threads.

R.



More information about the Python-list mailing list