parallel programming in Python
Jabba Laci
jabba.laci at gmail.com
Thu May 10 08:14:47 EDT 2012
Hi,
I would like to do some parallel programming with Python but I don't
know how to start. There are several ways to go but I don't know what
the differences are between them: threads, multiprocessing, gevent,
etc.
I want to use a single machine with several cores. I want to solve
problems like this: iterate over a loop (with millions of steps) and
do some work at each step. The steps are independent, so here I would
like to process several steps in parallel. I want to store the results
in a global list (which should be "synchronised"). Typical use case:
crawl webpages, extract images and collect the images in a list.
What's the best way?
Thanks,
Laszlo
More information about the Python-list
mailing list