multiprocessing module and matplotlib.pyplot/PdfPages
davea at davea.name
Tue Apr 21 12:26:36 CEST 2015
On 04/20/2015 10:14 PM, Paulo da Silva wrote:
> I have program that generates about 100 relatively complex graphics and
> writes then to a pdf book.
> It takes a while!
> Is there any possibility of using multiprocessing to build the graphics
> and then use several calls to savefig(), i.e. some kind of graphic's
To know if this is practical, we have to guess about the code you
describe, and about the machine you're using.
First, if you don't have multiple cores on your machine, then it's
probably not going to be any faster, and might be substantially slower.
Ditto if the code is so large that multiple copies of it will cause
But if you have 4 cores, and a processor=bound algorithm, it can indeed
save time to run 3 or 4 processes in parallel. You'll have to write
code to parcel out the parts that can be done in parallel, and make a
queue that each process can grab its next assignment from.
There are other gotchas, such as common code that has to be run before
any of the subprocesses. If you discover that each of these 100 pieces
has to access data from earlier pieces, then you could get bogged down
in communications and coordination.
If the 100 plots are really quite independent, you could also consider
recruiting time from multiple machines. As long as the data that needs
to go between them is not too large, it can pay off big time.
More information about the Python-list