PyMPI comm.gather problem

[Peyman Askari]

Hi

I have run into a serious problem with PyMPI (Python bindings for the Message Passing Interface). Unfortunately I can not provide any example code as it is a massive program (38,000+ lines) and it is very difficult to break the program down due to multiple inheritance.

When I run the program (on multiple nodes), there is a linear increase in time per iteration. Every 1,500 iterations, it gains a tenth of a second per iteration. I timed different regions of my code which were engaged in messaging (broadcasting and gathering) and I found the increase to be in a gather statement, much like so

IO_list=self.comm.gather(IO,root=0)

each iteration this single command requires more and more time to pass information. I checked the size of IO being sent from every node (including root node) and it always remains constant. I also checked the size of the IO_list, and it too is always constant. Furthermore, I added a sleep(5) command before the line to make sure it was not a synchronization issue. 

Has anyone got any possible ideas as to what could possibly be going wrong? 

Cheers


Peyman Askari



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20100511/d25b02af/attachment.html>