Hello, I'm doing a simple analysis of looking for a gravitationally bound sphere centred at the most dense point in each level of clumps by iteratively increasing the radius from 0.5pc to 4pc until the derived quantity "IsBound" is True. I'm trying to use mpi4py as this iteration takes a very long time on serial but I'm getting an error very similar to the error I received when I tried mpi4py with clump finding routine: Traceback (most recent call last): bound = sp_cand.quantities["IsBound"](truncate=True, include_thermal_energy=False) File "/home/elee/accretion/analysis/codes/clump_excise_sph.py", line 71, in <module> File "/home/jchong/ytdev/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 86, in __call__ self.func(e, *args, **kwargs) self.func(e, *args, **kwargs) File "/home/jchong/ytdev/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 351, in _IsBound File "/home/jchong/ytdev/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 351, in _IsBound self.func(e, *args, **kwargs) File "/home/jchong/ytdev/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 351, in _IsBound bound = sp_cand.quantities["IsBound"](truncate=True, include_thermal_energy=False) File "/home/jchong/ytdev/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 86, in __call__ bv_x,bv_y,bv_z = data.quantities["BulkVelocity"]() bv_x,bv_y,bv_z = data.quantities["BulkVelocity"]() AttributeError: 'FieldDetector' object has no attribute 'quantities' self.func(e, *args, **kwargs) AttributeError: 'FieldDetector' object has no attribute 'quantities' File "/home/jchong/ytdev/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 351, in _IsBound bv_x,bv_y,bv_z = data.quantities["BulkVelocity"]() AttributeError: 'FieldDetector' object has no attribute 'quantities' bv_x,bv_y,bv_z = data.quantities["BulkVelocity"]() AttributeError: 'FieldDetector' object has no attribute 'quantities' It seems like parallelization loses the attribute "quantities" associated with data, which, if I understand correctly, should be the sphere object. Does this mean it somehow loses the sphere object that was passed in as an argument? Or is this somehow related to the parallel contouring algorithm that Britton has mentioned before? Thanks. Best Regards, Eve
Hi Eve,
I'm trying to use mpi4py as this iteration takes a very long time on serial but I'm getting an error very similar to the error I received when I tried mpi4py with clump finding routine:
Clump finding does not work in parallel - at least the expensive part of it (where the gravitational boundedness of the clump is tested) is definitely not parallel. You might want to experiment with changing your opening angle to see if it speeds things up enough to justify the loss in accuracy: http://yt-project.org/doc/analysis_modules/clump_finding.html Good luck! -- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice)
Hi Stephen, But is the derived quantity "IsBound" also not parallel in general? The code I'm using does not use clump object at all; all it's doing is picking out a sphere object centred around some point in the box (i.e. I have a separate code to find all the clumps and output all the relevant quantities such as the x,y,z of the densest point and the centre of mass, etc. onto a text file. In other words, the code I'm using is only retrieving the coordinates). I thought the reason why clump finding does not work in parallel was because parallel contour algorithm was not yet implemented, which, if I understand correctly, is separate from "IsBound". In any case, I'll try changing the opening angle. Thanks! Best Regards, Eve On 09/28/2011 11:19 AM, Stephen Skory wrote:
Hi Eve,
I'm trying to use mpi4py as this iteration takes a very long time on serial but I'm getting an error very similar to the error I received when I tried mpi4py with clump finding routine: Clump finding does not work in parallel - at least the expensive part of it (where the gravitational boundedness of the clump is tested) is definitely not parallel. You might want to experiment with changing your opening angle to see if it speeds things up enough to justify the loss in accuracy:
http://yt-project.org/doc/analysis_modules/clump_finding.html
Good luck!
Hi Eve,
But is the derived quantity "IsBound" also not parallel in general? The code I'm using does not use clump object at all; all it's doing is picking out a sphere object centred around some point in the box
"IsBound" is precisely the expensive operation I mentioned in my last email that is not parallel. That calls the functionality that calculates the gravitational attraction between all cells in the object and tells you if the object is bound or not. When someone is sufficiently motivated, there are parallel treecodes that could be included in yt, but that would be a significant project, and is not likely to happen any time soon. Unless you want to volunteer! :)
I thought the reason why clump finding does not work in parallel was because parallel contour algorithm was not yet implemented, which, if I understand correctly, is separate from "IsBound".
Yes, that is another reason why clump finding is not truly parallel. The only way clump finding is parallelized now is over individual clumps. That is, given a bunch of different objects, each object can be processed by a single core, and therefore many are done at the same time, but all the heavy lifting (IsBound) is done in serial. -- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice)
On Wed, Sep 28, 2011 at 12:03 PM, Stephen Skory <s@skory.us> wrote:
Hi Eve,
But is the derived quantity "IsBound" also not parallel in general? The code I'm using does not use clump object at all; all it's doing is picking out a sphere object centred around some point in the box
"IsBound" is precisely the expensive operation I mentioned in my last email that is not parallel. That calls the functionality that calculates the gravitational attraction between all cells in the object and tells you if the object is bound or not. When someone is sufficiently motivated, there are parallel treecodes that could be included in yt, but that would be a significant project, and is not likely to happen any time soon. Unless you want to volunteer! :)
I thought the reason why clump finding does not work in parallel was because parallel contour algorithm was not yet implemented, which, if I understand correctly, is separate from "IsBound".
Yes, that is another reason why clump finding is not truly parallel.
To echo what Stephen says, parallelizing clump finding is actually easier than parallelizing the IsBound operation. There is some extant code currently in testing to work with parallel clump finding, but I do not have an estimate of when it could be included; likely not for several months.
The only way clump finding is parallelized now is over individual clumps. That is, given a bunch of different objects, each object can be processed by a single core, and therefore many are done at the same time, but all the heavy lifting (IsBound) is done in serial.
One can implement this with mpi4py, if you have saved clumps. You will have to run yt in serial mode, but something like this could work: from yt.mods import * from mpi4py import MPI rank = MPI.COMM_WORLD.rank size = MPI.COMM_WORLD.size all_my_clumps = get_my_list_of_clumps(somehow) for clump in all_my_clumps[rank::size]: binding_energy = clump.quantities["IsBound"]() We're hoping to add helper functions to make this kind of thing easier in the future. -Matt
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
I have little to no experience with clump finding, but for the IsBound part perhaps you could make use of symmetry? No idea how spherical (or ellipsoidal) your clumps are, but you might consider first obtaining a radial density profile (perhaps with ellipsoidal radius?) and then using that to calculate the gravitational potential as a function of radius, which you'd add to the specific kinetic energy of a given grid cell to determine if it's bound. YMMV. Mike On Sep 28, 2011 9:14 AM, "Matthew Turk" <matthewturk@gmail.com> wrote:
On Wed, Sep 28, 2011 at 12:03 PM, Stephen Skory <s@skory.us> wrote:
Hi Eve,
But is the derived quantity "IsBound" also not parallel in general? The code I'm using does not use clump object at all; all it's doing is picking out a sphere object centred around some point in the box
"IsBound" is precisely the expensive operation I mentioned in my last email that is not parallel. That calls the functionality that calculates the gravitational attraction between all cells in the object and tells you if the object is bound or not. When someone is sufficiently motivated, there are parallel treecodes that could be included in yt, but that would be a significant project, and is not likely to happen any time soon. Unless you want to volunteer! :)
I thought the reason why clump finding does not work in parallel was because parallel contour algorithm was not yet implemented, which, if I understand correctly, is separate from "IsBound".
Yes, that is another reason why clump finding is not truly parallel.
To echo what Stephen says, parallelizing clump finding is actually easier than parallelizing the IsBound operation. There is some extant code currently in testing to work with parallel clump finding, but I do not have an estimate of when it could be included; likely not for several months.
The only way clump finding is parallelized now is over individual clumps. That is, given a bunch of different objects, each object can be processed by a single core, and therefore many are done at the same time, but all the heavy lifting (IsBound) is done in serial.
One can implement this with mpi4py, if you have saved clumps. You will have to run yt in serial mode, but something like this could work:
from yt.mods import * from mpi4py import MPI rank = MPI.COMM_WORLD.rank size = MPI.COMM_WORLD.size
all_my_clumps = get_my_list_of_clumps(somehow)
for clump in all_my_clumps[rank::size]: binding_energy = clump.quantities["IsBound"]()
We're hoping to add helper functions to make this kind of thing easier in the future.
-Matt
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Thank you for all the suggestions. I'll give them a try. :) Best Regards, Eve On 09/28/2011 12:54 PM, Michael Kuhlen wrote:
I have little to no experience with clump finding, but for the IsBound part perhaps you could make use of symmetry? No idea how spherical (or ellipsoidal) your clumps are, but you might consider first obtaining a radial density profile (perhaps with ellipsoidal radius?) and then using that to calculate the gravitational potential as a function of radius, which you'd add to the specific kinetic energy of a given grid cell to determine if it's bound. YMMV.
Mike
On Wed, Sep 28, 2011 at 12:03 PM, Stephen Skory <s@skory.us <mailto:s@skory.us>> wrote:
Hi Eve,
But is the derived quantity "IsBound" also not parallel in general? The code I'm using does not use clump object at all; all it's doing is
On Sep 28, 2011 9:14 AM, "Matthew Turk" <matthewturk@gmail.com <mailto:matthewturk@gmail.com>> wrote: picking out a
sphere object centred around some point in the box
"IsBound" is precisely the expensive operation I mentioned in my last email that is not parallel. That calls the functionality that calculates the gravitational attraction between all cells in the object and tells you if the object is bound or not. When someone is sufficiently motivated, there are parallel treecodes that could be included in yt, but that would be a significant project, and is not likely to happen any time soon. Unless you want to volunteer! :)
I thought the reason why clump finding does not work in parallel was because parallel contour algorithm was not yet implemented, which, if I understand correctly, is separate from "IsBound".
Yes, that is another reason why clump finding is not truly parallel.
To echo what Stephen says, parallelizing clump finding is actually easier than parallelizing the IsBound operation. There is some extant code currently in testing to work with parallel clump finding, but I do not have an estimate of when it could be included; likely not for several months.
The only way clump finding is parallelized now is over individual clumps. That is, given a bunch of different objects, each object can be processed by a single core, and therefore many are done at the same time, but all the heavy lifting (IsBound) is done in serial.
One can implement this with mpi4py, if you have saved clumps. You will have to run yt in serial mode, but something like this could work:
from yt.mods import * from mpi4py import MPI rank = MPI.COMM_WORLD.rank size = MPI.COMM_WORLD.size
all_my_clumps = get_my_list_of_clumps(somehow)
for clump in all_my_clumps[rank::size]: binding_energy = clump.quantities["IsBound"]()
We're hoping to add helper functions to make this kind of thing easier in the future.
-Matt
-- Stephen Skory s@skory.us <mailto:s@skory.us> http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org <mailto:yt-users@lists.spacepope.org> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org <mailto:yt-users@lists.spacepope.org> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
participants (4)
-
Eve Lee -
Matthew Turk -
Michael Kuhlen -
Stephen Skory