Hi all, A paper, and affiliated GPL-licensed public code, was posted today that may be of interest: http://arxiv.org/abs/1105.0370 http://www.astro.rug.nl/~voronoi/DTFE/dtfe.html The paper itself has examples for gridding SPH particles, and it is OpenMP parallel. This could be the basis for an interesting mechanism for loading SPH data; one could imagine the initial procedure being decomposition based on a fairly coarse PH-curve (allowing for overlap between processors), feeding individual regions to the DTFE, and then gridding them. It also includes TSC as well as SPH interpolation. This could also be useful for a variety of other gridding processes that yt does. -Matt
uses Boost. -10000000. On Mon, May 2, 2011 at 5:30 PM, Matthew Turk <matthewturk@gmail.com> wrote:
Hi all,
A paper, and affiliated GPL-licensed public code, was posted today that may be of interest:
http://arxiv.org/abs/1105.0370 http://www.astro.rug.nl/~voronoi/DTFE/dtfe.html
The paper itself has examples for gridding SPH particles, and it is OpenMP parallel. This could be the basis for an interesting mechanism for loading SPH data; one could imagine the initial procedure being decomposition based on a fairly coarse PH-curve (allowing for overlap between processors), feeding individual regions to the DTFE, and then gridding them. It also includes TSC as well as SPH interpolation.
This could also be useful for a variety of other gridding processes that yt does.
-Matt _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
uses Boost. -10000000.
Seconded. Is there anything not awful that uses the name Boost? Boost Mobile? Boost Energy Drinks? -- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice)
Okay, okay, I missed that it uses Boost. But there's also Voro++: http://math.lbl.gov/voro++/ which to my reading does not use Boost. It could be another contender, although it does not perform the identical operationset as DTFE. -Matt On Tue, May 3, 2011 at 1:10 PM, Stephen Skory <s@skory.us> wrote:
uses Boost. -10000000.
Seconded. Is there anything not awful that uses the name Boost? Boost Mobile? Boost Energy Drinks?
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi all, I've been looking at this paper closely and here are my initial thoughts: - It's unclear to me how the code treats the smoothing length in SPH particles (I don't think it does.) I thought it'd be cool if tried to maintain a minimum S/N, like a 3D version of this http://www-astro.physics.ox.ac.uk/~mxc/idl/#binning . - It calculates the interpolated values of fields and gradients, but it'd be hard to calculate stuff like the velocity dispersion or higher moments. + You can sample at arbitrary points, which I guess would be the centers of the adaptive mesh + Already parallelized, and it seems ready to use as an external reference Personally, the inability to calculate the velocity dispersion is a serious drawback. Perhaps if it returned the particle IDs that belong to every tessellation, then we could define a field that is a function of those particles, and not some aggregate field estimate. Their method implements the CGAL libraries to do the tessellations (which is where the Boost dep comes in from) - I'm not sure what their DTFE method adds to CGAL. So the choices are: 1. qhull and voro++ have few dependencies. 2. DTFE seems marginally better than CGAL, and has the same Boost problem. It is the only OpenMP option though. chris On Tuesday, May 3, 2011 at 1:31 PM, Matthew Turk wrote: Okay, okay, I missed that it uses Boost. But there's also Voro++:
which to my reading does not use Boost. It could be another contender, although it does not perform the identical operationset as DTFE.
-Matt
On Tue, May 3, 2011 at 1:10 PM, Stephen Skory <s@skory.us> wrote:
uses Boost. -10000000.
Seconded. Is there anything not awful that uses the name Boost? Boost Mobile? Boost Energy Drinks?
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi Chris, Thanks for your thoughtful comments. On Tue, May 3, 2011 at 2:29 PM, Christopher Moody <juxtaposicion@gmail.com> wrote:
Hi all, I've been looking at this paper closely and here are my initial thoughts:
- It's unclear to me how the code treats the smoothing length in SPH particles (I don't think it does.) I thought it'd be cool if tried to maintain a minimum S/N, like a 3D version of this http://www-astro.physics.ox.ac.uk/~mxc/idl/#binning . - It calculates the interpolated values of fields and gradients, but it'd be hard to calculate stuff like the velocity dispersion or higher moments. + You can sample at arbitrary points, which I guess would be the centers of the adaptive mesh + Already parallelized, and it seems ready to use as an external reference
Personally, the inability to calculate the velocity dispersion is a serious drawback. Perhaps if it returned the particle IDs that belong to every tessellation, then we could define a field that is a function of those particles, and not some aggregate field estimate. Their method implements the CGAL libraries to do the tessellations (which is where the Boost dep comes in from) - I'm not sure what their DTFE method adds to CGAL.
Interesting point. I suppose that the DTFE is mainly the implementation of the smoothing kernel?
So the choices are: 1. qhull and voro++ have few dependencies. 2. DTFE seems marginally better than CGAL, and has the same Boost problem. It is the only OpenMP option though.
Good point. While yt is not currently OpenMP parallelized, this is on the radar. (Specifically, utilizing recent efforst by Cython.) I think we might benefit from some cost/benefit analysis, though -- is the cost of having only MPI-parallel (manually parallelized, by us) worth the benefit of not relying on Boost+CGAL? If so, perhaps Voro++ could be a good route to go. qhull would also be an interesting mechanism, moving forward. But, again, these only provide the tesselation step -- on top of that, we will still need to apply a meshing operation. And this is a stopgap that is necessary for now. Thinking longer term, I am not certain that this method of gridding SPH is going to be sustainable. It could potentially keep us set for a while -- couple years even -- but I think long term we will need better conceptual separation between geometry and quantities. Right now everything is assumed to be a regular mesh, which enables very fast locating of data and construction of visualizations. However, we'll probably need to move to a generalized architecture for data point selection for a "3.0" release. We will need to implement some of this for spherical coordinates, as well. This would essentially mean separating out geometry from the data objects. As it stands, we utilize geometric selection as defined by the hierarchy (mesh?) for most of the data objects, and I think we could extend this, but we would need to be careful. This would likely be easily applied to the 3D data types, but 2D datatypes I could see being challenging for particle-based systems. But, that's getting a bit off-track. Suffice it to say, I think we could stage our development of SPH/particle support into both making-it-look-meshed and accessing-it-natively. What do you think, Chris? Could we just grab voro++ and have a shot at the gridding of SPH particles? Gabe is also listening in, I think, and he has substantial experience with SPH particles and might have feelings on the right way to proceed. -Matt
chris On Tuesday, May 3, 2011 at 1:31 PM, Matthew Turk wrote: Okay, okay, I missed that it uses Boost. But there's also Voro++:
which to my reading does not use Boost. It could be another contender, although it does not perform the identical operationset as DTFE.
-Matt
On Tue, May 3, 2011 at 1:10 PM, Stephen Skory <s@skory.us> wrote:
uses Boost. -10000000.
Seconded. Is there anything not awful that uses the name Boost? Boost Mobile? Boost Energy Drinks?
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi Chris, and others, As a quick note, I wrapped a very simple subset of the voro++ technology in the latest revision of my yt repo: http://bitbucket.org/MatthewTurk/yt/ https://bitbucket.org/MatthewTurk/yt/changeset/30f62efd8734 This exposes the ability to create a Voro++ container (rectangular prism), add particles to it, and then get back the volumes. A sample script is here: http://paste.enzotools.org/show/1616/ To get it to run, you will need to download voro++ from here: http://math.lbl.gov/voro++/download/dir/voro++_0.3.1.tar.gz untar it, and then make a symbolic link from its subdirectory "src" as yt/utilities/voro++. Then uncomment lines 175-178 in yt/utilities/setup.py. You do *not* need to compile voro++. (Which is pretty rad, I think.) Just rebuild yt as usual and paste #1616 should work. I think this could be the basis for a regridding mechanism. Chris, do you have any thoughts on possible next steps? The main voro++ website, including docs and examples: http://math.lbl.gov/voro++/ -Matt On Fri, May 6, 2011 at 1:26 PM, Matthew Turk <matthewturk@gmail.com> wrote:
Hi Chris,
Thanks for your thoughtful comments.
On Tue, May 3, 2011 at 2:29 PM, Christopher Moody <juxtaposicion@gmail.com> wrote:
Hi all, I've been looking at this paper closely and here are my initial thoughts:
- It's unclear to me how the code treats the smoothing length in SPH particles (I don't think it does.) I thought it'd be cool if tried to maintain a minimum S/N, like a 3D version of this http://www-astro.physics.ox.ac.uk/~mxc/idl/#binning . - It calculates the interpolated values of fields and gradients, but it'd be hard to calculate stuff like the velocity dispersion or higher moments. + You can sample at arbitrary points, which I guess would be the centers of the adaptive mesh + Already parallelized, and it seems ready to use as an external reference
Personally, the inability to calculate the velocity dispersion is a serious drawback. Perhaps if it returned the particle IDs that belong to every tessellation, then we could define a field that is a function of those particles, and not some aggregate field estimate. Their method implements the CGAL libraries to do the tessellations (which is where the Boost dep comes in from) - I'm not sure what their DTFE method adds to CGAL.
Interesting point. I suppose that the DTFE is mainly the implementation of the smoothing kernel?
So the choices are: 1. qhull and voro++ have few dependencies. 2. DTFE seems marginally better than CGAL, and has the same Boost problem. It is the only OpenMP option though.
Good point. While yt is not currently OpenMP parallelized, this is on the radar. (Specifically, utilizing recent efforst by Cython.) I think we might benefit from some cost/benefit analysis, though -- is the cost of having only MPI-parallel (manually parallelized, by us) worth the benefit of not relying on Boost+CGAL? If so, perhaps Voro++ could be a good route to go. qhull would also be an interesting mechanism, moving forward.
But, again, these only provide the tesselation step -- on top of that, we will still need to apply a meshing operation. And this is a stopgap that is necessary for now.
Thinking longer term, I am not certain that this method of gridding SPH is going to be sustainable. It could potentially keep us set for a while -- couple years even -- but I think long term we will need better conceptual separation between geometry and quantities. Right now everything is assumed to be a regular mesh, which enables very fast locating of data and construction of visualizations. However, we'll probably need to move to a generalized architecture for data point selection for a "3.0" release. We will need to implement some of this for spherical coordinates, as well.
This would essentially mean separating out geometry from the data objects. As it stands, we utilize geometric selection as defined by the hierarchy (mesh?) for most of the data objects, and I think we could extend this, but we would need to be careful. This would likely be easily applied to the 3D data types, but 2D datatypes I could see being challenging for particle-based systems.
But, that's getting a bit off-track. Suffice it to say, I think we could stage our development of SPH/particle support into both making-it-look-meshed and accessing-it-natively.
What do you think, Chris? Could we just grab voro++ and have a shot at the gridding of SPH particles? Gabe is also listening in, I think, and he has substantial experience with SPH particles and might have feelings on the right way to proceed.
-Matt
chris On Tuesday, May 3, 2011 at 1:31 PM, Matthew Turk wrote: Okay, okay, I missed that it uses Boost. But there's also Voro++:
which to my reading does not use Boost. It could be another contender, although it does not perform the identical operationset as DTFE.
-Matt
On Tue, May 3, 2011 at 1:10 PM, Stephen Skory <s@skory.us> wrote:
uses Boost. -10000000.
Seconded. Is there anything not awful that uses the name Boost? Boost Mobile? Boost Energy Drinks?
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
participants (4)
-
Christopher Moody -
j s oishi -
Matthew Turk -
Stephen Skory