Hi Nathan,
Thanks a ton for your help, and the guidance in the notebook! The slowness should really be caused by the large data file I have. I followed your notebook to pass only a subset of the dataset, and it ran much faster! (And yes I do have yt's parallelism enabled.)
I think letting users to provide a data object as the first argument in Streamlines will indeed be very helpful. Unfortunately I am currently quite swamped with other commitments, but I will make sure to file an issue on the issue tracker page.
On the tutorial page it says the "data access along the streamline" functionality has not been implemented in the 3.x versions of yt. How involved will it be to implement it? This functionality would be very useful, or even crucial, in extracting the science from my simulations.
Many thanks,
Benny
On Jun 30, 2017, at 3:45 PM, Nathan Goldbaum mailto:nathan12343@gmail.com> wrote:
Hi Benny,
Yes, this should work and will work (eventually) for your data. We mean octree AMR data like Ramses or ART, where each cell can be refined into 8 more cells, not like Flash, which implements an octee of grid patches.
The slow operation here is generating vertex centered data. We have plans to make this faster (grid visitors) but those plans have not yet been implemented.
Do you expect your streamline to span the full dataset? You might be able to get some speedups by only selecting from a subset of the dataset using a data object. Another option would be to tell the AMRKDtree to generate vertex centered data by extrapolating at grid boundaries instead of interpolating. This will be a lot faster, but will also be less accurate.
Looking at the Streamlines class, the API for doing this is a bit awkward right now. It looks like you need to create a custom AMRKDTree and pass that to the Streamlines class. To show you how to do that, I've put together an example notebook that shows how to generate a streamline with a custom AMRKDtree instance:
https://gist.github.com/anonymous/b80284a5c8fa09d94cf428163f97902d
I first show how to switch from interpolation to extrapolation, then show how to create streamlines from a subset of the simulation volume.
Finally, if you're able to run in parallel using yt's parallelism, constructing the AMRKDTree and generating vertex centered data should parallelize well.
There are probably some things we could do to make it easier to opt out of the default AMRKDTree that the Streamlines class instantiates internally if you don't pass it one. One would be to give the Streamlines class a use_ghost_zones keyword argument, right now the AMRKDTree that gets generated by the Streamlines class is hard-coded to allways generate vertex centered data using ghost zones, which can be slow. Another option would be to let the Streamlines class optionally accept a data object as the first argument instead of a dataset. If you'd be interested in implementing one or more of these suggestions and submitting a pull request on GitHub I think it would be a very welcome and also wouldn't be very hard to implement, you'd basically only need to edit the Streamlines class. If you're not interested, that's ok too, but you might want to file an issue on our issue tracker so we don't forget about this opportunity to simplify the API.
-Nathan
On Fri, Jun 30, 2017 at 3:19 PM, Benny Tsang mailto:bthtsang@astro.as.utexas.edu> wrote:
Hi yt friends,
I'm trying to produce 3D streamlines using the AMR data from FLASH following the yt-project tutorial http://yt-project.org/doc/visualizing/streamlines.html. However, the execution was very slow (in particular, at the 'Streamlines' call). The code didn't complete one single streamline after an hour.
Here's the excerpt of my code:
—————
c = ds.domain_center
streamlines = Streamlines(ds, c, 'velx', 'vely', 'velz', length=1.0*pc, get_magnitude=False)
streamlines.integrate_through_volume()
fig=plt.figure()
ax = Axes3D(fig)
for stream in streamlines.streamlines:
stream = stream[np.all(stream != 0.0, axis=1)]
ax.plot3D(stream[:,0], stream[:,1], stream[:,2], alpha=0.1)
plt.savefig('streamlines.png')
—————
I have two thoughts:
1. My FLASH file is quite large (~10GB). It may just be slow because of the large AMR grid size. Are there any ways to read only the velocities off the grid or only a subsection of the grid?
2. FLASH's octree data structure may be incompatible with the AMRKDTree construction (first step in streamlines creation). In the doc string of AMRKDTree, it says "Not applicable to particle or octree-based datasets.". If it's the case, I may have to find another way out.
Thank you very much for your time! Have a great weekend!
Best,
Benny
_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.orgmailto:yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.orgmailto:yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
