Summary of a yt sprint

Hi all, This last weekend, John, Nathan, Britton, Sam and I were fortunate enough to be in the same place for a couple days and we had a sprint on some yt functionality. Almost all of the changes have gone into the code base already, but we wanted to share them with you all. Feel free to chime in with questions, comments, or more information. = Parallelism Improvements = Britton implemented a pure-MPI task queuing system that works with the existing parallel_objects machinery. It is currently not threaded (and opinions vary as to whether that would be productive or not) and so one MPI task is dedicated as the controller node that distributes objects. This can either be manually used or you can simply supply "dynamic=True" to the parallel_objects machinery. I have to say, the changes to yt's parallelism in the last year or so have been pretty much my favorite improvements, and Britton really knocked this one out of the park, in my opinion. As I noted in my other email, with the pull request for Reason, Britton also added an MPI-parallel backend for Reason. I won't repeat my email here, but if you want to test it out you'll need Pyro4 and the launcher script in scripts/ . This will be documented in more detail when I finish up the remaining task, which is to deliver intermediate payloads from the MPI controller back to the web browser. = Plot Window = Perhaps Nathan would be better at describing this, but he's now implemented a mechanism (built on Jeff's initial design) for generating nice plots easily and complex plots straightforwardly. This was discussed here: http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/2012-June/002111.h... Nathan, anything you wanted to add? = Reason = This was described in my other email as well: http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/2012-June/002117.h... and this PR: https://bitbucket.org/yt_analysis/yt/pull-request/179/reason-gui-refactor-ex... If anybody can test this out, I'd appreciate it. I did note this weekend that when running simulations, I spied a few of the other sprinters using it. I think seeing developers using something means it has "arrived." (And I know Sam has a volume rendering widget in mind ...) = Ghost Zones = John spent a good amount of time implementing a fast sibling finder, as well as a ghost zone filling mechanism. This is designed to replace the existing mechanism of a smoothed covering grid, which requires interpolation from the coarsest level up to the requested level. This should lead to a considerable performance increase when doing anything having ghost zones. I think this is going to be PR'd sometime in the next while. -- Anyway, I think I speak for everybody when I say it was a pretty successful sprint, and we'd love to hear your feedback! -Matt

Hi all, I'm excited for how things are shaping up. If you haven't yet, you may want to try pulling in the changes from Matt's PR and check out the new interface. I'm particularly excited about the 3d scene viewer (which is still a bit rough in places, unfortunately), I think we'll be able to come up with a lot of cool functionality that will significantly ease the pain of producing nice-looking volume renderings.
Perhaps Nathan would be better at describing this, but he's now implemented a mechanism (built on Jeff's initial design) for generating nice plots easily and complex plots straightforwardly. This was discussed here:
http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/2012-June/002111.h...
Nathan, anything you wanted to add?
If you haven't seen, I shared an example from the new top-level plotting functions on google+: https://plus.google.com/109002259194298035269/posts/PJ8xqVfJuq3 I also just pushed some docstrings for the new plotting functions. It would be great if you could try to use the new plotting interface in favor of PlotCollection in new scripts, and please let me know if there are bugs I need to fix or use-cases I didn't anticipate. I think I'd like to make it so 'yt plot' on the command line interface uses the new plotting tools, but I'll hold off if there are objections. I didn't get a chance to take a look at Matt's ProfilePlotter, but hopefully we can come up with a way to integrate that into the new plotting interface. Once the all the new plotting functions can do everything PlotCollection does now, which will require porting over all of the existing plot callbacks, I'm strongly in favor of deprecating PlotCollection. It should be left in any yt 2.X releases for backward compatibility, but we should try to emphasize the new plotting interface in the docs and maybe even throw a DeprecationWarning if a script tries to use it. +/- 1? I'm Looking forward to releasing all this awesome code sometime soon :) -Nathan

On Wed, Jun 27, 2012 at 12:26 AM, Nathan Goldbaum <goldbaum@ucolick.org> wrote:
Hi all,
I'm excited for how things are shaping up. If you haven't yet, you may want to try pulling in the changes from Matt's PR and check out the new interface.
I'm particularly excited about the 3d scene viewer (which is still a bit rough in places, unfortunately), I think we'll be able to come up with a lot of cool functionality that will significantly ease the pain of producing nice-looking volume renderings.
Perhaps Nathan would be better at describing this, but he's now implemented a mechanism (built on Jeff's initial design) for generating nice plots easily and complex plots straightforwardly. This was discussed here:
http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/2012-June/002111.h...
Nathan, anything you wanted to add?
If you haven't seen, I shared an example from the new top-level plotting functions on google+:
https://plus.google.com/109002259194298035269/posts/PJ8xqVfJuq3
I also just pushed some docstrings for the new plotting functions. It would be great if you could try to use the new plotting interface in favor of PlotCollection in new scripts, and please let me know if there are bugs I need to fix or use-cases I didn't anticipate.
I think I'd like to make it so 'yt plot' on the command line interface uses the new plotting tools, but I'll hold off if there are objections. I didn't get a chance to take a look at Matt's ProfilePlotter, but hopefully we can come up with a way to integrate that into the new plotting interface.
Once the all the new plotting functions can do everything PlotCollection does now, which will require porting over all of the existing plot callbacks, I'm strongly in favor of deprecating PlotCollection. It should be left in any yt 2.X releases for backward compatibility, but we should try to emphasize the new plotting interface in the docs and maybe even throw a DeprecationWarning if a script tries to use it. +/- 1?
+1
I'm Looking forward to releasing all this awesome code sometime soon :)
-Nathan
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
participants (2)
-
Matthew Turk
-
Nathan Goldbaum