Hi all,
Matt and I have been working a bunch in the volume_refactor bookmark in our yt-refactor forks and have added a bunch of functionality and performance improvements. We are now entering a point where we will be issuing a PR soon. At that point it should be tested by as many people as possible to ensure coverage. I’m going to break this down into things that will change behavior and new things.
The underlying code in the volume renderer has been refactored, simplified and streamlined. Adding new types of volume rendering should now be much easier, since the act of walking the cells and the accumulation of values in the plane are now completely separated.
--Things that will change in some way:
*Off-axis Projections:
We have implemented a new way of calculating off-axis projections which no-longer requires ghost zones and interpolation. Instead, as the rays traverse the cells, they take the value directly from the value for that cell. This leads to several changes. First, it is way faster! Second, because we aren’t interpolating anymore, that means you won’t get quite as smooth of an image as you may have before. Below are before and after images. Note that the transpose of the image is due to changes in the default behavior of setting up the looking vectors and the writing of the png in write_image.
left: old right: new
The performance gains for a sample (largish) dataset:
Old: 186 seconds, including IO
New (single-threaded): 20 seconds, including IO
If you use 8 OpenMP threads (see later), the new drops down to 10 seconds, with an improvement from 18 to 8 in the section that is threaded. Not perfect scaling, but there’s not a lot of work per cell at that point.
*Isocontour/Other Renderings:
These may change in their appearance slightly, mostly because of the changes made to allow for opaque and semi-opaque renderings. If you were using transfer functions that have values close to 1.0 for the alpha, then things may start getting opaque. More on this later.
-- Things that are new:
*OpenMP Threading:
We’ve added support for OpenMP in some of the cython files such as grid_traversal.pyx. This means that by default your renderings will run with OpenMP threads. By default it will use the number of cores on whatever computer/node you are on. You can set this by either adjusting the environment variable OMP_NUM_THREADS or using the num_threads keyword in camera.snapshot(). Just be aware of it, and take a gander at top to see yt running at >100%! This will hopefully allow users to run without mpi for rendering on their desktops/laptops. Here’s a snapshot from John running on (presumably) 24 cores:
A side effect is that Cython 0.16 is now *required*. This is included in the install_script.sh, which can be re-run at any time over top of an existing installation to update dependencies.
*Opaque Rendering:
Support for opaque rendering is now possible. This is mostly enabled by a few changes to the allowed values in the transfer function. The opacity is dictated by the values in the rgb of the transfer function. This is a continuous behavior, so if you slowly ramp up the rgb values from 0.1 to 10, you’ll see it get more and more opaque. In general it works out that rgb values around 1.0 start making things opaque, but you may need to experiment to get the desired behavior. By the time 2.4 is released we plan on having a tutorial on how to make more and more complex renderings, starting from a simple rendering and moving to a more complicated example. Currently the behavior is set so that all colors are opaque to all other colors. The integration step (in yt/utilities/lib/field_interpolation_tables.pxd) is:
ttot = trgba[0] + trgba[1] + trgba[2]
ta = fmax(1.0 - dt*ttot, 0.0)
for i in range(3):
rgba[i] = (1.0-ta)*trgba[i] + ta*rgba[i]
trgba is the transfer function rgba
rgba is the image values
*Lighting:
We also now enable adding a light source to the rendering. The behavior is enabled by use_light=True in the camera() call. It is then controlled by cam.light_dir and cam.light_rgba. The light direction is then dotted with the gradient of the field being rendered to either add or subtract color from that region.
Here the integration is:
ta = fmax(1.0-dt*(trgba[0] + trgba[1] + trgba[2]), 0.0)
for i in range(3):
rgba[i] = (1.-ta)*trgba[i]*(1. + dot_prod*l_rgba[i]) + ta * rgba[i]
where dot_prod is the (gradient of field) dot (light_direction), and l_rgba is the light color.
What this does is amplify the emission when the dot product is positive, and decrease it when it is negative.
So, what would be awesome is if you could try out your old scripts, and then try out some of the new features. Remember, you may need to run the install script again to get an up-to-date cython. There are two simple options to try this out. The first is to pull the volume_refactor bookmark into your local repository.
hg pull -B volume_refactor https://bitbucket.org/MatthewTurk/yt-refactor
hg up volume_refactor
This will create a head, which you shouldn’t worry about merging with any other head, with the bookmark volume_refactor. Then a python setup.py develop or whatever you like should do it.
If you don’t want to use bookmarks (though you really should because they are awesome!), then maybe just cloning the whole repo is for you.
Once we let this simmer on yt-dev for a while, we will put up a PR to get this merged into the yt tip, and send out a compact version of this email to the yt-users list. And yes, the PR will have lot of updated docs to go with it. We are also working on a narrative example of how to go from a simple rendering like in the cookbook to something much more complex, and maybe even a screencast of one of us doing that in real time.
Cheers,
Sam & Matt
!DSPAM:10175,4fd2241f258181156422526!