Mailman 3 February 2017 - yt-dev

RFC: updating API docs
by Kacper Kowalik 04 Mar '17

04 Mar '17

Hi all! I'd like to bring PR 2526 to your attention. It introduces small, but significant changes to the way how the API reference is built. Most notably: it generates a one html page per *module* with classes and methods as subsections, instead of a one page per method (as we do now). An example deployment can be browsed here: http://yt-project.org/docs/mock There are couple of pros to this approach: * The build time is greatly reduced. It shaves a 0.5h from the total build time on our server, also makes it feasible to build the API doc locally (it takes 2m on my laptop vs over 1h before) * The lower number of pages creates a better searchindex. As an example try searching for an 'AxisAlignedSlicePlot' phrase on 'docs/dev and' on 'docs/mock'. A one con that Nathan identified is that methods of a parent class are not explicitly listed for derived classes (it can be also observed on the aforementioned AxisAlignedSlicePlot's page). Instead there's a link to the parent class definition at the top of doc entry (Bases: yt.visualization.plot_window.PWViewerMPL for AxisAlignedSlicePlot). For someone heavily relying on a previous API doc structure, that's going to be something to get used to. I'd love to hear from y'all, either on the ml or directly on the pull request. Cheers, Kacper

2 2

Issue #1326: Inconsistency between code_length / code_time and code_velocity in enzo_tiny_cosmology (yt_analysis/yt)
by John ZuHone 27 Feb '17

27 Feb '17

New issue 1326: Inconsistency between code_length / code_time and code_velocity in enzo_tiny_cosmology https://bitbucket.org/yt_analysis/yt/issues/1326/inconsistency-between-code… John ZuHone: `code_velocity` and `code_length/code_time` should be the same units, but in `enzo_tiny_cosmology` this does not appear to be the case. Someone with more knowledge of Enzo should have a look at this--I tried to look but for cosmo sims the units are set in a way that is rather enzo-specific. ```python import yt ds = yt.load("enzo_tiny_cosmology/DD0046/DD0046") dd = ds.all_data() print(dd["velocity_divergence"].to("1/code_time")) print(dd["velocity_divergence"].to("code_velocity/code_length")) print(dd["velocity_magnitude"].to("code_velocity")) print(dd["velocity_magnitude"].to("code_length/code_time")) ``` ```pycon # velocity_divergence check YTArray([ 0.00186079, 0.00200255, 0.00191645, ..., -0.03508909, -0.04609586, -0.02762048]) 1/code_time YTArray([ 0.09485985, 0.10208633, 0.09769683, ..., -1.78877676, -2.34988201, -1.40804124]) code_velocity/code_length # velocity_magnitude check YTArray([ 0.01237452, 0.01096058, 0.00971304, ..., 0.02039839, 0.0223968 , 0.02642769]) code_velocity YTArray([ 0.00024274, 0.00021501, 0.00019053, ..., 0.00040014, 0.00043934, 0.00051841]) code_length/code_time ```

1 0

RFI: Anybody making big changes?
by Matthew Turk 24 Feb '17

24 Feb '17

Hi folks, There are a couple big, backwards-incompatible changes that are coming up in the future for yt. None of these will come as a surprise, but here they are: * The Demeshening: removing the global octree index for particle datasets and replacing with a bitmap index. This also includes changing how particle fields are regarded, and makes the SPH implementation much cleaner and faster. * Non-Spatial Coordinates: this one is a bit different than the title suggests. For example, ds.domain_left_edge and ds.domain_right_edge will no longer be YTArray instances, instead they will be YTIndexArray instances. YTIndexArray differs from YTArray in that instead of having a single unit associated with it, it has a tuple of units, one for each column of the array. This will not, in its initial form, be a big change to non-Cartesian coordinates, but that will certainly be possible. YTEPs are either done or being prepped for both of these. Those are kind of outside the main topic of this email, which is much more social. >From discussions, it seems pretty likely that these two would need to be a yt 4.0. So let's take that as a given. Both of these are designed to be *as backwards compatible as possible*. The Demeshening will have many more breaking changes for people that rely on yt for SPH data, but ideally the *surface area* of those breakages will be small. For the non-spatial changes, we absolutely want that to be completely backwards compatible for cartesian, spatial datasets. So if we take as given that we want disruption to both users and developers to be minimal, we're still kind of stuck with the possibility that this will be a difficult transition. We absolutely want everything that worked before to work again. What I'm getting at is this: how can we manage this change, when the development from 3.x to 4.0 may be long? We're at a different stage than we were when we managed this for yt 3.0, which I did *not* do a good job of managing. We now encourage people to use binary distributions a lot more, and we have a reliable bugfix release system. If we were to merge in both of those two big things, there is still the possibility that unforeseen difficulties would crop up -- but, the userbase will be much more isolated than they were when we did 3.0. So the main problem is going to be if folks want to do big development, but that big development conflicts with one of those two things. Here's the first thing I want to ask: == Is anybody planning or working toward some big development that they would be nervous about doing in the codebase while we shake out any lingering issues from those? == Ultimately, the question we need to figure out is, "how do we minimize disruption to cartesian, grid-based codes while we fix this stuff up?" And I think until we know if folks have anything pretty big planned, we can't *quite* answer that. -Matt

3 2

Issue #1325: Fields cannot be retrieved properly from ray object created from reloaded ytdata (yt_analysis/yt)
by Bili Dong 21 Feb '17

21 Feb '17

New issue 1325: Fields cannot be retrieved properly from ray object created from reloaded ytdata https://bitbucket.org/yt_analysis/yt/issues/1325/fields-cannot-be-retrieved… Bili Dong: I'm trying to create a ray object from a saved ytdata and access its fields using the following script: ```python import yt ds = yt.load('FIRE_M12i_ref11/snapshot_600.hdf5') ad = ds.all_data() ad['PartType0', 'Density'] path = ad.save_as_dataset() ds_reload = yt.load(path) ray_reload = ds_reload.ray(ds_reload.domain_left_edge, ds_reload.domain_right_edge) print(ray_reload['PartType0', 'Density']) print(ray_reload['t']) ``` And the outputs are: ``` [] code_mass/code_length**3 [ 0. 0.25 0.5 0.5078125 0.515625 0.5234375 0.53125 0.546875 0.5625 0.59375 0.625 0.6875 0.75 0.875 ] dimensionless ``` Somehow `ray_reload['PartType0', 'Density']` is empty, which is not what I expected.

1 0

Fwd: [Matplotlib-devel] Announcing: Docathon, week of 6 March 2017
by Nathan Goldbaum 17 Feb '17

17 Feb '17

Hi all, I think I will be going to this at BIDS to work on yt documentation. If anyone else is interested in working on yt documentation with me that week (doesn't have to be at BIDS, could be over the internet) let me know. I will likely be adding new documentation issues. I'd particularly like to take a close look at our quickstart notebooks to hopefully make them less focused on cosmology and AMR data. If I can find some sphinx experts, it would also be awesome to figure out a way to speed up building the API docs. -Nathan ---------- Forwarded message ---------- From: Nelle Varoquaux <nelle.varoquaux(a)gmail.com> Date: Thu, Feb 16, 2017 at 4:42 PM Subject: [Matplotlib-devel] Announcing: Docathon, week of 6 March 2017 To: matplotlib development list <matplotlib-devel(a)python.org>, matplotlib-users <matplotlib-users(a)lists.sourceforge.net> Hi everyone, As some of you already know, some of us are involved in the organization of a docathon. Several of us will be meeting up to sprint on documentation or documentation-related projects at Berkeley, New York and Seattle. If you are interested in joining us, either remotely or on campus, don't hesitate to join! Cheers, Nelle *What's a Docathon?* It's a week-long sprint where we focus our efforts on improving the state of documentation in the open-source and open-science world. This means writing better documentation, building tools, and sharing skills. *Who’s this for?* Anyone who is interested in improving the understandability, accessibility, and clarity of software! This might mean developers with a particular project, or individuals who would like to contribute to a project. You don’t need to use a specific language (though there will be many Python and R developers) and you don’t need to be a core developer in order to help out. *Where can I sign up?* Check out the *Docathon website* <https://bids.github.io/docathon/>. You can sign up as a *participant* <https://goo.gl/forms/AaW2b24mMxOutxt02>, *suggest a project* <https://goo.gl/forms/0cPpw01zehrEyDDE3> to work on, or sign up *to host your own* <https://bids.github.io/docathon/pages/hosting.html> remote Docathon wherever you like. You don’t have to use a specific language - we’ll be as accommodating as possible! *When is the Docathon?* The Docathon will be held *March 6 through March 10*. For those coming to BIDS at UC Berkeley, on the first day we'll have tutorials about documentation and demos of documentation tools, followed by a few hours of hacking. During the middle of the week, we'll set aside a few hours each afternoon for hacking as a group at BIDS. On the last day, we'll have a wrap-up event to show off what everybody worked on. *Where will the Docathon take place?* There are a *few docathons being held simultaneously* <https://bids.github.io/docathon/pages/hosts.html>, each with their own schedule. At Berkeley we'll have a physical presence at BIDS over the week, and we encourage you to show up for the hours we set aside for doc hacking. However, it is totally fine to work remotely; we will coordinate people via email/GitHub, too. *Where can I get more information?* Check out an updated schedule, list of tutorials, and more information at our website here: *bids.github.io/docathon* <http://bids.github.io/docathon> . *Contact* If you have any questions, open an issue on our *GitHub repo* <https://github.com/BIDS/docathon>. We look forward to hearing from you! Please feel free to forward this email to anyone who may be interested. We'd love for other institutions/groups to get involved. _______________________________________________ Matplotlib-devel mailing list Matplotlib-devel(a)python.org https://mail.python.org/mailman/listinfo/matplotlib-devel

1 0

Issue #1324: Shouldn't need to demote 2nd order tets in CartesianCoordinateHandler.pixelize (yt_analysis/yt)
by Alexander Lindsay 16 Feb '17

16 Feb '17

New issue 1324: Shouldn't need to demote 2nd order tets in CartesianCoordinateHandler.pixelize https://bitbucket.org/yt_analysis/yt/issues/1324/shouldnt-need-to-demote-2n… Alexander Lindsay: Hypothetically, #2401 should make it so we don't have to demote second order tets when slicing through a 3D mesh, e.g. this check: `elif field_data.shape[1] == 10` should be removable. I would like to test this first though.

1 0

Issue #1323: Change in particle data (Gizmo) from yt-3.2 to yt-3.3? (yt_analysis/yt)
by Kyle Stewart 15 Feb '17

15 Feb '17

New issue 1323: Change in particle data (Gizmo) from yt-3.2 to yt-3.3? https://bitbucket.org/yt_analysis/yt/issues/1323/change-in-particle-data-gi… Kyle Stewart: Apologies if this information is available in the reading list and I simply couldn't find it. I recently went back to re-do some calculations I made last year using yt-3.2, and as I redo the exact same calculations, I keep finding some very different answers on calculations made purely from the particle fields in a Gizmo dataset. Namely, if you look at ('PartType0','temperature') in the dataset using yt-3.2, and again using yt-3.3, (exact same calculation / same dataset / same region) you get a noticeably different distribution of particle temperatures. I've reproduced the issue here with the test dataset: FIRE_M12i_ref11/snapshot_600.hdf5 by plotting the mass-weighted temperature distribution within 300 kpc of the galaxy (i.e. maximum value of "all_densitiy"). Did something drastic change in yt-3.3 about the way the *particle* data was read in? Hopefully this isn't a dumb question but shouldn't the raw particle data for a particle-based dataset be identical? Was there a bug in yt-3.2 that got fixed, perhaps? Or has a bug been introduced? In short, since I'm re-doing some calculations and getting different answers, I'm just hoping to know which of them is correct... Thanks so much! Kyle

1 0

Issue #1322: Adding several sources to VR doesn't work (yt_analysis/yt)
by Maximilian Witt 15 Feb '17

15 Feb '17

New issue 1322: Adding several sources to VR doesn't work https://bitbucket.org/yt_analysis/yt/issues/1322/adding-several-sources-to-… Maximilian Witt: Hello, I'm currently having the problem that when I want to add another source to any Scene, it only shows the last added source and the others disappear. To make things clearer, I'm trying to plot 3D FLASH hdf5 files and want to get a plot like here on page 5, with some entropy rendering but the neutron star in the center as marble defined by density=1e11: http://iopscience.iop.org/article/10.1088/0004-637X/799/1/5/pdf I can get both pictures the way I want them separately, but when having two sc.add_source statements it only shows the last one and drops the first one. Also, I can't reproduce this example with the very same code (only with a FLASH file from me): http://yt-project.org/doc/cookbook/complex_plots.html#cookbook-vol-points As soon as sc.add_source(points) is commented in, the Volume Rendering from create_scene disappears. Looking at sc.sources or sc.get_source shows all sources correctly, they just don't appear in the image. Please see my code below. Other question I have in mind: Is it possible to add a normal bar-scale like in annotate_scale to see how big everything in the current image is? Just a simple 1D bar with its corresponding length... Thanks for any help! ``` #!python import yt from yt.visualization.volume_rendering.api import Scene, VolumeSource, OpaqueSource, PointSource, LineSource from yt.units import cm, m, km filePath = "/work/scratch/aj19ugij/3D/test/ccsn3d_hdf5_chk_0271" ds = yt.load(filePath) #sc = yt.create_scene(ds, lens_type='perspective') sc = Scene() PNS = VolumeSource(ds,field='density') PNS_bounds = (10**(10.99), 10**(11.01)) PNS.set_log(True) PNS_tf = yt.ColorTransferFunction(np.log10(PNS_bounds)) PNS_tf.add_gaussian(10,1,[1,1,1,0.3]) # PNS contour PNS.tfh.tf = PNS_tf PNS.tfh.bounds = PNS_bounds ENT = VolumeSource(ds,field='entr') ENT_bounds = (0,10) ENT.set_log(False) ENT_tf = yt.ColorTransferFunction(ENT_bounds) ENT_tf.add_gaussian(8,1,[0.4,1,0.4,0.3]) ENT.tfh.tf = ENT_tf ENT.tfh.bounds = ENT_bounds sc.add_source(PNS,keyname='PNS') #sc[0] = PNS # Makes no difference either sc.add_source(ENT) sc.add_source(PNS) #vertices = np.random.random([1000, 3])*100*km #colors = np.random.random([1000, 4]) #points = PointSource(vertices,colors=colors) #sc.add_source(points) #source.tfh.plot('transfer_function.png', profile_field='density') cam = sc.add_camera(ds, lens_type='perspective') cam.resolution = [400, 400] cam.position = ds.arr([300, 300, 300], 'km') cam.switch_orientation() #sc.save('rendering.png', sigma_clip=6) sc.render() sc ```

1 0

Issue #1321: Gadget Temperature field assumes zero ionization if ElectronAbundance field is not present. (yt_analysis/yt)
by John ZuHone 14 Feb '17

14 Feb '17

New issue 1321: Gadget Temperature field assumes zero ionization if ElectronAbundance field is not present. https://bitbucket.org/yt_analysis/yt/issues/1321/gadget-temperature-field-a… John ZuHone: In the Gadget frontend, it is assumed that if the `("PartType0", "ElectronAbundance")` field is not present, that the gas is not ionized and therefore this affects the mean molecular weight and conversely the computation of the temperature. Lines 71-89 of `yt/frontends/gadget/fields.py`: ```python def setup_gas_particle_fields(self, ptype): if (ptype, "ElectronAbundance") in self.ds.field_list: def _temperature(field, data): # Assume cosmic abundances x_H = 0.76 gamma = 5.0/3.0 a_e = data[ptype, 'ElectronAbundance'] mu = 4.0 / (3.0 * x_H + 1.0 + 4.0 * x_H * a_e) ret = data[ptype, "InternalEnergy"]*(gamma-1)*mu*mp/kb return ret.in_units(self.ds.unit_system["temperature"]) else: def _temperature(field, data): # Assume cosmic abundances x_H = 0.76 gamma = 5.0/3.0 # Assume zero ionization mu = 4.0 / (3.0 * x_H + 1.0) ret = data[ptype, "InternalEnergy"]*(gamma-1)*mu*mp/kb return ret.in_units(self.ds.unit_system["temperature"]) ``` It seems as if this is too restrictive of an assumption--there is no reason to think that simulations would not be perhaps fully ionized instead. We should find a more flexible way to define this field, perhaps by setting a field parameter for `mu`. Responsible: jzuhone

1 0

Issue #1320: Annotating halos fails in yt-3.3.4 (yt_analysis/yt)
by Kacper Kowalik 14 Feb '17

14 Feb '17

New issue 1320: Annotating halos fails in yt-3.3.4 https://bitbucket.org/yt_analysis/yt/issues/1320/annotating-halos-fails-in-… Kacper Kowalik: Following code: ``` #!python import yt from yt.analysis_modules.halo_analysis.halo_catalog import HaloCatalog data_ds = yt.load('Enzo_64/RD0006/RedshiftOutput0006') halos_ds = yt.load('rockstar_halos/halos_0.0.bin') hc = HaloCatalog(halos_ds=halos_ds) hc.create() prj = yt.ProjectionPlot(data_ds, 'z', 'density') prj.annotate_halos(hc, annotate_field='particle_identifier') prj.save() ``` yields: ``` #!bash Traceback (most recent call last): File "yt/visualization/plot_window.py", line 1007, in run_callbacks callback(cbw) File "yt/visualization/plot_modifications.py", line 54, in _check_geometry return func(self, plot) File "yt/visualization/plot_modifications.py", line 1473, in __call__ radius = halo_data[self.radius_field][:].in_units(units) AttributeError: 'HaloCatalogCallback' object has no attribute 'radius_field' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "foo.py", line 12, in <module> prj.save() File "yt/visualization/plot_container.py", line 78, in newfunc args[0]._setup_plots() File "yt/visualization/plot_window.py", line 956, in _setup_plots self.run_callbacks() File "yt/visualization/plot_window.py", line 1013, in run_callbacks sys.exc_info()[2]) File "six.py", line 685, in reraise raise value.with_traceback(tb) File "yt/visualization/plot_window.py", line 1007, in run_callbacks callback(cbw) File "yt/visualization/plot_modifications.py", line 54, in _check_geometry return func(self, plot) File "yt/visualization/plot_modifications.py", line 1473, in __call__ radius = halo_data[self.radius_field][:].in_units(units) yt.utilities.exceptions.YTPlotCallbackError: annotate_halos callback failed with the following error: 'HaloCatalogCallback' object has no attribute 'radius_field' ```

1 0