Mailman 3 - yt-users

announcing yt 2.0-alpha
by j s oishi 30 Aug '10

30 Aug '10

Hi all, After a few days of hard work, Britton, Matt, and I are pleased to announce that a major reorganization of yt is complete. This is the biggest part of what will become yt-2.0, and encompasses nearly all of the codebase. We have pushed it to the tip of the yt branch in the hg repository today. PLEASE NOTE you MUST reclone (not hg pull!) your mercurial repository from hg.enzotools.org! Just move your old repository out of the way and do hg clone http://hg.enzotools.org/yt Finally, please note that as of today SVN is read only. You can still use the SVN repository for now, but it will no longer acquire any new changes. We have done a good amount of testing, but there are sure to be places where things break. Please report anything you find to the issue tracker (http://yt.enzotools.org/newticket) as soon as you can. The changes have almost all been organizational, except for the addition of a new QuadTree method for projections, which Matt pushed to the yt branch ahead of this. Furthermore, unless you do something **other** than from yt.mods import * *everything* should be exactly the same for all of your scripts. We would really appreciate it if you could file a ticket (http://yt.enzotools.org/newticket) if you find anything that imports only from yt.mods doesn't work. The biggest change is that the old module Snow Crash-derived module names (lagos, raven, fido, enki) have been removed, and the code has been divided up into new set of modules, with descriptive names: drwxr-xr-x 21 joishi joishi 714 Aug 27 11:49 analysis_modules drwxr-xr-x 31 joishi joishi 1.0k Aug 28 08:45 data_objects drwxr-xr-x 15 joishi joishi 510 Aug 26 16:57 frontends drwxr-xr-x 9 joishi joishi 306 Aug 28 08:45 gui drwxr-xr-x 46 joishi joishi 1.5k Aug 28 08:45 utilities drwxr-xr-x 38 joishi joishi 1.3k Aug 28 16:09 visualization The second biggest change, and perhaps the more important one (especially to yt developers) is the redesign of how we access component pieces within and outside of yt. Previously, within yt, importing modules from one part of the code to another was accomplished by things like from lagos import * in the __init__.py file of another module. This has two negative effects: first, it pollutes the namespaces with many unnecessary classes and functions (greatly raising the possibility of a collision), and second, it creates circular imports which can slow down import times for yt. In order to solve these two problems, we emptied out __init__.py of everything except a docstring in every module and replaced them with api.py files. Each module has a api.py file containing a list of imports of each class or function wihthin that module imported individually to the API. For example, here is the visualization api.py: from color_maps import \ add_cmap from plot_collection import \ PlotCollection, \ PlotCollectionInteractive, \ concatenate_pdfs, \ get_multi_plot from fixed_resolution import \ FixedResolutionBuffer, \ ObliqueFixedResolutionBuffer from image_writer import \ multi_image_composite, \ write_bitmap, \ write_image, \ map_to_colors, \ splat_points from plot_modifications import \ PlotCallback, \ callback_registry In order to use a function from the yt/visualization package, you would add the following import lines to your script: from yt.mods import * # this hasn't changed from yt.visualization.api import PlotCollection pf = load("AwesomeTown0132") pc = PlotCollection(pf) The third major change was to move extensions to analysis_modules. This will likely affect many people in a purely cosmetic way; for example, the halo finder is now imported by adding the following line to your script: from yt.analysis_modules.halo_finding.api import HaloFinder a bit wordier, but much cleaner. If you are developing yt, please note that the api.py file is for EXTERNAL scripts only. Anything INTERNAL to yt must directly import any class or function necessary by directly referencing the file it resides in, e.g.: from .fixed_resolution import FixedResolutionBuffer. Finally, Volume Rendering has been relocated to the new visualization module. It should otherwise be unchanged (except the api.py change, of course). Full notes on all these changes can be found on the wiki: http://yt.enzotools.org/wiki/yt-2.0 We hope this will lead to a much cleaner, more hackable, more understandable yt. yt-2.0 will also include a new, simpler plotting interface. Please feel free to contact Britton, Matt, or me if you have any questions or any problems with the new yt. Thanks! Jeff Matt Britton

1 0

list of clumps?
by Stephen Skory 29 Aug '10

29 Aug '10

Hi All, according to this recipe: http://yt.enzotools.org/doc/cookbook/recipes.html#find-clumps it should be possible to pass a list of clumps to the 'clumps' callback. How do I do that? How do I turn 'master_clump' into a list of clumps? In particular, I'd like to pass all the clumps found using that recipe above to the plotting callback. Thanks! _______________________________________________________ sskory(a)physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________

2 2

Basic units quesions
by Andrew Davis 26 Aug '10

26 Aug '10

Hi yt users, I have some basic questions about the units. I'm trying to follow through the cookbook and help site, but some of it isn't working. Namely, the part about having it tell me what units some of the fields are in. When I follow the commands on the page: http://yt.enzotools.org/doc/howto/objects.html This is what happens: In: pf=load("EZ2_0000") In: print(pf.h.field_list) Out: ['z-velocity', 'Temperature', 'Density', 'particle_velocity_z', 'TotalEnergy', 'Dark_Matter_Density', 'particle_position_z', 'x-velocity', 'particle_position_y', 'y-velocity', 'particle_index', 'particle_position_x', 'particle_mass', 'particle_velocity_x', 'particle_velocity_y'] So, I see DM particle fields, which is what I'm interested in. However, when I try to find their units, I get: In: print(pf.field_info["particle_velocity_x"].units) Out: --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) /home/adavis/Research/YT/src/yt-1.7-svn/scripts/iyt in <module>() ----> 1 2 3 4 5 AttributeError: 'DerivedField' object has no attribute 'units' I get the same error for the following fields: [xyz]-velocity, particle_velocity_[xyz], particle_position_[xyz], Density, TotalEnergy, Dark_Matter_Density, and particle_index (ok, this shouldn't have units) When I try the other method of getting at the units: pf.field_info["particle_velocity_x"].get_source() , there is nothing there that says anything about units. Out: def _Particles(field, data): io = data.hierarchy.io if not data.NumberOfParticles > 0: return na.array([], dtype=dtype) try: return io._read_data_set(data, p_field).astype(dtype) except io._read_exception: pass # This is bad. But it's the best idea I have right now. return data._read_data(p_field.replace("_"," ")).astype(dtype) And, while field_info["Density"].get_source() works, the same command for Temperature tells me that the units are g/cm^3, which is obviously wrong. So, my major question: particle positions seem to be in the [0-1] unit system, so conversion is simple for them. Particle velocities, however, are unclear. They look like they're in cm/s, but I'd like to be sure. thanks! Andrew ************************************************************* ** Andrew J. Davis andrew.davis(a)yale.edu ** ** Dept. of Astronomy 203-432-5119 ** ** Yale University www.astro.yale.edu/adavis ** *************************************************************

6 15

volume rendering in parallel
by Luigi Iapichino 23 Aug '10

23 Aug '10

Dear all, I attach to this e-mail a sample script for volume rendering. With YT 1.6 I used the old version of volume rendering (also in parallel) without problems, but now (I run on trunk, with an updated revision) I can run this script only on one CPU. If I try a parallel run, I get the following output: -------------- mpirun -np 4 python2.6 ./sample-vr.py --parallel yt INFO 2010-08-20 16:53:08,959 Parallel computation enabled: 0 / 4 yt INFO 2010-08-20 16:53:08,959 Parallel computation enabled: 2 / 4 yt INFO 2010-08-20 16:53:08,959 Parallel computation enabled: 1 / 4 yt INFO 2010-08-20 16:53:08,959 Parallel computation enabled: 3 / 4 ... etc etc... P001 yt INFO 2010-08-20 16:54:51,234 Finishing 'Ray casting ' P002 yt INFO 2010-08-20 16:54:51,383 Finishing 'Ray casting ' P003 yt INFO 2010-08-20 16:55:02,526 Finishing 'Ray casting ' P000 yt INFO 2010-08-20 16:55:02,791 Finishing 'Ray casting ' Traceback (most recent call last): File "./sample-vr.py", line 29, in <module> vp.ray_cast() File "/home/hlrb2/pr95he/lu79qey3/remotefs/yt-trunk-svn/yt/extensions/volume_rendering/software_sampler.py", line 117, in ray_cast if finalize: self._finalize() File "/home/hlrb2/pr95he/lu79qey3/remotefs/yt-trunk-svn/yt/extensions/volume_rendering/software_sampler.py", line 121, in _finalize im, f = self._mpi_catrgb((self.image, self.resolution)) File "/home/hlrb2/pr95he/lu79qey3/remotefs/yt-trunk-svn/yt/lagos/ParallelTools.py", line 165, in passage return func(self, data) File "/home/hlrb2/pr95he/lu79qey3/remotefs/yt-trunk-svn/yt/lagos/ParallelTools.py", line 852, in _mpi_catrgb new_image[0:nsize[0],0:nsize[1],:] = data[:] ValueError: shape mismatch: objects cannot be broadcast to a single shape ---------------- Any suggestion? Thanks in advance, Luigi -- --------------------------------------------------------------- Luigi Iapichino Zentrum fuer Astronomie der Universitaet Heidelberg Institut fuer Theoretische Astrophysik Albert-Ueberle-Str. 2, D-69120 Heidelberg, Germany Tel: +49 6221 548983, Fax: +49 6221 544221 e-mail: luigi(a)ita.uni-heidelberg.de URL: http://www.ita.uni-heidelberg.de/~luigi/

3 3

new feature in EnzoSimulation class
by Britton Smith 20 Aug '10

20 Aug '10

Greetings, I've just added a new feature to the EnzoSimulation class that a number of people have been asking for. If you don't know what EnzoSimulation does, see here: http://yt.enzotools.org/doc/extensions/analyzing_an_entire_simulation.html For people who have been setting Enzo to output every constant number of cycles, rather than a constant amount of time, the EnzoSimulation class did not work, as there is no way to know from the parameter file exactly how many datadumps will get made during the simulation and at what times. Also, if you do collapse simulations where you are outputting data every time a new AMR level is reached, it doesn't work for you either. To fix that, I've added a new keyword, get_data_by_force, that if set to True, will search the working directory for all directories that match the data dir name convention. It will then open them all and get times and redshifts by hand. It takes slightly longer, but only a second or two compared to previously less than a second. So to use this, you simply need to do, for example: import yt.extensions.enzo_simulation as ES my_run = ES.EnzoSimulation(parameter_file, get_data_by_force=True, initial_time=0) And my_run.allOutputs will have your list of datasets. Britton

2 1

Re: [yt-users] Dark matter + yt
by Matthew Turk 20 Aug '10

20 Aug '10

Hi Elizabeth, > Does yt have trouble if there is no gas but only particles in an output? I'm > seeing: > > In [2]: fn="Gasoline_32static_dmonly_0000" > > In [3]: pf = load(fn) > > In [4]: pc = PlotCollection(pf) > yt INFO 2010-08-20 17:32:36,447 Getting the binary hierarchy > yt INFO 2010-08-20 17:32:36,451 Finished with binary hierarchy > reading > --------------------------------------------------------------------------- > NeedsDataField Traceback (most recent call last) > > /1/home/taskere/yt/src/yt-1.7-svn/scripts/iyt in <module>() > ----> 1 > 2 > 3 > 4 > 5 > > /1/home/taskere/yt/src/yt-1.7-svn/yt/raven/plot_collection.py in > __init__(self, pf, center) > 81 self.pf = pf > 82 if center == None: > ---> 83 v,self.c = pf.h.find_max("Density") # @todo: ensure no > caching > 84 elif center == "center" or center == "c": > 85 self.c = (pf["DomainRightEdge"] + > pf["DomainLeftEdge"])/2.0 > > /1/home/taskere/yt/src/yt-1.7-svn/yt/lagos/object_finding_mixin.py in > find_max(self, field, finest_levels) > 53 if (field, finest_levels) in self._max_locations: > 54 return self._max_locations[(field, finest_levels)] > ---> 55 mg, mc, mv, pos = self.find_max_cell_location(field, > finest_levels) > 56 self._max_locations[(field, finest_levels)] = (mv, pos) > 57 return mv, pos > > /1/home/taskere/yt/src/yt-1.7-svn/yt/lagos/object_finding_mixin.py in > find_max_cell_location(self, field, finest_levels) > 66 len(source._grids), field) > 67 max_val, maxi, mx, my, mz, mg = \ > ---> 68 source.quantities["MaxLocation"]( field, > lazy_reader=True) > 69 max_grid = self.grids[mg] > 70 mc = na.unravel_index(maxi, max_grid.ActiveDimensions) > > /1/home/taskere/yt/src/yt-1.7-svn/yt/lagos/DerivedQuantities.py in > __call__(self, *args, **kwargs) > 70 self._data_source.pf.h.io) > 71 if lazy_reader and not self.force_unlazy: > ---> 72 return self._call_func_lazy(args, kwargs) > 73 else: > 74 return self._call_func_unlazy(args, kwargs) > > /1/home/taskere/yt/src/yt-1.7-svn/yt/lagos/DerivedQuantities.py in > _call_func_lazy(self, args, kwargs) > 77 self.retvals = [ [] for i in range(self.n_ret)] > 78 for gi,g in enumerate(self._get_grids()): > ---> 79 rv = self.func(GridChildMaskWrapper(g, > self._data_source), *args, **kwargs) > 80 for i in range(self.n_ret): self.retvals[i].append(rv[i]) > 81 g.clear_data() > > /1/home/taskere/yt/src/yt-1.7-svn/yt/lagos/DerivedQuantities.py in > _MaxLocation(data, field) > 445 """ > 446 ma, maxi, mx, my, mz, mg = -1e90, -1, -1, -1, -1, -1 > --> 447 if data[field].size > 0: > 448 maxi = na.argmax(data[field]) > 449 ma = data[field][maxi] > > /1/home/taskere/yt/src/yt-1.7-svn/yt/lagos/DerivedQuantities.py in > __getitem__(self, item) > 41 return getattr(self.grid, attr) > 42 def __getitem__(self, item): > ---> 43 return self.data_source._get_data_from_grid(self.grid, item) > 44 > 45 class DerivedQuantity(ParallelAnalysisInterface): > > /1/home/taskere/yt/src/yt-1.7-svn/yt/lagos/BaseDataTypes.py in > save_state(self, grid, field) > 41 old_keys = grid.data.keys() > 42 grid.field_parameters = self.field_parameters > ---> 43 tr = func(self, grid, field) > 44 grid.field_parameters = old_params > 45 grid.data = dict( [(k, grid.data[k]) for k in old_keys] ) > > /1/home/taskere/yt/src/yt-1.7-svn/yt/lagos/BaseDataTypes.py in > _get_data_from_grid(self, grid, field) > 1681 else: > 1682 pointI = self._get_point_indices(grid) > -> 1683 if grid[field].size == 1: # dx, dy, dz, cellvolume > 1684 t = grid[field] * na.ones(grid.ActiveDimensions, > dtype='float64') > 1685 return t[pointI].ravel() > > /1/home/taskere/yt/src/yt-1.7-svn/yt/lagos/BaseGridType.py in > __getitem__(self, key) > 133 """ > 134 if not self.data.has_key(key): > --> 135 self.get_data(key) > 136 return self.data[key] > 137 > > /1/home/taskere/yt/src/yt-1.7-svn/yt/lagos/BaseGridType.py in get_data(self, > field) > 176 else: raise > 177 else: > --> 178 self._generate_field(field) > 179 return self.data[field] > 180 > > /1/home/taskere/yt/src/yt-1.7-svn/yt/lagos/BaseGridType.py in > _generate_field(self, field) > 111 # First we check the validator > 112 try: > --> 113 self.pf.field_info[field].check_available(self) > 114 except NeedsGridType, ngt_exception: > 115 # This is only going to be raised if n_gz > 0 > > /1/home/taskere/yt/src/yt-1.7-svn/yt/lagos/field_info_container.py in > check_available(self, data) > 283 """ > 284 for validator in self.validators: > --> 285 validator(data) > 286 # If we don't get an exception, we're good to go > 287 return True > > /1/home/taskere/yt/src/yt-1.7-svn/yt/lagos/field_info_container.py in > __call__(self, data) > 381 doesnt_have.append(f) > 382 if len(doesnt_have) > 0: > --> 383 raise NeedsDataField(doesnt_have) > 384 return True > 385 > > NeedsDataField: (['Density']) > > > Which looks like it is requiring a density field? > > Elizabeth > > Yup, it's trying to center on the most dense point. You can specify a center with: pc = PlotCollection(pf, center = [0.5, 0.5, 0.5]) I believe it has correctly guessed the data type, but if you run into another issue with this try: pf = EnzoStaticOutput(fn, data_style="enzo_packed_3d") Best, Matt

1 0

YT installation issues
by Agarwal, Shankar 20 Aug '10

20 Aug '10

Hi, I have been trying to build yt on Legion cluster at University College London. My default modules are... Currently Loaded Modulefiles: 1) shared 5) infinipath/base/2.1 9) nedit/5.6 2) intel/fce/10.1.008 6) infinipath/mpi/intel/2.1 10) mrxvt/0.5.4 3) intel/cce/10.1.008 7) torque/2.2.1 11) hdf5/1.6.6 4) intel/mkl/64emt/10.0.2.018 8) moab/5.3.0 I did a module purge. And added gcc module... Currently Loaded Modulefiles: 1) GCC/4.4.1 But, the install_script.sh crashes when building matplot with the following warning in the yt_install.log... gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DPY_ARRAY_UNIQUE_SYMBOL=MPL_ARRAY_API -I/home/ucapsag/enzo3/src/yt/doc/yt-x86_64/lib/python2.6/site-packages/numpy/core/include -I/usr/local/include -I/usr/include -I. -I/home/ucapsag/enzo3/src/yt/doc/yt-x86_64/lib/python2.6/site-packages/numpy/core/include -Isrc -Iagg24/include -I. -I/home/ucapsag/enzo3/src/yt/doc/yt-x86_64/include/python2.6 -c src/path.cpp -o build/temp.linux-x86_64-2.6/src/path.o cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++ In file included from /shared/ucl/test/GCC4/lib/gcc/x86_64-unknown-linux-gnu/4.4.1/../../../../include/c++/4.4.1/ext/hash_map:59, from ./CXX/Extensions.hxx:68, from src/path.cpp:8: /shared/ucl/test/GCC4/lib/gcc/x86_64-unknown-linux-gnu/4.4.1/../../../../include/c++/4.4.1/backward/backward_warning.h:28:2: warning: #warning This file includes at least one deprecated or antiquated header which may be removed without further notice at a future date. Please use a non-deprecated interface with equivalent functionality instead. For a listing of replacement headers and interfaces, consult the file backward_warning.h. To disable this warning use -Wno-deprecated. Please let me know what could be happening here. Shankar

2 5

ubuntu 10.04 installation issue
by Robert Thompson 17 Aug '10

17 Aug '10

I just put Ubuntu 10.04 64bit on my desktop and wanted to get yt going and ran into an issue with the linux/unix install script. It gets up to matplotlib and errors out: Installation will be to /home/bob/local/yt-unknown and I'll be logging the installation in /home/bob/local/yt-unknown/yt_install.log I think that about wraps it up. If you want to continue, hit enter. If you'd rather stop, maybe think things over, even grab a sandwich, hit Ctrl-C. ======================================================================== [hit enter] Awesome! Here we go. Downloading HDF5 Downloading hdf5-1.6.9.tar.gz from yt.enzotools.org Downloading zlib-1.2.3.tar.bz2 from yt.enzotools.org Downloading Python-2.6.3.tgz from yt.enzotools.org Downloading numpy-1.4.1.tar.gz from yt.enzotools.org Downloading matplotlib-0.99.3.tar.gz from yt.enzotools.org Downloading ipython-0.10.tar.gz from yt.enzotools.org Downloading h5py-1.2.0.tar.gz from yt.enzotools.org Setting YT_DIR=/home/bob/local/yt-unknown/src/yt-1.7-svn/ Installing distribute Installing pip Installing matplotlib-0.99.3 (arguments: 'matplotlib-0.99.3') Failure. Check /home/bob/local/yt-unknown/yt_install.log. bob@keplernix:~/local$ I've attached the yt_install.log to lower the spam content. Thanks in advance! -Robert Thompson

2 2

simple calculation question
by Robert Thompson 17 Aug '10

17 Aug '10

Hi yt users, I'm still very new at python syntax so please forgive me =) I've ran a few n-body simulations and I'd like to simply calculate the pairwise velocity between the halo pairs. I simply need to read in the x,y,z positions & vx,vy,vz velocities from the halo finder output, do some calculations, and output the result. I've done this in C but I'd love to learn how to do this in yt. Here's my calculation in C: for(i=0; i<NUM; i++) { for(j=i+1; j<NUM; j++) { dXctr = periodic(Xctr[i] - Xctr[j]) / 1000.; /* in Mpc/h */ dYctr = periodic(Yctr[i] - Yctr[j]) / 1000.; dZctr = periodic(Zctr[i] - Zctr[j]) / 1000.; d = sqrt(dXctr*dXctr + dYctr*dYctr + dZctr*dZctr); /* in Mpc/h */ if(d < 10.) /* less than 10 Mpc/h */ { dvx = vx[i] - vx[j]; dvy = vy[i] - vy[j]; dvz = vz[i] - vz[j]; vrel = sqrt(dvx*dvx + dvy*dvy + dvz*dvz); } } How would I go about doing this in yt given the halo finder outputs. Thanks for the assistance! -Robert Thompson

3 3

questions about halo profiling
by Irina Dvorkin 15 Aug '10

15 Aug '10

Hello All, After reading the halo profiler python files, I got the impression that it reads the position of the halo center and then approximates the halo to a sphere with this center, and so includes some mass that is actually not in the halo (for example if the halo is an ellipsoid). My knowledge of python, however, is rather poor, so it would help me a lot if you could tell me whether my understanding is correct. Also, I noticed that both the halo profiler and the halo finder use a non-standard definition of the virial radius: usually it is defined as the radius that encompasses a region that is, say, 200 (or other number) times denser than the *critical* density at that redshift. The definition in the halo profiler is 200 (or other number) times the density of *matter*. So I would suggest to specifically mention this definition in the manual because it can cause confusion. Thanks, Irina

2 2