yt-users August 2014

yt-users@python.org

35 participants
44 discussions

Problems using Rockstar in yt
by David Sullivan 14 Aug '14

14 Aug '14

Hi Britton, all, That took care of my issues, now it’s run successfully. Just one more question: as I’m using RAMSES I create a DatasetSeries using: es = yt.load('../output_*/info_*.txt’) rh = RockstarHaloFinder(es, num_readers=1, num_writers=2) Is there a way for me to add a dark matter only filter as done here for an enzo dataset: # create a particle filter to remove star particles @yt.particle_filter("dark_matter", requires=["creation_time"]) def _dm_filter(pfilter, data): return data["creation_time"] < 0.0 def setup_ds(ds): ds.add_particle_filter("dark_matter”) es = yt.simulation("enzo_tiny_cosmology/32Mpc_32.enzo", "Enzo") es.get_time_series(setup_function=setup_ds, redshift_data=False) rh = RockstarHaloFinder(es, num_readers=1, num_writers=2, particle_type="dark_matter") I’ve looked at the docs on particle filtering but I can’t add filters to the DatasetSeries object I have. Many thanks for the help so far! David On 12 Aug 2014, at 12:44, yt-users-request(a)lists.spacepope.org wrote: > Hi Cameron, David, > > Cameron, you are correct, on Mac systems you need to set DYLD_LIBRARY_PATH > instead of LD_LIBRARY_PATH. The fact that this causes vi and other things > to fail is apparently an OS X bug (I found some discussions of it on the > internet) and has nothing to do with yt. I went through a similar thing > and had to resort to having a separate terminal for running rockstar that I > didn't try to do anything else in. For what it's worth, this behavior > seems to have been fixed. I am on OS X 10.8.5 and no longer have the issue > of things crashing in a terminal where I have edited DYLD_LIBRARY_PATH. > > On the second issue that you both reported, the problem is the way in which > one tells yt to run in parallel depends on how you import yt. > > If you do "from yt.mods import *", the parallelism is enabled by adding the > --parallel flag on the command line. However, if you do "import yt", then > parallelism is enabled by doing "yt.enable_parallelism()" directly below > the import statement within your script. > > I looked at the latest documentation on parallelism ( > http://yt-project.org/doc/analyzing/parallel_computation.html) and is > discusses the use of yt.enable_parallelism(), but not the old way that > works with "from yt.mods import *". There should probably be a short > discussion added of how this has changed. Additionally, let's make sure > that the yt-rockstar docs get any needed additions there as well. > > Britton

3 2

yt/mpi4py on oak ridge cray machines
by David Sullivan 13 Aug '14

13 Aug '14

Hi all, I’ve been trying to build yt and mpi4py on the cray machines at oak ridge and came across this old thread in the mailing list: http://lists.spacepope.org/pipermail/yt-users-spacepope.org/2013-December/0… I was just wondering if anyone had managed this since? If so, what the best approach is (system python or install script)? Many thanks, David

6 6

Re: [yt-users] Problems using Rockstar in yt
by David Sullivan 12 Aug '14

12 Aug '14

Hi all, I also get the same error as Cameron when trying to run rockstar with mpi: http://paste.yt-project.org/show/5019/ I’m using a ramses dataset (not sure if this is supported, but couldn’t see any clear evidence that it isn’t) and the following script: mpirun -np 4 --mca btl ^openib python run_rockstar.py --parallel http://paste.yt-project.org/show/5020/ I followed the same install instructions as Cameron, but I’m using a Linux cluster running CentOS. Appreciate any help! Cheers, David On 12 Aug 2014, at 10:04, yt-users-request(a)lists.spacepope.org wrote: > Message: 3 > Date: Tue, 12 Aug 2014 02:04:10 -0700 > From: Sam Skillman <samskillman(a)gmail.com> > To: Discussion of the yt analysis package > <yt-users(a)lists.spacepope.org> > Subject: Re: [yt-users] Problems using Rockstar in yt > Message-ID: > <CAB4=1s1oHK0uPFo88rkSaEa-W6EASMLoyOiEFeniV-xSm6SPaQ(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hi Cameron, > > Are you running in MPI? You need to run with at least 3 MPI tasks for > Rockstar to work properly. > > Sam > > > On Mon, Aug 11, 2014 at 6:41 PM, Cameron Hummels <chummels(a)gmail.com> wrote: > >> Hello, >> >> I'm trying to use Rockstar with the new HaloCatalog functionality in yt >> 3.0, but I'm encountering some problems. For reference, this is an OS X >> box with the "yt" branch. I've tested it on two OS X systems to the same >> result. >> >> I modified my install_script.py to have it install Rockstar as described >> here: >> >> >> http://yt-project.org/docs/dev/analyzing/analysis_modules/halo_finders.html… >> >> Then, after rerunning my install script for the `yt` branch (ie yt-3.1 >> dev) I tried to run the script provided here in order to generate a halo >> catalog using HOP: >> >> >> http://yt-project.org/docs/dev/analyzing/analysis_modules/halo_transition.h… >> >> It can create the catalog fine with finder_method='hop', but when I change >> it to finder_method='rockstar', I get failures. This is the traceback: >> >> http://paste.yt-project.org/show/TE2DW8gc50urchv4JnUk >> >> In the traceback, it claims that it cannot access librockstar.so, but this >> library exists in the yt-x86_64/lib directory, which is explicitly in from >> my $LD_LIBRARY_PATH. I can seemingly get it to work, if I add that path to >> my $DYLD_LIBRARY_PATH environment variable, but then a bunch of other >> things break (like vi). Not sure what to do here. >> >> But assuming there is no vi problem here, even then, when I run the script >> above, I get failures. Here is the traceback: >> >> http://paste.yt-project.org/show/rCOT1eVT1szR1T6DEUDb >> >> I'm not sure what to do here. I've also tried running rockstar external >> to yt on an enzo dataset, but it doesn't appear that the version we install >> as part of yt allows it to work on HDF5 datasets. >> >> Has anyone run into any of these problems before? I'm at a bit of a loss >> as to how to proceed on any of these fronts. >> >> Cameron >> >> >> -- >> Cameron Hummels >> Postdoctoral Researcher >> Steward Observatory >> University of Arizona >> http://chummels.org >> >> _______________________________________________ >> yt-users mailing list >> yt-users(a)lists.spacepope.org >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >>

6 7

ANN: YT v0.1.0, a Julia wrapper for yt 3.0
by John ZuHone 12 Aug '14

12 Aug '14

Hi yt-er's, I just wanted to email the list and let everyone know about a new package I've developed called YT (yes, in caps), which as of today has its first release as v0.1.0. YT is a wrapper for yt for the Julia programming language. Julia is a new(-ish) high-level, high-performance dynamic programming language for technical computing. If you want to learn more about Julia, visit http://julialang.org. One of the most important things to note about Julia is that it has a "just-in-time" compiler, so it can be very fast (within a factor of few of C speed in some cases). There is also a lot of interoperability between Julia and Python, including wrappers for Matplotlib and Sympy, and even an extension of the IPython (soon to be Jupyter) notebook! A short summary of what YT exposes from yt in Julia: * Datasets * Some data objects (e.g., spheres, rectangular regions, slices, projections, profiles, etc.) * YTArrays and YTQuantities * Simple visualization tools (e.g., SlicePlot, ProjectionPlot, FixedResolutionBuffer) To find out more about the YT package, visit the documentation: http://www.jzuhone.com/yt_julia or the GitHub project page: http://github.com/jzuhone/YT.jl Thanks for reading, and sorry for the spam. Best, John Z

1 0

Problems using Rockstar in yt
by Cameron Hummels 12 Aug '14

12 Aug '14

Hello, I'm trying to use Rockstar with the new HaloCatalog functionality in yt 3.0, but I'm encountering some problems. For reference, this is an OS X box with the "yt" branch. I've tested it on two OS X systems to the same result. I modified my install_script.py to have it install Rockstar as described here: http://yt-project.org/docs/dev/analyzing/analysis_modules/halo_finders.html… Then, after rerunning my install script for the `yt` branch (ie yt-3.1 dev) I tried to run the script provided here in order to generate a halo catalog using HOP: http://yt-project.org/docs/dev/analyzing/analysis_modules/halo_transition.h… It can create the catalog fine with finder_method='hop', but when I change it to finder_method='rockstar', I get failures. This is the traceback: http://paste.yt-project.org/show/TE2DW8gc50urchv4JnUk In the traceback, it claims that it cannot access librockstar.so, but this library exists in the yt-x86_64/lib directory, which is explicitly in from my $LD_LIBRARY_PATH. I can seemingly get it to work, if I add that path to my $DYLD_LIBRARY_PATH environment variable, but then a bunch of other things break (like vi). Not sure what to do here. But assuming there is no vi problem here, even then, when I run the script above, I get failures. Here is the traceback: http://paste.yt-project.org/show/rCOT1eVT1szR1T6DEUDb I'm not sure what to do here. I've also tried running rockstar external to yt on an enzo dataset, but it doesn't appear that the version we install as part of yt allows it to work on HDF5 datasets. Has anyone run into any of these problems before? I'm at a bit of a loss as to how to proceed on any of these fronts. Cameron -- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org

2 1

ANN: DistArray 0.5 release
by Kurt Smith 11 Aug '14

11 Aug '14

==================================================== DistArray 0.5 release ==================================================== **Documentation:** http://distarray.readthedocs.org **License:** Three-clause BSD **Python versions:** 2.7, 3.3, and 3.4 **OS support:** \*nix and Mac OS X What is DistArray? ------------------ DistArray aims to bring the ease-of-use of NumPy to data-parallel high-performance computing. It provides distributed multi-dimensional NumPy arrays, distributed ufuncs, and distributed IO capabilities. It can efficiently interoperate with external distributed libraries like Trilinos. DistArray works with NumPy and builds on top of it in a flexible and natural way. 0.5 Release ----------- Noteworthy improvements in this release include: * closer alignment with NumPy's API, * support for Python 3.4 (existing support for Python 2.7 and 3.3), * a performance-oriented MPI-only mode for deployment on clusters and supercomputers, * a way to register user-defined functions to be callable locally on worker processes, * more consistent naming of sub-packages, * testing with MPICH2 (already tested against OpenMPI), * improved and expanded examples, * installed version testable via ``distarray.test()``, and * performance and scaling improvements. With this release, DistArray ready for real-world testing and deployment. The project is still evolving rapidly and we appreciate the continued input from the larger scientific-Python community. Existing features ----------------- DistArray: * supports NumPy-like slicing, reductions, and ufuncs on distributed multidimensional arrays; * has a client-engine process design -- data resides on the worker processes, commands are initiated from master; * allows full control over what is executed on the worker processes and integrates transparently with the master process; * allows direct communication between workers, bypassing the master process for scalability; * integrates with IPython.parallel for interactive creation and exploration of distributed data; * supports distributed ufuncs (currently without broadcasting); * builds on and leverages MPI via MPI4Py in a transparent and user-friendly way; * has basic support for unstructured arrays; * supports user-controllable array distributions across workers (block, cyclic, block-cyclic, and unstructured) on a per-axis basis; * has a straightforward API to control how an array is distributed; * has basic plotting support for visualization of array distributions; * separates the array’s distribution from the array’s data -- useful for slicing, reductions, redistribution, broadcasting, and other operations; * implements distributed random arrays; * supports ``.npy``-like flat-file IO and hdf5 parallel IO (via ``h5py``); leverages MPI-based IO parallelism in an easy-to-use and transparent way; and * supports the distributed array protocol [protocol]_, which allows independently developed parallel libraries to share distributed arrays without copying, analogous to the PEP-3118 new buffer protocol. Planned features and roadmap ---------------------------- Near-term features and improvements include: * array re-distribution capabilities; * lazy evaluation and deferred computation for latency hiding; * interoperation with Trilinos [Trilinos]_; and * distributed broadcasting support. The longer-term roadmap includes: * Integration with other packages [petsc]_ that subscribe to the distributed array protocol [protocol]_; * Distributed fancy indexing; * Out-of-core computations; * Support for distributed sorting and other non-trivial distributed algorithms; and * End-user control over communication and temporary array creation, and other performance aspects of distributed computations. History and funding ------------------- Brian Granger started DistArray as a NASA-funded SBIR project in 2008. Enthought picked it up as part of a DOE Phase II SBIR [SBIR]_ to provide a generally useful distributed array package. It builds on NumPy, MPI, MPI4Py, IPython, IPython.parallel, and interfaces with the Trilinos suite of distributed HPC solvers (via PyTrilinos [Trilinos]_). This material is based upon work supported by the Department of Energy under Award Number DE-SC0007699. This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor any agency thereof, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof. .. [protocol] http://distributed-array-protocol.readthedocs.org/en/rel-0.10.0/ .. [Trilinos] http://trilinos.org/ .. [petsc] http://www.mcs.anl.gov/petsc/ .. [SBIR] http://www.sbir.gov/sbirsearch/detail/410257 -- Kurt W. Smith, Ph.D. <ksmith(a)enthought.com> Enthought, Inc. <http://www.enthought.com> | 512.536.1057

1 0

Slice Plot Gadget Problem
by Jared Coughlin 11 Aug '14

11 Aug '14

Hello! I have a vanilla gadget snapshot (one from the lcdm sample run that the code comes with) that I was able to plot by following the instructions given here: http://yt-project.org/docs/3.0/cookbook/tipsy_notebook.html#tipsy-notebook I then moved on to the 'Making Smoothed Images' section of that page and tried the SlicePlot command. However, I get the following error: IndexError: index 32767 is out of bounds for axis 0 with size 32767 Basically, I'm not really sure what that means. It seems like it's overstepping the bounds of an array, as, for a zero-indexed array of size 32767, index 32767 would be out of bounds by one. I just wanted to see if anyone else has had this problem, and if so, how they fixed it? Thank you very much! -Jared

4 11

will yt 2.6 analysis scripts works for yt 3.0 ?
by Reju Sam John 11 Aug '14

11 Aug '14

Dear All, My yt version is --- Version = 2.6.1 Changeset = c7bae05dd4ef --- If I update it to 3.0 with the command: yt update --all , will my old analysis scripts(written for 2.6 ) works? Thank you. -- Reju Sam John

2 2

lagrangian volume of ICs?
by Brian O'Shea 09 Aug '14

09 Aug '14

Hi folks, I'm going back to resimulate some cosmological simulations with a nested-grid setup, and can't seem to figure out how to get yt to tell me where all of the dark matter particles in a given halo start out at high redshift. It seems that something like this used to exist in the Hub (i.e., http://lists.spacepope.org/pipermail/yt-users-spacepope.org/2011-August/001…), but that is not around any more. Would somebody mind sharing such a tool if they have already coded it up? Thanks! --Brian

3 4

yt-3.0 density gradient
by Lile Wang 08 Aug '14

08 Aug '14

Hi all, I am just worried about the problem associated to the density gradient field in yt-3.0. When I was trying to get the gradient of density field along z (actually any) direction, I got this error message (last few lines) when calling data[ 'density_gradient_z' ]: ~/lilew/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.pyc in selector(self) 529 if sclass is None: 530 raise YTDataSelectorNotImplemented(self._type_name) --> 531 self._selector = sclass(self) 532 return self._selector 533 ~/lilew/yt-x86_64/src/yt-hg/yt/geometry/selection_routines.so in yt.geometry.selection_routines.RegionSelector.__init__ (yt/geometry/selection_routines.c:20276)() RuntimeError: Error: bad Region in non-periodic domain along dimension 0. Region left edge = -0.01171875 code_length, Region right edge = 0.13671875 code_lengthDataset left edge = 0. 0 code_length, Dataset right edge = 1.0 code_length Similar things happen for the pressure gradient. The data input was generated by Enzo with AMR. In comparison, it works well with yt-2.6, where I refer to the gradient by data[ 'gradDensityZ' ]. Any ideas are appreciated. Thanks! Cheers, Lile ----------------------------------------------------------- Lile 'Patrick' Wang Dept. of Astroph. Sci., Princeton University, NJ 08544 USA Website : http://www.astro.princeton.edu/~lilew/ <http://www.astro.princeton.edu/%7Elilew/> Tel (Mobile): +1-(734)-834-3936 Tel (GVoice): +1-(510)-926-6831 -----------------------------------------------------------

2 2

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

yt-users August 2014