Mailman 3 May 2009 - yt-users

Re: [yt-users] Field existence check
by Matthew Turk 28 May '09

28 May '09

Hi Dave, Quick followup. I realized we do this already for Gas_Energy and Total_Energy, as you can see in yt/lagos/EnzoFields.py on line 71. Note that this might get sticky if you are deriving from StaticOutput or EnzoStaticOutput with a new field container and want these to be communicated up and down the hierarchy of possibilities. I think that is probably an extreme corner case, though. So, in short, it should work well, and if it doesn't, it is a bug and it should be fixed. -Matt On Wed, May 27, 2009 at 3:39 PM, Matthew Turk <matthewturk(a)gmail.com> wrote: > Hi Dave, > > It should be the same call to add field. However, the more I think > about it, the more I think it will default to using the data field > over the derived field, and so you should be safe adding it as a > derived field in *all* cases. I have conducted a few experiments and > this seems to be true over here. > > You should check this by adding a field that supplies bad values, or > raises an exception. However, from my reading of the > AMRGridPatch.get_data function, I believe it will first try the data > reader and then fall back to generating it on the fly. > hierarchy.field_list *only* contains fields it finds in the data > files, and that is what the grid type checks against. > AMRData.get_data also checks against this list and will generate the > field if it is not there. > > Can you give it a shot, let us know if this is not the case? > > -Matt > > > > On May 27, 2009, at 3:14 PM, David Collins <dcollins(a)physics.ucsd.edu> > wrote: > >>> pf = load("my_data") >>> if not "DavesField" in pf.h.field_list: >>> add_field(...) >> >> >> add_field is the same one I've been using from lagos? >> >> I have some datasets that have this field, some that don't, all being >> used at the same time. Can I sometimes add the field, sometimes not? >> >> Thanks a bunch, >> >> d. >> >>> >>> Hope that helps! >>> >>> -Matt >>> >>> On Wed, May 27, 2009 at 2:45 PM, david collins >>> <antpuncher(a)gmail.com> wrote: >>>> Hi-- >>>> >>>> I have a derived quantity that I only write to disk in some >>>> simulations. Is there a way to check for the existence of a field >>>> in >>>> the data set, so that if it doesn't exist I can derive it? >>>> >>>> Thanks, >>>> d. >>>> _______________________________________________ >>>> yt-users mailing list >>>> yt-users(a)lists.spacepope.org >>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >>>> >>> _______________________________________________ >>> yt-users mailing list >>> yt-users(a)lists.spacepope.org >>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >>> >>> >> _______________________________________________ >> yt-users mailing list >> yt-users(a)lists.spacepope.org >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >

1 0

Field existence check
by david collins 27 May '09

27 May '09

Hi-- I have a derived quantity that I only write to disk in some simulations. Is there a way to check for the existence of a field in the data set, so that if it doesn't exist I can derive it? Thanks, d.

3 3

Unigrid datasets, covering grid
by david collins 20 May '09

20 May '09

Hi, Everyone-- I have a unigrid run that I want to compare to another one, run on a different number of processors. I'd like to extract them both to a single numpy array (for each run) so I can difference. Does the covering grid do any interpolation if I want a single N^3 numpy array from an N^3 enzo dump on X processors? Thanks, d.

2 1

Halo finder changes
by Matthew Turk 13 May '09

13 May '09

Hi everyone, Stephen Skory has added the Friends-of-friends halo finder to the yt trunk (not yt-1.0) and we've now enabled it as a possibility for finding halos in a simulation. Because there are now two mechanisms for finding halos, we've also changed the API a little bit and made the halo finders available through yt.mods. So to run the HOP halo finder you would do the following: from yt.mods import * pf = load("data0001") halo_list = HaloFinder(pf) and to run FOF you have to call it slightly differently: from yt.mods import * pf = load("data0001") halo_list = FOFHaloFinder(pf) To get FOF to compile, you'll have to rerun "python setup.py develop" or "python setup.py install" depending on how yt was installed initially. (The install_script does the former, but most hand-installs do the latter.) With this new API, both halo finders run in parallel. I'll save any discussion of that for Stephen, however, as there are some caveats to running HOP in parallel. Additionally, as of right now they dont scale as well as we'd like -- but we're working on it. Additionally, as a result of this change we have intentionally broken compatibility by removing the import of the 'hop' module in yt/mods.py. Hopefully this has not caused too many problems; all of the old, largely duplicated-in-multiple-places classes and interfaces have now been consolidated and placed in yt/lagos/HaloFinding.py. Thanks to Stephen for his hard work adding in FOF and working with the parallel interface! Best wishes, and let us know if there are any problems, Matt

3 5

scripts, the write_out method, plot_options, and 1D profiles, oh my!
by David A. Ventimiglia 12 May '09

12 May '09

Hello, I've a few more yt newbie questions, and I'm grateful for the patience of the yt gods. 1. How do I write scripts? Presumably, they're Python scripts with the appropriate imports, but what do I import? When I try, say, 'import yt.lagos as lagos' from within a vanilla Python interpreter, I get this error from with numpy: ...undefined symbol: PyUnicodeUCS2_FromUnicode 2. How do I use the write_out() method to export data? Specifically, I tried the following, which generated an error: x = pf.h.ray(0, [0.5, 0.5], "Density") x.write_out("x.dat",fields=["Density"]) Eventually, I'll probably learn how to do everything I need to do from within yt, so this is a lower priority for me. 3. The background to question no. 2 is that I'd like to generate, not a radial profile, but a 1D profile along one axis through the center of a box, and I thought I'd see what my options are using other plotting tools. But maybe EnzoOrthoRay isn't even the best way to go about this. What is the best way to make non-radial, 1D profiles along one axis? 4. I did generate 1D line profiles using PlotCollection.add_ortho_ray (though I guess they're not binned), which may be just what I need. In that case (and for other plots), how does one change the look of the plot, such as plot ranges, line weights, colors, types, etc.? As always, thanks in advance! Best, David

2 2

yt install script on NICS Kraken
by David A. Ventimiglia 11 May '09

11 May '09

Hello, I'm new to yt, so I apologize if I'm doing something dumb. But, I'm trying to install yt on Kraken and am having a little trouble. install_script.sh appeared to run fine, but when I run iyt I get these errors: yt.lagos WARNING 2009-05-11 15:09:47,175 No HDF4 support yt.lagos WARNING 2009-05-11 15:09:47,185 No h5py. Data serialization will fail. Traceback (most recent call last): File "/nics/b/home/ventimig/scratch/yt-x86_64/bin/iyt", line 7, in <module> execfile(__file__) File "/nics/b/home/ventimig/scratch/yt-x86_64/src/yt-trunk-svn/scripts/iyt", line 3, in <module> from yt.mods import * File "/nics/b/home/ventimig/scratch/yt-x86_64/src/yt-trunk-svn/yt/mods.py", line 32, in <module> import yt.lagos as lagos File "/nics/b/home/ventimig/scratch/yt-x86_64/src/yt-trunk-svn/yt/lagos/__init__.py", line 74, in <module> import HDF5LightReader ImportError: libpgc.so: cannot open shared object file: No such file or directory Evidently it can't find one of the Portland Group libraries, but I thought all that'd be taken care of by having the PrgEnv-pgi module loaded. What am I doing wrong? Thanks! Best, David Ventimiglia ventimig(a)msu.edu

2 5

a quick note on yt, h5py, and hdf5
by Britton Smith 10 May '09

10 May '09

Hi everyone, I just want to mention a few things about h5py, now that we have moved to it for yt. h5py works with both the hdf5 1.6 and 1.8 api's. Currently, yt installs with hdf5 1.6.8 and uses the 1.6 api. If you experience any problems trying to use yt with hdf5 1.8, I would recommend moving back to hdf5 1.6. hdf5 1.8 does have added functionality over 1.6, but currently yt does not use any of it. Britton

1 0

h5py and PyTables problem with HDF5 1.8.3
by Rick Wagner 09 May '09

09 May '09

Hi, This is just a word of caution if anyone finds themselves getting an error from h5py or PyTables about H5Zregister. Apparently something changed in that release (1.8.3) which trips up both of the packages. 1.8.2 seems to be fine. --Rick Messages: Installation errors when building with `--prefix' http://tinyurl.com/q6mrwf From: Craig Bookwalter <craig.bookwal...(a)gmail.com> Date: Fri, 8 May 2009 10:13:00 -0400 Subject: Re: Installation errors when building with `--prefix' Hi Andrew, I wouldn't worry about the `--prefix' option just yet--seems that it's a problem with HDF5 1.8.3 -- here's a quote from a guy who works on PyTables (Francesc): Mmh, I've just tried HDF5 1.8.3 and had errors too: EHDF5-DIAG: Error detected in HDF5 (1.8.3) thread 0: #000: H5Dio.c line 174 in H5Dread(): can't read data major: Dataset minor: Read failed #001: H5Dio.c line 404 in H5D_read(): can't read data major: Dataset minor: Read failed #002: H5Dchunk.c line 1548 in H5D_chunk_read(): unable to read raw data chunk major: Low-level I/O minor: Read failed #003: H5Dchunk.c line 2561 in H5D_chunk_lock(): data pipeline read failed major: Data filters minor: Filter operation failed #004: H5Z.c line 1006 in H5Z_pipeline(): required filter is not registered major: Data filters minor: Read failed I'm afraid that the LZO compressor is not correctly detected (they have probably changed the way to register filters). (end quote) Hope that helps! Thanks for the quick reply. --cb

1 0

yt moving to h5py
by Britton Smith 06 May '09

06 May '09

Currently, the hdf5 operations for object serialization in yt are handled by pytables. However, pytables is not the only python hdf5 module out there. For my own non-yt code, I have been using h5py ( http://code.google.com/p/h5py/). For simple hdf5 i/o, I find h5py to be far more intuitive and flexible than pytables. If you want to do hdf5 in python, I strongly recommend h5py over pytables. It's for these reasons that we're moving yt from using pytables as a data-serialization backend to using h5py. Apart from casual use, h5py offers additional benefits over pytables that are relevant to yt. h5py is faster and relies on fewer python object than pytables. h5py works better with yt in parallel and its design is generally better suited to yt. Switching dependencies is something we want to do as little as possible, but it seems worth it in this case and will most likely continue to pay off as yt grows. This evening, we will be committing the switch from pytables to h5py dependency to the yt trunk. We have altered the install script accordingly so rerunning that will do everything that is needed. h5py can also be easily installed on it own. Set the following environment variables: HDF5_DIR=path to hdf5 HDF5_API=16 Then do: sudo easy_install h5py (You may not need to use sudo, depending on where easy_install is installed.) None of the yt function calls have changed. Only the internal calls to tables functionality have been replaced with h5py calls. Additionally, it should be noted that this does not affect any of the dataset i/o as this is handled by Matt's specially built hdf5 reader. So far, we have tested this on various machines and it seems to be working. If anyone encounters any problems that may be related to this or has any problems installing h5py, please contact us as soon as possible. Britton Smith

1 0

Announcing the EnzoTools Barn
by Britton Smith 04 May '09

04 May '09

Dear Enzo users, I'm very pleased to announce the opening of the EnzoTools Barn: http://barn.enzotools.org In their time working with Enzo, users have developed a host of useful tools, scripts, and snippets of code for performing any number of tasks related to Enzo. The Barn is the place to share these creations with the larger Enzo community. Each submission will be made into a Mercurial repository to which only the author will have write access. This will allow the author to make updates and changes to their code any time they like. An entry will be automatically generated on the Barn main page with a short description and screenshot. Each entry will provide links to downloadable tarballs and the code's revision history. Submission instructions are available at: http://barn.enzotools.org/contribute.html If you've written something you find useful, odds are that it will be useful to many others, so please share! More information on the submission guidelines is available at: http://barn.enzotools.org/guidelines.html , and you can follow the entire EnzoTools collective at http://planet.enzotools.org/ , where all of the different changelogs and feeds from the EnzoTools projects are aggregated. Sincerely, Britton Smith http://barn.enzotools.org/ http://planet.enzotools.org/ p.s. - A very special thanks to Matt Turk for providing the technical know how to make this all possible.

1 0