Re: [yt-users] Field existence check
by Matthew Turk
Hi Dave,
Quick followup. I realized we do this already for Gas_Energy and
Total_Energy, as you can see in yt/lagos/EnzoFields.py on line 71.
Note that this might get sticky if you are deriving from StaticOutput
or EnzoStaticOutput with a new field container and want these to be
communicated up and down the hierarchy of possibilities. I think that
is probably an extreme corner case, though.
So, in short, it should work well, and if it doesn't, it is a bug and
it should be fixed.
-Matt
On Wed, May 27, 2009 at 3:39 PM, Matthew Turk <matthewturk(a)gmail.com>
wrote:
> Hi Dave,
>
> It should be the same call to add field. However, the more I think
> about it, the more I think it will default to using the data field
> over the derived field, and so you should be safe adding it as a
> derived field in *all* cases. I have conducted a few experiments and
> this seems to be true over here.
>
> You should check this by adding a field that supplies bad values, or
> raises an exception. However, from my reading of the
> AMRGridPatch.get_data function, I believe it will first try the data
> reader and then fall back to generating it on the fly.
> hierarchy.field_list *only* contains fields it finds in the data
> files, and that is what the grid type checks against.
> AMRData.get_data also checks against this list and will generate the
> field if it is not there.
>
> Can you give it a shot, let us know if this is not the case?
>
> -Matt
>
>
>
> On May 27, 2009, at 3:14 PM, David Collins <dcollins(a)physics.ucsd.edu>
> wrote:
>
>>> pf = load("my_data")
>>> if not "DavesField" in pf.h.field_list:
>>> add_field(...)
>>
>>
>> add_field is the same one I've been using from lagos?
>>
>> I have some datasets that have this field, some that don't, all being
>> used at the same time. Can I sometimes add the field, sometimes not?
>>
>> Thanks a bunch,
>>
>> d.
>>
>>>
>>> Hope that helps!
>>>
>>> -Matt
>>>
>>> On Wed, May 27, 2009 at 2:45 PM, david collins
>>> <antpuncher(a)gmail.com> wrote:
>>>> Hi--
>>>>
>>>> I have a derived quantity that I only write to disk in some
>>>> simulations. Is there a way to check for the existence of a field
>>>> in
>>>> the data set, so that if it doesn't exist I can derive it?
>>>>
>>>> Thanks,
>>>> d.
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users(a)lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users(a)lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users(a)lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
9 years, 8 months
Field existence check
by david collins
Hi--
I have a derived quantity that I only write to disk in some
simulations. Is there a way to check for the existence of a field in
the data set, so that if it doesn't exist I can derive it?
Thanks,
d.
9 years, 8 months
Unigrid datasets, covering grid
by david collins
Hi, Everyone--
I have a unigrid run that I want to compare to another one, run on a
different number of processors. I'd like to extract them both to a
single numpy array (for each run) so I can difference. Does the
covering grid do any interpolation if I want a single N^3 numpy array
from an N^3 enzo dump on X processors?
Thanks,
d.
9 years, 9 months
Halo finder changes
by Matthew Turk
Hi everyone,
Stephen Skory has added the Friends-of-friends halo finder to the yt
trunk (not yt-1.0) and we've now enabled it as a possibility for
finding halos in a simulation. Because there are now two mechanisms
for finding halos, we've also changed the API a little bit and made
the halo finders available through yt.mods. So to run the HOP halo
finder you would do the following:
from yt.mods import *
pf = load("data0001")
halo_list = HaloFinder(pf)
and to run FOF you have to call it slightly differently:
from yt.mods import *
pf = load("data0001")
halo_list = FOFHaloFinder(pf)
To get FOF to compile, you'll have to rerun "python setup.py develop"
or "python setup.py install" depending on how yt was installed
initially. (The install_script does the former, but most
hand-installs do the latter.) With this new API, both halo finders
run in parallel. I'll save any discussion of that for Stephen,
however, as there are some caveats to running HOP in parallel.
Additionally, as of right now they dont scale as well as we'd like --
but we're working on it.
Additionally, as a result of this change we have intentionally broken
compatibility by removing the import of the 'hop' module in
yt/mods.py. Hopefully this has not caused too many problems; all of
the old, largely duplicated-in-multiple-places classes and interfaces
have now been consolidated and placed in yt/lagos/HaloFinding.py.
Thanks to Stephen for his hard work adding in FOF and working with the
parallel interface!
Best wishes, and let us know if there are any problems,
Matt
9 years, 9 months
scripts, the write_out method, plot_options, and 1D profiles, oh my!
by David A. Ventimiglia
Hello,
I've a few more yt newbie questions, and I'm grateful for the patience
of the yt gods.
1. How do I write scripts? Presumably, they're Python scripts with the
appropriate imports, but what do I import? When I try, say, 'import
yt.lagos as lagos' from within a vanilla Python interpreter, I get this
error from with numpy:
...undefined symbol: PyUnicodeUCS2_FromUnicode
2. How do I use the write_out() method to export data? Specifically, I
tried the following, which generated an error:
x = pf.h.ray(0, [0.5, 0.5], "Density")
x.write_out("x.dat",fields=["Density"])
Eventually, I'll probably learn how to do everything I need to do from
within yt, so this is a lower priority for me.
3. The background to question no. 2 is that I'd like to generate, not a
radial profile, but a 1D profile along one axis through the center of a
box, and I thought I'd see what my options are using other plotting
tools. But maybe EnzoOrthoRay isn't even the best way to go about this.
What is the best way to make non-radial, 1D profiles along one axis?
4. I did generate 1D line profiles using PlotCollection.add_ortho_ray
(though I guess they're not binned), which may be just what I need. In
that case (and for other plots), how does one change the look of the
plot, such as plot ranges, line weights, colors, types, etc.?
As always, thanks in advance!
Best,
David
9 years, 9 months
yt install script on NICS Kraken
by David A. Ventimiglia
Hello,
I'm new to yt, so I apologize if I'm doing something dumb. But, I'm
trying to install yt on Kraken and am having a little trouble.
install_script.sh appeared to run fine, but when I run iyt I get these
errors:
yt.lagos WARNING 2009-05-11 15:09:47,175 No HDF4 support
yt.lagos WARNING 2009-05-11 15:09:47,185 No h5py. Data serialization will fail.
Traceback (most recent call last):
File "/nics/b/home/ventimig/scratch/yt-x86_64/bin/iyt", line 7, in <module>
execfile(__file__)
File "/nics/b/home/ventimig/scratch/yt-x86_64/src/yt-trunk-svn/scripts/iyt", line 3, in <module>
from yt.mods import *
File "/nics/b/home/ventimig/scratch/yt-x86_64/src/yt-trunk-svn/yt/mods.py", line 32, in <module>
import yt.lagos as lagos
File "/nics/b/home/ventimig/scratch/yt-x86_64/src/yt-trunk-svn/yt/lagos/__init__.py", line 74, in <module>
import HDF5LightReader
ImportError: libpgc.so: cannot open shared object file: No such file or directory
Evidently it can't find one of the Portland Group libraries, but I
thought all that'd be taken care of by having the PrgEnv-pgi module
loaded. What am I doing wrong? Thanks!
Best,
David Ventimiglia
ventimig(a)msu.edu
9 years, 9 months
a quick note on yt, h5py, and hdf5
by Britton Smith
Hi everyone,
I just want to mention a few things about h5py, now that we have moved to it
for yt. h5py works with both the hdf5 1.6 and 1.8 api's. Currently, yt
installs with hdf5 1.6.8 and uses the 1.6 api. If you experience any
problems trying to use yt with hdf5 1.8, I would recommend moving back to
hdf5 1.6. hdf5 1.8 does have added functionality over 1.6, but currently yt
does not use any of it.
Britton
9 years, 9 months
h5py and PyTables problem with HDF5 1.8.3
by Rick Wagner
Hi,
This is just a word of caution if anyone finds themselves getting an
error from h5py or PyTables about H5Zregister. Apparently something
changed in that release (1.8.3) which trips up both of the packages.
1.8.2 seems to be fine.
--Rick
Messages:
Installation errors when building with `--prefix'
http://tinyurl.com/q6mrwf
From: Craig Bookwalter <craig.bookwal...(a)gmail.com>
Date: Fri, 8 May 2009 10:13:00 -0400
Subject: Re: Installation errors when building with `--prefix'
Hi Andrew,
I wouldn't worry about the `--prefix' option just yet--seems that
it's a
problem with HDF5 1.8.3 -- here's a quote from a guy who works on
PyTables
(Francesc):
Mmh, I've just tried HDF5 1.8.3 and had errors too:
EHDF5-DIAG: Error detected in HDF5 (1.8.3) thread 0:
#000: H5Dio.c line 174 in H5Dread(): can't read data
major: Dataset
minor: Read failed
#001: H5Dio.c line 404 in H5D_read(): can't read data
major: Dataset
minor: Read failed
#002: H5Dchunk.c line 1548 in H5D_chunk_read(): unable to read raw
data chunk
major: Low-level I/O
minor: Read failed
#003: H5Dchunk.c line 2561 in H5D_chunk_lock(): data pipeline read
failed
major: Data filters
minor: Filter operation failed
#004: H5Z.c line 1006 in H5Z_pipeline(): required filter is not
registered
major: Data filters
minor: Read failed
I'm afraid that the LZO compressor is not correctly detected (they have
probably changed the way to register filters).
(end quote)
Hope that helps! Thanks for the quick reply.
--cb
9 years, 9 months
yt moving to h5py
by Britton Smith
Currently, the hdf5 operations for object serialization in yt are handled by
pytables. However, pytables is not the only python hdf5 module out there.
For my own non-yt code, I have been using h5py (
http://code.google.com/p/h5py/). For simple hdf5 i/o, I find h5py to be far
more intuitive and flexible than pytables. If you want to do hdf5 in
python, I strongly recommend h5py over pytables. It's for these reasons
that we're moving yt from using pytables as a data-serialization backend to
using h5py.
Apart from casual use, h5py offers additional benefits over pytables that
are relevant to yt. h5py is faster and relies on fewer python object than
pytables. h5py works better with yt in parallel and its design is generally
better suited to yt. Switching dependencies is something we want to do as
little as possible, but it seems worth it in this case and will most likely
continue to pay off as yt grows. This evening, we will be committing the
switch from pytables to h5py dependency to the yt trunk. We have altered
the install script accordingly so rerunning that will do everything that is
needed.
h5py can also be easily installed on it own. Set the following environment
variables:
HDF5_DIR=path to hdf5
HDF5_API=16
Then do: sudo easy_install h5py
(You may not need to use sudo, depending on where easy_install is
installed.)
None of the yt function calls have changed. Only the internal calls to
tables functionality have been replaced with h5py calls. Additionally, it
should be noted that this does not affect any of the dataset i/o as this is
handled by Matt's specially built hdf5 reader. So far, we have tested this
on various machines and it seems to be working. If anyone encounters any
problems that may be related to this or has any problems installing h5py,
please contact us as soon as possible.
Britton Smith
9 years, 9 months
Announcing the EnzoTools Barn
by Britton Smith
Dear Enzo users,
I'm very pleased to announce the opening of the EnzoTools Barn:
http://barn.enzotools.org
In their time working with Enzo, users have developed a host of useful
tools, scripts, and snippets of code for performing any number of tasks
related to Enzo. The Barn is the place to share these creations with the
larger Enzo community.
Each submission will be made into a Mercurial repository to which only the
author will have write access. This will allow the author to make updates
and changes to their code any time they like. An entry will be
automatically generated on the Barn main page with a short description and
screenshot. Each entry will provide links to downloadable tarballs and the
code's revision history. Submission instructions are available at:
http://barn.enzotools.org/contribute.html
If you've written something you find useful, odds are that it will be useful
to many others, so please share!
More information on the submission guidelines is available at:
http://barn.enzotools.org/guidelines.html , and you can follow the entire
EnzoTools collective at http://planet.enzotools.org/ , where all of the
different changelogs and feeds from the EnzoTools projects are aggregated.
Sincerely,
Britton Smith
http://barn.enzotools.org/
http://planet.enzotools.org/
p.s. - A very special thanks to Matt Turk for providing the technical know
how to make this all possible.
9 years, 9 months
