
Hi everyone, Currently, the hdf5 operations in yt are handled by pytables. However, pytables is not the only python hdf5 module out there. For my own non-yt code, I have been using h5py (http://code.google.com/p/h5py/). For simple hdf5 i/o, I find h5py to be far more intuitive and flexible than pytables. I want to propose to people here that we switch the yt trunk over to h5py before the yt 1.5 release. Before I enter my sales pitch, I want to make it known that the work is already done. I converted a version of the yt-trunk to h5py this weekend. Matt and I have been testing it on various machines and it seems to be working. None of the yt function calls have changed. Only the internal calls to tables functionality have been replaced with h5py calls. Other than the ease of casual use, some things in h5py's favor include: - h5py is faster and relies on fewer python objects than tables - tables is not designed for our style of IO - better with reloading nodes than pytables (something very relevant to yt, esp. in parallel) - h5py installs easily: I did it from the source, which requires Cython, but it should be installable with easy_install with these environment variables: HDF5_DIR=path to hdf5 HDF5_API=16 I know that switching dependencies is something we want to do as little as possible, but I think it's worth it in this case. Matt will alter the install script so this change should be relatively seemless. If anyone has any major objections, please raise them. Matt, please chime in if I missed anything or presented anything incorrectly. Britton

I want to propose to people here that we switch the yt trunk over to h5py before the yt 1.5 release.
No complaints from me! _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________

Awesome. Here's a patch I just created to apply to trunk to see the effects: http://paste.enzotools.org/show/119/ I ask that if you apply this you NOT commit it. If we could get at least one person to apply this patch and run some simple calculations, that would be outstanding, and good enough for me. It includes an updated script that will also optionally install mercurial, as with the now-launched barn we want to encourage more of that. When we move to h5py over tables, Britton will send an email announcing this to yt-users, including the note about easy_install. -Matt On Tue, May 5, 2009 at 1:05 PM, Stephen Skory <stephenskory@yahoo.com> wrote:
I want to propose to people here that we switch the yt trunk over to h5py before the yt 1.5 release.
No complaints from me!
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________ _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

Also, as a quick sidenote, remember that you can always download pastes on the command line: yt_lodgeit.py --download=119 will get you this patch. On Tue, May 5, 2009 at 4:58 PM, Matthew Turk <matthewturk@gmail.com> wrote:
Awesome.
Here's a patch I just created to apply to trunk to see the effects:
http://paste.enzotools.org/show/119/
I ask that if you apply this you NOT commit it. If we could get at least one person to apply this patch and run some simple calculations, that would be outstanding, and good enough for me.
It includes an updated script that will also optionally install mercurial, as with the now-launched barn we want to encourage more of that. When we move to h5py over tables, Britton will send an email announcing this to yt-users, including the note about easy_install.
-Matt
On Tue, May 5, 2009 at 1:05 PM, Stephen Skory <stephenskory@yahoo.com> wrote:
I want to propose to people here that we switch the yt trunk over to h5py before the yt 1.5 release.
No complaints from me!
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________ _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

It all works for me? Anybody have problems so far? If not, I say we do this this afternoon. On Tue, May 5, 2009 at 5:58 PM, Matthew Turk <matthewturk@gmail.com> wrote:
Also, as a quick sidenote, remember that you can always download pastes on the command line:
yt_lodgeit.py --download=119
will get you this patch.
Awesome.
Here's a patch I just created to apply to trunk to see the effects:
http://paste.enzotools.org/show/119/
I ask that if you apply this you NOT commit it. If we could get at least one person to apply this patch and run some simple calculations, that would be outstanding, and good enough for me.
It includes an updated script that will also optionally install mercurial, as with the now-launched barn we want to encourage more of that. When we move to h5py over tables, Britton will send an email announcing this to yt-users, including the note about easy_install.
-Matt
On Tue, May 5, 2009 at 1:05 PM, Stephen Skory <stephenskory@yahoo.com> wrote:
I want to propose to people here that we switch the yt trunk over to
On Tue, May 5, 2009 at 4:58 PM, Matthew Turk <matthewturk@gmail.com> wrote: h5py before the yt 1.5 release.
No complaints from me!
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ <http://physics.ucsd.edu/%7Esskory/>_.>/ _Graduate Student ________________________________(_)_\(_)_______________ _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

I agree. On Wed, May 6, 2009 at 12:22 PM, Britton Smith <brittonsmith@gmail.com> wrote:
It all works for me? Anybody have problems so far? If not, I say we do this this afternoon.
On Tue, May 5, 2009 at 5:58 PM, Matthew Turk <matthewturk@gmail.com> wrote:
Also, as a quick sidenote, remember that you can always download pastes on the command line:
yt_lodgeit.py --download=119
will get you this patch.
On Tue, May 5, 2009 at 4:58 PM, Matthew Turk <matthewturk@gmail.com> wrote:
Awesome.
Here's a patch I just created to apply to trunk to see the effects:
http://paste.enzotools.org/show/119/
I ask that if you apply this you NOT commit it. If we could get at least one person to apply this patch and run some simple calculations, that would be outstanding, and good enough for me.
It includes an updated script that will also optionally install mercurial, as with the now-launched barn we want to encourage more of that. When we move to h5py over tables, Britton will send an email announcing this to yt-users, including the note about easy_install.
-Matt
On Tue, May 5, 2009 at 1:05 PM, Stephen Skory <stephenskory@yahoo.com> wrote:
I want to propose to people here that we switch the yt trunk over to h5py before the yt 1.5 release.
No complaints from me!
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________ _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

Matt, please chime in if I missed anything or presented anything incorrectly.
Hi Britton -- I think the only things I'd like to add are that we've put the repository up here: http://hg.enzotools.org/yt-trunk-h5py/ (might be a few SVN revisions out of date) and to say THANKS for your hard work on this. Guys, I really think H5Py is the way to go. Note that we are *not* changing away from the HDF5LightReader for IO that we use to read Enzo data -- tables is the only thing being replaced for *now*, which affects data serialization (This may change in the future, though.) -Matt
participants (3)
-
Britton Smith
-
Matthew Turk
-
Stephen Skory