Hello,
Just a quick question that jumps out at me, it is possible to not use yt's
own hdf5 opening code and do it all in the frontend because I have a
library that abstracts the two data formats we have for the simulations,
only one is hdf5.
Stuart
On 24 August 2013 06:51, Nathan Goldbaum
Hey Stewart,
Take a look at the FLASH frontend, specifically yt/frontends/flash/data_structures.py.
In FLASH, the IO is handled by relying on h5py. If you look at the beginning of FLASHStaticOutput.__init__, you'll see that a _handle attribute is assigned to the instance. This _handle is a reference to the open h5py File object. Optionally, FLASH allows a particle file which is written to disk seperately from the main hydro data file, so that is optionally opened as well and the File handle is assigned to _particle_handle.
FLASHStaticOutput._handle is then used many times elsewhere to read data from the output file and to create an in-memory view of the FLASH AMR data structure. The AMR hierarchy is handled by FLASHHierarchy, and data IO is also logically handled there, although via a FLASHIOHandler instance. The IOHandler is defined using a bit of metaclass magic via the io_registry, which is set up in the IOHandler base class in yt/utilities/io_handler.py. If you look in yt/frontends/flash/io.py, you'll find the definition for FLASHIOHandler, which implements all the IO for hydro data. Since it uses h5py, this all happens via a dict-like interface.
I'm not sure if there is an exhaustive list of things that a StaticOutput object needs to implement. There is probably a minimal number of parameters that the rest of the code needs to know about and that list of things should probably be written down somewhere. I would look at a few of the frontends (FLASH and Orion are simple, relatively clean examples; the enzo frontend is unfortunately a bit messy, there's been a lot of work recently on the Ramses, SPH, ART, and ARTIO frontends) and try to implement functionality that is common between the codes. A couple examples spring to mind. All StaticOutput sublcasses need to define a parameters dict and a minimal set of parameters needed elsewhere. Similarly, there needs to be a units dict which defines unit conversions.
Frontends also define a fields.py file. Right now, frontends needs to implement fields with enzo field names (so, 'Density', 'Pressure', 'x-velocity', 'GasEnergy', 'TotalEnergy', etc.) because universal_fields.py assumes these field names are defined and in CGS units. There's an effort to make this situation better, both by providing an updated list of field names, and a unit system, but that's currently on the backburner while Matt pushes ahead on necessary yt-3.0 infrastructure work.
You should also supply on-disk fields without conversions to CGS and with native field names - this is how we distinguish between 'yt' fields, which always have units and enzo field names, and on-disk fields, which are always in code units. As a curious aside, this ad hoc system is broken for enzo, making it a little non-trivial to access on-disk enzo data. Like I said, the units refactor will make this better :)
I'll let Matt explain the distinction between known fields and unknown fields... ;)
Anyway, like Matt said on IRC, please let us know about sticking points in this process. We really should have better documentation for how to write a frontend and it would be cool if we could streamline the process or simplify by refactoring and cleaning up the frontend code.
HTH,
-Nathan
On Fri, Aug 23, 2013 at 4:37 PM, Stuart Mumford
wrote: Hello,
First an introduction, I am Stuart Mumford I am doing a solar physics PhD at Sheffield University in the UK. I work primarily on MHD wave simulations of the solar atmosphere studying energy propagation and wave excitation. The code SAC (http://www.aanda.org/10.1051/0004-6361:200809800) solves for perturbations ontop of a fixed background to deal with the very wide range of the parameters e.g. density and temperature in the solar atmosphere. I also attended SciPy this year, where I met some of you and discovered yt, mainly as a consequence of my contributions to SunPy ( www.sunpy.org) which is a solar physics data analysis library.
So that's that over with, now having been playing with yt on and off since SciPy I started using it properly this week, as the need arose. I am very impressed with yt's capabilities and I want to be able to make better use of it. To that end I am trying to write a frontend for SAC, as I have been speaking to some of you about in IRC (I am Cadair in IRC and on GH).
I have some questions regarding this, as far as I can ascertain a frontend provides a mapper between the stored data and a abstracted format that yt understands? I have started looking in the code for FLASH and Stream to try and get an idea where to start and not got very far. So some questions to start:
Where should the file handling be done? Where is the file opened? What things should the StaticOutput subclass define?
Thanks once again for your help on IRC over the last few days and I hope you can point me in the right direction.
Stuart
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org