The hook for load() is in your StaticOutput class's _is_valid() method. Take a look at the definition of the load function, in yt/convenience.py. Also keep in mind that load isn't strictly necessary, it's just there for syntactic sugar. On Saturday, August 24, 2013, Stuart Mumford wrote:
Cool,
So how can I get the filename?
Ok so more generally what is the pipeline for reading a file that I need to replicate?
how to go from
load('myfile') ... ... ...
to returning a subclass of StaticOutput?
Stuart
On 24 August 2013 08:50, Kacper Kowalik
wrote: On 24.08.2013 09:46, Stuart Mumford wrote:
Hello,
Just a quick question that jumps out at me, it is possible to not use yt's own hdf5 opening code and do it all in the frontend because I have a library that abstracts the two data formats we have for the simulations, only one is hdf5.
Hi Stuart, I doesn't matter what you use to get data from disk to memory in your_frontend/io.py as long as you get the job done ;) Cheers, Kacper
Stuart
On 24 August 2013 06:51, Nathan Goldbaum
wrote: Hey Stewart,
Take a look at the FLASH frontend, specifically yt/frontends/flash/data_structures.py.
In FLASH, the IO is handled by relying on h5py. If you look at the beginning of FLASHStaticOutput.__init__, you'll see that a _handle attribute is assigned to the instance. This _handle is a reference to
open h5py File object. Optionally, FLASH allows a particle file which is written to disk seperately from the main hydro data file, so that is optionally opened as well and the File handle is assigned to _particle_handle.
FLASHStaticOutput._handle is then used many times elsewhere to read data from the output file and to create an in-memory view of the FLASH AMR data structure. The AMR hierarchy is handled by FLASHHierarchy, and data IO is also logically handled there, although via a FLASHIOHandler instance. The IOHandler is defined using a bit of metaclass magic via the io_registry, which is set up in the IOHandler base class in yt/utilities/io_handler.py. If you look in yt/frontends/flash/io.py, you'll find the definition for FLASHIOHandler, which implements all the IO for hydro data. Since it uses h5py, this all happens via a dict-like interface.
I'm not sure if there is an exhaustive list of things that a StaticOutput object needs to implement. There is probably a minimal number of parameters that the rest of the code needs to know about and that list of things should probably be written down somewhere. I would look at a few of the frontends (FLASH and Orion are simple, relatively clean examples;
the the
enzo frontend is unfortunately a bit messy, there's been a lot of work recently on the Ramses, SPH, ART, and ARTIO frontends) and try to implement functionality that is common between the codes. A couple examples spring to mind. All StaticOutput sublcasses need to define a parameters dict and a minimal set of parameters needed elsewhere. Similarly, there needs to be a units dict which defines unit conversions.
Frontends also define a fields.py file. Right now, frontends needs to implement fields with enzo field names (so, 'Density', 'Pressure', 'x-velocity', 'GasEnergy', 'TotalEnergy', etc.) because universal_fields.py assumes these field names are defined and in CGS units. There's an effort to make this situation better, both by providing an updated list of field names, and a unit system, but that's currently on the backburner while Matt pushes ahead on necessary yt-3.0 infrastructure work.
You should also supply on-disk fields without conversions to CGS and with native field names - this is how we distinguish between 'yt' fields, which always have units and enzo field names, and on-disk fields, which are always in code units. As a curious aside, this ad hoc system is broken for enzo, making it a little non-trivial to access on-disk enzo data. Like I said, the units refactor will make this better :)
I'll let Matt explain the distinction between known fields and unknown fields... ;)
Anyway, like Matt said on IRC, please let us know about sticking points in this process. We really should have better documentation for how to write a frontend and it would be cool if we could streamline the pro