This week, after Cameron announced and Jeff demo'd the new GUI for yt, we've had some thinking and thoughts about some other obvious places to go with the interface. Tonight Tom and I did some brainstorming about what a personal database of simulation outputs might look like; this isn't so much about large, comprehensive databases of simulations, but rather about a pragmatic convenience for people who run their data on a single file system.
As it stands, yt dumps the last 200 parameter files into a .csv file in your ~/.yt directory. These include the unique id, the parameter file name on the file system, and some other misc data that's not terribly important.
We were thinking it might be neat to have a really simple one that the files never really left. It could be ephemeral -- so that files that get removed or moved around would simply be removed from the DB -- and maybe it would contain information (if available from the simulation code) about the previous output in the simulation. Enzo has this, and it may be coming to other codes, in the form of UUIDs that get tracked and carried along by the sim.
So there are a couple ways this might work. We came up with a few:
What do people think? With the new GUI, this becomes I think a lot more interesting -- particularly if you can re-assemble a graph of the outputs, maybe even querying them, if you were to store the parameter file in full.
Anyway, just a handful of thoughts. Stephen, you have quite a bit of experience with SQLite; do you think this could be a fun application of them? Or would that be overkill? Maybe the whole system could be handled with just filenames?