For the data format I strongly recommend NetCDF and especially NetCDF4 
(allows for compression)

Actually, why not hdf5, which seems to be used by NetCDF4, but is also 
largely used across many scientific comunities, and very well supported
under python (pytables)?

My 2 cents,

Gaël

Seconded. We use HDF5 [1] in a parallel simulation code written in C++ and Fortran, and find it to be incredibly useful. I personally use PyTables to access the data for analysis and plotting. If you have any questions, feel free to email me.

--Rick

[1] - http://hdfgroup.org/


-------------------------------------------------------------------------
Rick Wagner, Graduate Student Researcher
UCSD Physics
9500 Gilman Drive
La Jolla, CA  92093-0424
(858) 822-4784 Phone
-------------------------------------------------------------------------
Measuring programming progress by lines of code is
like measuring aircraft building progress by weight.
--Bill Gates
-------------------------------------------------------------------------