[SciPy-User] tool for running simulations

[Dan Goodman]

On 25/06/2011 00:49, Gael Varoquaux wrote:
> On Fri, Jun 24, 2011 at 01:22:53AM +0200, Dan Goodman wrote:
>>>> * Can it be used on multiple computers?
>
>>> If you have an NFS share between the computers, yes. The code works OK in
>>> parallel. You will have race conditions, but it captures them, and falls
>>> back on its feets.
>
>> Ah nice, how does it do that?
>
> Breaking up the storage in many different files, located in directories,
> and enforcing consistency only localy.

Ah yes, so I'm using a similar system, and indeed it won't extend well 
to working with HDF5 as the backend.

>>>> * Can you browse the generated data easily?
>
>>> No. This is something that could/should be improved (want to organize a
>>> sprint in Paris, if you still are in Paris?).
>
>> If we used HDF5 as the backend then you'd get this for free, so maybe
>> that's the better way?
>
> I wouldn't call it a 'better way'. It is a different way that brings in
> some good things. I do not think that HDF5 can be as robust as my current
> implementation to crashes. I addition, it enforces a bug depency. I want
> joblib to work with no dependencies on Python. It can have optional
> dependencies though.

Good points.

>> I'd possibly be interested in doing a sprint, I'm in Paris until the
>> end of July but I'm finishing up here so I'll probably have quite a lot
>> of things to finish.
>
> OK. I am abroad till the end of July :).

Ah OK, so maybe some time after September then? Unless you'll be in 
London in August? :)

Dan