Thank you very much for the help. But I was more looking for some coding solution (furthermore, I'm not using Linux). My point in not to make some real arrays looking like they are saved on files (and use for it some files in memory), but at the contrary, to make some "fake" arrays, saved on disk, to look like real arrays. In other words, to make them behave like if they were inherited from ndarrays. In pytables, if you use my_node[:] it returns an array. It's part of what I want to do. Plus : - if I call my_node[:] ten times, only the first call will read to disk (it matters, even if pytables is very very fast) ; - the same class can represent a node or a numpy array. Jean-Baptiste Rudant ________________________________ De : Francesc Alted <faltet@pytables.org> À : Discussion of Numerical Python <numpy-discussion@scipy.org> Envoyé le : Lun 17 mai 2010, 20h 13min 35s Objet : Re: [Numpy-discussion] Saving an array on disk to free memory - Pickling A Monday 17 May 2010 13:03:19 Jean-Baptiste Rudant escrigué:
Hello,
I tried to create an object : - which behave just like a numpy array ; - which can be saved on disk in an efficient way (numpy.save in my example but with pytables in my real program) ; - which can be "unloaded" (if it is saved) to free memory : it can exsit has an empty stuff which knows how to retrieve real values ; it will be loaded only when we need to work with it ; - which unloads itself before being pickled (values are already saved and don't have to be pickled). [clip]
Well, if you are using Linux, you can make use of /dev/shm in order to save your files in-memory instead of disk. There are some considerations to have in mind for doing this though: http://superuser.com/questions/45342/when-should-i-use-dev-shm-and-when- should-i-use-tmp However, I don't know the equivalent to this in Win, Mac OSX or other UNICES. -- Francesc Alted _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion