Sure. So one would need to re-instantiate if something gets bumped off?
d.
On Tue, Mar 3, 2009 at 2:15 PM, Matthew Turk
Okay, how about this.
We make a 'freshness' column in the CSV. This is a timestamp of the last time the parameter file was used. This can be the sort field, too. We'll set a variable -- default it to, say, 300 -- that includes the freshness cutoff in number of parameter files. Thus, sorted by freshness, we take the last N, where N is set in the parameter file.
Sound good?
On Tue, Mar 3, 2009 at 1:47 PM, david collins
wrote: Wow, you've been analyzing 10,000 parameter files? Okay.
Ci. Making time sequences. Hundreds/thousands of snapshots from a dozen sims, and comparing. (This is why uber)
Deleting parameter_files.csv is a good workaround for me now, so don't spend more than 45 seconds on it. I did this recently, so it's not huge, only 68 Mb.
kraken /nics/b/home/collins/.yt/parameter_files.csv
d.
One thing I can do is implement a maximum number in the store. Then we can only guarantee objects will be automatically retrieved (without having *recently* instantiated the PF) if they were in the last N, where N is the number of files we keep in the store.
Can you send me one of your parameter file stores? Or put it on kraken?
-Matt - Show quoted text - On Tue, Mar 3, 2009 at 1:04 PM, david collins
wrote: Hey--
Not sure if this belongs on the dev list--
I've been reading and analyzing large-ish ( O(10,000) ) parameter files, and the parameter_file.csv gets large (300 Mb), which isn't a problem, but when that happens, doing anything (even importing YT) takes forever (> 1 minute to import yt.lagos).
I did install the "unboudned growth" fix you sent, still happens.
I just noticed that it happens after your fix yesterday, or I'da said something earlier.
d.
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org