Wow, you've been analyzing 10,000 parameter files? Okay.
One thing I can do is implement a maximum number in the store. Then
we can only guarantee objects will be automatically retrieved (without
having *recently* instantiated the PF) if they were in the last N,
where N is the number of files we keep in the store.
Can you send me one of your parameter file stores? Or put it on kraken?
-Matt
On Tue, Mar 3, 2009 at 1:04 PM, david collins
Hey--
Not sure if this belongs on the dev list--
I've been reading and analyzing large-ish ( O(10,000) ) parameter files, and the parameter_file.csv gets large (300 Mb), which isn't a problem, but when that happens, doing anything (even importing YT) takes forever (> 1 minute to import yt.lagos).
I did install the "unboudned growth" fix you sent, still happens.
I just noticed that it happens after your fix yesterday, or I'da said something earlier.
d.
Wow, you've been analyzing 10,000 parameter files? Okay.
Ci. Making time sequences. Hundreds/thousands of snapshots from a dozen sims, and comparing. (This is why uber) Deleting parameter_files.csv is a good workaround for me now, so don't spend more than 45 seconds on it. I did this recently, so it's not huge, only 68 Mb. kraken /nics/b/home/collins/.yt/parameter_files.csv d.
One thing I can do is implement a maximum number in the store. Then we can only guarantee objects will be automatically retrieved (without having *recently* instantiated the PF) if they were in the last N, where N is the number of files we keep in the store.
Can you send me one of your parameter file stores? Or put it on kraken?
-Matt - Show quoted text - On Tue, Mar 3, 2009 at 1:04 PM, david collins
wrote: Hey--
Not sure if this belongs on the dev list--
I've been reading and analyzing large-ish ( O(10,000) ) parameter files, and the parameter_file.csv gets large (300 Mb), which isn't a problem, but when that happens, doing anything (even importing YT) takes forever (> 1 minute to import yt.lagos).
I did install the "unboudned growth" fix you sent, still happens.
I just noticed that it happens after your fix yesterday, or I'da said something earlier.
d.
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Okay, how about this.
We make a 'freshness' column in the CSV. This is a timestamp of the
last time the parameter file was used. This can be the sort field,
too. We'll set a variable -- default it to, say, 300 -- that includes
the freshness cutoff in number of parameter files. Thus, sorted by
freshness, we take the last N, where N is set in the parameter file.
Sound good?
On Tue, Mar 3, 2009 at 1:47 PM, david collins
Wow, you've been analyzing 10,000 parameter files? Okay.
Ci. Making time sequences. Hundreds/thousands of snapshots from a dozen sims, and comparing. (This is why uber)
Deleting parameter_files.csv is a good workaround for me now, so don't spend more than 45 seconds on it. I did this recently, so it's not huge, only 68 Mb.
kraken /nics/b/home/collins/.yt/parameter_files.csv
d.
One thing I can do is implement a maximum number in the store. Then we can only guarantee objects will be automatically retrieved (without having *recently* instantiated the PF) if they were in the last N, where N is the number of files we keep in the store.
Can you send me one of your parameter file stores? Or put it on kraken?
-Matt - Show quoted text - On Tue, Mar 3, 2009 at 1:04 PM, david collins
wrote: Hey--
Not sure if this belongs on the dev list--
I've been reading and analyzing large-ish ( O(10,000) ) parameter files, and the parameter_file.csv gets large (300 Mb), which isn't a problem, but when that happens, doing anything (even importing YT) takes forever (> 1 minute to import yt.lagos).
I did install the "unboudned growth" fix you sent, still happens.
I just noticed that it happens after your fix yesterday, or I'da said something earlier.
d.
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Sure. So one would need to re-instantiate if something gets bumped off?
d.
On Tue, Mar 3, 2009 at 2:15 PM, Matthew Turk
Okay, how about this.
We make a 'freshness' column in the CSV. This is a timestamp of the last time the parameter file was used. This can be the sort field, too. We'll set a variable -- default it to, say, 300 -- that includes the freshness cutoff in number of parameter files. Thus, sorted by freshness, we take the last N, where N is set in the parameter file.
Sound good?
On Tue, Mar 3, 2009 at 1:47 PM, david collins
wrote: Wow, you've been analyzing 10,000 parameter files? Okay.
Ci. Making time sequences. Hundreds/thousands of snapshots from a dozen sims, and comparing. (This is why uber)
Deleting parameter_files.csv is a good workaround for me now, so don't spend more than 45 seconds on it. I did this recently, so it's not huge, only 68 Mb.
kraken /nics/b/home/collins/.yt/parameter_files.csv
d.
One thing I can do is implement a maximum number in the store. Then we can only guarantee objects will be automatically retrieved (without having *recently* instantiated the PF) if they were in the last N, where N is the number of files we keep in the store.
Can you send me one of your parameter file stores? Or put it on kraken?
-Matt - Show quoted text - On Tue, Mar 3, 2009 at 1:04 PM, david collins
wrote: Hey--
Not sure if this belongs on the dev list--
I've been reading and analyzing large-ish ( O(10,000) ) parameter files, and the parameter_file.csv gets large (300 Mb), which isn't a problem, but when that happens, doing anything (even importing YT) takes forever (> 1 minute to import yt.lagos).
I did install the "unboudned growth" fix you sent, still happens.
I just noticed that it happens after your fix yesterday, or I'da said something earlier.
d.
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
That's right, it's completely re-creatable.
On Tue, Mar 3, 2009 at 2:19 PM, david collins
Sure. So one would need to re-instantiate if something gets bumped off?
d.
On Tue, Mar 3, 2009 at 2:15 PM, Matthew Turk
wrote: Okay, how about this.
We make a 'freshness' column in the CSV. This is a timestamp of the last time the parameter file was used. This can be the sort field, too. We'll set a variable -- default it to, say, 300 -- that includes the freshness cutoff in number of parameter files. Thus, sorted by freshness, we take the last N, where N is set in the parameter file.
Sound good?
On Tue, Mar 3, 2009 at 1:47 PM, david collins
wrote: Wow, you've been analyzing 10,000 parameter files? Okay.
Ci. Making time sequences. Hundreds/thousands of snapshots from a dozen sims, and comparing. (This is why uber)
Deleting parameter_files.csv is a good workaround for me now, so don't spend more than 45 seconds on it. I did this recently, so it's not huge, only 68 Mb.
kraken /nics/b/home/collins/.yt/parameter_files.csv
d.
One thing I can do is implement a maximum number in the store. Then we can only guarantee objects will be automatically retrieved (without having *recently* instantiated the PF) if they were in the last N, where N is the number of files we keep in the store.
Can you send me one of your parameter file stores? Or put it on kraken?
-Matt - Show quoted text - On Tue, Mar 3, 2009 at 1:04 PM, david collins
wrote: Hey--
Not sure if this belongs on the dev list--
I've been reading and analyzing large-ish ( O(10,000) ) parameter files, and the parameter_file.csv gets large (300 Mb), which isn't a problem, but when that happens, doing anything (even importing YT) takes forever (> 1 minute to import yt.lagos).
I did install the "unboudned growth" fix you sent, still happens.
I just noticed that it happens after your fix yesterday, or I'da said something earlier.
d.
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi Dave,
Here's a patch:
http://paste.enzotools.org/show/68/
You can get this with yt_lodgeit.py --download=68 > patch and apply it
with patch -p1 < patch . Conversely, you can download it from the
bitbucket repo or the mirror on hg.enzotools.org. It defaults to
keeping the last 300 touched files around.
If this works I'll merge back to trunk. You can test it just by
instantiated a crapton of them in a row.
-Matt
On Tue, Mar 3, 2009 at 2:25 PM, Matthew Turk
That's right, it's completely re-creatable.
On Tue, Mar 3, 2009 at 2:19 PM, david collins
wrote: Sure. So one would need to re-instantiate if something gets bumped off?
d.
On Tue, Mar 3, 2009 at 2:15 PM, Matthew Turk
wrote: Okay, how about this.
We make a 'freshness' column in the CSV. This is a timestamp of the last time the parameter file was used. This can be the sort field, too. We'll set a variable -- default it to, say, 300 -- that includes the freshness cutoff in number of parameter files. Thus, sorted by freshness, we take the last N, where N is set in the parameter file.
Sound good?
On Tue, Mar 3, 2009 at 1:47 PM, david collins
wrote: Wow, you've been analyzing 10,000 parameter files? Okay.
Ci. Making time sequences. Hundreds/thousands of snapshots from a dozen sims, and comparing. (This is why uber)
Deleting parameter_files.csv is a good workaround for me now, so don't spend more than 45 seconds on it. I did this recently, so it's not huge, only 68 Mb.
kraken /nics/b/home/collins/.yt/parameter_files.csv
d.
One thing I can do is implement a maximum number in the store. Then we can only guarantee objects will be automatically retrieved (without having *recently* instantiated the PF) if they were in the last N, where N is the number of files we keep in the store.
Can you send me one of your parameter file stores? Or put it on kraken?
-Matt - Show quoted text - On Tue, Mar 3, 2009 at 1:04 PM, david collins
wrote: Hey--
Not sure if this belongs on the dev list--
I've been reading and analyzing large-ish ( O(10,000) ) parameter files, and the parameter_file.csv gets large (300 Mb), which isn't a problem, but when that happens, doing anything (even importing YT) takes forever (> 1 minute to import yt.lagos).
I did install the "unboudned growth" fix you sent, still happens.
I just noticed that it happens after your fix yesterday, or I'da said something earlier.
d.
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Any luck?
On Tue, Mar 3, 2009 at 3:19 PM, Matthew Turk
Hi Dave,
Here's a patch:
http://paste.enzotools.org/show/68/
You can get this with yt_lodgeit.py --download=68 > patch and apply it with patch -p1 < patch . Conversely, you can download it from the bitbucket repo or the mirror on hg.enzotools.org. It defaults to keeping the last 300 touched files around.
If this works I'll merge back to trunk. You can test it just by instantiated a crapton of them in a row.
-Matt
On Tue, Mar 3, 2009 at 2:25 PM, Matthew Turk
wrote: That's right, it's completely re-creatable.
On Tue, Mar 3, 2009 at 2:19 PM, david collins
wrote: Sure. So one would need to re-instantiate if something gets bumped off?
d.
On Tue, Mar 3, 2009 at 2:15 PM, Matthew Turk
wrote: Okay, how about this.
We make a 'freshness' column in the CSV. This is a timestamp of the last time the parameter file was used. This can be the sort field, too. We'll set a variable -- default it to, say, 300 -- that includes the freshness cutoff in number of parameter files. Thus, sorted by freshness, we take the last N, where N is set in the parameter file.
Sound good?
On Tue, Mar 3, 2009 at 1:47 PM, david collins
wrote: Wow, you've been analyzing 10,000 parameter files? Okay.
Ci. Making time sequences. Hundreds/thousands of snapshots from a dozen sims, and comparing. (This is why uber)
Deleting parameter_files.csv is a good workaround for me now, so don't spend more than 45 seconds on it. I did this recently, so it's not huge, only 68 Mb.
kraken /nics/b/home/collins/.yt/parameter_files.csv
d.
One thing I can do is implement a maximum number in the store. Then we can only guarantee objects will be automatically retrieved (without having *recently* instantiated the PF) if they were in the last N, where N is the number of files we keep in the store.
Can you send me one of your parameter file stores? Or put it on kraken?
-Matt - Show quoted text - On Tue, Mar 3, 2009 at 1:04 PM, david collins
wrote: > Hey-- > > Not sure if this belongs on the dev list-- > > I've been reading and analyzing large-ish ( O(10,000) ) parameter > files, and the parameter_file.csv gets large (300 Mb), which isn't a > problem, but when that happens, doing anything (even importing YT) > takes forever (> 1 minute to import yt.lagos). > > I did install the "unboudned growth" fix you sent, still happens. > > I just noticed that it happens after your fix yesterday, or I'da said > something earlier. > > d. > _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Dude.
Looking at time per parameter file read, reading the first 600 steps
from one sim.
Before the fix, it monotonically increased from 0.01 second/read to 3
seconds/read.
After the fix, it's roughly constant at 0.02. Huge improvement in
performance. That's really going to speed things up when I make my
core tracking analysis. Thanks!
d.
On Thu, Mar 5, 2009 at 10:49 PM, Matthew Turk
Any luck?
On Tue, Mar 3, 2009 at 3:19 PM, Matthew Turk
wrote: Hi Dave,
Here's a patch:
http://paste.enzotools.org/show/68/
You can get this with yt_lodgeit.py --download=68 > patch and apply it with patch -p1 < patch . Conversely, you can download it from the bitbucket repo or the mirror on hg.enzotools.org. It defaults to keeping the last 300 touched files around.
If this works I'll merge back to trunk. You can test it just by instantiated a crapton of them in a row.
-Matt
On Tue, Mar 3, 2009 at 2:25 PM, Matthew Turk
wrote: That's right, it's completely re-creatable.
On Tue, Mar 3, 2009 at 2:19 PM, david collins
wrote: Sure. So one would need to re-instantiate if something gets bumped off?
d.
On Tue, Mar 3, 2009 at 2:15 PM, Matthew Turk
wrote: Okay, how about this.
We make a 'freshness' column in the CSV. This is a timestamp of the last time the parameter file was used. This can be the sort field, too. We'll set a variable -- default it to, say, 300 -- that includes the freshness cutoff in number of parameter files. Thus, sorted by freshness, we take the last N, where N is set in the parameter file.
Sound good?
On Tue, Mar 3, 2009 at 1:47 PM, david collins
wrote: > Wow, you've been analyzing 10,000 parameter files? Okay.
Ci. Making time sequences. Hundreds/thousands of snapshots from a dozen sims, and comparing. (This is why uber)
Deleting parameter_files.csv is a good workaround for me now, so don't spend more than 45 seconds on it. I did this recently, so it's not huge, only 68 Mb.
kraken /nics/b/home/collins/.yt/parameter_files.csv
d.
> > One thing I can do is implement a maximum number in the store. Then > we can only guarantee objects will be automatically retrieved (without > having *recently* instantiated the PF) if they were in the last N, > where N is the number of files we keep in the store. > > Can you send me one of your parameter file stores? Or put it on kraken? > > -Matt > - Show quoted text - > On Tue, Mar 3, 2009 at 1:04 PM, david collins
wrote: >> Hey-- >> >> Not sure if this belongs on the dev list-- >> >> I've been reading and analyzing large-ish ( O(10,000) ) parameter >> files, and the parameter_file.csv gets large (300 Mb), which isn't a >> problem, but when that happens, doing anything (even importing YT) >> takes forever (> 1 minute to import yt.lagos). >> >> I did install the "unboudned growth" fix you sent, still happens. >> >> I just noticed that it happens after your fix yesterday, or I'da said >> something earlier. >> >> d. >> > _______________________________________________ > Yt-dev mailing list > Yt-dev@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org > _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Awesome!
So no problems? I should feel free to apply the patch to trunk?
On Fri, Mar 6, 2009 at 9:32 AM, david collins
Dude.
Looking at time per parameter file read, reading the first 600 steps from one sim.
Before the fix, it monotonically increased from 0.01 second/read to 3 seconds/read.
After the fix, it's roughly constant at 0.02. Huge improvement in performance. That's really going to speed things up when I make my core tracking analysis. Thanks!
d.
On Thu, Mar 5, 2009 at 10:49 PM, Matthew Turk
wrote: Any luck?
On Tue, Mar 3, 2009 at 3:19 PM, Matthew Turk
wrote: Hi Dave,
Here's a patch:
http://paste.enzotools.org/show/68/
You can get this with yt_lodgeit.py --download=68 > patch and apply it with patch -p1 < patch . Conversely, you can download it from the bitbucket repo or the mirror on hg.enzotools.org. It defaults to keeping the last 300 touched files around.
If this works I'll merge back to trunk. You can test it just by instantiated a crapton of them in a row.
-Matt
On Tue, Mar 3, 2009 at 2:25 PM, Matthew Turk
wrote: That's right, it's completely re-creatable.
On Tue, Mar 3, 2009 at 2:19 PM, david collins
wrote: Sure. So one would need to re-instantiate if something gets bumped off?
d.
On Tue, Mar 3, 2009 at 2:15 PM, Matthew Turk
wrote: Okay, how about this.
We make a 'freshness' column in the CSV. This is a timestamp of the last time the parameter file was used. This can be the sort field, too. We'll set a variable -- default it to, say, 300 -- that includes the freshness cutoff in number of parameter files. Thus, sorted by freshness, we take the last N, where N is set in the parameter file.
Sound good?
On Tue, Mar 3, 2009 at 1:47 PM, david collins
wrote: >> Wow, you've been analyzing 10,000 parameter files? Okay. > > Ci. Making time sequences. Hundreds/thousands of snapshots from a > dozen sims, and comparing. (This is why uber) > > Deleting parameter_files.csv is a good workaround for me now, so don't > spend more than 45 seconds on it. I did this recently, so it's not > huge, only 68 Mb. > > kraken /nics/b/home/collins/.yt/parameter_files.csv > > d. > >> >> One thing I can do is implement a maximum number in the store. Then >> we can only guarantee objects will be automatically retrieved (without >> having *recently* instantiated the PF) if they were in the last N, >> where N is the number of files we keep in the store. >> >> Can you send me one of your parameter file stores? Or put it on kraken? >> >> -Matt >> - Show quoted text - >> On Tue, Mar 3, 2009 at 1:04 PM, david collins wrote: >>> Hey-- >>> >>> Not sure if this belongs on the dev list-- >>> >>> I've been reading and analyzing large-ish ( O(10,000) ) parameter >>> files, and the parameter_file.csv gets large (300 Mb), which isn't a >>> problem, but when that happens, doing anything (even importing YT) >>> takes forever (> 1 minute to import yt.lagos). >>> >>> I did install the "unboudned growth" fix you sent, still happens. >>> >>> I just noticed that it happens after your fix yesterday, or I'da said >>> something earlier. >>> >>> d. >>> >> _______________________________________________ >> Yt-dev mailing list >> Yt-dev@lists.spacepope.org >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org >> > _______________________________________________ > Yt-dev mailing list > Yt-dev@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org > _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
So no problems? I should feel free to apply the patch to trunk?
Seems good so far. I re-read and re-plotted some clumps I made a while ago, and it matches a plot I made about a week ago. Patch away. d.
On Fri, Mar 6, 2009 at 9:32 AM, david collins
wrote: Dude.
Looking at time per parameter file read, reading the first 600 steps from one sim.
Before the fix, it monotonically increased from 0.01 second/read to 3 seconds/read.
After the fix, it's roughly constant at 0.02. Huge improvement in performance. That's really going to speed things up when I make my core tracking analysis. Thanks!
d.
On Thu, Mar 5, 2009 at 10:49 PM, Matthew Turk
wrote: Any luck?
On Tue, Mar 3, 2009 at 3:19 PM, Matthew Turk
wrote: Hi Dave,
Here's a patch:
http://paste.enzotools.org/show/68/
You can get this with yt_lodgeit.py --download=68 > patch and apply it with patch -p1 < patch . Conversely, you can download it from the bitbucket repo or the mirror on hg.enzotools.org. It defaults to keeping the last 300 touched files around.
If this works I'll merge back to trunk. You can test it just by instantiated a crapton of them in a row.
-Matt
On Tue, Mar 3, 2009 at 2:25 PM, Matthew Turk
wrote: That's right, it's completely re-creatable.
On Tue, Mar 3, 2009 at 2:19 PM, david collins
wrote: Sure. So one would need to re-instantiate if something gets bumped off?
d.
On Tue, Mar 3, 2009 at 2:15 PM, Matthew Turk
wrote: > Okay, how about this. > > We make a 'freshness' column in the CSV. This is a timestamp of the > last time the parameter file was used. This can be the sort field, > too. We'll set a variable -- default it to, say, 300 -- that includes > the freshness cutoff in number of parameter files. Thus, sorted by > freshness, we take the last N, where N is set in the parameter file. > > Sound good? > > On Tue, Mar 3, 2009 at 1:47 PM, david collins wrote: >>> Wow, you've been analyzing 10,000 parameter files? Okay. >> >> Ci. Making time sequences. Hundreds/thousands of snapshots from a >> dozen sims, and comparing. (This is why uber) >> >> Deleting parameter_files.csv is a good workaround for me now, so don't >> spend more than 45 seconds on it. I did this recently, so it's not >> huge, only 68 Mb. >> >> kraken /nics/b/home/collins/.yt/parameter_files.csv >> >> d. >> >>> >>> One thing I can do is implement a maximum number in the store. Then >>> we can only guarantee objects will be automatically retrieved (without >>> having *recently* instantiated the PF) if they were in the last N, >>> where N is the number of files we keep in the store. >>> >>> Can you send me one of your parameter file stores? Or put it on kraken? >>> >>> -Matt >>> - Show quoted text - >>> On Tue, Mar 3, 2009 at 1:04 PM, david collins wrote: >>>> Hey-- >>>> >>>> Not sure if this belongs on the dev list-- >>>> >>>> I've been reading and analyzing large-ish ( O(10,000) ) parameter >>>> files, and the parameter_file.csv gets large (300 Mb), which isn't a >>>> problem, but when that happens, doing anything (even importing YT) >>>> takes forever (> 1 minute to import yt.lagos). >>>> >>>> I did install the "unboudned growth" fix you sent, still happens. >>>> >>>> I just noticed that it happens after your fix yesterday, or I'da said >>>> something earlier. >>>> >>>> d. >>>> >>> _______________________________________________ >>> Yt-dev mailing list >>> Yt-dev@lists.spacepope.org >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org >>> >> _______________________________________________ >> Yt-dev mailing list >> Yt-dev@lists.spacepope.org >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org >> > _______________________________________________ > Yt-dev mailing list > Yt-dev@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org > _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
participants (2)
-
david collins
-
Matthew Turk