[Tutor] More efficient than glob?
Doug.Shawhan@gecits.ge.com
Doug.Shawhan@gecits.ge.com
Thu Nov 7 10:51:02 2002
>> Since there is well over a year's worth of logfiles (> 31536000 files)
the
>> following method:
>> >>> import glob
>> >>> files = glob.glob('\\tmp\\*')
>> ...would seem to use an undue amount of memory.
>> Is there a more efficient way of doing this?
> Usually I'd say generators, but in this particular case that won't work;
all
> functions that return a list of files do _not_ use generators.
Heh. Good, because that would have lead me to the embarassing question:
"What is a generator?" :-)
> So I can't give you a general solution, but if you logfiles have sensible
> names, you could use that to partition your logfiles.
Yep. Logfiles are named with a straight unix date string: i.e. 1036683980
So smaller bites seems to be the key...
Thanks for the advice!
d