[Tutor] More efficient than glob?

Doug.Shawhan@gecits.ge.com Doug.Shawhan@gecits.ge.com
Thu Nov 7 10:51:02 2002


>> Since there is well over a year's worth of logfiles (> 31536000 files)
the
>> following method:
>> >>> import glob
>> >>> files = glob.glob('\\tmp\\*')
>> ...would seem to use an undue amount of memory.
>> Is there a more efficient way of doing this? 

> Usually I'd say generators, but in this particular case that won't work;
all 
> functions that return a list of files do _not_ use generators.

Heh. Good, because that would have lead me to the embarassing question:
"What is a generator?" :-)

> So I can't give you a general solution, but if you logfiles have sensible 
> names, you could use that to partition your logfiles.

Yep. Logfiles are named with a straight unix date string: i.e. 1036683980

So smaller bites seems to be the key...

Thanks for the advice! 

d