walking a directory with very many files
ldo at geek-central.gen.new_zealand
Wed Jun 17 07:53:33 CEST 2009
In message <20090617091858.432f89ca at malediction>, Mike Kazantsev wrote:
> On Wed, 17 Jun 2009 14:52:28 +1200
> Lawrence D'Oliveiro <ldo at geek-central.gen.new_zealand> wrote:
>> In message
>> <234b19ac-7baf-4356-9fe5-37d00146d982 at z9g2000yqi.googlegroups.com>,
>> thebjorn wrote:
>> > Not proud of this, but...:
>> > [django] www4:~/datakortet/media$ ls bfpbilder|wc -l
>> > 174197
>> > all .jpg files between 40 and 250KB with the path stored in a
>> > database field... *sigh*
>> Why not put the images themselves into database fields?
>> > Oddly enough, I'm a relieved that others have had similar folder
>> > sizes ...
>> One of my past projects had 400000-odd files in a single folder. They
>> were movie frames, to allow assembly of movie sequences on demand.
> For both scenarios:
> Why not use hex representation of md5/sha1-hashed id as a path,
> arranging them like /path/f/9/e/95ea4926a4 ?
> That way, you won't have to deal with many-files-in-path problem ...
Why is that a problem?
More information about the Python-list