walking a directory with very many files

[Lawrence D'Oliveiro]

In message <20090617091858.432f89ca at malediction>, Mike Kazantsev wrote:

> On Wed, 17 Jun 2009 14:52:28 +1200
> Lawrence D'Oliveiro <ldo at geek-central.gen.new_zealand> wrote:
> 
>> In message
>> <234b19ac-7baf-4356-9fe5-37d00146d982 at z9g2000yqi.googlegroups.com>,
>> thebjorn wrote:
>> 
>> > Not proud of this, but...:
>> > 
>> > [django] www4:~/datakortet/media$ ls bfpbilder|wc -l
>> >  174197
>> > 
>> > all .jpg files between 40 and 250KB with the path stored in a
>> > database field... *sigh*
>> 
>> Why not put the images themselves into database fields?
>> 
>> > Oddly enough, I'm a relieved that others have had similar folder
>> > sizes ...
>> 
>> One of my past projects had 400000-odd files in a single folder. They
>> were movie frames, to allow assembly of movie sequences on demand.
> 
> For both scenarios:
> Why not use hex representation of md5/sha1-hashed id as a path,
> arranging them like /path/f/9/e/95ea4926a4 ?
> 
> That way, you won't have to deal with many-files-in-path problem ...

Why is that a problem?