No space left on device

Gustavo Cordova gcordova at hebmex.com
Wed Feb 20 18:51:37 EST 2002


> 
> So, I am assuming there is some problem with the FAT32 system 
> when a directory gets lots of files in it.  Since I can make the 100K 
> files if I don't use mktemp, I am also assuming there is some 
> problem with mktemp in this scenario.
> 
> If anyone figures this out, it would be interesting to know what is 
> going on.  Meanwhile, I guess I'll need to redesign my file 
> generation algorithm to split things up.
> 
> thanks for the input.
> 
> sue
> 

The only problem I see is that FAT32 is outright
dead-ugly-inefficient, in a major way, when dealing
with long filename directory entries.

Normally, a directory entry contains the 8+3 filename,
creation / modification date, attributes, and that's all.

So, in order to create and use long filename entries,
there's this hideous hack going to behind the scenes.

A long-filename entry consists of enough directory
entries consecutive to each other, to write the filename
using unicode. Immediately after that, there's a "normal"
short-filename entry, containing it's respective short
filename.

Also, FAT32 doesn't have any kind of directory sorting,
so all filename searches are sequential.

So, there's lots of jumping around when looking for a
filename, looking in both the long-filename entries
and the short-filename ones.

Added to that, mktemp first looks to check if the filename
exists, before opening the file.

So, you have a stat() call, followed by an open() call.
The open() call may be faster than the stat(), because
of OS filename-caching and all that, but still, it's
slow and ugly.

Can you run your program in a linux machine using ext2fs
and reiserfs?? ext2 uses a sequential file lookup mechanism,
only that there's no ugly short- and long-filename entries.
On the other hand, reiserfs uses a b-tree to save it's
directory entries, so searching for files, or opening then,
is quite a bit faster.

Nice stuff, this. :-)

-gustavo




More information about the Python-list mailing list