writing large files quickly

[Ivan Voras]

Jens Theisen wrote:

> cp bigfile bigfile2
> 
> cat bigfile > bigfile3
> 
> du bigfile*
> 8       bigfile2
> 1032    bigfile3
> 
> So it's not consumings 0's. It's just doesn't store unwritten data. And I  

Very possibly cp "understands" sparse file and cat (doint what it's 
meant to do) doesn't :)


>>I read somewhere that it has a use in database software, but the only
>>thing I can imagine for this is when using heap queues
>>(http://python.active-venture.com/lib/node162.html).
> 
> That's an article about the heap efficient data structure. Was it your  
> intention to link this?

Yes. The idea is that in implementing such a structure, in which each 
level is 2^x (x="level" of the structure, and it's depentent on the 
number of entries the structure must hold) wide, most of blocks could 
exist and never be written to (i.e. they'd be "empty"). Using sparse 
files would save space :)

(It has nothing to do with python; I remembered the article so I linked 
to it; The sparse-file issue is useful only when implementing heaps 
directly on file or in mmaped file).