[python-win32] Sparse files and Pickle
timr at probo.com
Mon Apr 4 18:29:34 CEST 2005
On Sun, 3 Apr 2005 12:37:40 +0200, Graeme Glass <graemeglass at gmail.com>
>Could anyone tell me if it is possible to create a sparse file on
>win32 platform using python, and if so how?
Sparse files are only supported on Windows 2000 and more, on NTFS
partitions. You will not be able to use the Python file I/O operations
to do this. You have to send an ioctl (FSCTL_SET_SPARSE) to the file
after it is opened, then you have to open a memory growable mapping of
the file, and use that memory mapping to do your writes.
CreateFile, DeviceIoControl, and the memory mapping methods are all
provided in the Win32 extensions, but it makes me wonder if it wouldn't
be easier to write a C extension for this.
>And just out of interest, what is the practical limit for pickling an object?
>I pickled a 89MB binary read from a disk file, and although it worked,
>hogged my PC for about 8minutes making it unusable.
I guess this depends on your threshhold of pain, and is likely to vary
from PC to PC. I view pickling as a way to encode small objects for
later resurrection. If I needed to refer to an 89MB disk file in an
object, I would replace the data with the file name before pickling. I
thought pickle recognized a magic method name so the object could "help"
put itself into a picklable state, but I don't see it now.
- Tim Roberts, timr at probo.com
Providenza & Boekelheide, Inc.
More information about the Python-win32