regd efficient methods to manipulate *large* files
paddy3118 at netscape.net
Mon May 1 23:19:21 CEST 2006
I take it that you have a binary file that takes a file name and
proceses the file contents.
Sometimes Unix binaries are written so that a file name of '-', (just a
dash), causes it to take input from stdin so that the piping mentioned
in a previous reply could work.
On some of our unix systems /tmp is set up as a 'virtual disk' It
quacks like a normal disk filesystem but is actually implimented in
RAM/virtual memory, and is faster than normal disk access.
(Unfortunately we are not allowed to save multi-gigabyte files there as
it affects other aspects of the OS).
Maybe you can mount a similar filesystem if you have the RAM.
More information about the Python-list