[Tutor] Efficient way to join files

[Danny Yoo]

On Wed, 1 Oct 2003, Bob Gailer wrote:

> At 07:39 PM 9/30/2003, H=E9ctor Villafuerte D. wrote:
>
> >I need to join multiple files.

Hi Hector,


Do you need to do this physically, or can you just use an iterator on
multiple files?  If you can use an iterator, then the 'itertools' module
will be useful:

    http://www.python.org/doc/current/lib/itertools-functions.html


In particular, we can "chain" files together by doing something like
this:

###
"""A small 'cat'-like utility that takes files from the command line,
and prints them all."""
import itertools
import sys
files =3D map(open, sys.argv[1:])
all_files =3D itertools.chain(*files)
for line in all_files:
    sys.stdout.write(line)
###


But that being said, the 'fileinput' module does something like this
already with its input() function!  *grin*  According to:

    http://www.python.org/doc/current/lib/module-fileinput.html

fileinput.input() can take in a list of files.  We can recode the above
as:


###
#!/usr/local/bin/python
"""A small 'cat'-like utility that takes files from the command line,
and prints them all."""
import fileinput
import sys
all_files =3D fileinput.input(sys.argv[1:])
for line in all_files:
    sys.stdout.write(line)
###



Please feel free to ask more questions.  Good luck to you!