Pipelining tar create and tar extract the "Python" way...

[Ray Van Dolson]

On Wed, Sep 23, 2009 at 04:22:36PM -0700, Ray Van Dolson wrote:
> On Wed, Sep 23, 2009 at 03:52:11PM -0700, Ray Van Dolson wrote:
> > Hi all;
> > 
> > In the land'o'shell, I can do something like the following:
> > 
> >   tar cvf - SrcDir | (cd /dest ; tar xvf -)
> > 
> 
> Bad form replying to my own post... while I'd still like to know if
> this is possible to do with the tarfile class, it seems like using
> subprocess.Popen() and calling tar from there with stdout set to PIPE
> is probably the way to go.
> 
> I think this will result in the fastest way to copy files around.
> Sounds like shutil.copytree() may not be all that robust (and probably
> not very fast) in my version of Python (2.4.3 on RHEL5).
> 
> Still open to creative suggestions... :)
> 

Never found a way to do this with the tarfile class directly, but used
Popen() to call tar:

  # Time for some fancy shmancy calls to tar.  First create the process that
  # will generate our tar file and set it to output to a PIPE.
  p1 = Popen(["tar", "cf", "-", "."], cwd=u['homedir'], stdout=PIPE)

  # Next, set up our consumer tar process.  This one should extract its data
  # in the destination directory.
  p2 = Popen(["tar", "xf", "-"], cwd=new_homedir, stdin=p1.stdout)

  # Go!
  err = p2.communicate()[1]

Nothing groundbreaking as this is from the examples in the
documentation, but just in case anyone else stumbles across this..

Ray