[Tutor] pipes and redirecting

[Adam Gold]

> On 27/05/14 21:01, Adam Gold wrote:
> 
>> "dd if=/home/adam/1 bs=4k conv=noerror,notrunc,sync | pbzip2 > 1.img.bz2"
>>
>> The first thing I do is break it into two assignments
> 
> And that's the start of the problem because it should be three:
> The first command, the second command and the output file.
> 
>> ddIf = shlex.split("dd if=/home/adam/1 bs=4k conv=noerror,notrunc,sync")
>> compress = shlex.split("pbzip2 > /home/adam/1.img.bz2")
> 
> compress = "pbzip2"
> outfile = open('/home/adam/1.img.bz2','w')
> 
> The redirection symbol is not something subprocess can
> use as an argument.
> 
>> p1 = subprocess.Popen(ddIf, stdout=subprocess.PIPE)
>> p2 = subprocess.Popen(compress, stdin=p1.stdout, stdout=subprocess.PIPE)
> 
> Use the output file here.
> 
> p2 = subprocess.Popen(compress, stdin=p1.stdout, stdout=outfile)
> 
> 
>> I think that the '>' redirect needs to be dealt with using the
>> subprocess module as well but I can't quite put the pieces together.
>> I'd appreciate any guidance.  Thanks.
> 
> Alternatively read the output into a variable using communicate but then
> write it out to the file manually at the end.
> 
> [You might also be able to use shell=TRUE but that introduces other
> issues. But I don't know whether using shell includes redirection.]
> 
Thanks Alan, yes, I realise now I needed a third assignment to make this
work.

I actually had an exchange with subscriber 'eryksun' yesterday who did a
great job of pointing me in the right direction.  As a bit of a noob, I
think I replied to the individual rather than the list, hence it doesn't
seem to be in the thread.  For the benefit of the archives I append
below eryksun's initial (there was a bit of follow up but nothing too
important) reply to me.
=====================================

Send p2's stdout to a file:

    import subprocess
    import shlex

    ddIf = shlex.split("dd if=/home/adam/1 bs=4k "
                       "conv=noerror,notrunc,sync")
    compress = "pbzip2"
    filename = "/home/adam/1.img.bz2"

    p1 = subprocess.Popen(ddIf, stdout=subprocess.PIPE)
    with p1.stdout as fin, open(filename, "w") as fout:
        p2 = subprocess.Popen(compress, stdin=fin, stdout=fout)

    ret1 = p1.wait()
    ret2 = p2.wait()

Does this work?