creating size-limited tar files
kushal.kumaran+python at gmail.com
Wed Nov 14 07:05:04 CET 2012
Ian Kelly <ian.g.kelly at gmail.com> writes:
> On Tue, Nov 13, 2012 at 3:31 AM, andrea crotti
> <andrea.crotti.0 at gmail.com> wrote:
>> but it's a bit ugly. I wonder if I can use the subprocess PIPEs to do
>> the same thing, is it going to be as fast and work in the same way??
> It'll look something like this:
>>>> p1 = subprocess.Popen(cmd1, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
>>>> p2 = subprocess.Popen(cmd2, shell=True, stdin=p1.stdout, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
> ('', '')
> ('', '')
> Note that there's a subtle potential for deadlock here. During the
> p1.communicate() call, if the p2 output buffer fills up, then it will
> stop accepting input from p1 until p2.communicate() can be called, and
> then if that buffer also fills up, p1 will hang. Additionally, if p2
> needs to wait on the parent process for some reason, then you end up
> effectively serializing the two processes.
> Solution would be to poll all the open-ended pipes in a select() loop
> instead of using communicate(), or perhaps make the two communicate
> calls simultaneously in separate threads.
Or, you could just change the p1's stderr to an io.BytesIO instance.
Then call p2.communicate *first*.
More information about the Python-list