Doubts related to subprocess.Popen()

[Jeff McNeil]

On Jan 20, 9:19 am, srinivasan srinivas <sri_anna... at yahoo.co.in>
wrote:
> Do parent process will have different file descriptor in it for each subprocesses or paprent uses a single file descriptor for all?
> I really want to know creation of each subprocess will occupy an entry in parents'file descriptor table. B'cos if i create more than 200 subprocesses, i am getting 'Too many open files' error.
>
> Thanks,
> Srini
>
> ----- Original Message ----
> From: Mark Wooding <m... at distorted.org.uk>
> To: python-l... at python.org
> Sent: Tuesday, 20 January, 2009 6:16:17 PM
> Subject: Re: Doubts related to subprocess.Popen()
>
> srinivasan srinivas <sri_anna... at yahoo.co.in> writes:
>
> > Does subprocess.Popen() count a new open file for each suprocess? I
> > mean does it occupy an entry in file descriptor table of parent
> > process?  If so, wat is each file descriptor connected to?
>
> On Unix, subprocess.Popen will use up a file descriptor in the parent
> for each use of subprocess.PIPE.  The descriptor in question is one end
> of a pipe; the child process holds the other end.
>
> I guess the situation is similar on Windows, but I don't know for sure.
>
> -- [mdw]
> --http://mail.python.org/mailman/listinfo/python-list
>
>       Add more friends to your messenger and enjoy! Go tohttp://messenger.yahoo.com/invite/

Have you upped your open files limit?  My test script:

import subprocess

procs = []
for i in xrange(400):
    procs.append(subprocess.Popen("/bin/cat",
        stdin=subprocess.PIPE, stdout=subprocess.PIPE,
            stderr=subprocess.PIPE))

By default, 'ulimit -n' returns 1024, which is the number of open
files allowed.  Running the test script without changing it results
in:

Traceback (most recent call last):
  File "test.py", line 9, in <module>
    stderr=subprocess.PIPE))
  File "/usr/lib/python2.5/subprocess.py", line 593, in __init__
    errread, errwrite)
  File "/usr/lib/python2.5/subprocess.py", line 1002, in
_execute_child
    errpipe_read, errpipe_write = os.pipe()
OSError: [Errno 24] Too many open files

Now, up that limit to 8192 via ulimit -n, and run the script again:

[root at marvin jeff]# ulimit -n 8192
[root at marvin jeff]# python test.py
[root at marvin jeff]#

HTH,

Jeff