popen2 made >7 times faster!

[Sverker Nilsson]

Hi,

I was using the popen2 module to call out to a program. I found it
suspiciously slow to run a small quick program so I began to look for
the reason. Turned out that there was a loop in __init__ in the Popen3
class that closes files numbered 3 to 255 before it exec's the new
program. This took a very long time compared to the other code and
also compared to the time for the OS's fork and exec functions.

When I rewrote this loop in C it became much faster:

The time to call out 100 times to a hello world program dropped from
15 seconds to 2 seconds (*).  The time for 100 calls to the actual
program I wanted to run (PGP) dropped from 18 seconds to 6 seconds.

I guess there are a number of ways to fix this, I made just a quick
hack to close the files 3 to 255 from C. Should be better to keep track
of the files that are actually open to avoid the system calls
altogether - probably not too significant if there could only be 256
files open but would become a problem if the OS allows more
files. Also, popen2 has an hard-coded assumption that there can only
be 256 files open, regardless of what the OS actually allows.

Anyway, maybe somebody is working on this already...? 

'yours

Sverker Nilsson

(*) Pentium 90 Linux 2.0