Question about unreasonable slowness

[Leif K-Brooks]

allenjo5 at mail.northgrum.com wrote:
> i = 0
> while (i < 20):
>   i = i + 1

for i in xrange(20):

>   (shellIn, shellOut) = os.popen4("/bin/sh -c ':'")  # for testing, the
> spawned shell does nothing
>   print 'next'
> #  for line in shellOut:
> #       print line
> 
> On my system (AIX 5.1 if it matters, with Python 2.4.3), this simple
> loop spawning 20 subshells takes .75 sec.  Ok, that's reasonable.  Now,
> if I uncomment the two commented lines, which loop over the empty
> shellOut array, the progam now takes 11 secs.   That slowdown seems
> very hard to believe.  Why should it slow down so much?

The key fact here is that shellOut isn't an array; it's a living,
breathing file object. If you don't iterate over it, you can run all 20
shell processes in parallel if necessary; but if you do iterate over it,
   you're waiting for sh's stdout pipe to reach EOF, which effectively
means you can only run one process at a time.

On my system (OS X 10.4 with Python 2.5 installed), your code runs in
.187 secs with the loop commented out, and in .268 secs otherwise. But I
guess AIX's sh is slower than OS X's.