object references/memory access

Karthik Gurusamy kar1107 at gmail.com
Tue Jul 3 18:22:56 EDT 2007


On Jul 3, 2:33 pm, "Martin v. Löwis" <mar... at v.loewis.de> wrote:
> > If the problem does not require two way communication, which is
> > typical of a producer-consumer, it is a lot faster to allow P to fully
> > run before C is started.
>
> Why do you say it's *a lot* faster. I find that it is a little faster.
> The only additional overhead from switching forth and back between
> consumer and producer is the overhead for context switching, which
> is typically negligible, compared to everything else that is going
> on.

True it needn't be *a lot*. I did observe 25% gain or more when there
were a chain of processes involved as in a shell pipeline. Again this
could be very problem specific. What I had, included something like 4
or 5 processes connected as in  p1 | p2 | p3 | p4 ... here I found the
back-n-forth context switching was slowing down quite a bit (some
thing like 2 mins task completed in under 40 seconds without the
piping)

If all you had is just two processes, P and C and the amount of data
flowing is less (say on the order of 10's of buffer-size ... e.g. 20
times 4k), *a lot* may  not be right quantifier. But if the data is
large and several processes are involved, I am fairly sure the
overhead of context-switching is very significant (not negligible) in
the final throughput.

Thanks,
Karthik

>
> Regards,
> Martin





More information about the Python-list mailing list