object references/memory access
Karthik Gurusamy
kar1107 at gmail.com
Tue Jul 3 18:22:56 EDT 2007
On Jul 3, 2:33 pm, "Martin v. Löwis" <mar... at v.loewis.de> wrote:
> > If the problem does not require two way communication, which is
> > typical of a producer-consumer, it is a lot faster to allow P to fully
> > run before C is started.
>
> Why do you say it's *a lot* faster. I find that it is a little faster.
> The only additional overhead from switching forth and back between
> consumer and producer is the overhead for context switching, which
> is typically negligible, compared to everything else that is going
> on.
True it needn't be *a lot*. I did observe 25% gain or more when there
were a chain of processes involved as in a shell pipeline. Again this
could be very problem specific. What I had, included something like 4
or 5 processes connected as in p1 | p2 | p3 | p4 ... here I found the
back-n-forth context switching was slowing down quite a bit (some
thing like 2 mins task completed in under 40 seconds without the
piping)
If all you had is just two processes, P and C and the amount of data
flowing is less (say on the order of 10's of buffer-size ... e.g. 20
times 4k), *a lot* may not be right quantifier. But if the data is
large and several processes are involved, I am fairly sure the
overhead of context-switching is very significant (not negligible) in
the final throughput.
Thanks,
Karthik
>
> Regards,
> Martin
More information about the Python-list
mailing list