[issue7946] Convoy effect with I/O bound threads and New GIL
Nir Aides
report at bugs.python.org
Tue Apr 27 17:01:00 CEST 2010
Nir Aides <nir at winpdb.org> added the comment:
On Tue, Apr 27, 2010 at 12:23 PM, Charles-Francois Natali wrote:
> @nirai
> I have some more remarks on your patch:
> - /* Diff timestamp capping results to protect against clock differences
> * between cores. */
> _LOCAL(long double) _bfs_diff_ts(long double ts1, long double ts0) {
>
> I'm not sure I understand. You can have problem with multiple cores when reading directly the
> TSC register, but that doesn't affect gettimeofday. gettimeofday should be reliable and accurate
> (unless the OS is broken of course), the only issue is that since it's wall clock time, if a process
> like ntpd is running, then you'll run into problem
I think gettimeofday() might return different results on different cores as result of kernel/hardware problems or clock drift issues in VM environments:
http://kbase.redhat.com/faq/docs/DOC-7864
https://bugzilla.redhat.com/show_bug.cgi?id=461640
In Windows the high-precision counter might return different results on different cores in some hardware configurations (older multi-core processors). I attempted to alleviate these problems by using capping and by using a "python time" counter constructed from accumulated slices, with the assumption that IO bound threads are unlikely to get migrated often between cores while running. I will add references to the patch docs.
> - did you experiment with the time slice ? I tried some higher values and got better results,
> without penalizing the latency. Maybe it could be interesting to look at it in more detail (and
> on various platforms).
Can you post more details on your findings? It is possible that by using a bigger slice, you helped the OS classify CPU bound threads as such and improved "synchronization" between BFS and the OS scheduler.
Notes on optimization of code taken, thanks.
----------
_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue7946>
_______________________________________
More information about the Python-bugs-list
mailing list