[Patches] [ python-Patches-1607149 ] bug# 1607041: Condition.wait timeout fails on clock change
SourceForge.net
noreply at sourceforge.net
Fri Apr 13 17:09:30 CEST 2007
Patches item #1607149, was opened at 2006-12-01 14:32
Message generated for change (Comment added) made by hashstat
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607149&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Library (Lib)
Group: Python 2.4
Status: Closed
Resolution: Rejected
Priority: 5
Private: No
Submitted By: BC (hashstat)
Assigned to: Nobody/Anonymous (nobody)
Summary: bug# 1607041: Condition.wait timeout fails on clock change
Initial Comment:
This patch if for bug# 1607041.
If the system clock is adjusted after Condition.wait is called with a timeout, the timeout does not expire as expected. This appears to be due to the threading.Condition class using the system clock to calculate the timeout expiration without taking system clock changes into account.
No matter what timeout is used, setting the system clock ahead reduces or eliminates the wait while setting the system clock back increases the wait. So if the clock is set back one hour in the middle of a 1 microsecond wait (c.wait(1)), wait will return in an hour and 1 microsecond rather than after 1 microsecond.
This patch modifies the Condition classes wait method to check for variations in the clock between calls to sleep and ajust for abnormalities.
----------------------------------------------------------------------
>Comment By: BC (hashstat)
Date: 2007-04-13 08:09
Message:
Logged In: YES
user_id=939959
Originator: YES
You are right loewis. DST does not have any effect on the wait time. I
haven't looked at this in a while and I shot off a reply too quickly. The
point is that a sleep (or wait) of any sort should not depend on the wall
clock. While I can undertand rejecting this patch, the underlying bug
still exists and I hope a solution can be provided. I plan to look into a
better fix myself. Thanks.
----------------------------------------------------------------------
Comment By: Martin v. Löwis (loewis)
Date: 2007-04-12 23:41
Message:
Logged In: YES
user_id=21627
Originator: NO
Please let me repeat that DST changes are *not* an issue. If the system
runs for many months, and has a condition timeout set that passes the DST
change boundary, the condition *will* work correctly in the code as is. I
don't know where you got the idea that DST changes are a problem, please
rest ensured that they are no problem whatsoever, for the issue under
discussion. The clock does *not* change when the time zone changes. The
timeout will *not* be off by an hour.
Rejecting the patch as "won't fix".
----------------------------------------------------------------------
Comment By: BC (hashstat)
Date: 2007-04-12 22:06
Message:
Logged In: YES
user_id=939959
Originator: YES
I admit that the patch isn't ideal. But it is a simple approach to fix
the issue using pure Python. A better solution would be to use
synchronization routines provided by the OS, such as the pthread condition
functions on *NIX systems and the Win32 API synchronization functions on
Windows.
I wrote this patch to use with a simulator that may run for months at a
time and will definitely run for days at a time. If the run happens to
occur when DST ends and the clock automatically changes, as is the case for
most systems, the simulation could stall for one hour and produce
undesirable results.
Having a sleeps depend on the wall clock is a bad idea. But since that is
what we are working with, wouldn't it be better to have results skewed by a
fraction of a minute or even a second rather than be off by an hour?
----------------------------------------------------------------------
Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-11 10:26
Message:
Logged In: YES
user_id=80475
Originator: NO
I think we should punt on this one. When the clock changes, it is hard to
detect when it happens, how much it changed, what the interactions are with
other time dependent routines (logging, etc), and what to do about it.
Much of this is pure guesswork.
It would be different if there we some API in-place for clock changes to
notify all of its clients when a change occurs. But there isn't. The
responsibility for handling clock changes has to lie with the changer, not
with all possible clients that call time() or sleep.
Recommend closing as WontFix.
----------------------------------------------------------------------
Comment By: Martin v. Löwis (loewis)
Date: 2007-02-11 04:55
Message:
Logged In: YES
user_id=21627
Originator: NO
Ah. I fixed the "changing stat result problem" in Python 2.5, by not using
the C library's stat implementation anymore.
----------------------------------------------------------------------
Comment By: Josiah Carlson (josiahcarlson)
Date: 2007-02-11 02:11
Message:
Logged In: YES
user_id=341410
Originator: NO
I've never run FAT in Windows 2000. While I am pretty sure that the
underlying platform doesn't have issues, Python keeps its DST offsets when
it first starts up, so if you have a Python process that persists across
DST changes, the modification time changes, even if the actual file
modification time didn't (quitting and restarting Python "fixes" the
issue).
----------------------------------------------------------------------
Comment By: Martin v. Löwis (loewis)
Date: 2007-02-11 01:03
Message:
Logged In: YES
user_id=21627
Originator: NO
Indeed, you would have issues with file stamps changing on Windows NT+ if
the file system is FAT. However, this is not because the system time moves
forth and back, but because FAT time stamps get reinterpreted according to
the current time zone, not according to the time zone in force when the
file was created (as NT tries to convert the FAT time stamps to UTC). On
NTFS, this is not an issue, as NTFS file stamps are in UTC already (units
of 100ns since Jan 1, 1601).
So indeed, the submitter complains that Condition.wait doesn't time out
correctly if the system clock is changed while the wait occurs. I think it
is reasonable to make this complaint, however, I don't like to the patch
proposed, and I do think this can't really be fixed without the help of the
operating system (so I wouldn't mind if it was unfixed on systems that
don't help). The point is that .wait() really takes a duration, not a time
stamp, for timeout, and we should honor that. The duration should expire
after the given amount of physical time has passed, independent of the
system clock. As I said, one way to implement this would be to use a
monotonic clock if the system has one.
----------------------------------------------------------------------
Comment By: Josiah Carlson (josiahcarlson)
Date: 2007-02-10 12:42
Message:
Logged In: YES
user_id=341410
Originator: NO
Hrm. I know I've had issues with file modification times "changing"
during a DST change on Windows in a Python process, but that may be due to
Python only checking the timezone at startup.
What you are saying sounds suspiciously like, "don't change the system
clock when your program is running", which seems reasonable to me.
----------------------------------------------------------------------
Comment By: Martin v. Löwis (loewis)
Date: 2007-02-10 11:26
Message:
Logged In: YES
user_id=21627
Originator: NO
DST is irrelevant for this discussion. On WIndows NT+, the system clock
(as returned by libc ftime/windows GetSystemTime) is always in UTC (broken
down to a Gregorian date). It is thus not affected by DST changes.
What does matter for this bug is administrator changes to the system time
(like manual changes), and time adjustments due to NTP (although the
Windows NTP implementation will perform a non-monototic change only during
boot, AFAIK).
----------------------------------------------------------------------
Comment By: Josiah Carlson (josiahcarlson)
Date: 2007-02-10 10:02
Message:
Logged In: YES
user_id=341410
Originator: NO
Really, it would be better if time.sleep was based off of a monotonic
clock, then everything could use the single working API, and we wouldn't
need to monkey-patch everything that uses sleep.
Unfortunately, due to the insanity that is the underlying Windows
platform, the bios clock is set to local time and is converted to UTC
(which is then used natively by NT). There is a work-around involving
setting the system clock to UTC and setting a registry setting, but it
doesn't always work, can cause 100% CPU usage during DST changes, and
doesn't propagate time changes from NT to the underlying clock.
http://www.cl.cam.ac.uk/~mgk25/mswish/ut-rtc.html
An alternate (or additional option) is to just disable DST changes,
manually adjust times as necessary (or using a script that runs only at
boot), and never run any software while adjusting the time.
http://www.pcreview.co.uk/forums/thread-531048-2.php
Considering the recent changes to DST, the latter link is probably more
applicable.
----------------------------------------------------------------------
Comment By: Martin v. Löwis (loewis)
Date: 2006-12-02 23:11
Message:
Logged In: YES
user_id=21627
Originator: NO
It would be better if this was based on a monotonic clock if available on
the system (such as the POSIX CLOCK_MONOTONIC argument to clock_gettime).
"Guessing" jumps in the time is inherently unreliable; in your code, you
won't notice changes that involve less than 10s.
Notice that the same problem exists for Thread.join; it would be good if
they were fixed together.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607149&group_id=5470
More information about the Patches
mailing list