Mailman 3 Happy leap second - Python-ideas

Happy leap second

Alexander Belopolsky

30 Jun 2012 30 Jun '12

2:23 p.m.

Even though many have hoped that the authorities would stop fiddling with our clocks, today a leap second will be inserted in UTC. Systems using Olson/IANA timezone database have a way to deal with this without adjusting their clocks, but few systems are configured that way: $ TZ=right/UTC date -d @1341100824 Sat Jun 30 23:59:60 UTC 2012 (1341100824 is the number of seconds since epoch including the leap seconds.) Python's time module works fine with the "right" timezones:

...

...
...
import time print(time.strftime('%T', time.localtime(1341100824))) 23:59:60

but the datetime module clips the leap second down to the previous second:

...

...
...
from datetime import datetime from datetime import datetime print(datetime.fromtimestamp(1341100824).strftime('%T')) 23:59:59 print datetime.fromtimestamp(1341100823).strftime('%T') 23:59:59

BDFL has been resisting adding support for leap seconds to the datetime module [1], but as the clocks become more accurate and synchronization requirements become stricter, we may want to revisit this issue. [1] http://mail.python.org/pipermail/python-ideas/2010-June/007307.html

Show replies by date

Guido van Rossum

30 Jun 30 Jun

2:57 p.m.

POSIX timestamps don't have leap seconds. Convince POSIX to change that and Python will follow suit. On Sat, Jun 30, 2012 at 7:23 AM, Alexander Belopolsky wrote:

...

Even though many have hoped that the authorities would stop fiddling with our clocks, today a leap second will be inserted in UTC. Systems using Olson/IANA timezone database have a way to deal with this without adjusting their clocks, but few systems are configured that way:

$ TZ=right/UTC date -d @1341100824 Sat Jun 30 23:59:60 UTC 2012

(1341100824 is the number of seconds since epoch including the leap seconds.)

Python's time module works fine with the "right" timezones:

...
...
...
import time print(time.strftime('%T', time.localtime(1341100824))) 23:59:60

but the datetime module clips the leap second down to the previous second:

...
...
...
from datetime import datetime from datetime import datetime print(datetime.fromtimestamp(1341100824).strftime('%T')) 23:59:59 print datetime.fromtimestamp(1341100823).strftime('%T') 23:59:59

BDFL has been resisting adding support for leap seconds to the datetime module [1], but as the clocks become more accurate and synchronization requirements become stricter, we may want to revisit this issue.

[1] http://mail.python.org/pipermail/python-ideas/2010-June/007307.html _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas

-- --Guido van Rossum (python.org/~guido)

Alexander Belopolsky

3:18 p.m.

On Sat, Jun 30, 2012 at 10:57 AM, Guido van Rossum wrote:

...

POSIX timestamps don't have leap seconds. Convince POSIX to change that and Python will follow suit.

POSIX (time_t) timestamps are mostly irrelevant for the users of the datetime module. POSIX type that is closest to datetime.datetime is struct tm and it does have leap seconds: """ The header shall declare the structure tm, which shall include at least the following members: int tm_sec Seconds [0,60]. ... """ - http://pubs.opengroup.org/onlinepubs/009696699/basedefs/time.h.html Note that that POSIX does require that a round-trip through time_t (localtime(mktime(x))) converts hh:59:60 to (hh+1):00:00, but datetime.timestamp() can still do the same if we make second=60 valid.

Guido van Rossum

4:29 p.m.

On Sat, Jun 30, 2012 at 8:18 AM, Alexander Belopolsky wrote:

...

On Sat, Jun 30, 2012 at 10:57 AM, Guido van Rossum wrote:

...
POSIX timestamps don't have leap seconds. Convince POSIX to change that and Python will follow suit.

POSIX (time_t) timestamps are mostly irrelevant for the users of the datetime module. POSIX type that is closest to datetime.datetime is struct tm and it does have leap seconds:

""" The header shall declare the structure tm, which shall include at least the following members:

int tm_sec Seconds [0,60]. ... """ - http://pubs.opengroup.org/onlinepubs/009696699/basedefs/time.h.html

Note that that POSIX does require that a round-trip through time_t (localtime(mktime(x))) converts hh:59:60 to (hh+1):00:00, but datetime.timestamp() can still do the same if we make second=60 valid.

The roundtrip requirement is telling though -- they have no way to actually represent a leap second in the underlying clock (which is a POSIX timestamp). -- --Guido van Rossum (python.org/~guido)

Alexander Belopolsky

5:17 p.m.

On Sat, Jun 30, 2012 at 12:29 PM, Guido van Rossum wrote: ..

...

The roundtrip requirement is telling though -- they have no way to actually represent a leap second in the underlying clock (which is a POSIX timestamp).

This correct: POSIX gettimeofday() cannot produce accurate UTC time during the leap second, but this does not mean that a python program should not be able to keep UTC time as accurately as the underlying hardware allows. Systems synchronized with official time using NTP, get notifications about leap seconds up to a day in advance and can prepare for a second during which NTP time stops. (As far as I understand, few systems actually stop their clocks or roll them back on a leap seconds - most slow the clocks down in various incompatible ways.) For example, during the leap second a software clock can use clock_gettime() (or Python's new time.monotonic()) function to get actual time. For better worse, legal time throughout the world is based on UTC and once every couple of years there is a second that has to be communicated as hh:mm:60. Today we are fortunate that it is inserted during the time when most of the world markets are closed, but next time we may see a lot of lawsuits between traders arguing over whose orders should have been filled first. While few systems report accurate UTC time during a leap second, there is no technological limitation that would prevent most systems from implementing it. One can even implement such UTC clock in python, but valid times produced by such clock cannot be stored in datetime objects.

Guido van Rossum

6:04 p.m.

There's no reason why you need to use date time objects for such extreme use cases. --Guido van Rossum (sent from Android phone) On Jun 30, 2012 10:17 AM, "Alexander Belopolsky" < alexander.belopolsky@gmail.com> wrote:

...

On Sat, Jun 30, 2012 at 12:29 PM, Guido van Rossum wrote: ..

...
The roundtrip requirement is telling though -- they have no way to actually represent a leap second in the underlying clock (which is a POSIX timestamp).

This correct: POSIX gettimeofday() cannot produce accurate UTC time during the leap second, but this does not mean that a python program should not be able to keep UTC time as accurately as the underlying hardware allows. Systems synchronized with official time using NTP, get notifications about leap seconds up to a day in advance and can prepare for a second during which NTP time stops. (As far as I understand, few systems actually stop their clocks or roll them back on a leap seconds - most slow the clocks down in various incompatible ways.) For example, during the leap second a software clock can use clock_gettime() (or Python's new time.monotonic()) function to get actual time.

For better worse, legal time throughout the world is based on UTC and once every couple of years there is a second that has to be communicated as hh:mm:60. Today we are fortunate that it is inserted during the time when most of the world markets are closed, but next time we may see a lot of lawsuits between traders arguing over whose orders should have been filled first. While few systems report accurate UTC time during a leap second, there is no technological limitation that would prevent most systems from implementing it. One can even implement such UTC clock in python, but valid times produced by such clock cannot be stored in datetime objects.

Antoine Pitrou

1 Jul 1 Jul

4:22 p.m.

On Sat, 30 Jun 2012 10:23:45 -0400 Alexander Belopolsky wrote:

...

Even though many have hoped that the authorities would stop fiddling with our clocks, today a leap second will be inserted in UTC. Systems using Olson/IANA timezone database have a way to deal with this without adjusting their clocks, but few systems are configured that way:

$ TZ=right/UTC date -d @1341100824 Sat Jun 30 23:59:60 UTC 2012

If I believe the Internet, many server systems got broken by the leap second (*). Even one of our buildbots seems affected: - before: http://buildbot.python.org/all/builders/AMD64%20Ubuntu%20LTS%203.x/builds/41... - after: http://buildbot.python.org/all/builders/AMD64%20Ubuntu%20LTS%203.x/builds/42... (weird errors with timeouts, and a crash) (*) https://linuxfr.org/users/nono/journaux/leap-second Regards Antoine.

Guido van Rossum

7:11 p.m.

So would "supporting" the leap second in Python have made any difference? On Sun, Jul 1, 2012 at 6:22 PM, Antoine Pitrou wrote:

...

On Sat, 30 Jun 2012 10:23:45 -0400 Alexander Belopolsky wrote:

...
Even though many have hoped that the authorities would stop fiddling with our clocks, today a leap second will be inserted in UTC. Systems using Olson/IANA timezone database have a way to deal with this without adjusting their clocks, but few systems are configured that way:

$ TZ=right/UTC date -d @1341100824 Sat Jun 30 23:59:60 UTC 2012

If I believe the Internet, many server systems got broken by the leap second (*). Even one of our buildbots seems affected: - before: http://buildbot.python.org/all/builders/AMD64%20Ubuntu%20LTS%203.x/builds/41... - after: http://buildbot.python.org/all/builders/AMD64%20Ubuntu%20LTS%203.x/builds/42... (weird errors with timeouts, and a crash)

(*) https://linuxfr.org/users/nono/journaux/leap-second

Regards

Antoine.

_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas

-- --Guido van Rossum (python.org/~guido)

Antoine Pitrou

7:43 p.m.

On Sun, 1 Jul 2012 21:11:25 +0200 Guido van Rossum wrote:

...

So would "supporting" the leap second in Python have made any difference?

No. These look like system (kernel?) bugs. Regards Antoine.

Nick Coghlan

10:14 p.m.

It was a kernel race condition (and subsequent live lock) after ntpd notified the clock subsystem of the coming leap second ( http://serverfault.com/questions/403732/anyone-else-experiencing-high-rates-...) The joys of fine grained locking :) Cheers, Nick. -- Sent from my phone, thus the relative brevity :) On Jul 2, 2012 5:45 AM, "Antoine Pitrou" wrote:

...

On Sun, 1 Jul 2012 21:11:25 +0200 Guido van Rossum wrote:

...
So would "supporting" the leap second in Python have made any difference?

No. These look like system (kernel?) bugs.

Regards

Antoine.

_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas

4310

Age (days ago)

4311

Last active (days ago)

List overview

Download

9 comments

4 participants

participants (4)

Alexander Belopolsky
Antoine Pitrou
Guido van Rossum
Nick Coghlan

Happy leap second

tags

participants (4)