Add UTC to 2.7 (PyCon sprint idea)
Hello, So far, Python timezone handling is far from "pythonic". There is no function to get current UTC offset, intuitive API to get DST of current time zone and whenever it is active, no functions to work with internet timestamps (RFC 3339). In my case [1] it took about one month and five people to get the right solution for a valid RFC 3339 timestamp. One of the reasons I see is that date/time functions are implemented in C, they expose C API, and there are not many people who can help and patch them. I am sure many current Python users will appreciate UTC functions and improved date/time API more than any new language features. That's why I would like to propose this enhancements for a coding spring on forthcoming PyCon. I am located in Europe and can't attend PyCon, but I summarized date/time issues related to UTC below. More open UTC-related Python issues: http://bugs.python.org/issue1647654 No obvious and correct way to get the time zone offset http://bugs.python.org/issue1667546 Time zone-capable variant of time.localtime http://bugs.python.org/issue7662 time.utcoffset() http://bugs.python.org/issue7229 [PATCH] Manual entry for time.daylight can be misleading http://bugs.python.org/issue5094 datetime lacks concrete tzinfo impl. for UTC http://bugs.python.org/issue7584 datetime.rfcformat() for Date and Time on the Internet (support RFC 3339, ISO 8601 datetime format) http://bugs.python.org/issue665194 datetime-RFC2822 roundtripping http://bugs.python.org/issue6280 calendar.timegm() belongs in time module, next to time.gmtime() All solutions require C expertise. If it will be impossible to find experts able to modify current implementation, then perhaps it could be real to create Python stub for coding solution in Python later? FWIW, this proposal is from my other issue about problems with Python date/time in separate tracker on Google Code [2]. [1] http://bugs.python.org/issue7582 [patch] diff.py to use iso timestamp [2] http://code.google.com/p/rainforce/issues/detail?id=10 python: date/time is a mess -- anatoly t.
Hi, Le Tuesday 16 February 2010 11:38:05 anatoly techtonik, vous avez écrit :
So far, Python timezone handling is far from "pythonic". There is no function to get current UTC offset, (...)
There is the time.timezone attribute: UTC offset in seconds.
Is it no possible to extend Python datetime module in Python? There are already 3rd party libraries: http://pypi.python.org/pypi/pytz http://pypi.python.org/pypi/django-timezones http://www.egenix.com/products/python/mxBase/mxDateTime/ ... Can't why integrating an existing module (well tested, with a documentation, an user base, etc.)?
I am sure many current Python users will appreciate UTC functions and improved date/time API more than any new language features.
Sure, the current API is complex and has few documentation.
That's why I would like to propose this enhancements for a coding spring on forthcoming PyCon.
Excellent idea :) -- There are also some interesting open issues about the datetime module: http://bugs.python.org/issue1289118 - timedelta multiply and divide by floating point http://bugs.python.org/issue1673409 - datetime module missing some important methods http://bugs.python.org/issue2706 - datetime: define division timedelta/timedelta http://bugs.python.org/issue2736 - datetime needs and "epoch" method Bugs about old timestamps: http://bugs.python.org/issue1726687 - Bug found in datetime for Epoch time = -1 http://bugs.python.org/issue1777412 - Python's strftime dislikes years before 1900 http://bugs.python.org/issue2494 - Can't round-trip datetimes<->timestamps prior to 1970 on Windows Victor
Victor Stinner wrote:
Splitting datetime into a datetime.py with an underlying _datetime.c is an idea definitely worth exploring - that module structure makes it much easier to accelerate things that need it, while allowing less critical or more complex aspects to be written in the higher level language. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
Maybe an alternate sprint idea would be to incorporate dateutil into the Python core: http://labix.org/python-dateutil Skip
Maybe an alternate sprint idea would be to incorporate dateutil into the Python core: http://labix.org/python-dateutil Whoops... (just waking up - still need that first cup of coffee) While incorporating dateutil into the core would be nice (in my opinion at least), I was really thinking of pytz: http://pytz.sourceforge.net/ Skip
skip@pobox.com wrote:
While incorporating dateutil into the core would be nice (in my opinion at least)
I believe that idea has come up before - as I recall, the major concern was with the heuristic nature of some of the 'natural language' date parsing. (I could be completely misremembering though...) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
On Tue, Feb 16, 2010 at 13:05, <skip@pobox.com> wrote:
I think dateutil is fairly heavy for the stdlib, but I think pytz would be a very good candidate for inclusion. Without it, the timezone support in datetime is hardly usable. I'd be happy to participate in a PyCon sprint to get datetime issues sorted out and/or work on pytz inclusion. Cheers, Dirkjan
On Tue, Feb 16, 2010 at 13:42, Dirkjan Ochtman <dirkjan@ochtman.nl> wrote:
The timezone database is updated several times per year. You can *not* include it in the standard library. On Tue, Feb 16, 2010 at 16:43, Dirkjan Ochtman <dirkjan@ochtman.nl> wrote:
By using what the OS provides.
The OS often does not.
The Windows timezone data sucks donkeyballs through a hose. Thus, if the timezone implementations from pytz was in the standard library, and the timezone data not, they would not be useable on Windows. So, no can do. Also, different Unices often have slightly different names and organisations of the Olsen database, which would create confusions and incompatibilities, so that's probably also not the best solution. == So, what to do? Use Pytz! == There is no need to stick Pytz in the standard library. It's available on PyPI, updated frequently, etc. What we can do is point to it from the documentation. But before that, it needs a fix. Pytz is great, but missing one thing: Wrappers for the current locale settings. This is necessary, because there is no way of realiably figuring out the current locale. See http://regebro.wordpress.com/2008/05/10/python-and-time-zones-part-2-the-bea... (and http://regebro.wordpress.com/2007/12/18/python-and-time-zones-fighting-the-b... for other timezone issues). These kinds of wrappers exist in dateutils.tz. It would be great if that type of functionality could get into Pytz as well. A sprint to do this and fix the issues in the tracker should solve the issues, I think. There is no need to move things into the core. An Pytz could use more maintainers, Stuart tends not to answer emails, I assume this is because he is overw -- Lennart Regebro: http://regebro.wordpress.com/ Python 3 Porting: http://python-incompatibility.googlecode.com/ +33 661 58 14 64
On Wed, Feb 17, 2010 at 12:32, Lennart Regebro <regebro@gmail.com> wrote:
Bloody gmail! I did NOT press send. Glah. Stuart tends not to answer emails, I assume this is because he is overworked, so more eyes on Pytz is probably a good idea. He is welcome to correct me if this is not so. :) -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
Lennart> The timezone database is updated several times per year. You Lennart> can *not* include it in the standard library. My guess is the data are updated several times per year, not the code. Can they not be separated? Skip
On Wed, Feb 17, 2010 at 23:46, <skip@pobox.com> wrote:
Yes, but that would mean we have an implementation in stdlib that relies on a dataset which may not exist. That is just going to be confusing. Moving pytz into the stdlib doesn't solve anything, really. So why do it? It's not like pytz is hard to install. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
On 17 February 2010 23:01, Lennart Regebro <regebro@gmail.com> wrote:
Some of the Linux distributions *already* patch pytz to use the system information, which they keep updated separately. That information is also available from the system on Mac OS and Windows. It would seem to be very useful to have a version of pytz that defaults to using the system information if available, has a mechanism for using separate data for systems that don't provide the information or raises an error when neither system information nor separate data is available. The data could then still be available and released regularly without being tied to the Python release schedule. That assumes that the author of pytz *wants* it to come into the standard library of course. Michael Foord
On Thu, Feb 18, 2010 at 03:48, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
Some of the Linux distributions *already* patch pytz to use the system information, which they keep updated separately.
Yes. And what problem does including pytz in the stdlib solve?
That information is also available from the system on Mac OS and Windows.
It is not available on Windows in any reasonable and useable form.
Pytz has mechanisms for that, perhaps they should be more easily useable. Perhaps it should even default to using the system Olsen database if there is one. But the discussion was if it should be included in the standard library, and nobody still has explain what problem that would solve. If it doesn't solve a problem, it shouldn't be done, as it also is going to create problems, because everything does. :)
The data could then still be available and released regularly without being tied to the Python release schedule.
Which it already is. So.... no problem solved. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
On Thu, Feb 18, 2010 at 4:38 PM, Lennart Regebro <regebro@gmail.com> wrote:
If it doesn't solve a problem, it shouldn't be done, as it also is going to create problems, because everything does. :)
I think a tzinfo implementation in the standard library that uses the system timezone database would be useful to a great many people, providing a standard way of mapping a string to a tzinfo instance. The number of frameworks requiring pytz as a dependency demonstrate this. It is unfortunate that those strings would be platform specific. For most applications this doesn't matter - you are reading the key from a config file or allowing the user to select from a list of possible values. For applications where that actually matters it would be simple enough to install and maintain a local zoneinfo database, for example by allowing pytz to plug itself in or just a well known location in the Python tree where valid compiled zoneinfo files can be copied in from a nearby unix-like system or pytz tarball. As the pytz maintainer, this would help me solve a long standing problem. Currently, pytz tzinfo instances don't really follow the documented tzinfo interface (in order to allow localized datetime arithmetic to be always correct instead of good enough). This is a source of confusion to many users who don't need this level of accuracy. It would be great if the standard library provided a tzinfo implementation that was good enough for the vast majority of users, and for people who do care they can continue to use pytz.timezone() to retrieve the anal tzinfo implementation. Users will be happier as they will have fewer bugs in their code. The alternative for me is to eventually split pytz, somehow providing the simpler interface that works exactly as documented in the Python reference and the anal interface that works per the pytz README (in hindsight, it should have been this way from day 1). I'm happy to work on this if there is agreement. I'm happy to relicense any pytz code used as a basis if necessary (currently MIT), and dateutil is already PSF licensed if that seems a better starting point. -- Stuart Bishop <stuart@stuartbishop.net> http://www.stuartbishop.net/
On Tue, Feb 16, 2010 at 1:52 PM, Victor Stinner <victor.stinner@haypocalc.com> wrote:
It is correct only if DST is not in effect. On Tue, Feb 16, 2010 at 5:26 PM, Tres Seaver <tseaver@palladion.com> wrote:
Actual TZ information can be shipped with every Python release, but.. If pytz package is available and it's newer - library functions may use its data instead. Of course, this should be documented as official way to maintain TZ info up-to-date. If pytz to be included in standard library - it should still be distributed as separate package to provide more frequent TZ updates and updates to older Python versions. On Wed, Feb 17, 2010 at 1:32 PM, Lennart Regebro <regebro@gmail.com> wrote:
It will still require workarounds and bridges to make API in user scripts convenient, i.e. try: import pytz mydatetime = PytzDatetime() catch ImportError: mydatetime = ClassicDatetime() The goal is to reduce workarounds and avoid repeated code in Python scripts. Leaving pytz aside, does everybody feel comfortable with setting a Wave for API design of date/time issues and the stuff to be done? If there will be an API draft and current list of stuff - I can try to do some work in "offline" mode. -- anatoly t.
On Thu, Feb 18, 2010 at 7:41 AM, anatoly techtonik <techtonik@gmail.com> wrote:
It will still require workarounds and bridges to make API in user scripts convenient, i.e.
I'm not entirely sure what you intended the "It" to refer to here. My take on this is that bundling a version of pytz in the standard library will simply generate more ways to deal with timezones. Using pytz as an external dependency is easy and provides a high level of update support. -Fred -- Fred L. Drake, Jr. <fdrake at gmail.com> "Chaos is the score upon which reality is written." --Henry Miller
On Thu, Feb 18, 2010 at 13:41, anatoly techtonik <techtonik@gmail.com> wrote:
Only if you want to work both with and without pytz. So don't. Just require pytz. I don't see the problem with that. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
On Thu, Feb 18, 2010 at 11:29, Stuart Bishop <stuart@stuartbishop.net> wrote:
Well, that wouldn't work on Windows, which would be a bit strange. So yes, on some systems it would mean you now have pytz in the standard library, while you don't on others. That's going to cause problems, while it doesn't actually solve any problem except "I need to install pytz", which isn't much of a problem.
The number of frameworks requiring pytz as a dependency demonstrate this.
They are going to still need to require pytz, or rather the data part of it.
Well, the problem in finding your won timezone has been documented in one of the links I sent before. But that's another problems, solved by the tzfile/tzwin implementations discussed previously.
I understand the need for different API's but can't the extended part that doesn't behave like timezone be separate methods? I don't *mind* pytz in the standardlibrary, I just don't really see how it solves any problems, while I can see how it creates them. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
On Thu, Feb 18, 2010 at 22:42, "Martin v. Löwis" <martin@v.loewis.de> wrote:
That's not true. The registry is readable by any user, and the format is fully documented.
Yes, but they use non-standard locations, and afaik, pytz does not support it. If a stdlib pytz would use this you would have to use different timezone names for Unix and Windows. I don't think that's a good idea. Also, the windows data contains only current timezone data, so for calendars stretching back in time, the Olsen database would be preferable as it keeps history. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
On Fri, Feb 19, 2010 at 4:45 AM, Lennart Regebro <regebro@gmail.com> wrote:
Under Windows the only backend API I'm aware of we could use is tzwin and I think any standard library inclusion would require this or something similar. For the standard library, I think it would be great if you could do 'datetime.timezone(some_string)' and get a tzinfo instance suitable for most use cases which seems no problem for tzwin to provide (platform default DST information, simple interface with potentially incorrect localized datetime arithmetic over DST transitions). It is unfortunate that the timezone strings are platform specific, but I feel that is still better than end users having to keep updating timezone databases or not supporting non-zoneinfo systems at all. Even with pytz, the timezone strings are version specific to an extent (we had a real issue where we updated pytz on our web applications, which changed a preferred timezone name from Asia/Calcutta to Asia/Kolkata per requests from our users and as a result our wiki's exploded for some users as they where on separate boxes using a different pytz release that didn't understand that timezone string). It would also be trivial to always look up timezone information from compiled zoneinfo files in a well known location if the system timezone database cannot find the requested timezone information. So datetime.timezone('US/Eastern') would work on Windows if you had installed pytz (I'd update pytz to install its zoneinfo files into the blessed location). Best of both worlds.
Sure. I'm not saying pytz will disappear or become unmaintained for people who need it. I believe most people who are using it now don't need it and I'm sure there are real bugs in real code out there because of this, as using pytz correctly requires reading and understanding the pytz documentation. If this is all too ambitious, tzinfo implementations in the standard library for UTC and the current system timezone would be a step forward and solve most real world use cases. -- Stuart Bishop <stuart@stuartbishop.net> http://www.stuartbishop.net/
But is the "You don't have to install it" really such a big problem so that it's worth the other problems like platform incompatibilities and such? In any case, since you want to make a version that can be included and uses the timezone API, I guess that's a moot question until we have that version. :) On Fri, Feb 19, 2010 at 05:59, Stuart Bishop <stuart@stuartbishop.net> wrote:
-- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
On Fri, Feb 19, 2010 at 12:11 PM, Lennart Regebro <regebro@gmail.com> wrote:
I don't think there are any platform incompatibilities - it will work as documented on all platforms. You lose the ability to assume that two identical pytz versions on different platforms can use the same timezone strings, but gain the ability that system timezone strings can be used with Python.
As I understand it dateutil pretty much already provides what I'm describing. -- Stuart Bishop <stuart@stuartbishop.net> http://www.stuartbishop.net/
On Fri, Feb 19, 2010 at 07:27, Stuart Bishop <stuart@stuartbishop.net> wrote:
Well, pretty much yes. I don't know how good it is at using the system data without an Olsen database, but it shouldn't be too much work to add that, I guess. But that changes the topic from moving pytz to stdlib into moving dateutil.tz into stdlib. :) Personally I like pytz "anal" timezone support though, and dateutil.tz doesn't have that, and I still think it would be possible to have both in one system, but using different API-calls. Also, people have uttered negativities about datetime.tz, but they have never been able to say what they don't like about it. I would like if we could look into making a timezone module that works on Python 2.5 to 3.2 that uses system data, unless there is also a "Olsen module" installed, and that has all the features of both dateutil.tz and pytz, ie: 1. Support for the standard API. 2. A Pytz extended API. 3. Using the system data. 4. Using a separate Olsen database installable by normal Python means. 5. Perhaps a timezone name alias map? That could map both old Olsen names and Windows names. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
Lennart> I would like if we could look into making a timezone module Lennart> that works on Python 2.5 to 3.2 that uses system data... 2.5, 2.6 and 3.1 are completely off the radar screen at this point. The best you could hope for is that someone backports whatever is created for 2.7 or 3.2 and distributes it outside the normal distribution channel (say, as a patch on PyPI). Skip
On Fri, Feb 19, 2010 at 19:37, <skip@pobox.com> wrote:
My argument was that we should create a module distributed on PyPI, and once that's stable, move it into stdlib. The suggestions in this thread of moving things into stdlib has included a lot of new features, and are as such not stable. I'm worrying that adding such a thing to stdlib will do so in an unfinished state, and we'll just en up with yet another state of semi-brokenness. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
On Fri, Feb 26, 2010 at 3:59 PM, Lennart Regebro <regebro@gmail.com> wrote:
I valid worry, and compelling. As I've alluded to before, leaving it out and allowing applications to just use pytz (or whatever else) is entirely reasonable. -Fred -- Fred L. Drake, Jr. <fdrake at gmail.com> "Chaos is the score upon which reality is written." --Henry Miller
skip@pobox.com writes:
My guess is the data are updated several times per year, not the code. Can they not be separated?
AIUI this discussion is about getting the ‘pytz’ library into the Python standard library. If the data is separate from the modules, the question then becomes how users on various platforms can update the data without installing a new version of the whole standard library. -- \ “The best ad-libs are rehearsed.” —Graham Kennedy | `\ | _o__) | Ben Finney
-On [20100217 23:48], skip@pobox.com (skip@pobox.com) wrote:
My guess is the data are updated several times per year, not the code. Can they not be separated?
The bulk of the original timezone package is data for the timezones. Last year saw close to 26 releases for this. -- Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェン http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B When you pass through, no one can pin you down, no one can call you back...
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 skip@pobox.com wrote:
Because timezones are defined politically, they change frequently. pytz is released frequently (multiple times per year) to accomodate those changes: I can't see any way to preserve that flexibility if the package were part of stdlib. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkt6uQwACgkQ+gerLs4ltQ4BiwCcDKbfmFmapdQZ188AbiiJ8iCD JvcAoMozT+bcXDCX1tQ5FuLqpCTTbxZe =OP1W -----END PGP SIGNATURE-----
On Tue, Feb 16, 2010 at 16:26, Tres Seaver <tseaver@palladion.com> wrote:
By using what the OS provides. At least on Linux, the basic timezone data is usually updated by other means (at least on the distro I'm familiar with, it's updated quite often, too; through the package manager). I'm assuming Windows and OS X would also be able to provide something like this. I think pytz already looks at this data if it's available (precisely because it might well be newer). Cheers, Dirkjan
On 2/16/2010 11:15 AM, exarkun@twistedmatrix.com wrote:
dateutil can use the system timezone data. See tzfile. http://labix.org/python-dateutil#head-4e4386d98006f1e3cb9290a04bff7e01e58450... or on windows see tzwin. http://labix.org/python-dateutil#head-566bbb3e75e621ac00d2cb1b54abc09036b994...
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dirkjan Ochtman wrote:
If that were so, I don't think Stuart would be going to the trouble to re-release the library 6 - 12 times per year. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkt6xVwACgkQ+gerLs4ltQ4q4ACdGRozE9rfoYkYGmNOiGTQIZyj CeMAoJlmEamyWUbHSQYA0Yq28t+YlbZT =UC3U -----END PGP SIGNATURE-----
On Tue, Feb 16, 2010 at 11:18 PM, Tres Seaver <tseaver@palladion.com> wrote:
The Debian, Ubuntu and I think Redhat packages all use the system zoneinfo database - there are hooks in there to support package maintainers that want to do this. This way the package can be included in the supported release but still receive timezone information updates via the OS (but no code updates, but these are rare and usually irrelevant unless you where the person who filed the bug ;) ). I'd be happy to rework pytz for the standard Library using the system installed zoneinfo database if it is available. I think for the standard library though, it needs to follow the documented API better rather than the .normalize() & .localize rubbish I needed to get localized datetime arithmetic working correctly. Having seen the confusion and bug reports over the last few years, I think people who need this are in the minority and pytz can still exist as a separate package to support them. tzwin could be used on Windows platforms - I'd need to look into that further to see if the API can remain consistent between *nix and Windows. I suspect that pytz without the .normalize() & .localize() rubbish may look remarkably similar to dateutil so that might be a better option to start from. We could consider extending the existing datetime library to support localized datetime arithmetic. This would either involve adding an extra bit to datetime instances to support the is_dst flag (originally deemed unacceptable as it increased the pickle size by a whole byte), or better support for tzinfo implementations to store the is_dst flag in the tzinfo instance (the approach pytz used). This requires a C programmer though and I'm so very, very rusty. I am not at pycon alas. Some of my coworkers from Canonical will be though and they might be interested as we use pytz for Launchpad and other Canonical projects. -- Stuart Bishop <stuart@stuartbishop.net> http://www.stuartbishop.net/
-On [20100217 03:19], Stuart Bishop (stuart@stuartbishop.net) wrote:
This can also work for all the BSDs since they include the Olson zoneinfo data in the base system as well. And that will probably mean Mac OS X as well, if they stuck to what FreeBSD had in place for that. Can anyone verify that? -- Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェン http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Anything becomes possible, after you find the courage to admit that nothing is certain.
On Tue, Feb 16, 2010 at 02:38, anatoly techtonik <techtonik@gmail.com> wrote:
Issue 5094 already has a patch that is nearly complete to provide a default UTC object (and requisite changes to functions to no longer be naive but to use UTC). I did a code review on it in Rietveld and it only has minor things to correct.
Probably worth doing as I am sure everyone would prefer to maintain a pure Python version when possible and only drop into C as needed. See heapq, warnings, and a couple of others if you don't know how to properly do a Python/C module split. -Brett
Hi, Le Tuesday 16 February 2010 11:38:05 anatoly techtonik, vous avez écrit :
So far, Python timezone handling is far from "pythonic". There is no function to get current UTC offset, (...)
There is the time.timezone attribute: UTC offset in seconds.
Is it no possible to extend Python datetime module in Python? There are already 3rd party libraries: http://pypi.python.org/pypi/pytz http://pypi.python.org/pypi/django-timezones http://www.egenix.com/products/python/mxBase/mxDateTime/ ... Can't why integrating an existing module (well tested, with a documentation, an user base, etc.)?
I am sure many current Python users will appreciate UTC functions and improved date/time API more than any new language features.
Sure, the current API is complex and has few documentation.
That's why I would like to propose this enhancements for a coding spring on forthcoming PyCon.
Excellent idea :) -- There are also some interesting open issues about the datetime module: http://bugs.python.org/issue1289118 - timedelta multiply and divide by floating point http://bugs.python.org/issue1673409 - datetime module missing some important methods http://bugs.python.org/issue2706 - datetime: define division timedelta/timedelta http://bugs.python.org/issue2736 - datetime needs and "epoch" method Bugs about old timestamps: http://bugs.python.org/issue1726687 - Bug found in datetime for Epoch time = -1 http://bugs.python.org/issue1777412 - Python's strftime dislikes years before 1900 http://bugs.python.org/issue2494 - Can't round-trip datetimes<->timestamps prior to 1970 on Windows Victor
Victor Stinner wrote:
Splitting datetime into a datetime.py with an underlying _datetime.c is an idea definitely worth exploring - that module structure makes it much easier to accelerate things that need it, while allowing less critical or more complex aspects to be written in the higher level language. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
Maybe an alternate sprint idea would be to incorporate dateutil into the Python core: http://labix.org/python-dateutil Skip
Maybe an alternate sprint idea would be to incorporate dateutil into the Python core: http://labix.org/python-dateutil Whoops... (just waking up - still need that first cup of coffee) While incorporating dateutil into the core would be nice (in my opinion at least), I was really thinking of pytz: http://pytz.sourceforge.net/ Skip
skip@pobox.com wrote:
While incorporating dateutil into the core would be nice (in my opinion at least)
I believe that idea has come up before - as I recall, the major concern was with the heuristic nature of some of the 'natural language' date parsing. (I could be completely misremembering though...) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
On Tue, Feb 16, 2010 at 13:05, <skip@pobox.com> wrote:
I think dateutil is fairly heavy for the stdlib, but I think pytz would be a very good candidate for inclusion. Without it, the timezone support in datetime is hardly usable. I'd be happy to participate in a PyCon sprint to get datetime issues sorted out and/or work on pytz inclusion. Cheers, Dirkjan
On Tue, Feb 16, 2010 at 13:42, Dirkjan Ochtman <dirkjan@ochtman.nl> wrote:
The timezone database is updated several times per year. You can *not* include it in the standard library. On Tue, Feb 16, 2010 at 16:43, Dirkjan Ochtman <dirkjan@ochtman.nl> wrote:
By using what the OS provides.
The OS often does not.
The Windows timezone data sucks donkeyballs through a hose. Thus, if the timezone implementations from pytz was in the standard library, and the timezone data not, they would not be useable on Windows. So, no can do. Also, different Unices often have slightly different names and organisations of the Olsen database, which would create confusions and incompatibilities, so that's probably also not the best solution. == So, what to do? Use Pytz! == There is no need to stick Pytz in the standard library. It's available on PyPI, updated frequently, etc. What we can do is point to it from the documentation. But before that, it needs a fix. Pytz is great, but missing one thing: Wrappers for the current locale settings. This is necessary, because there is no way of realiably figuring out the current locale. See http://regebro.wordpress.com/2008/05/10/python-and-time-zones-part-2-the-bea... (and http://regebro.wordpress.com/2007/12/18/python-and-time-zones-fighting-the-b... for other timezone issues). These kinds of wrappers exist in dateutils.tz. It would be great if that type of functionality could get into Pytz as well. A sprint to do this and fix the issues in the tracker should solve the issues, I think. There is no need to move things into the core. An Pytz could use more maintainers, Stuart tends not to answer emails, I assume this is because he is overw -- Lennart Regebro: http://regebro.wordpress.com/ Python 3 Porting: http://python-incompatibility.googlecode.com/ +33 661 58 14 64
On Wed, Feb 17, 2010 at 12:32, Lennart Regebro <regebro@gmail.com> wrote:
Bloody gmail! I did NOT press send. Glah. Stuart tends not to answer emails, I assume this is because he is overworked, so more eyes on Pytz is probably a good idea. He is welcome to correct me if this is not so. :) -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
Lennart> The timezone database is updated several times per year. You Lennart> can *not* include it in the standard library. My guess is the data are updated several times per year, not the code. Can they not be separated? Skip
On Wed, Feb 17, 2010 at 23:46, <skip@pobox.com> wrote:
Yes, but that would mean we have an implementation in stdlib that relies on a dataset which may not exist. That is just going to be confusing. Moving pytz into the stdlib doesn't solve anything, really. So why do it? It's not like pytz is hard to install. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
On 17 February 2010 23:01, Lennart Regebro <regebro@gmail.com> wrote:
Some of the Linux distributions *already* patch pytz to use the system information, which they keep updated separately. That information is also available from the system on Mac OS and Windows. It would seem to be very useful to have a version of pytz that defaults to using the system information if available, has a mechanism for using separate data for systems that don't provide the information or raises an error when neither system information nor separate data is available. The data could then still be available and released regularly without being tied to the Python release schedule. That assumes that the author of pytz *wants* it to come into the standard library of course. Michael Foord
On Thu, Feb 18, 2010 at 03:48, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
Some of the Linux distributions *already* patch pytz to use the system information, which they keep updated separately.
Yes. And what problem does including pytz in the stdlib solve?
That information is also available from the system on Mac OS and Windows.
It is not available on Windows in any reasonable and useable form.
Pytz has mechanisms for that, perhaps they should be more easily useable. Perhaps it should even default to using the system Olsen database if there is one. But the discussion was if it should be included in the standard library, and nobody still has explain what problem that would solve. If it doesn't solve a problem, it shouldn't be done, as it also is going to create problems, because everything does. :)
The data could then still be available and released regularly without being tied to the Python release schedule.
Which it already is. So.... no problem solved. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
On Thu, Feb 18, 2010 at 4:38 PM, Lennart Regebro <regebro@gmail.com> wrote:
If it doesn't solve a problem, it shouldn't be done, as it also is going to create problems, because everything does. :)
I think a tzinfo implementation in the standard library that uses the system timezone database would be useful to a great many people, providing a standard way of mapping a string to a tzinfo instance. The number of frameworks requiring pytz as a dependency demonstrate this. It is unfortunate that those strings would be platform specific. For most applications this doesn't matter - you are reading the key from a config file or allowing the user to select from a list of possible values. For applications where that actually matters it would be simple enough to install and maintain a local zoneinfo database, for example by allowing pytz to plug itself in or just a well known location in the Python tree where valid compiled zoneinfo files can be copied in from a nearby unix-like system or pytz tarball. As the pytz maintainer, this would help me solve a long standing problem. Currently, pytz tzinfo instances don't really follow the documented tzinfo interface (in order to allow localized datetime arithmetic to be always correct instead of good enough). This is a source of confusion to many users who don't need this level of accuracy. It would be great if the standard library provided a tzinfo implementation that was good enough for the vast majority of users, and for people who do care they can continue to use pytz.timezone() to retrieve the anal tzinfo implementation. Users will be happier as they will have fewer bugs in their code. The alternative for me is to eventually split pytz, somehow providing the simpler interface that works exactly as documented in the Python reference and the anal interface that works per the pytz README (in hindsight, it should have been this way from day 1). I'm happy to work on this if there is agreement. I'm happy to relicense any pytz code used as a basis if necessary (currently MIT), and dateutil is already PSF licensed if that seems a better starting point. -- Stuart Bishop <stuart@stuartbishop.net> http://www.stuartbishop.net/
On Tue, Feb 16, 2010 at 1:52 PM, Victor Stinner <victor.stinner@haypocalc.com> wrote:
It is correct only if DST is not in effect. On Tue, Feb 16, 2010 at 5:26 PM, Tres Seaver <tseaver@palladion.com> wrote:
Actual TZ information can be shipped with every Python release, but.. If pytz package is available and it's newer - library functions may use its data instead. Of course, this should be documented as official way to maintain TZ info up-to-date. If pytz to be included in standard library - it should still be distributed as separate package to provide more frequent TZ updates and updates to older Python versions. On Wed, Feb 17, 2010 at 1:32 PM, Lennart Regebro <regebro@gmail.com> wrote:
It will still require workarounds and bridges to make API in user scripts convenient, i.e. try: import pytz mydatetime = PytzDatetime() catch ImportError: mydatetime = ClassicDatetime() The goal is to reduce workarounds and avoid repeated code in Python scripts. Leaving pytz aside, does everybody feel comfortable with setting a Wave for API design of date/time issues and the stuff to be done? If there will be an API draft and current list of stuff - I can try to do some work in "offline" mode. -- anatoly t.
On Thu, Feb 18, 2010 at 7:41 AM, anatoly techtonik <techtonik@gmail.com> wrote:
It will still require workarounds and bridges to make API in user scripts convenient, i.e.
I'm not entirely sure what you intended the "It" to refer to here. My take on this is that bundling a version of pytz in the standard library will simply generate more ways to deal with timezones. Using pytz as an external dependency is easy and provides a high level of update support. -Fred -- Fred L. Drake, Jr. <fdrake at gmail.com> "Chaos is the score upon which reality is written." --Henry Miller
On Thu, Feb 18, 2010 at 13:41, anatoly techtonik <techtonik@gmail.com> wrote:
Only if you want to work both with and without pytz. So don't. Just require pytz. I don't see the problem with that. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
On Thu, Feb 18, 2010 at 11:29, Stuart Bishop <stuart@stuartbishop.net> wrote:
Well, that wouldn't work on Windows, which would be a bit strange. So yes, on some systems it would mean you now have pytz in the standard library, while you don't on others. That's going to cause problems, while it doesn't actually solve any problem except "I need to install pytz", which isn't much of a problem.
The number of frameworks requiring pytz as a dependency demonstrate this.
They are going to still need to require pytz, or rather the data part of it.
Well, the problem in finding your won timezone has been documented in one of the links I sent before. But that's another problems, solved by the tzfile/tzwin implementations discussed previously.
I understand the need for different API's but can't the extended part that doesn't behave like timezone be separate methods? I don't *mind* pytz in the standardlibrary, I just don't really see how it solves any problems, while I can see how it creates them. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
On Thu, Feb 18, 2010 at 22:42, "Martin v. Löwis" <martin@v.loewis.de> wrote:
That's not true. The registry is readable by any user, and the format is fully documented.
Yes, but they use non-standard locations, and afaik, pytz does not support it. If a stdlib pytz would use this you would have to use different timezone names for Unix and Windows. I don't think that's a good idea. Also, the windows data contains only current timezone data, so for calendars stretching back in time, the Olsen database would be preferable as it keeps history. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
On Fri, Feb 19, 2010 at 4:45 AM, Lennart Regebro <regebro@gmail.com> wrote:
Under Windows the only backend API I'm aware of we could use is tzwin and I think any standard library inclusion would require this or something similar. For the standard library, I think it would be great if you could do 'datetime.timezone(some_string)' and get a tzinfo instance suitable for most use cases which seems no problem for tzwin to provide (platform default DST information, simple interface with potentially incorrect localized datetime arithmetic over DST transitions). It is unfortunate that the timezone strings are platform specific, but I feel that is still better than end users having to keep updating timezone databases or not supporting non-zoneinfo systems at all. Even with pytz, the timezone strings are version specific to an extent (we had a real issue where we updated pytz on our web applications, which changed a preferred timezone name from Asia/Calcutta to Asia/Kolkata per requests from our users and as a result our wiki's exploded for some users as they where on separate boxes using a different pytz release that didn't understand that timezone string). It would also be trivial to always look up timezone information from compiled zoneinfo files in a well known location if the system timezone database cannot find the requested timezone information. So datetime.timezone('US/Eastern') would work on Windows if you had installed pytz (I'd update pytz to install its zoneinfo files into the blessed location). Best of both worlds.
Sure. I'm not saying pytz will disappear or become unmaintained for people who need it. I believe most people who are using it now don't need it and I'm sure there are real bugs in real code out there because of this, as using pytz correctly requires reading and understanding the pytz documentation. If this is all too ambitious, tzinfo implementations in the standard library for UTC and the current system timezone would be a step forward and solve most real world use cases. -- Stuart Bishop <stuart@stuartbishop.net> http://www.stuartbishop.net/
But is the "You don't have to install it" really such a big problem so that it's worth the other problems like platform incompatibilities and such? In any case, since you want to make a version that can be included and uses the timezone API, I guess that's a moot question until we have that version. :) On Fri, Feb 19, 2010 at 05:59, Stuart Bishop <stuart@stuartbishop.net> wrote:
-- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
On Fri, Feb 19, 2010 at 12:11 PM, Lennart Regebro <regebro@gmail.com> wrote:
I don't think there are any platform incompatibilities - it will work as documented on all platforms. You lose the ability to assume that two identical pytz versions on different platforms can use the same timezone strings, but gain the ability that system timezone strings can be used with Python.
As I understand it dateutil pretty much already provides what I'm describing. -- Stuart Bishop <stuart@stuartbishop.net> http://www.stuartbishop.net/
On Fri, Feb 19, 2010 at 07:27, Stuart Bishop <stuart@stuartbishop.net> wrote:
Well, pretty much yes. I don't know how good it is at using the system data without an Olsen database, but it shouldn't be too much work to add that, I guess. But that changes the topic from moving pytz to stdlib into moving dateutil.tz into stdlib. :) Personally I like pytz "anal" timezone support though, and dateutil.tz doesn't have that, and I still think it would be possible to have both in one system, but using different API-calls. Also, people have uttered negativities about datetime.tz, but they have never been able to say what they don't like about it. I would like if we could look into making a timezone module that works on Python 2.5 to 3.2 that uses system data, unless there is also a "Olsen module" installed, and that has all the features of both dateutil.tz and pytz, ie: 1. Support for the standard API. 2. A Pytz extended API. 3. Using the system data. 4. Using a separate Olsen database installable by normal Python means. 5. Perhaps a timezone name alias map? That could map both old Olsen names and Windows names. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
Lennart> I would like if we could look into making a timezone module Lennart> that works on Python 2.5 to 3.2 that uses system data... 2.5, 2.6 and 3.1 are completely off the radar screen at this point. The best you could hope for is that someone backports whatever is created for 2.7 or 3.2 and distributes it outside the normal distribution channel (say, as a patch on PyPI). Skip
On Fri, Feb 19, 2010 at 19:37, <skip@pobox.com> wrote:
My argument was that we should create a module distributed on PyPI, and once that's stable, move it into stdlib. The suggestions in this thread of moving things into stdlib has included a lot of new features, and are as such not stable. I'm worrying that adding such a thing to stdlib will do so in an unfinished state, and we'll just en up with yet another state of semi-brokenness. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
On Fri, Feb 26, 2010 at 3:59 PM, Lennart Regebro <regebro@gmail.com> wrote:
I valid worry, and compelling. As I've alluded to before, leaving it out and allowing applications to just use pytz (or whatever else) is entirely reasonable. -Fred -- Fred L. Drake, Jr. <fdrake at gmail.com> "Chaos is the score upon which reality is written." --Henry Miller
skip@pobox.com writes:
My guess is the data are updated several times per year, not the code. Can they not be separated?
AIUI this discussion is about getting the ‘pytz’ library into the Python standard library. If the data is separate from the modules, the question then becomes how users on various platforms can update the data without installing a new version of the whole standard library. -- \ “The best ad-libs are rehearsed.” —Graham Kennedy | `\ | _o__) | Ben Finney
-On [20100217 23:48], skip@pobox.com (skip@pobox.com) wrote:
My guess is the data are updated several times per year, not the code. Can they not be separated?
The bulk of the original timezone package is data for the timezones. Last year saw close to 26 releases for this. -- Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェン http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B When you pass through, no one can pin you down, no one can call you back...
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 skip@pobox.com wrote:
Because timezones are defined politically, they change frequently. pytz is released frequently (multiple times per year) to accomodate those changes: I can't see any way to preserve that flexibility if the package were part of stdlib. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkt6uQwACgkQ+gerLs4ltQ4BiwCcDKbfmFmapdQZ188AbiiJ8iCD JvcAoMozT+bcXDCX1tQ5FuLqpCTTbxZe =OP1W -----END PGP SIGNATURE-----
On Tue, Feb 16, 2010 at 16:26, Tres Seaver <tseaver@palladion.com> wrote:
By using what the OS provides. At least on Linux, the basic timezone data is usually updated by other means (at least on the distro I'm familiar with, it's updated quite often, too; through the package manager). I'm assuming Windows and OS X would also be able to provide something like this. I think pytz already looks at this data if it's available (precisely because it might well be newer). Cheers, Dirkjan
On 2/16/2010 11:15 AM, exarkun@twistedmatrix.com wrote:
dateutil can use the system timezone data. See tzfile. http://labix.org/python-dateutil#head-4e4386d98006f1e3cb9290a04bff7e01e58450... or on windows see tzwin. http://labix.org/python-dateutil#head-566bbb3e75e621ac00d2cb1b54abc09036b994...
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dirkjan Ochtman wrote:
If that were so, I don't think Stuart would be going to the trouble to re-release the library 6 - 12 times per year. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkt6xVwACgkQ+gerLs4ltQ4q4ACdGRozE9rfoYkYGmNOiGTQIZyj CeMAoJlmEamyWUbHSQYA0Yq28t+YlbZT =UC3U -----END PGP SIGNATURE-----
On Tue, Feb 16, 2010 at 11:18 PM, Tres Seaver <tseaver@palladion.com> wrote:
The Debian, Ubuntu and I think Redhat packages all use the system zoneinfo database - there are hooks in there to support package maintainers that want to do this. This way the package can be included in the supported release but still receive timezone information updates via the OS (but no code updates, but these are rare and usually irrelevant unless you where the person who filed the bug ;) ). I'd be happy to rework pytz for the standard Library using the system installed zoneinfo database if it is available. I think for the standard library though, it needs to follow the documented API better rather than the .normalize() & .localize rubbish I needed to get localized datetime arithmetic working correctly. Having seen the confusion and bug reports over the last few years, I think people who need this are in the minority and pytz can still exist as a separate package to support them. tzwin could be used on Windows platforms - I'd need to look into that further to see if the API can remain consistent between *nix and Windows. I suspect that pytz without the .normalize() & .localize() rubbish may look remarkably similar to dateutil so that might be a better option to start from. We could consider extending the existing datetime library to support localized datetime arithmetic. This would either involve adding an extra bit to datetime instances to support the is_dst flag (originally deemed unacceptable as it increased the pickle size by a whole byte), or better support for tzinfo implementations to store the is_dst flag in the tzinfo instance (the approach pytz used). This requires a C programmer though and I'm so very, very rusty. I am not at pycon alas. Some of my coworkers from Canonical will be though and they might be interested as we use pytz for Launchpad and other Canonical projects. -- Stuart Bishop <stuart@stuartbishop.net> http://www.stuartbishop.net/
-On [20100217 03:19], Stuart Bishop (stuart@stuartbishop.net) wrote:
This can also work for all the BSDs since they include the Olson zoneinfo data in the base system as well. And that will probably mean Mac OS X as well, if they stuck to what FreeBSD had in place for that. Can anyone verify that? -- Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェン http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Anything becomes possible, after you find the courage to admit that nothing is certain.
On Tue, Feb 16, 2010 at 02:38, anatoly techtonik <techtonik@gmail.com> wrote:
Issue 5094 already has a patch that is nearly complete to provide a default UTC object (and requisite changes to functions to no longer be naive but to use UTC). I did a code review on it in Rietveld and it only has minor things to correct.
Probably worth doing as I am sure everyone would prefer to maintain a pure Python version when possible and only drop into C as needed. See heapq, warnings, and a couple of others if you don't know how to properly do a Python/C module split. -Brett
participants (18)
-
"Martin v. Löwis" -
anatoly techtonik -
Ben Finney -
Brett Cannon -
Dirkjan Ochtman -
exarkun＠twistedmatrix.com -
Fred Drake -
Greg Ewing -
Jeroen Ruigrok van der Werven -
Lennart Regebro -
Michael Foord -
Nick Coghlan -
Pop User -
skip＠pobox.com -
Stuart Bishop -
Toshio Kuratomi -
Tres Seaver -
Victor Stinner