Stdlib and timezones, again
With 3.3 out, it's time to bring up something for 3.4. And it's about pytz and stdlib, basically. And we have been over that again, but I have a proposal anyway. The problem with including pytz in the stdlib is that it contains the tz/zoneinfo/Olson database, and it updates much more often than Python does. I propose to solve this problem in the following way: 1. Merge the functionality of pytz into the stdlib datetime module. That means extending datetime.tzinfo with the localize() function and and adding a top-level datetime.timezone() function. any other extra functionality pytz has. 2. The datetime.timezone() function will work as the pytz.timezone() function, ie return a tzinfo object from the tz database. 3. However, Python will not include the database. Instead it will use the one that the OS provides, if it provides one (ie Unices, including OS X if I remember correctly). 4. On any other OS, datetime.timezone() will return an error explaining that no tz database can be found, but that it is available in the module 'pytzdata' which then will need to be created. If this module is installed, the datetime.timezone() function will prefer that database before the OS database. 5. We can also consider implementing something like the get_localzone() function of my module tzlocal. This solves two problems. i. We have real, useful timezone support in Python. For Unix, it's included, for all other systems, it's just an extra module. The extra module can also be installed to give Python up to date tz information even when the operating system is outdated and unsupported and no longer receiving updates. ii. pytz is_dst flag becomes included in stdlib, solving one of the few imperfections in the current library. What do you say? Is this a path worth pursuing? //Lennart
On Sun, Sep 30, 2012 at 2:47 PM, Lennart Regebro <regebro@gmail.com> wrote:
What do you say? Is this a path worth pursuing?
+1. It's the kind of low-level thing that should be solved in the stdlib as far as possible, and the pytz interface is as stable as the stdlib's. Cheers, Dirkjan
On Sun, 30 Sep 2012 14:47:28 +0200 Lennart Regebro <regebro@gmail.com> wrote:
Can't we simply include the Olson database in Windows installers? Regards Antoine. -- Software development and contracting: http://pro.pitrou.net
On Sun, Sep 30, 2012 at 3:03 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
Can't we simply include the Olson database in Windows installers?
We probably can, but the problem is that it's updated quite often (for example, in 2011, there were about 14 releases; in 2009, there were 21). So you'd want to have a mechanism to override the data that is included in the stdlib. Cheers, Dirkjan
On Sun, 30 Sep 2012 15:10:06 +0200 Dirkjan Ochtman <dirkjan@ochtman.nl> wrote:
Probably, but for most purposes I would guess a 2-year old database is still good enough? After all, you don't see many people complaining about the outdated Unicode database that is hard-wired in past Pythons. Regards Antoine.
2012/9/30 Xavier Morel <python-dev@masklinn.net>:
But at worst, an outdated unicode database will be missing data right?
Doesn't an outdated timezone db have the risk of returning *incorrect* data?
Unicode updates also include corrections; however, it seems there are not significant enough or about obscure enough scripts that not many notice. :) -- Regards, Benjamin
On Sun, Sep 30, 2012 at 8:33 AM, Benjamin Peterson <benjamin@python.org>wrote:
We never hear anyone complain because the corrections are not for English or other "western" languages that the majority of us speak. ;) Regardless, I think including a version of the database on windows releases makes sense. Update it on a best effort basis before each .x minor release. The documentation can make it clear that it is a changing database how to use an updated version with your application. One additional thing I'd like to see: Don't let the successful importing of a 'pytzdata' module be the only way to get an updated tz database. Create an API for someone to supply one themselves at runtime that is cleaner than shoving something into sys.modules['pytzdata']. And define in which order they'll be used: priority: 1) api call supplying tz data to the process. 2) pytzdata module if it exists 3) tz data from the underlying operating system 4) error. -gps PS Unicode data is political as well, just less politically active than stupid time zones and "savings" times.
On Sun, Sep 30, 2012 at 3:17 PM, Matthias Klose <doko@ubuntu.com> wrote:
Agreed, but that is why anyone writing code for Linux will simply never use 1) or 2). Those exist primarily to handle the Windows and people with software running on non well managed systems. I'd add a 3.5) to the above list: tz data bundled with the Python distribution. Bundled tz data would likely simply not even be included in a Python package on a Linux system. -gps
On 01.10.2012 00:50, Gregory P. Smith wrote:
I would add - a 2.5) if pytzdata does exist and uses the tz data from the underlying operating system. - a 3.5) if pytzdata does exist and doesn't use the tz data from the underlying operating system. however I see our patch to pytzdata to use the os data is a local patch, and not found upstream. Matthias
On 01.10.2012 00:51, Lennart Regebro wrote:
but not overriding with an outdated database. the new venv stuff makes the distinction to just use the core python environment, or use the core and site/dist-packages. I don't want to see an exception to not use something which is not in the core. Matthias
On Mon, Oct 1, 2012 at 8:17 AM, Matthias Klose <doko@ubuntu.com> wrote:
There's no guarantee that an individual sysadmin will have OS updates up-to-date. If, on Linux, the pytzdata module is not installed unless explicitly called for, that would define pytzdata as high specificity, ergo it should override the lower specificity of the OS-provided data. The normal case on Linux will happily use the well-managed and frequently-updated tzdata. ChrisA
Matthias Klose wrote:
The order exists this way to allow for customization. If I have 1) downloaded the most recent, or 2) made customizations (for whatever reason) then Python needs to use it. Can I break stuff this way? Sure. Is it Python's responsibility to stop me? Nope. Consenting adults, and all that. ~Ethan~
Am 30.09.2012 20:18, schrieb Gregory P. Smith:
I like your basic approach but I'm suggesting a slightly different approach. Before I explain my proposal I like to get a naming convention straight. integrated tz database ---------------------- A copy of the Olson database that is shipped with every version of Python (not just Windows). The integrated database is updated with every feature release of Python. system tz database ------------------ That's an interface to the operating system's or platform's tz database. We probably have to provide multiple backends for different OSes, Java etc. update tz database ------------------ A PyPI package that contains a current version of the Olson database. A new version of the update tz should be provided by the Python core developer team every time a new Olson database is available. The updatetz must never be distributed with the Python source code and shall not be installed by a distributor. Optionally we can include the code that creates an update tz package from a Olson database. By default Python tries to load the updatetz first, then systemtz (if available) and finally the integratedtz. A user can query the status and version of the databases, set and get the currently used database with three functions. The used database can also be set with an environment variable: datetime.gettzdatabase() -> "integrated" or "system" or "update" datetime.settzdatabase(name) datetime.listtzdatabases() -> {"integrated": "version", "system": "???" "update": "version", # only if an update tz db is installed } PYTHON_TZDB = "integrated" or "system" or "update" With this setup users get the full benefit of system updates but can use the integrated or update database if they don't like the operating system's data. Christian
On Mon, Oct 1, 2012 at 9:02 AM, Lennart Regebro <regebro@gmail.com> wrote:
I think that would be a little too pure to be practical. There are many practical usages of tz data that would work fine with a year-old timezone database. Personally, I'd want to not ship any tzdata with non-Windows Python packages on the assumption that they have good up-to-date OS tzdata (or it should be easy to disable it in the configure phase). Also, I wonder if it would be possible to select the copy of the data that is the most recent (e.g. on Unix, pick the OS version if tzupdate is installed but older than the OS version). Cheers, Dirkjan
On Mon, Oct 1, 2012 at 10:28 AM, Dirkjan Ochtman <dirkjan@ochtman.nl> wrote:
A year is no age for a Python installation. A customer of mine has one website developed in 2003, still running on the same server. It runs Python 2.3, I don't remember which version, but I'd be surprised if it is 2.3.7 from 2008.
I don't think so. As far as I can tell, the data files contain no version information (only information on the version of the file format, which currently is 2). It also contains no information on what the name of the timezone is. This lack of information is unfortunate, but we'll have to live with it. The format hasn't changed since 1989, it is unlikely that we'll get anyone to change it anytime soon. //Lennart
On Mon, Oct 1, 2012 at 10:47 AM, Lennart Regebro <regebro@gmail.com> wrote:
Right. If they don't keep their Python up-to-date, why would they bother with their tzupdate? My point is that there is not much of a difference in the incentive for upgrading your timezone data whether an initial version of it came with Python or not. Having to manually install it might make you slightly more aware that it helps if you upgrade it once in a while, but it seems more likely to be a fire and forget type of operation, in which case it's basically the same as shipping a version of the timezone data with Python (which is much easier, of course). To put it crudely, you seem to think that most developers keep careful track of what packages they need for their apps and actively assess the risk for upgrading each of the packages involved. On the other hand, I would assume more developers just get something working and then fix any bugs that come up. Cheers, Dirkjan
On Mon, Oct 1, 2012 at 11:16 AM, Dirkjan Ochtman <dirkjan@ochtman.nl> wrote:
I'm sorry, is there a new releases of Python 2.3 made last year I don't know about? My point is that there is not much of a difference in the incentive
for upgrading your timezone data whether an initial version of it came with Python or not.
Incentive, no. But there is a difference in awareness and likelyhood.
Well, me at least would include that package in the buildout or the instructions, etc. It is therefore much more likely to be updated if it is not included with Python. To put it crudely, you seem to think that most developers keep careful
No, I assume there are developers of both types, and in between. If somebody just installed pytzdata and then doesn't upgrade it, fine, that's his problem. He doesn't become *more* likely to upgrade it because it's included in the standard library. But many developers are more likely to keep it updated and upgrade it if it is *not* included, at least once in a while. For example, if it's included in a buildout it could get updated when the buildout is re-run because some of the custom modules have been updated. While if it's included, it will never end up in the buildout and never get updated. //Lennart
On Mon, 1 Oct 2012 12:11:41 +0200 Lennart Regebro <regebro@gmail.com> wrote:
Python 2.3 has been EOL'ed for years. It definitely is not up-to-date, for any reasonable definition of the term. For example, it will have many unplugged security holes. So will that user's version of OpenSSL and other libraries. If they don't want to apply security fixes, why should we even care about their timezones' freshness? Regards Antoine. -- Software development and contracting: http://pro.pitrou.net
On Mon, Oct 1, 2012 at 5:23 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
Exactly. If a service provider has a bug in their application because they're using an old timezone database, that is their problem, so long as *we* ensure there is a clear and obvious mechanism for upgrading *just* the TZ database, without upgrading Python itself. There's nothing forcing people to keep their installed version of pytz up to date, either. If a timezone database is bundled into the standard library, there are 3 clear mechanisms for encouraging the use of fresh TZ data: 1. Consider TZ database updates to be bug fixes, and thus include them in maintenance releases. This will keep the provided version reasonably fresh for Python versions that are still in maintenance mode. 2. Provide a mechanism to prefer the database from PyPI. 3. Provide a mechanism to prefer the OS database for platforms that provide an Olson compatible interface (I briefly looked into that for Windows a while back - it doesn't seem like a practical idea, since Microsoft went off and did their own thing. It works for Linux and other platforms that use the Olson database natively, though) Since explicit is better than implicit, I *wouldn't* want to see magical side affects where merely installing the database from PyPI, or switching from Windows to Linux caused different behaviour. However, it should be very easy for an application or environment to *explicitly request* the use of the pytz database or the OS database in preference to the bundled database. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Mon, Oct 1, 2012 at 3:22 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I think that would be a pain. What you're proposing means that Linux installations have to use the Python-installed copy by default (because you want them to do the same thing as on Windows), even though they have a perfectly good copy, often newer, of the database installed on the system. Cheers, Dirkjan
On Mon, Oct 1, 2012 at 3:22 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I proposed 2 and 3, and I don't really see much magical side-effects with those. As mentioned we can also include a database in the standardlib, but since that will almost always be out of date, I don't really see the point. It is of course only an issue on Windows, but still. //Lennart
Re: to bundle or not to bundle What about having an included database that issues a (silence-able) warning any time it is used/imported/etc.? Have it say something to the effect of "Warning, the Olson database included with Python is likely to be outdated, please see <webpage> for information" where <webpage> (which may just be the doc page for the module) spells out why it's outdated, why it's not possible for it to be kept up to date, that this version may still work for you depending on application and how to silence the warning, and how to get the latest version via pypi or otherwise. As far as preference of database, I would think the best route would be to provide the ability to set the order you want to look in, with the default being: 1) user specified source (usually not one of the below) 2) updated tzdb from pypi 3) OS's tzdb 4) included tzdb (with warning) My $0.02USD, for what they're worth :)
On Mon, Oct 1, 2012 at 3:57 PM, Zachary Ware <zachary.ware+pydev@gmail.com>wrote:
Actually, that's not a bad idea. My original idea was to warn if it *was* outdated, but since there is no way to check that, I scratched that idea. But as I have pointed out several times, a database that is shipped with Python is almost guaranteed to be outdated, so yeah, we could just warn *all the time*. :-) I like this idea. It gives an incentive to update: Get rid of the annoying warning. It also will be silent on Unix, as we'll use the OS database, so this will only happen on Windows or if you embed Python or similar. //Lennart
On Mon, 1 Oct 2012 16:06:18 +0200 Lennart Regebro <regebro@gmail.com> wrote:
Well, no, it is just silly. If we ship a database, that's because we think it is good enough. A warning is just a nuisance here. We don't display warnings when the installed Python version is too old. Regards Antoine. -- Software development and contracting: http://pro.pitrou.net
On Mon, Oct 1, 2012 at 9:12 AM, Antoine Pitrou <solipsis@pitrou.net> wrote:
My thought was that it's better to have *something* always available, that has a decent chance of being "good enough" in a lot of cases (and if it's good enough for you, just silence the warning), than to noisily fail because we can't provide a perfect solution due to political idiocy. Or worse, to *silently* be wrong because someone assumed we had provided a perfect solution without looking too hard. I had no idea the Olson database was updated so often until Dirkjan posted about there being 21 updates in 2009 alone. For most of my uses, I would probably be a warning-silencer. And that wouldn't bother me; I would actually appreciate being reminded now and then that things may have changed since the last time I did something with timezones, and that I need to be careful of such changes. But, of course, that's just me, and it was my idea anyway ;)
On Mon, 1 Oct 2012 09:52:09 -0500 Zachary Ware <zachary.ware+pydev@gmail.com> wrote:
We can, and should, mention potential pitfalls in the documentation. But I don't think a warning is warranted, anymore than for other known issues (there are many of them at http://bugs.python.org/ :-)). Regards Antoine. -- Software development and contracting: http://pro.pitrou.net
On Oct 1, 2012 10:06 AM, "Antoine Pitrou" <solipsis@pitrou.net> wrote:
With large red text and blink tags :-P
But I don't think a warning is warranted, anymore than for other known issues (there are many of them at http://bugs.python.org/ :-)).
Just curious (and a bit off topic), what exactly does warrant a warning in Python? I've been messing around with it for a couple years (since shortly before 3.1, and always on 3.x) and I'm pretty sure I have yet to see a warning for anything. Which I suppose could be counted as a good thing...
On Mon, 1 Oct 2012 10:16:16 -0500 Zachary Ware <zachary.ware@gmail.com> wrote:
We don't really have a well-defined policy, except for one warning category: DeprecationWarnings, whose purpose are quite clear. But they are now silenced by default :-) Generally, warnings are not easy to deal with for the end-user, so we are reluctant to add any. We'd rather document annoyances and pitfalls; and, moreover, we try to design our APIs so that they don't have unexpected effects. Regards Antoine. -- Software development and contracting: http://pro.pitrou.net
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/01/2012 10:12 AM, Antoine Pitrou wrote:
Using a bundled version would be the Wrong Thing (TM) for almost any non-toy application (one where non-programmers actually care about the timezones in which dates / times are entered / displayed): the fact that the errors would infrequent / subtle / hard to diagnose is an argument *against* inclusion. - -1 for including a copy in Python at all: if your app needs it, and the OS on which you deploy doesn't provide it, you have a straightforward way of installing it. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iEUEARECAAYFAlBp36IACgkQ+gerLs4ltQ4mbACY60cQ1SbUgRh0hNtUmxScRM68 ZwCg125QzeMmhFV0VYYZGmNVmEqpajo= =ehg8 -----END PGP SIGNATURE-----
Reminder to everyone: the current state of the art for getting up to date tz info for Python is "pip install pytz". If any proposal is more complicated than that, there's absolutely no point in changing anything. The version I liked best so far is for Python to just install a copy of pytz automatically (shipping it in the installer rather than downloading it). OS packagers would then take it out (replacing it with a dependency on a pytz emulator that used the system database instead). Cheers, Nick.
On Tue, 02 Oct 2012 00:18:10 +0530, Nick Coghlan <ncoghlan@gmail.com> wrote:
Emulator? That makes no sense, I'm afraid. I think we are talking here about incorporating pytz into the stdlib. The only question is how to manage the Olson database on Windows, which has *always* been the question. --David
It occurred to me this morning that Python already ships a set of timezone data with the Windows installer: Tcl/Tk's. Is there any way we could use that as the default on Windows?
On Thu, Oct 4, 2012 at 12:01 PM, Zachary Ware <zachary.ware+pydev@gmail.com> wrote:
I would say no. You could choose not to include Tcl/Tk in your installation, or you could build your own Python and not include those sources. In either case, the lack of a GUI toolkit is now affecting your use of timezone data - weird/bad. Plus we only pull updated Tcl/Tk every few years.
On Mon, Oct 1, 2012 at 12:02 PM, Barry Warsaw <barry@python.org> wrote:
I think those are all suggestions included in Nick's intention. The main points are that (a) pytz and some not-too-ancient Olson database are included by default, (b) vendor distributions can tweak the packaging so that the platform's copy of the Olson database is used automatically; (c) users not benefitting from such vendor support can update the database as easily as "pip install pytz" (though not necessarily with that specific command). -- --Guido van Rossum (python.org/~guido)
On Mon, Oct 1, 2012 at 8:48 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
But that would then mean that when you install from source, as I typically do to avoid depending on Python, it would use another database than the OS version. I don't want that, i want Python on Unix to use the OS supplied database, unless another one is explicitly installed/configured. //Lennart
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/01/2012 02:48 PM, Nick Coghlan wrote:
Lennart's original proposal was to land the machinery from the current pytz package into the stdlib, leaving the data part as a separately-installable package. Lennart also proposed some kind of (maybe configurable) policy for looking up the Olson-based timezonedb. I'm fine with Lennart's proposal, but want to argue against including any copy of the Olson db in Python itself: I don't believe the set of folks who need it but can't do the equivalent of 'pip / easy_install <lennarts-new-name-for-the-data-part>' is big enough to outweigh the downside of bundling immediately-stale data, and hoping that everybody else remembers to configure it out. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iEYEARECAAYFAlBp6OkACgkQ+gerLs4ltQ4fZwCfUtz9lnkjxM6rdTaO0io1yzIP f2cAn0YfQt+n09Znuh6BZ/Culixpf7eE =ecDk -----END PGP SIGNATURE-----
Gah, ignore half of that last post. I think Lennart is talking about incorporating the missing pytz *functionality* into the stdlib. --David
On 2012-10-01, at 17:32 , Terry Reedy wrote:
There are several: there's a message on a dedicated mailing list and there are HTTP, FTP and rsync repositories with both all releases and a "latest" archive for both tzdata and tzcode. The HTTP repositories seems to handle time-based conditional requests correctly (an If-Modified-Since with the date of the latest release or later will result in a 304 response, earlier will result in a 200) The HTTP URIs are https://www.iana.org/time-zones/repository/tzdata-latest.tar.gz and https://www.iana.org/time-zones/repository/tzcode-latest.tar.gz For some reason, IANA does not seem to publish a feed.
On Mon, Oct 1, 2012 at 5:32 PM, Terry Reedy <tjreedy@udel.edu> wrote:
Sure, but there is no way to get a last updated time from the database that's installed. And making an HTTP request when you call localize() to check if the database is outdated or not doesn't seem to be a great idea. //Lennart
On Oct 01, 2012, at 03:34 PM, Lennart Regebro wrote:
I agree. I don't think the Python RM should have to worry about tz updates, given how frequent or unpredictable they can be. For OSes that provide the database, I can't think of any reason not to prefer that, except if your OS version is no longer being maintained, and then it seems like updating your tz database is the least of your worries. However, if someone wants to maintain a Cheeseshop package with updates, I can't stop them. My biggest concern there is that eventually the maintainers will lose interest and this package will bitrot, and then we'll have obsolete tz info out there that people will still rely on. Oh well, I guess. I completely agree that just installing the Cheeseshop tz package should *not* be enough to prefer it over the system tz data. Cheers, -Barry
On Oct 01, 2012, at 05:15 PM, Lennart Regebro wrote:
Yes. Using the script I mentioned in an different response, if someone installed the database to some location (TBD), then there would probably be a config file sitting next to it. A simple .ini file with an 'enable' flag would be needed to turn on the override. Cheers, -Barry
On Mon, Oct 1, 2012 at 6:21 PM, Larry Hastings <larry@hastings.org> wrote:
The most current one is likely to be the one provided by the operating system, which does not contain any .ini file, nor, as far as I can tell, any information about the database version or any timestamps. //Lennart
On 10/1/2012 12:39 PM, Lennart Regebro wrote:
But Windows does not provide one, or at least, the proposal seems to be not use whatever it does have. I think your PEP should propose one api but conditional tz db access code for systems with and without the tz db already provided. -- Terry Jan Reedy
On Mon, 1 Oct 2012 11:11:46 -0400 Barry Warsaw <barry@python.org> wrote:
I agree. I don't think the Python RM should have to worry about tz updates, given how frequent or unpredictable they can be.
I don't understand what makes them specifically "unpredictable". We statically link OpenSSL and other libraries in the Windows builds. I don't think they have very predictable updates either (especially OpenSSL, actually). Regards Antoine. -- Software development and contracting: http://pro.pitrou.net
On Sun, Sep 30, 2012 at 11:15 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
An out-of-date timezone database will bite someone at some point. They'll be running code that expects some locality to be on/off DST when it isn't, and some customer will be confused or annoyed. I suspect most people won't much care, though. It's only a few who'd explicitly upgrade their tz database. ChrisA
On 30.09.12 16:15, Antoine Pitrou wrote:
In 2011 Ukrainian timezone data was changed twice for year. And perhaps even change in 2013. Russian timezones were changed over the last few years and most likely will change in the near future. Correct time is critical for many applications.
On Sun, 30 Sep 2012 21:55:34 +0300 Serhiy Storchaka <storchaka@gmail.com> wrote:
Perhaps, but that's the responsibility of governements. Just because some governments have erratic policies shouldn't be a reason for residents of other countries not to enjoy the benefits of their stable timezones. Regards Antoine. -- Software development and contracting: http://pro.pitrou.net
On Sun, 30 Sep 2012 22:35:54 +0300 Serhiy Storchaka <storchaka@gmail.com> wrote:
Well, no, this isn't similar. Choosing one's timezone policies is a contemporary political decision, while choosing a language and its alphabet is not really a decision people ever make (it's just an aspect of a society's long-term evolution) - except Atatürk, perhaps :-) Furthermore, the proposal I'm making does *not* disadvantage residents of Russia and Ukraine: whether our Windows installer provides a database or not, they have to download a new database if they want up-to-date information. And they have to download it afresh every few months, if I'm following you. Regards Antoine. -- Software development and contracting: http://pro.pitrou.net
On 30.09.12 22:51, Antoine Pitrou wrote:
Oh, no. Choosing of alphabet (and sometimes language) is also a contemporary political decision. For the last 25 years new letter Ґ has been added to the Ukrainian alphabet, and the letter Ь changed its place in the alphabet. There were at least 4 family of absolutely different character sets for Ukrainian (not counting the Unicode), some of them contains several incompatible variants. In several neighboring countries the alphabet was changed completely (from Cyrillic-based to Latin). Why ASCII is not enough for all?
Who will update the database? The developer which distributes the application with embedded Python can forget about the tz updates, as well as about non-ascii encodings. Native Unicode support in Python makes the second error less likely. Why not use the system data which are updated by the OS? I know that Windows also changes the clock for local DST.
On Sun, Sep 30, 2012 at 11:28 PM, Serhiy Storchaka <storchaka@gmail.com>wrote:
Why not use the system data which are updated by the OS? I know that Windows also changes the clock for local DST.
The Windows timezone information does not include any historical information, as it's designed primarily to keep your computers clock correctly, not to convert date times between different timezones. As a result using to to convert data in the past can give incorrect information. We therefore want to consistently use the tz database. //Lennart
On Mon, 01 Oct 2012 00:28:41 +0300 Serhiy Storchaka <storchaka@gmail.com> wrote:
Well, yeah, but it's not like you can do it on a whim either, and you can't change the large body of existing text.
If that's possible, then it sounds the ideal solution indeed. Regards Antoine. -- Software development and contracting: http://pro.pitrou.net
On 2012-09-30 19:55, Serhiy Storchaka wrote:
That's what UTC is for! :-) I think that it would be a good idea to provide a database with the release plus a tool for updating it, the updates being announced by email or RSS, and the ability to use the system's database if there's one.
When people use pytz they have to reinstall pytz too if thet want to
benefit from the updates. (Or depend on automated updates via some vendor
On Sep 30, 2012 10:34 PM, "Guido van Rossum" <guido@python.org> wrote: package management system.) If we can ensure that with pytz in the stdlib, updates to the Olson database can be installed just as easily as before (probably through some new custom mechanism, as a new Python bugfix release is too heavy-handed), I see no problem with adopting pytz into the stdlib. The mechanism would be exactly the same, but a different modulename. I´ll go forward and make a PEP on this soonish.
On Sep 30, 2012, at 02:47 PM, Lennart Regebro wrote:
Why include the database in Python at all? If you have an OS that keeps the system tz data up-to-date, I can't think of any reason why you wouldn't want to use it. If you don't have the data, why not include information in the documentation for how to download and install the database in a location that Python will search for, along with information on how to enable that? You could even provide a nice script that would download, install, and optionally enable that tz data's use. I think that would cover all the bases: * My OS keeps the tz data up-to-date. I don't have to do nuthin', and Python gives me a nice API for using all the world's timezones on my superb OS. * My OS keeps the tz data up-to-date, but I'm skeptical. I run the update script whenever is appropriate, adding the --enable flag, and the tz data is grabbed from the intarwebs, installed, and Python starts using it instead of the system data. * I am sad because I use an OS that has no tz data. I run the update script once in a while, adding the --enable flag, and my Python is timezonally happy. * I'm sad and lazy. Oh well, Python throws an exception when I try to use a timezone that isn't UTC. Cheers, -Barry
On Mon, Oct 1, 2012 at 5:25 PM, Barry Warsaw <barry@python.org> wrote:
Exactly my point.
Right, I was going to do that by bundling the files in a package, tentatively called pytzdata, available on PyPI, so it can be easy_installed/pip installed, etc.
We seem to be on the same page here. //Lennart
On Mon, 1 Oct 2012 11:25:08 -0400 Barry Warsaw <barry@python.org> wrote:
Well, for one, that doesn't really fit in the "batteries included" approach. Having to download separate data files is an annoyance.
This doesn't solve the update problem at all, since that tz data would still be outdated after a couple of months. In other words, it has no benefit for the user, but is more complicated to use. Besides, that script would have to care about security issues. For example you'd use a HTTPS-enabled source, and then bundle the required CA certificates with Python. But wait, now Python has to ship some data that will possibly become outdated in a year or two :-) Regards Antoine. -- Software development and contracting: http://pro.pitrou.net
On Sun, Sep 30, 2012 at 2:47 PM, Lennart Regebro <regebro@gmail.com> wrote:
What do you say? Is this a path worth pursuing?
+1. It's the kind of low-level thing that should be solved in the stdlib as far as possible, and the pytz interface is as stable as the stdlib's. Cheers, Dirkjan
On Sun, 30 Sep 2012 14:47:28 +0200 Lennart Regebro <regebro@gmail.com> wrote:
Can't we simply include the Olson database in Windows installers? Regards Antoine. -- Software development and contracting: http://pro.pitrou.net
On Sun, Sep 30, 2012 at 3:03 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
Can't we simply include the Olson database in Windows installers?
We probably can, but the problem is that it's updated quite often (for example, in 2011, there were about 14 releases; in 2009, there were 21). So you'd want to have a mechanism to override the data that is included in the stdlib. Cheers, Dirkjan
On Sun, 30 Sep 2012 15:10:06 +0200 Dirkjan Ochtman <dirkjan@ochtman.nl> wrote:
Probably, but for most purposes I would guess a 2-year old database is still good enough? After all, you don't see many people complaining about the outdated Unicode database that is hard-wired in past Pythons. Regards Antoine.
2012/9/30 Xavier Morel <python-dev@masklinn.net>:
But at worst, an outdated unicode database will be missing data right?
Doesn't an outdated timezone db have the risk of returning *incorrect* data?
Unicode updates also include corrections; however, it seems there are not significant enough or about obscure enough scripts that not many notice. :) -- Regards, Benjamin
On Sun, Sep 30, 2012 at 8:33 AM, Benjamin Peterson <benjamin@python.org>wrote:
We never hear anyone complain because the corrections are not for English or other "western" languages that the majority of us speak. ;) Regardless, I think including a version of the database on windows releases makes sense. Update it on a best effort basis before each .x minor release. The documentation can make it clear that it is a changing database how to use an updated version with your application. One additional thing I'd like to see: Don't let the successful importing of a 'pytzdata' module be the only way to get an updated tz database. Create an API for someone to supply one themselves at runtime that is cleaner than shoving something into sys.modules['pytzdata']. And define in which order they'll be used: priority: 1) api call supplying tz data to the process. 2) pytzdata module if it exists 3) tz data from the underlying operating system 4) error. -gps PS Unicode data is political as well, just less politically active than stupid time zones and "savings" times.
On Sun, Sep 30, 2012 at 3:17 PM, Matthias Klose <doko@ubuntu.com> wrote:
Agreed, but that is why anyone writing code for Linux will simply never use 1) or 2). Those exist primarily to handle the Windows and people with software running on non well managed systems. I'd add a 3.5) to the above list: tz data bundled with the Python distribution. Bundled tz data would likely simply not even be included in a Python package on a Linux system. -gps
On 01.10.2012 00:50, Gregory P. Smith wrote:
I would add - a 2.5) if pytzdata does exist and uses the tz data from the underlying operating system. - a 3.5) if pytzdata does exist and doesn't use the tz data from the underlying operating system. however I see our patch to pytzdata to use the os data is a local patch, and not found upstream. Matthias
On 01.10.2012 00:51, Lennart Regebro wrote:
but not overriding with an outdated database. the new venv stuff makes the distinction to just use the core python environment, or use the core and site/dist-packages. I don't want to see an exception to not use something which is not in the core. Matthias
On Mon, Oct 1, 2012 at 8:17 AM, Matthias Klose <doko@ubuntu.com> wrote:
There's no guarantee that an individual sysadmin will have OS updates up-to-date. If, on Linux, the pytzdata module is not installed unless explicitly called for, that would define pytzdata as high specificity, ergo it should override the lower specificity of the OS-provided data. The normal case on Linux will happily use the well-managed and frequently-updated tzdata. ChrisA
Matthias Klose wrote:
The order exists this way to allow for customization. If I have 1) downloaded the most recent, or 2) made customizations (for whatever reason) then Python needs to use it. Can I break stuff this way? Sure. Is it Python's responsibility to stop me? Nope. Consenting adults, and all that. ~Ethan~
Am 30.09.2012 20:18, schrieb Gregory P. Smith:
I like your basic approach but I'm suggesting a slightly different approach. Before I explain my proposal I like to get a naming convention straight. integrated tz database ---------------------- A copy of the Olson database that is shipped with every version of Python (not just Windows). The integrated database is updated with every feature release of Python. system tz database ------------------ That's an interface to the operating system's or platform's tz database. We probably have to provide multiple backends for different OSes, Java etc. update tz database ------------------ A PyPI package that contains a current version of the Olson database. A new version of the update tz should be provided by the Python core developer team every time a new Olson database is available. The updatetz must never be distributed with the Python source code and shall not be installed by a distributor. Optionally we can include the code that creates an update tz package from a Olson database. By default Python tries to load the updatetz first, then systemtz (if available) and finally the integratedtz. A user can query the status and version of the databases, set and get the currently used database with three functions. The used database can also be set with an environment variable: datetime.gettzdatabase() -> "integrated" or "system" or "update" datetime.settzdatabase(name) datetime.listtzdatabases() -> {"integrated": "version", "system": "???" "update": "version", # only if an update tz db is installed } PYTHON_TZDB = "integrated" or "system" or "update" With this setup users get the full benefit of system updates but can use the integrated or update database if they don't like the operating system's data. Christian
On Mon, Oct 1, 2012 at 9:02 AM, Lennart Regebro <regebro@gmail.com> wrote:
I think that would be a little too pure to be practical. There are many practical usages of tz data that would work fine with a year-old timezone database. Personally, I'd want to not ship any tzdata with non-Windows Python packages on the assumption that they have good up-to-date OS tzdata (or it should be easy to disable it in the configure phase). Also, I wonder if it would be possible to select the copy of the data that is the most recent (e.g. on Unix, pick the OS version if tzupdate is installed but older than the OS version). Cheers, Dirkjan
On Mon, Oct 1, 2012 at 10:28 AM, Dirkjan Ochtman <dirkjan@ochtman.nl> wrote:
A year is no age for a Python installation. A customer of mine has one website developed in 2003, still running on the same server. It runs Python 2.3, I don't remember which version, but I'd be surprised if it is 2.3.7 from 2008.
I don't think so. As far as I can tell, the data files contain no version information (only information on the version of the file format, which currently is 2). It also contains no information on what the name of the timezone is. This lack of information is unfortunate, but we'll have to live with it. The format hasn't changed since 1989, it is unlikely that we'll get anyone to change it anytime soon. //Lennart
On Mon, Oct 1, 2012 at 10:47 AM, Lennart Regebro <regebro@gmail.com> wrote:
Right. If they don't keep their Python up-to-date, why would they bother with their tzupdate? My point is that there is not much of a difference in the incentive for upgrading your timezone data whether an initial version of it came with Python or not. Having to manually install it might make you slightly more aware that it helps if you upgrade it once in a while, but it seems more likely to be a fire and forget type of operation, in which case it's basically the same as shipping a version of the timezone data with Python (which is much easier, of course). To put it crudely, you seem to think that most developers keep careful track of what packages they need for their apps and actively assess the risk for upgrading each of the packages involved. On the other hand, I would assume more developers just get something working and then fix any bugs that come up. Cheers, Dirkjan
On Mon, Oct 1, 2012 at 11:16 AM, Dirkjan Ochtman <dirkjan@ochtman.nl> wrote:
I'm sorry, is there a new releases of Python 2.3 made last year I don't know about? My point is that there is not much of a difference in the incentive
for upgrading your timezone data whether an initial version of it came with Python or not.
Incentive, no. But there is a difference in awareness and likelyhood.
Well, me at least would include that package in the buildout or the instructions, etc. It is therefore much more likely to be updated if it is not included with Python. To put it crudely, you seem to think that most developers keep careful
No, I assume there are developers of both types, and in between. If somebody just installed pytzdata and then doesn't upgrade it, fine, that's his problem. He doesn't become *more* likely to upgrade it because it's included in the standard library. But many developers are more likely to keep it updated and upgrade it if it is *not* included, at least once in a while. For example, if it's included in a buildout it could get updated when the buildout is re-run because some of the custom modules have been updated. While if it's included, it will never end up in the buildout and never get updated. //Lennart
On Mon, 1 Oct 2012 12:11:41 +0200 Lennart Regebro <regebro@gmail.com> wrote:
Python 2.3 has been EOL'ed for years. It definitely is not up-to-date, for any reasonable definition of the term. For example, it will have many unplugged security holes. So will that user's version of OpenSSL and other libraries. If they don't want to apply security fixes, why should we even care about their timezones' freshness? Regards Antoine. -- Software development and contracting: http://pro.pitrou.net
On Mon, Oct 1, 2012 at 5:23 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
Exactly. If a service provider has a bug in their application because they're using an old timezone database, that is their problem, so long as *we* ensure there is a clear and obvious mechanism for upgrading *just* the TZ database, without upgrading Python itself. There's nothing forcing people to keep their installed version of pytz up to date, either. If a timezone database is bundled into the standard library, there are 3 clear mechanisms for encouraging the use of fresh TZ data: 1. Consider TZ database updates to be bug fixes, and thus include them in maintenance releases. This will keep the provided version reasonably fresh for Python versions that are still in maintenance mode. 2. Provide a mechanism to prefer the database from PyPI. 3. Provide a mechanism to prefer the OS database for platforms that provide an Olson compatible interface (I briefly looked into that for Windows a while back - it doesn't seem like a practical idea, since Microsoft went off and did their own thing. It works for Linux and other platforms that use the Olson database natively, though) Since explicit is better than implicit, I *wouldn't* want to see magical side affects where merely installing the database from PyPI, or switching from Windows to Linux caused different behaviour. However, it should be very easy for an application or environment to *explicitly request* the use of the pytz database or the OS database in preference to the bundled database. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Mon, Oct 1, 2012 at 3:22 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I think that would be a pain. What you're proposing means that Linux installations have to use the Python-installed copy by default (because you want them to do the same thing as on Windows), even though they have a perfectly good copy, often newer, of the database installed on the system. Cheers, Dirkjan
On Mon, Oct 1, 2012 at 3:22 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I proposed 2 and 3, and I don't really see much magical side-effects with those. As mentioned we can also include a database in the standardlib, but since that will almost always be out of date, I don't really see the point. It is of course only an issue on Windows, but still. //Lennart
Re: to bundle or not to bundle What about having an included database that issues a (silence-able) warning any time it is used/imported/etc.? Have it say something to the effect of "Warning, the Olson database included with Python is likely to be outdated, please see <webpage> for information" where <webpage> (which may just be the doc page for the module) spells out why it's outdated, why it's not possible for it to be kept up to date, that this version may still work for you depending on application and how to silence the warning, and how to get the latest version via pypi or otherwise. As far as preference of database, I would think the best route would be to provide the ability to set the order you want to look in, with the default being: 1) user specified source (usually not one of the below) 2) updated tzdb from pypi 3) OS's tzdb 4) included tzdb (with warning) My $0.02USD, for what they're worth :)
On Mon, Oct 1, 2012 at 3:57 PM, Zachary Ware <zachary.ware+pydev@gmail.com>wrote:
Actually, that's not a bad idea. My original idea was to warn if it *was* outdated, but since there is no way to check that, I scratched that idea. But as I have pointed out several times, a database that is shipped with Python is almost guaranteed to be outdated, so yeah, we could just warn *all the time*. :-) I like this idea. It gives an incentive to update: Get rid of the annoying warning. It also will be silent on Unix, as we'll use the OS database, so this will only happen on Windows or if you embed Python or similar. //Lennart
On Mon, 1 Oct 2012 16:06:18 +0200 Lennart Regebro <regebro@gmail.com> wrote:
Well, no, it is just silly. If we ship a database, that's because we think it is good enough. A warning is just a nuisance here. We don't display warnings when the installed Python version is too old. Regards Antoine. -- Software development and contracting: http://pro.pitrou.net
On Mon, Oct 1, 2012 at 9:12 AM, Antoine Pitrou <solipsis@pitrou.net> wrote:
My thought was that it's better to have *something* always available, that has a decent chance of being "good enough" in a lot of cases (and if it's good enough for you, just silence the warning), than to noisily fail because we can't provide a perfect solution due to political idiocy. Or worse, to *silently* be wrong because someone assumed we had provided a perfect solution without looking too hard. I had no idea the Olson database was updated so often until Dirkjan posted about there being 21 updates in 2009 alone. For most of my uses, I would probably be a warning-silencer. And that wouldn't bother me; I would actually appreciate being reminded now and then that things may have changed since the last time I did something with timezones, and that I need to be careful of such changes. But, of course, that's just me, and it was my idea anyway ;)
On Mon, 1 Oct 2012 09:52:09 -0500 Zachary Ware <zachary.ware+pydev@gmail.com> wrote:
We can, and should, mention potential pitfalls in the documentation. But I don't think a warning is warranted, anymore than for other known issues (there are many of them at http://bugs.python.org/ :-)). Regards Antoine. -- Software development and contracting: http://pro.pitrou.net
On Oct 1, 2012 10:06 AM, "Antoine Pitrou" <solipsis@pitrou.net> wrote:
With large red text and blink tags :-P
But I don't think a warning is warranted, anymore than for other known issues (there are many of them at http://bugs.python.org/ :-)).
Just curious (and a bit off topic), what exactly does warrant a warning in Python? I've been messing around with it for a couple years (since shortly before 3.1, and always on 3.x) and I'm pretty sure I have yet to see a warning for anything. Which I suppose could be counted as a good thing...
On Mon, 1 Oct 2012 10:16:16 -0500 Zachary Ware <zachary.ware@gmail.com> wrote:
We don't really have a well-defined policy, except for one warning category: DeprecationWarnings, whose purpose are quite clear. But they are now silenced by default :-) Generally, warnings are not easy to deal with for the end-user, so we are reluctant to add any. We'd rather document annoyances and pitfalls; and, moreover, we try to design our APIs so that they don't have unexpected effects. Regards Antoine. -- Software development and contracting: http://pro.pitrou.net
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/01/2012 10:12 AM, Antoine Pitrou wrote:
Using a bundled version would be the Wrong Thing (TM) for almost any non-toy application (one where non-programmers actually care about the timezones in which dates / times are entered / displayed): the fact that the errors would infrequent / subtle / hard to diagnose is an argument *against* inclusion. - -1 for including a copy in Python at all: if your app needs it, and the OS on which you deploy doesn't provide it, you have a straightforward way of installing it. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iEUEARECAAYFAlBp36IACgkQ+gerLs4ltQ4mbACY60cQ1SbUgRh0hNtUmxScRM68 ZwCg125QzeMmhFV0VYYZGmNVmEqpajo= =ehg8 -----END PGP SIGNATURE-----
Reminder to everyone: the current state of the art for getting up to date tz info for Python is "pip install pytz". If any proposal is more complicated than that, there's absolutely no point in changing anything. The version I liked best so far is for Python to just install a copy of pytz automatically (shipping it in the installer rather than downloading it). OS packagers would then take it out (replacing it with a dependency on a pytz emulator that used the system database instead). Cheers, Nick.
On Tue, 02 Oct 2012 00:18:10 +0530, Nick Coghlan <ncoghlan@gmail.com> wrote:
Emulator? That makes no sense, I'm afraid. I think we are talking here about incorporating pytz into the stdlib. The only question is how to manage the Olson database on Windows, which has *always* been the question. --David
It occurred to me this morning that Python already ships a set of timezone data with the Windows installer: Tcl/Tk's. Is there any way we could use that as the default on Windows?
On Thu, Oct 4, 2012 at 12:01 PM, Zachary Ware <zachary.ware+pydev@gmail.com> wrote:
I would say no. You could choose not to include Tcl/Tk in your installation, or you could build your own Python and not include those sources. In either case, the lack of a GUI toolkit is now affecting your use of timezone data - weird/bad. Plus we only pull updated Tcl/Tk every few years.
On Mon, Oct 1, 2012 at 12:02 PM, Barry Warsaw <barry@python.org> wrote:
I think those are all suggestions included in Nick's intention. The main points are that (a) pytz and some not-too-ancient Olson database are included by default, (b) vendor distributions can tweak the packaging so that the platform's copy of the Olson database is used automatically; (c) users not benefitting from such vendor support can update the database as easily as "pip install pytz" (though not necessarily with that specific command). -- --Guido van Rossum (python.org/~guido)
On Mon, Oct 1, 2012 at 8:48 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
But that would then mean that when you install from source, as I typically do to avoid depending on Python, it would use another database than the OS version. I don't want that, i want Python on Unix to use the OS supplied database, unless another one is explicitly installed/configured. //Lennart
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/01/2012 02:48 PM, Nick Coghlan wrote:
Lennart's original proposal was to land the machinery from the current pytz package into the stdlib, leaving the data part as a separately-installable package. Lennart also proposed some kind of (maybe configurable) policy for looking up the Olson-based timezonedb. I'm fine with Lennart's proposal, but want to argue against including any copy of the Olson db in Python itself: I don't believe the set of folks who need it but can't do the equivalent of 'pip / easy_install <lennarts-new-name-for-the-data-part>' is big enough to outweigh the downside of bundling immediately-stale data, and hoping that everybody else remembers to configure it out. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iEYEARECAAYFAlBp6OkACgkQ+gerLs4ltQ4fZwCfUtz9lnkjxM6rdTaO0io1yzIP f2cAn0YfQt+n09Znuh6BZ/Culixpf7eE =ecDk -----END PGP SIGNATURE-----
Gah, ignore half of that last post. I think Lennart is talking about incorporating the missing pytz *functionality* into the stdlib. --David
On 2012-10-01, at 17:32 , Terry Reedy wrote:
There are several: there's a message on a dedicated mailing list and there are HTTP, FTP and rsync repositories with both all releases and a "latest" archive for both tzdata and tzcode. The HTTP repositories seems to handle time-based conditional requests correctly (an If-Modified-Since with the date of the latest release or later will result in a 304 response, earlier will result in a 200) The HTTP URIs are https://www.iana.org/time-zones/repository/tzdata-latest.tar.gz and https://www.iana.org/time-zones/repository/tzcode-latest.tar.gz For some reason, IANA does not seem to publish a feed.
On Mon, Oct 1, 2012 at 5:32 PM, Terry Reedy <tjreedy@udel.edu> wrote:
Sure, but there is no way to get a last updated time from the database that's installed. And making an HTTP request when you call localize() to check if the database is outdated or not doesn't seem to be a great idea. //Lennart
On Oct 01, 2012, at 03:34 PM, Lennart Regebro wrote:
I agree. I don't think the Python RM should have to worry about tz updates, given how frequent or unpredictable they can be. For OSes that provide the database, I can't think of any reason not to prefer that, except if your OS version is no longer being maintained, and then it seems like updating your tz database is the least of your worries. However, if someone wants to maintain a Cheeseshop package with updates, I can't stop them. My biggest concern there is that eventually the maintainers will lose interest and this package will bitrot, and then we'll have obsolete tz info out there that people will still rely on. Oh well, I guess. I completely agree that just installing the Cheeseshop tz package should *not* be enough to prefer it over the system tz data. Cheers, -Barry
On Oct 01, 2012, at 05:15 PM, Lennart Regebro wrote:
Yes. Using the script I mentioned in an different response, if someone installed the database to some location (TBD), then there would probably be a config file sitting next to it. A simple .ini file with an 'enable' flag would be needed to turn on the override. Cheers, -Barry
On Mon, Oct 1, 2012 at 6:21 PM, Larry Hastings <larry@hastings.org> wrote:
The most current one is likely to be the one provided by the operating system, which does not contain any .ini file, nor, as far as I can tell, any information about the database version or any timestamps. //Lennart
On 10/1/2012 12:39 PM, Lennart Regebro wrote:
But Windows does not provide one, or at least, the proposal seems to be not use whatever it does have. I think your PEP should propose one api but conditional tz db access code for systems with and without the tz db already provided. -- Terry Jan Reedy
On Mon, 1 Oct 2012 11:11:46 -0400 Barry Warsaw <barry@python.org> wrote:
I agree. I don't think the Python RM should have to worry about tz updates, given how frequent or unpredictable they can be.
I don't understand what makes them specifically "unpredictable". We statically link OpenSSL and other libraries in the Windows builds. I don't think they have very predictable updates either (especially OpenSSL, actually). Regards Antoine. -- Software development and contracting: http://pro.pitrou.net
On Sun, Sep 30, 2012 at 11:15 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
An out-of-date timezone database will bite someone at some point. They'll be running code that expects some locality to be on/off DST when it isn't, and some customer will be confused or annoyed. I suspect most people won't much care, though. It's only a few who'd explicitly upgrade their tz database. ChrisA
On 30.09.12 16:15, Antoine Pitrou wrote:
In 2011 Ukrainian timezone data was changed twice for year. And perhaps even change in 2013. Russian timezones were changed over the last few years and most likely will change in the near future. Correct time is critical for many applications.
On Sun, 30 Sep 2012 21:55:34 +0300 Serhiy Storchaka <storchaka@gmail.com> wrote:
Perhaps, but that's the responsibility of governements. Just because some governments have erratic policies shouldn't be a reason for residents of other countries not to enjoy the benefits of their stable timezones. Regards Antoine. -- Software development and contracting: http://pro.pitrou.net
On Sun, 30 Sep 2012 22:35:54 +0300 Serhiy Storchaka <storchaka@gmail.com> wrote:
Well, no, this isn't similar. Choosing one's timezone policies is a contemporary political decision, while choosing a language and its alphabet is not really a decision people ever make (it's just an aspect of a society's long-term evolution) - except Atatürk, perhaps :-) Furthermore, the proposal I'm making does *not* disadvantage residents of Russia and Ukraine: whether our Windows installer provides a database or not, they have to download a new database if they want up-to-date information. And they have to download it afresh every few months, if I'm following you. Regards Antoine. -- Software development and contracting: http://pro.pitrou.net
On 30.09.12 22:51, Antoine Pitrou wrote:
Oh, no. Choosing of alphabet (and sometimes language) is also a contemporary political decision. For the last 25 years new letter Ґ has been added to the Ukrainian alphabet, and the letter Ь changed its place in the alphabet. There were at least 4 family of absolutely different character sets for Ukrainian (not counting the Unicode), some of them contains several incompatible variants. In several neighboring countries the alphabet was changed completely (from Cyrillic-based to Latin). Why ASCII is not enough for all?
Who will update the database? The developer which distributes the application with embedded Python can forget about the tz updates, as well as about non-ascii encodings. Native Unicode support in Python makes the second error less likely. Why not use the system data which are updated by the OS? I know that Windows also changes the clock for local DST.
On Sun, Sep 30, 2012 at 11:28 PM, Serhiy Storchaka <storchaka@gmail.com>wrote:
Why not use the system data which are updated by the OS? I know that Windows also changes the clock for local DST.
The Windows timezone information does not include any historical information, as it's designed primarily to keep your computers clock correctly, not to convert date times between different timezones. As a result using to to convert data in the past can give incorrect information. We therefore want to consistently use the tz database. //Lennart
On Mon, 01 Oct 2012 00:28:41 +0300 Serhiy Storchaka <storchaka@gmail.com> wrote:
Well, yeah, but it's not like you can do it on a whim either, and you can't change the large body of existing text.
If that's possible, then it sounds the ideal solution indeed. Regards Antoine. -- Software development and contracting: http://pro.pitrou.net
On 2012-09-30 19:55, Serhiy Storchaka wrote:
That's what UTC is for! :-) I think that it would be a good idea to provide a database with the release plus a tool for updating it, the updates being announced by email or RSS, and the ability to use the system's database if there's one.
When people use pytz they have to reinstall pytz too if thet want to
benefit from the updates. (Or depend on automated updates via some vendor
On Sep 30, 2012 10:34 PM, "Guido van Rossum" <guido@python.org> wrote: package management system.) If we can ensure that with pytz in the stdlib, updates to the Olson database can be installed just as easily as before (probably through some new custom mechanism, as a new Python bugfix release is too heavy-handed), I see no problem with adopting pytz into the stdlib. The mechanism would be exactly the same, but a different modulename. I´ll go forward and make a PEP on this soonish.
On Sep 30, 2012, at 02:47 PM, Lennart Regebro wrote:
Why include the database in Python at all? If you have an OS that keeps the system tz data up-to-date, I can't think of any reason why you wouldn't want to use it. If you don't have the data, why not include information in the documentation for how to download and install the database in a location that Python will search for, along with information on how to enable that? You could even provide a nice script that would download, install, and optionally enable that tz data's use. I think that would cover all the bases: * My OS keeps the tz data up-to-date. I don't have to do nuthin', and Python gives me a nice API for using all the world's timezones on my superb OS. * My OS keeps the tz data up-to-date, but I'm skeptical. I run the update script whenever is appropriate, adding the --enable flag, and the tz data is grabbed from the intarwebs, installed, and Python starts using it instead of the system data. * I am sad because I use an OS that has no tz data. I run the update script once in a while, adding the --enable flag, and my Python is timezonally happy. * I'm sad and lazy. Oh well, Python throws an exception when I try to use a timezone that isn't UTC. Cheers, -Barry
On Mon, Oct 1, 2012 at 5:25 PM, Barry Warsaw <barry@python.org> wrote:
Exactly my point.
Right, I was going to do that by bundling the files in a package, tentatively called pytzdata, available on PyPI, so it can be easy_installed/pip installed, etc.
We seem to be on the same page here. //Lennart
On Mon, 1 Oct 2012 11:25:08 -0400 Barry Warsaw <barry@python.org> wrote:
Well, for one, that doesn't really fit in the "batteries included" approach. Having to download separate data files is an annoyance.
This doesn't solve the update problem at all, since that tz data would still be outdated after a couple of months. In other words, it has no benefit for the user, but is more complicated to use. Besides, that script would have to care about security issues. For example you'd use a HTTPS-enabled source, and then bundle the required CA certificates with Python. But wait, now Python has to ship some data that will possibly become outdated in a year or two :-) Regards Antoine. -- Software development and contracting: http://pro.pitrou.net
participants (23)
-
Antoine Pitrou -
Barry Warsaw -
Benjamin Peterson -
Brian Curtin -
Chris Angelico -
Christian Heimes -
Dirkjan Ochtman -
Ethan Furman -
Gregory P. Smith -
Guido van Rossum -
Larry Hastings -
Lennart Regebro -
Mark Lawrence -
Matthias Klose -
MRAB -
Nick Coghlan -
R. David Murray -
Serhiy Storchaka -
Terry Reedy -
Tres Seaver -
Xavier Morel -
Zachary Ware -
Zachary Ware