Mailman 3 February 2013 - Python-Dev

Re: [Python-Dev] cpython (2.7): - Issue #17086: Backport thepatches from the 3.3 branch to cross-build
by a.cavallo＠cavallinux.eu Feb. 7, 2013

Feb. 7, 2013

> (btw, nothing to do with this discussion, Nick, but it appears your > RHEL buildbot is offline) > > Regards Btw rpm build and packaging can be done on the suse build service: it is a very complete (and free) service. https://build.opensuse.org

1 0

Re: [Python-Dev] cpython (2.7): - Issue #17086: Backport the patches from the 3.3 branch to cross-build
by Antoine Pitrou Feb. 7, 2013

Feb. 7, 2013

On Thu, 31 Jan 2013 23:52:27 +0100 (CET) matthias.klose <python-checkins(a)python.org> wrote: > http://hg.python.org/cpython/rev/8ee6d96a1019 > changeset: 81859:8ee6d96a1019 > branch: 2.7 > parent: 81855:df9f8feb7444 > user: doko(a)python.org > date: Thu Jan 31 23:52:03 2013 +0100 > summary: > - Issue #17086: Backport the patches from the 3.3 branch to cross-build > the package. You aren't supposed to add new features to bugfix branches. Did you have a specific reason to do this? Regards Antoine.

7 13

PEP-0431 update 4
by Lennart Regebro Feb. 7, 2013

Feb. 7, 2013

Here is the update of PEP-0431. The major changes are that the new class now has a lowercase name, to conform to the rest of the datetime modeule: dsttimezone (i decided against dst_timezone, as non of the other classes have underscores). Also the timezone name sets are gone. Thanks to Nick Coghlan for finding a whole bunch of minor mistakes. //Lennart ---- PEP: 431 Title: Time zone support improvements Version: $Revision$ Last-Modified: $Date$ Author: Lennart Regebro <regebro(a)gmail.com> BDFL-Delegate: Barry Warsaw <barry(a)python.org> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 11-Dec-2012 Post-History: 11-Dec-2012, 28-Dec-2012, 28-Jan-2013 Abstract ======== This PEP proposes the implementation of concrete time zone support in the Python standard library, and also improvements to the time zone API to deal with ambiguous time specifications during DST changes. Proposal ======== Concrete time zone support -------------------------- The time zone support in Python has no concrete implementation in the standard library outside of a tzinfo baseclass that supports fixed offsets. To properly support time zones you need to include a database over all time zones, both current and historical, including daylight saving changes. But such information changes frequently, so even if we include the last information in a Python release, that information would be outdated just a few months later. Time zone support has therefore only been available through two third-party modules, ``pytz`` and ``dateutil``, both who include and wrap the "zoneinfo" database. This database, also called "tz" or "The Olsen database", is the de-facto standard time zone database over time zones, and it is included in most Unix and Unix-like operating systems, including OS X. This gives us the opportunity to include the code that supports the zoneinfo data in the standard library, but by default use the operating system's copy of the data, which typically will be kept updated by the updating mechanism of the operating system or distribution. For those who have an operating system that does not include the zoneinfo database, for example Windows, the Python source distribution will include a copy of the zoneinfo database, and a distribution containing the latest zoneinfo database will also be available at the Python Package Index, so it can be easily installed with the Python packaging tools such as ``easy_install`` or ``pip``. This could also be done on Unices that are no longer receiving updates and therefore have an outdated database. With such a mechanism Python would have full time zone support in the standard library on any platform, and a simple package installation would provide an updated time zone database on those platforms where the zoneinfo database isn't included, such as Windows, or on platforms where OS updates are no longer provided. The time zone support will be implemented by making the ``datetime`` module into a package, and adding time zone support to ``datetime`` based on Stuart Bishop's ``pytz`` module. Getting the local time zone --------------------------- On Unix there is no standard way of finding the name of the time zone that is being used. All the information that is available is the time zone abbreviations, such as ``EST`` and ``PDT``, but many of those abbreviations are ambiguous and therefore you can't rely on them to figure out which time zone you are located in. There is however a standard for finding the compiled time zone information since it's located in ``/etc/localtime``. Therefore it is possible to create a local time zone object with the correct time zone information even though you don't know the name of the time zone. A function in ``datetime`` should be provided to return the local time zone. The support for this will be made by integrating Lennart Regebro's ``tzlocal`` module into the new ``datetime`` module. For Windows it will look up the local Windows time zone name, and use a mapping between Windows time zone names and zoneinfo time zone names provided by the Unicode consortium to convert that to a zoneinfo time zone. The mapping should be updated before each major or bugfix release, scripts for doing so will be provided in the ``Tools/`` directory. Ambiguous times --------------- When changing over from daylight savings time (DST) the clock is turned back one hour. This means that the times during that hour happens twice, once with DST and then once without DST. Similarly, when changing to daylight savings time, one hour goes missing. The current time zone API can not differentiate between the two ambiguous times during a change from DST. For example, in Stockholm the time of 2012-11-28 02:00:00 happens twice, both at UTC 2012-11-28 00:00:00 and also at 2012-11-28 01:00:00. The current time zone API can not disambiguate this and therefore it's unclear which time should be returned:: # This could be either 00:00 or 01:00 UTC: >>> dt = datetime(2012, 10, 28, 2, 0, tzinfo=zoneinfo('Europe/Stockholm')) # But we can not specify which: >>> dt.astimezone(zoneinfo('UTC')) datetime.datetime(2012, 10, 28, 1, 0, tzinfo=<UTC>) ``pytz`` solved this problem by adding ``is_dst`` parameters to several methods of the tzinfo objects to make it possible to disambiguate times when this is desired. This PEP proposes to add these ``is_dst`` parameters to the relevant methods of the ``datetime`` API, and therefore add this functionality directly to ``datetime``. This is likely the hardest part of this PEP as this involves updating the C version of the ``datetime`` library with this functionality, as this involved writing new code, and not just reorganizing existing external libraries. Implementation API ================== The zoneinfo database --------------------- The latest version of the zoneinfo database should exist in the ``Lib/tzdata`` directory of the Python source control system. This copy of the database should be updated before every Python feature and bug-fix release, but not for releases of Python versions that are in security-fix-only-mode. Scripts to update the database will be provided in ``Tools/``, and the release instructions will be updated to include this update. New configure options ``--enable-internal-timezone-database`` and ``--disable-internal-timezone-database`` will be implemented to enable and disable the installation of this database when installing from source. A source install will default to installing them. Binary installers for systems that have a system-provided zoneinfo database may skip installing the included database since it would never be used for these platforms. For other platforms, for example Windows, binary installers must install the included database. Changes in the ``datetime``-module ---------------------------------- The public API of the new time zone support contains one new class, one new function, one new exception and four new collections. In addition to this, several methods on the datetime object gets a new ``is_dst`` parameter. New class ``dsttimezone`` ^^^^^^^^^^^^^^^^^^^^^^^^ This class provides a concrete implementation of the ``tzinfo`` base class that implements DST support. New function ``zoneinfo(name=None, db_path=None)`` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This function takes a name string that must be a string specifying a valid zoneinfo time zone, i.e. "US/Eastern", "Europe/Warsaw" or "Etc/GMT". If not given, the local time zone will be looked up. If an invalid zone name is given, or the local time zone can not be retrieved, the function raises `UnknownTimeZoneError`. The function also takes an optional path to the location of the zoneinfo database which should be used. If not specified, the function will look for databases in the following order: 1. Check if the `tzdata-update` module is installed, and then use that database. 2. Use the database in ``/usr/share/zoneinfo``, if it exists. 3. Use the Python-provided database in ``Lib/tzdata``. If no database is found an ``UnknownTimeZoneError`` or subclass thereof will be raised with a message explaining that no zoneinfo database can be found, but that you can install one with the ``tzdata-update`` package. New parameter ``is_dst`` ^^^^^^^^^^^^^^^^^^^^^^^^ A new ``is_dst`` parameter is added to several methods to handle time ambiguity during DST changeovers. * ``tzinfo.utcoffset(dt, is_dst=False)`` * ``tzinfo.dst(dt, is_dst=False)`` * ``tzinfo.tzname(dt, is_dst=False)`` * ``datetime.astimezone(tz, is_dst=False)`` The ``is_dst`` parameter can be ``False`` (default), ``True``, or ``None``. ``False`` will specify that the given datetime should be interpreted as not happening during daylight savings time, i.e. that the time specified is after the change from DST. This is default to preserve existing behavior. ``True`` will specify that the given datetime should be interpreted as happening during daylight savings time, i.e. that the time specified is before the change from DST. ``None`` will raise an ``AmbiguousTimeError`` exception if the time specified was during a DST change over. It will also raise a ``NonExistentTimeError`` if a time is specified during the "missing time" in a change to DST. New exceptions ^^^^^^^^^^^^^^ * ``UnknownTimeZoneError`` This exception is a subclass of KeyError and raised when giving a time zone specification that can't be found:: >>> datetime.zoneinfo('Europe/New_York') Traceback (most recent call last): ... UnknownTimeZoneError: There is no time zone called 'Europe/New_York' * ``InvalidTimeError`` This exception serves as a base for ``AmbiguousTimeError`` and ``NonExistentTimeError``, to enable you to trap these two separately. It will subclass from ValueError, so that you can catch these errors together with inputs like the 29th of February 2011. * ``AmbiguousTimeError`` This exception is raised when giving a datetime specification that is ambiguous while setting ``is_dst`` to None:: >>> datetime(2012, 11, 28, 2, 0, tzinfo=zoneinfo('Europe/Stockholm'), is_dst=None) >>> Traceback (most recent call last): ... AmbiguousTimeError: 2012-10-28 02:00:00 is ambiguous in time zone Europe/Stockholm * ``NonExistentTimeError`` This exception is raised when giving a datetime specification for a time that due to daylight saving does not exist, while setting ``is_dst`` to None:: >>> datetime(2012, 3, 25, 2, 0, tzinfo=zoneinfo('Europe/Stockholm'), is_dst=None) >>> Traceback (most recent call last): ... NonExistentTimeError: 2012-03-25 02:00:00 does not exist in time zone Europe/Stockholm New collections ^^^^^^^^^^^^^^^ * ``all_timezones`` is the exhaustive list of the time zone names that can be used, listed alphabethically. * ``common_timezones`` is a list of useful, current time zones, listed alphabethically. The ``tzdata-update``-package ----------------------------- The zoneinfo database will be packaged for easy installation with ``easy_install``/``pip``/``buildout``. This package will not install any Python code, and will not contain any Python code except that which is needed for installation. It will be kept updated with the same tools as the internal database, but released whenever the ``zoneinfo``-database is updated, and use the same version schema. Differences from the ``pytz`` API ================================= * ``pytz`` has the functions ``localize()`` and ``normalize()`` to work around that ``tzinfo`` doesn't have is_dst. When ``is_dst`` is implemented directly in ``datetime.tzinfo`` they are no longer needed. * The ``timezone()`` function is called ``zoneinfo()`` to avoid clashing with the ``timezone`` class introduced in Python 3.2. * ``zoneinfo()`` will return the local time zone if called without arguments. * The class ``pytz.StaticTzInfo`` is there to provide the ``is_dst`` support for static time zones. When ``is_dst`` support is included in ``datetime.tzinfo`` it is no longer needed. * ``InvalidTimeError`` subclasses from ``ValueError``. Resources ========= * http://pytz.sourceforge.net/ * http://pypi.python.org/pypi/tzlocal * http://pypi.python.org/pypi/python-dateutil * http://unicode.org/cldr/data/common/supplemental/windowsZones.xml Copyright ========= This document has been placed in the public domain.

1 0

Moderating the severity of the distutils freeze
by Nick Coghlan Feb. 7, 2013

Feb. 7, 2013

On Thu, Feb 7, 2013 at 6:45 AM, Éric Araujo <merwok(a)netwok.org> wrote: > Bug fixes are possible; changes to keep working with Python (i.e. pyc > files are generated in pycache directories after 3.2); changes to be > able to build on current OSes (e.g. following Mac dev tools location > change, introduction of Debian multiarch, etc.). > > Some bugs have been here for so long that everybody depends on them or > work around them, or they would be extremely painful to fix (e.g. in the > option parsing code) for little benefit, so they are wontfix. > > Cleanups, refactorings and improvements were banned by the feature freeze. I think the points to be clarified are that *new* internal APIs (including new commands, like bdist_wheel) are legitimate in a feature release, and internal changes which don't affect APIs (like some of the tweaks Dave Malcolm and I made to try to make building Python 3 RPMs on RHEL 6 with bdist_rpm less broken) are legitimate in maintenance releases. distutils is going to be around for a long time for backwards compatibility reasons, so if people want to improve it within those constraints they should be allowed to - we just need to be extra careful with maintenance releases, by treating even nominally internal APIs as if they were public. The current interpretation of the situation as a complete "no few features" freeze is overkill, when the setuptools/distribute developers will have plenty of time to send up warning flares during the alpha/beta/rc cycle. An appropriate rule would be more like "no major refactorings of distutils internals" (which may indeed mean distutils never supports metadata versions past 1.1). Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

1 0

importlib hooray!
by Thomas Heller Feb. 6, 2013

Feb. 6, 2013

I have become a fan of the new python 3.3 importlib in the last few days. It has allowed me to write a ModuleMapper which I put into sys.metapath (in sitecustomize.py, for Python 3.3). This mapper currently does rename modules like 'Queue' or '_winreg' to the Python3 modules 'queue' or 'winreg' without the need to change my 2.7 sourcecode (changing the sourcecode is required when using six.moves to write 2.x/3.x compatible code). The six.moves approach has another problem with freeze tools (py2exe for example): ModuleFinder does not find the moved modules because it cannot track __import__(). I have also started a new modulefinder which uses importlib to find modules; this was quite easy since I could copy a lot of code from importlib._bootstrap. The great thing is that this new modulefinder is much better than the old one: It finds modules in zipped eggs or other zip-archives which are on sys.path; also it finds the renamed modules correctly that my ModuleMapper has mapped. The only thing that I am missing is that it is a bit of guesswork to find out the type of the module that importlib.find_loader() has found. Previously imp.find_module returned a tuple containing the type of module found (like imp.PY_SOURCE), I have to poke around in some attributes of the result of importlib.find_loader() to guess the type. Thomas

2 1

update on 2.7.4
by Benjamin Peterson Feb. 4, 2013

Feb. 4, 2013

As you may have noticed, no 2.7.4 rc has been created yet. Yesterday, the buildbots were all red, and release blocker issues had to be dealt with. Today, I was not as availabIe and people were fixing important-looking crashers. In general, there seems to have been a lot more last-minute scrambling around than usual for a bugfix release. I'm afraid I'm still going to have to delay longer to see if we can get a few security patches in. It could be next week. Benjamin

4 4

Interested in GSoC 2013
by Jainit Purohit Feb. 4, 2013

Feb. 4, 2013

Hi, I'm Jainit and I'm planning to apply for GSoC 2013 for the PSF. I was also part of GSoC 2012 in interface ecology lab, Texas A&M university. I just gone though Python developer's guide and how to become core contributor document. And I just compiled CPython on my machine. Since I'm new to this community, Can anyone assign me some task/issues/bug to work on to get the better idea of how things works. I would also like to know if any of you have any ideas which can be implemented this summer. Thanks!, Jainit

3 2

2.7.4
by Benjamin Peterson Feb. 3, 2013

Feb. 3, 2013

It's been almost a year since 2.7.3, so it's time for another 2.7 bugfix release. 2013-02-02 - 2.7.4 release branch created; rc released 2013-02-16 - 2.7.4 released Does this work for you, Martin and Ned? -- Regards, Benjamin

8 8

Re: [Python-Dev] [Python-checkins] cpython (3.3): Issue #17098: Be more stringent of setting __loader__ on early imported
by Brett Cannon Feb. 3, 2013

Feb. 3, 2013

FYI, Raymond let me know that this is causing the buildbots to occasionally fail thanks to xml.parsers.expat.errors somehow circumventing import. I'm going to try and figure out what's going on. On Fri, Feb 1, 2013 at 4:38 PM, brett.cannon <python-checkins(a)python.org>wrote: > http://hg.python.org/cpython/rev/19ea454ccdf7 > changeset: 81884:19ea454ccdf7 > branch: 3.3 > parent: 81882:4a4688b865ff > user: Brett Cannon <brett(a)python.org> > date: Fri Feb 01 15:31:49 2013 -0500 > summary: > Issue #17098: Be more stringent of setting __loader__ on early imported > modules. Also made test more rigorous. > > files: > Lib/importlib/_bootstrap.py | 7 +- > Lib/test/test_importlib/test_api.py | 6 + > Python/importlib.h | 591 ++++++++------- > 3 files changed, 310 insertions(+), 294 deletions(-) > > > diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py > --- a/Lib/importlib/_bootstrap.py > +++ b/Lib/importlib/_bootstrap.py > @@ -1704,10 +1704,13 @@ > BYTECODE_SUFFIXES = DEBUG_BYTECODE_SUFFIXES > > module_type = type(sys) > - for module in sys.modules.values(): > + for name, module in sys.modules.items(): > if isinstance(module, module_type): > if not hasattr(module, '__loader__'): > - module.__loader__ = BuiltinImporter > + if name in sys.builtin_module_names: > + module.__loader__ = BuiltinImporter > + elif _imp.is_frozen(name): > + module.__loader__ = FrozenImporter > > self_module = sys.modules[__name__] > for builtin_name in ('_io', '_warnings', 'builtins', 'marshal'): > diff --git a/Lib/test/test_importlib/test_api.py > b/Lib/test/test_importlib/test_api.py > --- a/Lib/test/test_importlib/test_api.py > +++ b/Lib/test/test_importlib/test_api.py > @@ -184,6 +184,12 @@ > if isinstance(module, types.ModuleType): > self.assertTrue(hasattr(module, '__loader__'), > '{!r} lacks a __loader__ attribute'.format(name)) > + if name in sys.builtin_module_names: > + self.assertEqual(importlib.machinery.BuiltinImporter, > + module.__loader__) > + elif imp.is_frozen(name): > + self.assertEqual(importlib.machinery.FrozenImporter, > + module.__loader__) > > def test_main(): > from test.support import run_unittest > diff --git a/Python/importlib.h b/Python/importlib.h > --- a/Python/importlib.h > +++ b/Python/importlib.h > [stripped] > > -- > Repository URL: http://hg.python.org/cpython > > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > http://mail.python.org/mailman/listinfo/python-checkins > >

1 0

Release or not release the GIL
by Victor Stinner Feb. 1, 2013

Feb. 1, 2013

Hi, While working on the implementation of the PEP 433, I saw different places where Python asks for operating resources without releasing the GIL. In my implementation, I released the GIL in the following places. Is it correct? - os.dup() - os.dup2() - os.pipe() - socket.socketpair() os.listdir() releases the GIL to call dup(), whereas os.dup() doesn't. Victor

8 10