Python-Dev June 2016

python-dev@python.org

109 participants
66 discussions

ValuesView abc: why doesn't it (officially) inherit from Iterable?

by Alan Franzoni

Hello, I hope not to bother anyone with a somewhat trivial question, I was unable to get an answer from other channels. I was just checking out some docs on ABCs for a project of mine, where I need to do some type-related work. Those are the official docs about the ValuesView type, in both Python 2 and 3: https://docs.python.org/2/library/collections.html#collections.ValuesView https://docs.python.org/3/library/collections.abc.html and this is the source (Python 2, but same happens in Python 3) https://hg.python.org/releases/2.7.11/file/9213c70c67d2/Lib/_abcoll.py#l479 I was very puzzled about the ValuesView interface, because from a logical standpoint it should inherit from Iterable, IMHO (it's even got the __iter__ Mixin method); on the contrary the docs say that it just inherits from MappingView, which inherits from Sized, which doesn't inherit from Iterable. So I fired up my 2.7 interpreter: >>> from collections import Iterable >>> d = {1:2, 3:4} >>> isinstance(d.viewvalues(), Iterable) True >>> It looks iterable, after all, because of Iterable's own subclasshook. But I don't understand why ValuesView isn't explicitly Iterable. Other ABCs, like Sequence, are explicitly inheriting Iterable. Is there some arcane reason behind that, or it's just a documentation+implementation shortcoming (with no real-world impact) for a little-used feature? Bye, -- www.franzoni.eu - Twitter: @alanfranz contact me at public(a)[mysurname].eu

4 years, 10 months

mod_python compilation error in VS 2008 for py2.7.1

by asimkon

I would like to ask you a technical question regarding python module compilation for python 2.7.1. I want to compile mod_python <https://app.box.com/s/orsffo3t4g6h9ftkq6p1>library for Apache 2.2 and py2.7 <https://www.python.org/downloads/> on Win32 in order to use it for psp - py scripts that i have written. I tried to compile it using VS 2008 (VC++) and unfortunately i get an error on pyconfig.h (Py2.7/include) error C2632: int followed by int is illegal. This problem occurs when i try to run the bat file that exists on mod_python/dist folder. Any idea or suggestion what should i do in order to run it on Win 7 Pro (win 32) environment and produce the final apache executable module (.so). For your better assistance, i attach you the necessary files and error_log (ouput that i get during compilation process). I have posted the same question here <http://stackoverflow.com/questions/37696936/vc-compilation-error-in-pyconfi…>, but unfortunately i had had no luck! Additionally i give you the compilation instructions that i follow (used also MinGW-w64 and get the same error) in order to produce the final output! Compiling Open a command prompt with VS2008 support. The easiest way to do this is to use "Start | All Programs | Microsoft Visual Studio 2008 | Visual Studio Tools | Visual Studio 2008 Command Prompt". (This puts the VS2008 binaries in the path and sets up the lib/include environmental variables for the Platform SDK.) 1.cd to the mod_python\dist folder. 2.Tell mod_python where Apache is: set APACHESRC=C:\Apache 3. Run build_installer.bat. If it succeeds, an installer.exe will be created in a subfolder. Run that install the module. Kind Regards Kostas Asimakopoulos

4 years, 10 months

PEP 468

by zreed＠fastmail.com

Is there any further thoughts on including this in 3.6? Similar to the recent discussion on OrderedDict namespaces for metaclasses, this would simplify / enable a number of type factory use cases where proper metaclasses are overkill. This feature would also be quite nice in say pandas where the (currently unspecified) field order used in the definition of frames is preserved in user-visible displays.

4 years, 10 months

Current Python 3.2 status?

by Chi Hsuan Yen

Hello all, Georg said in February that 3.2.7 is going to be released, and now it's June. Will it ever be released? pip [2], virtualenv [3] and setuptools [4] have all dropped Python 3.2 support, and there's no new commits since 2016/01/15 on CPython's 3.2 branch. I'd like to know CPython's attitude against Python 3.2. Is it still maintained? Or is it dead? [1] https://mail.python.org/pipermail/python-dev/2016-February/143300.html [2] https://github.com/pypa/pip/commit/b11cb019a47ff0cf3d8a37a0c89d8ae4cf25282f [3] https://github.com/pypa/virtualenv/commit/8132fa3a826ff1ba0c0c065563b9733c2… [4] https://github.com/pypa/setuptools/commit/ae6c73f07680da77345f5ccfac4facde3…

4 years, 10 months

Python 3.6.0a2 is now available

by Ned Deily

On behalf of the Python development community and the Python 3.6 release team, I'm happy to announce the availability of Python 3.6.0a2. 3.6.0a2 is the first of four planned alpha releases of Python 3.6, the next major release of Python. During the alpha phase, Python 3.6 remains under heavy development: additional features will be added and existing features may be modified or deleted. Please keep in mind that this is a preview release and its use is not recommended for production environments. You can find Python 3.6.0a2 here: https://www.python.org/downloads/release/python-360a2/ The next release of Python 3.6 will be 3.6.0a3, currently scheduled for 2016-07-11. Enjoy! --Ned -- Ned Deily nad(a)python.org -- []

4 years, 10 months

PEP 520: Ordered Class Definition Namespace (round 3)

by Eric Snow

I've updated the PEP to reflect feedback up to this point. The reception has been positive. The only change to the original proposal has been that a manually set __definition_order__ must be a tuple of identifiers or None (rather that using the value as-is). All other updates to the PEP have been clarification. Guido, at this point I believe the PEP is ready for pronouncement. * I've included the most recent copy of the text below. Thanks. -eric ============================== PEP: 520 Title: Ordered Class Definition Namespace Version: $Revision$ Last-Modified: $Date$ Author: Eric Snow <ericsnowcurrently(a)gmail.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 7-Jun-2016 Python-Version: 3.6 Post-History: 7-Jun-2016 Abstract ======== When a class is defined using a ``class`` statement, the class body is executed within a namespace. After the execution completes, that namespace is copied into new ``dict`` and the original definition namespace is discarded. The new copy is stored away as the class's namespace and is exposed as ``__dict__`` through a read-only proxy. This PEP changes the default class definition namespace to ``OrderedDict``. The long-lived class namespace (``__dict__``) will remain a ``dict``. Furthermore, the order in which the attributes are defined in each class body will now be preserved in the ``__definition_order__`` attribute of the class. This allows introspection of the original definition order, e.g. by class decorators. Motivation ========== Currently the namespace used during execution of a class body defaults to ``dict``. If the metaclass defines ``__prepare__()`` then the result of calling it is used. Thus, before this PEP, if you needed your class definition namespace to be ``OrderedDict`` you had to use a metaclass. Metaclasses introduce an extra level of complexity to code and in some cases (e.g. conflicts) are a problem. So reducing the need for them is worth doing when the opportunity presents itself. Given that we now have a C implementation of ``OrderedDict`` and that ``OrderedDict`` is the common use case for ``__prepare__()``, we have such an opportunity by defaulting to ``OrderedDict``. The usefulness of ``OrderedDict``-by-default is greatly increased if the definition order is directly introspectable on classes afterward, particularly by code that is independent of the original class definition. One of the original motivating use cases for this PEP is generic class decorators that make use of the definition order. Changing the default class definition namespace has been discussed a number of times, including on the mailing lists and in PEP 422 and PEP 487 (see the References section below). Specification ============= * the default class *definition* namespace is now ``OrderdDict`` * the order in which class attributes are defined is preserved in the new ``__definition_order__`` attribute on each class * "dunder" attributes (e.g. ``__init__``, ``__module__``) are ignored * ``__definition_order__`` is a ``tuple`` (or ``None``) * ``__definition_order__`` is a read-only attribute * ``__definition_order__`` is always set: 1. if ``__definition_order__`` is defined in the class body then it must be a ``tuple`` of identifiers or ``None``; any other value will result in ``TypeError`` 2. classes that do not have a class definition (e.g. builtins) have their ``__definition_order__`` set to ``None`` 3. classes for which `__prepare__()`` returned something other than ``OrderedDict`` (or a subclass) have their ``__definition_order__`` set to ``None`` (except where #1 applies) The following code demonstrates roughly equivalent semantics for the default behavior:: class Meta(type): def __prepare__(cls, *args, **kwargs): return OrderedDict() class Spam(metaclass=Meta): ham = None eggs = 5 __definition_order__ = tuple(k for k in locals() if not (k.startswith('__') and k.endswith('__'))) Note that [pep487_] proposes a similar solution, albeit as part of a broader proposal. Why a tuple? ------------ Use of a tuple reflects the fact that we are exposing the order in which attributes on the class were *defined*. Since the definition is already complete by the time ``definition_order__`` is set, the content and order of the value won't be changing. Thus we use a type that communicates that state of immutability. Why a read-only attribute? -------------------------- As with the use of tuple, making ``__definition_order__`` a read-only attribute communicates the fact that the information it represents is complete. Since it represents the state of a particular one-time event (execution of the class definition body), allowing the value to be replaced would reduce confidence that the attribute corresponds to the original class body. If a use case for a writable (or mutable) ``__definition_order__`` arises, the restriction may be loosened later. Presently this seems unlikely and furthermore it is usually best to go immutable-by-default. Note that ``__definition_order__`` is centered on the class definition body. The use cases for dealing with the class namespace (``__dict__``) post-definition are a separate matter. ``__definition_order__`` would be a significantly misleading name for a feature focused on more than class definition. See [nick_concern_] for more discussion. Why ignore "dunder" names? -------------------------- Names starting and ending with "__" are reserved for use by the interpreter. In practice they should not be relevant to the users of ``__definition_order__``. Instead, for nearly everyone they would only be clutter, causing the same extra work for everyone. Why None instead of an empty tuple? ----------------------------------- A key objective of adding ``__definition_order__`` is to preserve information in class definitions which was lost prior to this PEP. One consequence is that ``__definition_order__`` implies an original class definition. Using ``None`` allows us to clearly distinquish classes that do not have a definition order. An empty tuple clearly indicates a class that came from a definition statement but did not define any attributes there. Why None instead of not setting the attribute? ---------------------------------------------- The absence of an attribute requires more complex handling than ``None`` does for consumers of ``__definition_order__``. Why constrain manually set values? ---------------------------------- If ``__definition_order__`` is manually set in the class body then it will be used. We require it to be a tuple of identifiers (or ``None``) so that consumers of ``__definition_order__`` may have a consistent expectation for the value. That helps maximize the feature's usefulness. Why is __definition_order__ even necessary? ------------------------------------------- Since the definition order is not preserved in ``__dict__``, it is lost once class definition execution completes. Classes *could* explicitly set the attribute as the last thing in the body. However, then independent decorators could only make use of classes that had done so. Instead, ``__definition_order__`` preserves this one bit of info from the class body so that it is universally available. Compatibility ============= This PEP does not break backward compatibility, except in the case that someone relies *strictly* on ``dict`` as the class definition namespace. This shouldn't be a problem. Changes ============= In addition to the class syntax, the following expose the new behavior: * builtins.__build_class__ * types.prepare_class * types.new_class Other Python Implementations ============================ Pending feedback, the impact on Python implementations is expected to be minimal. If a Python implementation cannot support switching to `OrderedDict``-by-default then it can always set ``__definition_order__`` to ``None``. Implementation ============== The implementation is found in the tracker. [impl_] Alternatives ============ <class>.__dict__ as OrderedDict ------------------------------- Instead of storing the definition order in ``__definition_order__``, the now-ordered definition namespace could be copied into a new ``OrderedDict``. This would then be used as the mapping proxied as ``__dict__``. Doing so would mostly provide the same semantics. However, using ``OrderedDict`` for ``__dict__`` would obscure the relationship with the definition namespace, making it less useful. Additionally, doing this would require significant changes to the semantics of the concrete ``dict`` C-API. A "namespace" Keyword Arg for Class Definition ---------------------------------------------- PEP 422 introduced a new "namespace" keyword arg to class definitions that effectively replaces the need to ``__prepare__()``. [pep422_] However, the proposal was withdrawn in favor of the simpler PEP 487. References ========== .. [impl] issue #24254 (https://bugs.python.org/issue24254) .. [nick_concern] Nick's concerns about mutability (https://mail.python.org/pipermail/python-dev/2016-June/144883.html) .. [pep422] PEP 422 (https://www.python.org/dev/peps/pep-0422/#order-preserving-classes) .. [pep487] PEP 487 (https://www.python.org/dev/peps/pep-0487/#defining-arbitrary-namespaces) .. [orig] original discussion (https://mail.python.org/pipermail/python-ideas/2013-February/019690.html) .. [followup1] follow-up 1 (https://mail.python.org/pipermail/python-dev/2013-June/127103.html) .. [followup2] follow-up 2 (https://mail.python.org/pipermail/python-dev/2015-May/140137.html) Copyright =========== This document has been placed in the public domain.

4 years, 10 months

Stop using timeit, use perf.timeit!

by Victor Stinner

Hi, Last weeks, I made researchs on how to get stable and reliable benchmarks, especially for the corner case of microbenchmarks. The first result is a serie of article, here are the first three: https://haypo.github.io/journey-to-stable-benchmark-system.html https://haypo.github.io/journey-to-stable-benchmark-deadcode.html https://haypo.github.io/journey-to-stable-benchmark-average.html The second result is a new perf module which includes all "tricks" discovered in my research: compute average and standard deviation, spawn multiple worker child processes, automatically calibrate the number of outter-loop iterations, automatically pin worker processes to isolated CPUs, and more. The perf module allows to store benchmark results as JSON to analyze them in depth later. It helps to configure correctly a benchmark and check manually if it is reliable or not. The perf documentation also explains how to get stable and reliable benchmarks (ex: how to tune Linux to isolate CPUs). perf has 3 builtin CLI commands: * python -m perf: show and compare JSON results * python -m perf.timeit: new better and more reliable implementation of timeit * python -m metadata: display collected metadata Python 3 is recommended to get time.perf_counter(), use the new accurate statistics module, automatic CPU pinning (I will implement it on Python 2 later), etc. But Python 2.7 is also supported, fallbacks are implemented when needed. Example with the patched telco benchmark (benchmark for the decimal module) on a Linux with two isolated CPUs. First run the benchmark: --- $ python3 telco.py --json-file=telco.json ......................... Average: 26.7 ms +- 0.2 ms --- Then show the JSON content to see all details: --- $ python3 -m perf -v show telco.json Metadata: - aslr: enabled - cpu_affinity: 2, 3 - cpu_count: 4 - cpu_model_name: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz - hostname: smithers - loops: 10 - platform: Linux-4.4.9-300.fc23.x86_64-x86_64-with-fedora-23-Twenty_Three - python_executable: /usr/bin/python3 - python_implementation: cpython - python_version: 3.4.3 Run 1/25: warmup (1): 26.9 ms; samples (3): 26.8 ms, 26.8 ms, 26.7 ms Run 2/25: warmup (1): 26.8 ms; samples (3): 26.7 ms, 26.7 ms, 26.7 ms Run 3/25: warmup (1): 26.9 ms; samples (3): 26.8 ms, 26.9 ms, 26.8 ms (...) Run 25/25: warmup (1): 26.8 ms; samples (3): 26.7 ms, 26.7 ms, 26.7 ms Average: 26.7 ms +- 0.2 ms (25 runs x 3 samples; 1 warmup) --- Note: benchmarks can be analyzed with Python 2. I'm posting my email to python-dev because providing timeit results is commonly requested in review of optimization patches. The next step is to patch the CPython benchmark suite to use the perf module. I already forked the repository and started to patch some benchmarks. If you are interested by Python performance in general, please join us on the speed mailing list! https://mail.python.org/mailman/listinfo/speed Victor

4 years, 10 months

[RELEASE] Python 2.7.12 release candidate 1

by Benjamin Peterson

Python 2.7.12 release candidate 1 is now available for download. This is a preview release of the next bugfix release in the Python 2.7.x series. Assuming no horrible regressions are located, a final release will follow in two weeks. Downloads for 2.7.12rc1 can be found python.org: https://www.python.org/downloads/release/python-2712rc1/ The complete changelog may be viewed at https://hg.python.org/cpython/raw-file/v2.7.12rc1/Misc/NEWS Please test the pre-release and report any bugs to https://bugs.python.org Servus, Benjamin

4 years, 10 months

[RELEASED] Python 3.4.5rc1 and Python 3.5.2rc1 are now available

by Larry Hastings

On behalf of the Python development community and the Python 3.4 and Python 3.5 release teams, I'm pleased to announce the availability of Python 3.4.5rc1 and Python 3.5.2rc1. Python 3.4 is now in "security fixes only" mode. This is the final stage of support for Python 3.4. All changes made to Python 3.4 since Python 3.4.4 should be security fixes only; conventional bug fixes are not accepted. Also, Python 3.4.5rc1 and all future releases of Python 3.4 will only be released as source code--no official binary installers will be produced. Python 3.5 is still in active "bug fix" mode. Python 3.5.2rc1 contains many incremental improvements over Python 3.5.1. Both these releases are "release candidates". They should not be considered the final releases, although the final releases should contain only minor differences. Python users are encouraged to test with these releases and report any problems they encounter. You can find Python 3.4.5rc1 here: https://www.python.org/downloads/release/python-345rc1/ And you can find Python 3.5.2rc1 here: https://www.python.org/downloads/release/python-352rc1/ Python 3.4.5 final and Python 3.5.2 final are both scheduled for release on June 26th, 2016. Happy Pythoneering, //arry/

4 years, 10 months

Reminder: 3.6.0a2 snapshot 2016-06-13 12:00 UTC

by Ned Deily

Just a quick reminder that the next alpha snapshot for the 3.6 release cycle is coming up in a couple of days. This is the second of four alphas we have planned. Alpha 2 follows the development sprints at the PyCon US 2016 in Portland. Thanks to all of you who were able to be there and contribute! And to all of you who continue to contribue from afar. While there are still plenty of proposed patches awaiting review, nearly 300 commits have been pushed to the default branch (for 3.6.0) in the four weeks since alpha 1. As a reminder, alpha releases are intended to make it easier for the wider community to test the current state of new features and bug fixes for an upcoming Python release as a whole and for us to test the release process. During the alpha phase, features may be added, modified, or deleted up until the start of the beta phase. Alpha users beware! Also note that Larry has announced plans to do a 3.5.2 release candidate sometime this weekend and Benjamin plans to do a 2.7.12 release candidate. So get important maintenance release fixes in ASAP. Looking ahead, the next alpha release, 3.6.0a3, will follow in about a month on 2016-07-11. 2016-06-13 ~12:00 UTC: code snapshot for 3.6.0 alpha 1 now to 2016-09-07: Alpha phase (unrestricted feature development) 2016-09-07: 3.6.0 feature code freeze, 3.7.0 feature development begins 2016-09-07 to 2016-12-04: 3.6.0 beta phase (bug and regression fixes, no new features) 2016-12-04 3.6.0 release candidate 1 (3.6.0 code freeze) 2016-12-16 3.6.0 release (3.6.0rc1 plus, if necessary, any dire emergency fixes) --Ned P.S. Just to be clear, this upcoming alpha snapshot will *not* contain a resolution for 3.6.0 of the current on-going discussions about the behavior of os.urandom(), the secrets module, and friends (Issue26839, Issue27288, et al). I think the focus should be on getting 3.5.2 settled and then we can decide on and implement any changes for 3.6.0 in an upcoming alpha prior to beta 1. https://www.python.org/dev/peps/pep-0494/ -- Ned Deily nad(a)python.org -- []

4 years, 10 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

Python-Dev June 2016