Mailman 3 July 2015 - Python-Dev

Summary of Python tracker Issues
by Python tracker July 10, 2015

July 10, 2015

ACTIVITY SUMMARY (2015-07-03 - 2015-07-10) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 4928 ( +6) closed 31439 (+40) total 36367 (+46) Open issues with patches: 2257 Issues opened (24) ================== #24563: Encoding declaration: doc supported encodings http://bugs.python.org/issue24563 opened by terry.reedy #24564: shutil.copytree fails … [View More]

1 0

modules dependencies issues
by Cyril Scetbon July 10, 2015

July 10, 2015

Hi, I use pip to install modules and setuptools to install dependencies, and generate a console_script using the entry_point parameter of setup. Here is the issue : my current sources depend on modules, let's say A=1.0, B=1.0, C=2.0. And C depends on B=1.1 I have no problem with using pip to install dependencies. However setuptools complain that 2 versions are conflicting : Installed /private/tmp/test/my-module Processing dependencies for my-module==0.0.1 error: B 1.0 is installed but B==1.1 … [View More]

2 2

Issue #15014 - SMTP AUTH initial-response (beta exception requested)
by Barry Warsaw July 9, 2015

July 9, 2015

Larry and others, I'd like to bring your attention to issue #15014. This issue added arbitrary auth methods to smtplib, which is a good thing. Implicitly though, a regression was introduced w.r.t. RFC 4954's optional initial-response for the AUTH command, for authentication methods that support it. An example is AUTH PLAIN, which does not require a challenge. It's possible that SMTP servers are not prepared to handle challenge/responses for authentication methods that support initial-… [View More]

3 4

OpenSSL Security Advisory [9 Jul 2015]
by Christian Heimes July 9, 2015

July 9, 2015

Hi, this just came in. According to Zachary all Windows builds use 1.0.2c. The version is vulnerable to a critical bug in the CA validation code of OpenSSL. The bug can be abused to turn any valid server certificate into a CA cert. We should consider a security release of Python ASAP. Alternative chains certificate forgery (CVE-2015-1793) ====================================================== Severity: High During certificate verification, OpenSSL (starting from version 1.0.1n and 1.0.2b)… [View More]

1 1

PEP 493: Redistributor guidance for Python 2.7 HTTPS
by Nick Coghlan July 9, 2015

July 9, 2015

Hi folks, As previously discussed on python-ideas, Red Hat has been looking at ways to provide a smoother migration path for system administrators to get to a point where system Python installations are verifying HTTPS by default. While we're not proposing that these changes be implemented upstream (given that the change in default behaviour was already implemented for 2.7.9), we *would* like to pursue an explicit upstream recommendation regarding how deviations from the upstream behaviour … [View More]should be handled. Otherwise we run the risk of different redistributors pursuing mutually incompatible approaches to the migration, which would be a rather unfortunate outcome. The proposal in PEP 493 offers two recommendations: * one option that leaves the default behaviour alone, but provides an easy environment variable based way to revert to the legacy behaviour on a per-application basis (based on a design suggested by MAL) * a second option designed for backporting the changes to versions that advertise themselves as older than 2.7.9 that permits opting in to the new behaviour on a system wide basis (based on a design suggested by Robert Kuska) The main change from the last version discussed on python-ideas is that in both cases there's now a required attribute that redistributors are expected to add to the SSL module to signal the presence of the feature (and to provide some useful information about its implementation). I think this is mature enough now for me to request pronouncement (if Guido's interested in doing that himself), or volunteers to be BDFL-Delegate (if Guido's would prefer someone else handle it). Regards, Nick. ====================== PEP: 493 Title: HTTPS verification recommendations for Python 2.7 redistributors Version: $Revision$ Last-Modified: $Date$ Author: Nick Coghlan <ncoghlan(a)gmail.com>, Robert Kuska <rkuska(a)redhat.com>, Marc-André Lemburg <mal(a)lemburg.com> Status: Draft Type: Informational Content-Type: text/x-rst Created: 10-May-2015 Post-History: 06-Jul-2015 Abstract ======== PEP 476 updated Python's default handling of HTTPS certificates to be appropriate for communication over the public internet. The Python 2.7 long term maintenance series was judged to be in scope for this change, with the new behaviour introduced in the Python 2.7.9 maintenance release. This PEP provides recommendations to downstream redistributors wishing to provide a smoother migration experience when helping their users to manage this change in Python's default behaviour. *Note that this PEP is not currently accepted, so it is a *proposed* recommendation, rather than an active one.* Rationale ========= PEP 476 changed Python's default behaviour to better match the needs and expectations of developers operating over the public internet, a category which appears to include most new Python developers. It is the position of the authors of this PEP that this was a correct decision. However, it is also the case that this change *does* cause problems for infrastructure administrators operating private intranets that rely on self-signed certificates, or otherwise encounter problems with the new default certificate verification settings. The long term answer for such environments is to update their internal certificate management to at least match the standards set by the public internet, but in the meantime, it is desirable to offer these administrators a way to continue receiving maintenance updates to the Python 2.7 series, without having to gate that on upgrades to their certificate management infrastructure. PEP 476 did attempt to address this question, by covering how to revert the new settings process wide by monkeypatching the ``ssl`` module to restore the old behaviour. Unfortunately, the ``sitecustomize.py`` based technique proposed to allow system administrators to disable the feature by default in their Standard Operating Environment definition has been determined to be insufficient in at least some cases. The specific case of interest to the authors of this PEP is the one where a Linux distributor aims to provide their users with a `smoother migration path <https://bugzilla.redhat.com/show_bug.cgi?id=1173041>`__ than the standard one provided by consuming upstream CPython 2.7 releases directly, but other potential challenges have also been pointed out with updating embedded Python runtimes and other user level installations of Python. Rather than allowing a plethora of mutually incompatibile migration techniques to bloom, this PEP proposes two alternative approaches that redistributors may take when addressing these problems. Redistributors may choose to implement one, both, or neither of these approaches based on their assessment of the needs of their particular userbase. These designs are being proposed as a recommendation for redistributors, rather than as new upstream features, as they are needed purely to support legacy environments migrating from older versions of Python 2.7. Neither approach is being proposed as an upstream Python 2.7 feature, nor as a feature in any version of Python 3 (whether published directly by the Python Software Foundation or by a redistributor). Requirements for capability detection ===================================== As these recommendations are intended to cover backports to earlier Python versions, the Python version number cannot be used as a reliable means for detecting them. Instead, the recommendations are defined to allow the presence or absence of the feature to be determined using the following technique:: python -c "import ssl; ssl._relevant_attribute" This will fail with `AttributeError` (and hence a non-zero return code) if the relevant capability is not available. The marker attributes are prefixed with an underscore to indicate the implementation dependent nature of these capabilities - not all Python distributions will offer them, only those that are providing a multi-stage migration process from the legacy HTTPS handling to the new default behaviour. Recommendation for an environment variable based security downgrade =================================================================== Some redistributors may wish to provide a per-application option to disable certificate verification in selected applications that run on or embed CPython without needing to modify the application itself. In these cases, a configuration mechanism is needed that provides: * an opt-out model that allows certificate verification to be selectively turned off for particular applications after upgrading to a version of Python that verifies certificates by default * the ability for all users to configure this setting on a per-application basis, rather than on a per-system, or per-Python-installation basis This approach may be used for any redistributor provided version of Python 2.7, including those that advertise themselves as providing Python 2.7.9 or later. Required marker attribute ------------------------- The required marker attribute on the ``ssl`` module when implementing this recommendation is:: _https_verify_envvar = 'PYTHONHTTPSVERIFY' This not only makes it straightforward to detect the presence (or absence) of the capability, it also makes it possible to programmatically determine the relevant environment variable name. Recommended modifications to the Python standard library -------------------------------------------------------- The recommended approach to providing a per-application configuration setting for HTTPS certificate verification that doesn't require modifications to the application itself is to: * modify the ``ssl`` module to read the ``PYTHONHTTPSVERIFY`` environment variable when the module is first imported into a Python process * set the ``ssl._create_default_https_context`` function to be an alias for ``ssl._create_unverified_context`` if this environment variable is present and set to ``'0'`` * otherwise, set the ``ssl._create_default_https_context`` function to be an alias for ``ssl.create_default_context`` as usual Example implementation ---------------------- :: _https_verify_envvar = 'PYTHONHTTPSVERIFY' def _get_https_context_factory(): config_setting = os.environ.get(_https_verify_envvar) if config_setting == '0': return _create_unverified_context return create_default_context _create_default_https_context = _get_https_context_factory() Security Considerations ----------------------- Relative to an unmodified version of CPython 2.7.9 or later, this approach does introduce a new downgrade attack against the default security settings that potentially allows a sufficiently determined attacker to revert Python to the vulnerable configuration used in CPython 2.7.8 and earlier releases. However, such an attack requires the ability to modify the execution environment of a Python process prior to the import of the ``ssl`` module, and any attacker with such access would already be able to modify the behaviour of the underlying OpenSSL implementation. Recommendation for backporting to earlier Python versions ========================================================= Some redistributors, most notably Linux distributions, may choose to backport the PEP 476 HTTPS verification changes to modified Python versions based on earlier Python 2 maintenance releases. In these cases, a configuration mechanism is needed that provides: * an opt-in model that allows the decision to enable HTTPS certificate verification to be made independently of the decision to upgrade to the Python version where the feature was first backported * the ability for system administrators to set the default behaviour of Python applications and scripts run directly in the system Python installation * the ability for the redistributor to consider changing the default behaviour of *new* installations at some point in the future without impacting existing installations that have been explicitly configured to skip verifying HTTPS certificates by default This approach should not be used for any Python installation that advertises itself as providing Python 2.7.9 or later, as most Python users will have the reasonable expectation that all such environments will validate HTTPS certificates by default. Required marker attribute ------------------------- The required marker attribute on the ``ssl`` module when implementing this recommendation is:: _cert_verification_config = '<path to configuration file>' This not only makes it straightforward to detect the presence (or absence) of the capability, it also makes it possible to programmatically determine the relevant configuration file name. Recommended modifications to the Python standard library -------------------------------------------------------- The recommended approach to backporting the PEP 476 modifications to an earlier point release is to implement the following changes relative to the default PEP 476 behaviour implemented in Python 2.7.9+: * modify the ``ssl`` module to read a system wide configuration file when the module is first imported into a Python process * define a platform default behaviour (either verifying or not verifying HTTPS certificates) to be used if this configuration file is not present * support selection between the following three modes of operation: * ensure HTTPS certificate verification is enabled * ensure HTTPS certificate verification is disabled * delegate the decision to the redistributor providing this Python version * set the ``ssl._create_default_https_context`` function to be an alias for either ``ssl.create_default_context`` or ``ssl._create_unverified_context`` based on the given configuration setting. Recommended file location ------------------------- This approach is currently only defined for \*nix system Python installations. The recommended configuration file name is ``/etc/python/cert-verification.cfg``. The ``.cfg`` filename extension is recommended for consistency with the ``pyvenv.cfg`` used by the ``venv`` module in Python 3's standard library. Recommended file format ----------------------- The configuration file should use a ConfigParser ini-style format with a single section named ``[https]`` containing one required setting ``verify``. Permitted values for ``verify`` are: * ``enable``: ensure HTTPS certificate verification is enabled by default * ``disable``: ensure HTTPS certificate verification is disabled by default * ``platform_default``: delegate the decision to the redistributor providing this particular Python version If the ``[https]`` section or the ``verify`` setting are missing, or if the ``verify`` setting is set to an unknown value, it should be treated as if the configuration file is not present. Example implementation ---------------------- :: _cert_verification_config = '/etc/python/cert-verification.cfg' def _get_https_context_factory(): # Check for a system-wide override of the default behaviour context_factories = { 'enable': create_default_context, 'disable': _create_unverified_context, 'platform_default': _create_unverified_context, # For now :) } import ConfigParser config = ConfigParser.RawConfigParser() config.read(_cert_verification_config) try: verify_mode = config.get('https', 'verify') except (ConfigParser.NoSectionError, ConfigParser.NoOptionError): verify_mode = 'platform_default' default_factory = context_factories.get('platform_default') return context_factories.get(verify_mode, default_factory) _create_default_https_context = _get_https_context_factory() Security Considerations ----------------------- The specific recommendations for the backporting case are designed to work for privileged, security sensitive processes, even those being run in the following locked down configuration: * run from a locked down administrator controlled directory rather than a normal user directory (preventing ``sys.path[0]`` based privilege escalation attacks) * run using the ``-E`` switch (preventing ``PYTHON*`` environment variable based privilege escalation attacks) * run using the ``-s`` switch (preventing user site directory based privilege escalation attacks) * run using the ``-S`` switch (preventing ``sitecustomize`` based privilege escalation attacks) The intent is that the *only* reason HTTPS verification should be getting turned off system wide when using this approach is because: * an end user is running a redistributor provided version of CPython rather than running upstream CPython directly * that redistributor has decided to provide a smoother migration path to verifying HTTPS certificates by default than that being provided by the upstream project * either the redistributor or the local infrastructure administrator has determined that it is appropriate to override the default upstream behaviour (at least for the time being) Using an administrator controlled configuration file rather than an environment variable has the essential feature of providing a smoother migration path, even for applications being run with the ``-E`` switch. Combining the recommendations ============================= If a redistributor chooses to implement both recommendations, then the environment variable should take precedence over the system-wide configuration setting. This allows the setting to be changed for a given user, virtual environment or application, regardless of the system-wide default behaviour. In this case, if the ``PYTHONHTTPSVERIFY`` environment variable is defined, and set to anything *other* than ``'0'``, then HTTPS certificate verification should be enabled. Example implementation ---------------------- :: _https_verify_envvar = 'PYTHONHTTPSVERIFY' _cert_verification_config = '/etc/python/cert-verification.cfg' def _get_https_context_factory(): # Check for am environmental override of the default behaviour config_setting = os.environ.get(_https_verify_envvar) if config_setting is not None: if config_setting == '0': return _create_unverified_context return create_default_context # Check for a system-wide override of the default behaviour context_factories = { 'enable': create_default_context, 'disable': _create_unverified_context, 'platform_default': _create_unverified_context, # For now :) } import ConfigParser config = ConfigParser.RawConfigParser() config.read(_cert_verification_config) try: verify_mode = config.get('https', 'verify') except (ConfigParser.NoSectionError, ConfigParser.NoOptionError): verify_mode = 'platform_default' default_factory = context_factories.get('platform_default') return context_factories.get(verify_mode, default_factory) _create_default_https_context = _get_https_context_factory() Copyright ========= This document has been placed into the public domain. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia [View Less]

7 14

Review/CI infrastructure status (was Re: cpython: Tighten-up code in the set iterator to use an entry pointer rather than)
by Nick Coghlan July 7, 2015

July 7, 2015

On 7 July 2015 at 22:31, Guido van Rossum <guido(a)python.org> wrote: > A simple way to address this would have been to CC Raymond on the issue. But > the reply-to header probably makes that hard. Agreed that post-commit > reviews don't really scale to our size. It's hard to teach old dogs new > tricks though. I now realize that my main point, however, was that Raymond > and Serhiy might have sorted this out offline already without either of them > bothering to CC python-… [View More]

1 0

Importance of "async" keyword
by Sven R. Kunze July 7, 2015

July 7, 2015

Hi everybody, recently, I stumbled over the new 3.5 release and in doing so over PEP 0492. After careful consideration and after reading many blog posts of various coders, I first would like to thank Yury Selivanov and everybody else who brought PEP 0492 to its final state. I therefore considered usage within our projects, however, still find hazy items in PEP 0492. So, I would like to contribute my thoughts on this in order to either increase my understanding or even improve Python's … [View More]

13 38

Re: [Python-Dev] cpython: Minor bit of factoring-out common code.
by Serhiy Storchaka July 7, 2015

July 7, 2015

On 07.07.15 05:08, raymond.hettinger wrote: > https://hg.python.org/cpython/rev/5088f2cd6293 > changeset: 96866:5088f2cd6293 > user: Raymond Hettinger <python(a)rcn.com> > date: Mon Jul 06 19:08:49 2015 -0700 > summary: > Minor bit of factoring-out common code. > + if (rv < 0) > + goto error; > if (rv) { > - if (set_add_entry(result, key, hash)) { > - Py_DECREF(it); > - … [View More]

1 0

raw_input prompt not printed on sys.stderr
by Clement Rouault July 7, 2015

July 7, 2015

Hi, While playing with non-standard sys.stdout/stderr, I noticed that the prompt of raw_input was printed on stderr (not sys.stderr) (see Parser/myreadline.c:120). I found an issue (http://bugs.python.org/issue1927) from 2008 talking about changing stderr to stdout. But nobody in the thread seems bothered by the use of stdout/err over the ones in the sys module. So, is there any good reason I might not be aware of that justifies the use of stderr over sys.stderr ? -- Clement "Hakril" Rouault

2 1

[RELEASED] Python 3.5.0b3 is now available
by Larry Hastings July 6, 2015

July 6, 2015

On behalf of the Python development community and the Python 3.5 release team, I'm relieved to announce the availability of Python 3.5.0b3. Python 3.5 has now entered "feature freeze". By default new features may no longer be added to Python 3.5. This is a preview release, and its use is not recommended for production settings. An important reminder for Windows users about Python 3.5.0b3: if installing Python 3.5.0b2 as a non-privileged user, you may need to escalate to administrator … [View More]

2 1