Mailman 3 December 2015 - Python-Dev

Re: [Python-Dev] Python semantic: Is it ok to replace not x == y with x != y? (no)
by Adam Bartoš 15 Dec '15

15 Dec '15

Hello, the comparisons >=, <=, >, < cannot be optimized this way. Not every order is a total order. For example, sets a = {1, 2} and b = {2, 3} are incomparable (in the sense that both a >= b and a <= b is False), and it is no pathology. Regards, Adam Bartoš

1 0

Urgent: Last call for the CfP of PythonFOSDEM 2016
by Stephane Wirtel 15 Dec '15

15 Dec '15

Hi all Because the deadline is imminent and because we have only received some proposals, we have extended the current deadline. The new submission deadline is 2015-12-20. Call For Proposals ================== This is the official call for sessions for the Python devroom at FOSDEM 2016. FOSDEM is the Free and Open source Software Developers' European Meeting, a free and non-commercial two-day week-end that offers open source contributors a place to meet, share ideas and collaborate. It's the biggest event in Europe with +5000 hackers, +400 speakers. For this edition, Python will be represented by its Community. If you want to discuss with a lot of Python Users, it's the place to be! Important dates =============== * Submission deadlines: 2015-12-20 * Acceptance notifications: 2015-12-24 Practical ========= * The duration for talks will be 30 minutes, including presentations and questions and answers. * Presentation can be recorded and streamed, sending your proposal implies giving permission to be recorded. * A mailing list for the Python devroom is available for discussions about devroom organisation. You can register at this address: https://lists.fosdem.org/listinfo/python-devroom How to submit ============= All submissions are made in the Pentabarf event planning tool at https://penta.fosdem.org/submission/FOSDEM16 When submitting your talk in Pentabarf, make sure to select the Python devroom as the Track. Of course, if you already have a user account, please reuse it. Questions ========= Any questions, please sned an email to info AT python-fosdem DOT org Thank you for submitting your sessions and see you soon in Brussels to talk about Python. If you want to keep informed for this edition, you can follow our twitter account @PythonFOSDEM. * FOSDEM 2016: https://fosdem.org/2016 * Python Devroom: http://python-fosdem.org * Twitter: https://twitter.com/PythonFOSDEM Thank you so much, Stephane -- Stéphane Wirtel - http://wirtel.be - @matrixise

1 0

Urgent: Last call for the CfP of PythonFOSDEM 2016
by Stephane Wirtel 15 Dec '15

15 Dec '15

Hi all Because the deadline is imminent and because we have only received some proposals, we have extended the current deadline. The new submission deadline is 2015-12-20. Call For Proposals ================== This is the official call for sessions for the Python devroom at FOSDEM 2016. FOSDEM is the Free and Open source Software Developers' European Meeting, a free and non-commercial two-day week-end that offers open source contributors a place to meet, share ideas and collaborate. It's the biggest event in Europe with +5000 hackers, +400 speakers. For this edition, Python will be represented by its Community. If you want to discuss with a lot of Python Users, it's the place to be! Important dates =============== * Submission deadlines: 2015-12-20 * Acceptance notifications: 2015-12-24 Practical ========= * The duration for talks will be 30 minutes, including presentations and questions and answers. * Presentation can be recorded and streamed, sending your proposal implies giving permission to be recorded. * A mailing list for the Python devroom is available for discussions about devroom organisation. You can register at this address: https://lists.fosdem.org/listinfo/python-devroom How to submit ============= All submissions are made in the Pentabarf event planning tool at https://penta.fosdem.org/submission/FOSDEM16 When submitting your talk in Pentabarf, make sure to select the Python devroom as the Track. Of course, if you already have a user account, please reuse it. Questions ========= Any questions, please sned an email to info AT python-fosdem DOT org Thank you for submitting your sessions and see you soon in Brussels to talk about Python. If you want to keep informed for this edition, you can follow our twitter account @PythonFOSDEM. * FOSDEM 2016: https://fosdem.org/2016 * Python Devroom: http://python-fosdem.org * Twitter: https://twitter.com/PythonFOSDEM Thank you so much, Stephane -- Stéphane Wirtel - http://wirtel.be - @matrixise

1 0

Summary of Python tracker Issues
by Python tracker 11 Dec '15

11 Dec '15

ACTIVITY SUMMARY (2015-12-04 - 2015-12-11) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 5297 ( +8) closed 32303 (+37) total 37600 (+45) Open issues with patches: 2332 Issues opened (28) ================== #25624: shutil.make_archive makes invalid directory entries http://bugs.python.org/issue25624 reopened by serhiy.storchaka #25798: Update python.org installers to use OpenSSL 1.0.2e http://bugs.python.org/issue25798 opened by ned.deily #25799: 2.7.11rc1 not added to Win10 app list (start menu) http://bugs.python.org/issue25799 opened by terry.reedy #25801: ResourceWarning in test_zipfile64 http://bugs.python.org/issue25801 opened by SilentGhost #25802: Finish deprecating load_module() http://bugs.python.org/issue25802 opened by brett.cannon #25803: pathlib.Path('/').mkdir() raises wrong error type http://bugs.python.org/issue25803 opened by Daniel Lepage #25804: Make Profile.print_stats support sorting by mutiple values http://bugs.python.org/issue25804 opened by wdv4758h #25805: Failure in test_pkgutil run from command-line http://bugs.python.org/issue25805 opened by SilentGhost #25809: "Invalid" tests on locales http://bugs.python.org/issue25809 opened by bapt #25810: Python 3 documentation for eval is incorrect http://bugs.python.org/issue25810 opened by aroberge #25812: locale.nl_langinfo() can't decode value http://bugs.python.org/issue25812 opened by serhiy.storchaka #25813: co_flags section of inspect module docs out of date http://bugs.python.org/issue25813 opened by BreamoreBoy #25817: modsupport: 'countformat' does not handle strings without brac http://bugs.python.org/issue25817 opened by Myron Walker #25821: Documentation for threading.enumerate / threading.Thread.is_al http://bugs.python.org/issue25821 opened by anthonygreen #25822: Add docstrings to fields of urllib.parse results http://bugs.python.org/issue25822 opened by serhiy.storchaka #25823: Speed-up oparg decoding on little-endian machines http://bugs.python.org/issue25823 opened by rhettinger #25824: 32-bit 2.7.11 installer creates registry keys that are incompa http://bugs.python.org/issue25824 opened by aundro #25825: AIX shared library extension modules installation broken http://bugs.python.org/issue25825 opened by David.Edelsohn #25827: Support ICC in configure http://bugs.python.org/issue25827 opened by zach.ware #25828: PyCode_Optimize() (peephole optimizer) doesn't handle Keyboard http://bugs.python.org/issue25828 opened by haypo #25829: Mixing multiprocessing pool and subprocess may create zombie p http://bugs.python.org/issue25829 opened by amikoren(a)yahoo.com #25830: _TypeAlias: Discrepancy between docstring and behavior http://bugs.python.org/issue25830 opened by flying sheep #25833: pyvenv: venvs cannot be moved because activate scripts hard-co http://bugs.python.org/issue25833 opened by moorecm #25834: getpass falls back when sys.stdin is changed http://bugs.python.org/issue25834 opened by Drekin #25836: Documentation of MAKE_FUNCTION is still incorrect http://bugs.python.org/issue25836 opened by freakboy3742 #25838: Lib/httplib.py: Resend http request on server close connection http://bugs.python.org/issue25838 opened by gmixo #25841: In FancyURLopener error in example with http address. http://bugs.python.org/issue25841 opened by Denis Savenko #25842: Installer does not set permissions correctly? http://bugs.python.org/issue25842 opened by lac Most recent 15 issues with no replies (15) ========================================== #25842: Installer does not set permissions correctly? http://bugs.python.org/issue25842 #25841: In FancyURLopener error in example with http address. http://bugs.python.org/issue25841 #25836: Documentation of MAKE_FUNCTION is still incorrect http://bugs.python.org/issue25836 #25834: getpass falls back when sys.stdin is changed http://bugs.python.org/issue25834 #25830: _TypeAlias: Discrepancy between docstring and behavior http://bugs.python.org/issue25830 #25812: locale.nl_langinfo() can't decode value http://bugs.python.org/issue25812 #25805: Failure in test_pkgutil run from command-line http://bugs.python.org/issue25805 #25802: Finish deprecating load_module() http://bugs.python.org/issue25802 #25791: Raise an ImportWarning when __spec__.parent/__package__ isn't http://bugs.python.org/issue25791 #25785: TimedRotatingFileHandler missing rotations http://bugs.python.org/issue25785 #25776: More compact pickle of iterators etc http://bugs.python.org/issue25776 #25774: [benchmarks] Adjust to allow uploading benchmark data to codes http://bugs.python.org/issue25774 #25773: Deprecate deleting with PyObject_SetAttr, PyObject_SetAttrStri http://bugs.python.org/issue25773 #25766: __bytes__ doesn't work for str subclasses http://bugs.python.org/issue25766 #25753: Reference leaks in test_smtplib http://bugs.python.org/issue25753 Most recent 15 issues waiting for review (15) ============================================= #25838: Lib/httplib.py: Resend http request on server close connection http://bugs.python.org/issue25838 #25827: Support ICC in configure http://bugs.python.org/issue25827 #25823: Speed-up oparg decoding on little-endian machines http://bugs.python.org/issue25823 #25822: Add docstrings to fields of urllib.parse results http://bugs.python.org/issue25822 #25809: "Invalid" tests on locales http://bugs.python.org/issue25809 #25804: Make Profile.print_stats support sorting by mutiple values http://bugs.python.org/issue25804 #25801: ResourceWarning in test_zipfile64 http://bugs.python.org/issue25801 #25794: __setattr__ does not always overload operators http://bugs.python.org/issue25794 #25789: py launcher stderr is not piped to subprocess.Popen.stderr http://bugs.python.org/issue25789 #25786: contextlib.ExitStack introduces a cycle in exception __context http://bugs.python.org/issue25786 #25782: CPython hangs on error __context__ set to the error itself http://bugs.python.org/issue25782 #25780: Add support for CAN_RAW_JOIN_FILTERS http://bugs.python.org/issue25780 #25778: winreg.EnumValue does not truncate strings correctly http://bugs.python.org/issue25778 #25776: More compact pickle of iterators etc http://bugs.python.org/issue25776 #25774: [benchmarks] Adjust to allow uploading benchmark data to codes http://bugs.python.org/issue25774 Top 10 most discussed issues (10) ================================= #25810: Python 3 documentation for eval is incorrect http://bugs.python.org/issue25810 10 msgs #25823: Speed-up oparg decoding on little-endian machines http://bugs.python.org/issue25823 9 msgs #24682: Add Quick Start: Communications section to devguide http://bugs.python.org/issue24682 8 msgs #25089: Can't run Python Launcher on Windows http://bugs.python.org/issue25089 7 msgs #25638: Verify the etree_parse and etree_iterparse benchmarks are work http://bugs.python.org/issue25638 7 msgs #25809: "Invalid" tests on locales http://bugs.python.org/issue25809 7 msgs #25701: Document that tp_setattro and tp_setattr are used for deleting http://bugs.python.org/issue25701 6 msgs #25817: modsupport: 'countformat' does not handle strings without brac http://bugs.python.org/issue25817 6 msgs #25698: The copy_reg module becomes unexpectedly empty in test_cpickle http://bugs.python.org/issue25698 5 msgs #25716: typeobject.c call_method & call_maybe can leak references on ' http://bugs.python.org/issue25716 5 msgs Issues closed (38) ================== #12509: test_gdb fails on debug build when builddir != srcdir http://bugs.python.org/issue12509 closed by martin.panter #14285: Traceback wrong on ImportError while executing a package http://bugs.python.org/issue14285 closed by ncoghlan #15858: tarfile missing entries due to omitted uid/gid fields http://bugs.python.org/issue15858 closed by martin.panter #16458: subprocess.py throw "The handle is invalid" error on duplicati http://bugs.python.org/issue16458 closed by eryksun #17772: test_gdb doesn't detect a gdb built with python3.3 (or higher) http://bugs.python.org/issue17772 closed by martin.panter #21240: Add an abstactmethod directive to the Python ReST domain http://bugs.python.org/issue21240 closed by berker.peksag #22341: Python 3 crc32 documentation clarifications http://bugs.python.org/issue22341 closed by martin.panter #22758: Regression in Python 3.2 cookie parsing http://bugs.python.org/issue22758 closed by Tim.Graham #23936: Wrong references to deprecated find_module instead of find_spe http://bugs.python.org/issue23936 closed by brett.cannon #24903: Do not verify destdir argument to compileall http://bugs.python.org/issue24903 closed by r.david.murray #24934: django_v2 benchmark not working in Python 3.6 http://bugs.python.org/issue24934 closed by brett.cannon #25039: Docs: Link to Stackless Python in Design and History FAQ secti http://bugs.python.org/issue25039 closed by zach.ware #25492: subprocess with redirection fails after FreeConsole http://bugs.python.org/issue25492 closed by eryksun #25500: docs claim __import__ checked for in globals, but IMPORT_NAME http://bugs.python.org/issue25500 closed by brett.cannon #25591: improve test coverage for the imaplib http://bugs.python.org/issue25591 closed by maciej.szulik #25715: Python 3.5.1 installer shows wrong upgrade path http://bugs.python.org/issue25715 closed by larry #25717: tempfile.TemporaryFile fails when dir option set to directory http://bugs.python.org/issue25717 closed by martin.panter #25764: PyObject_Call() is called with an exception set in subprocess http://bugs.python.org/issue25764 closed by martin.panter #25771: importlib: '.submodule' is not a relative name (no leading dot http://bugs.python.org/issue25771 closed by brett.cannon #25795: test_fork1 cannot be run directly: ./python Lib/test/test_fork http://bugs.python.org/issue25795 closed by python-dev #25800: errors running test_capi from command line http://bugs.python.org/issue25800 closed by python-dev #25806: ResourceWarning in test_tasks http://bugs.python.org/issue25806 closed by SilentGhost #25807: test_multiprocessing_fork.test_mymanager fails and hangs http://bugs.python.org/issue25807 closed by SilentGhost #25808: The Python Tutorial 5.3. Tuples and Sequences http://bugs.python.org/issue25808 closed by SilentGhost #25811: return from random.shuffle http://bugs.python.org/issue25811 closed by serhiy.storchaka #25814: Propagate all errors from ElementTree.iterparse http://bugs.python.org/issue25814 closed by serhiy.storchaka #25815: Improper subprocess output of arguments with braces in them on http://bugs.python.org/issue25815 closed by ebarry #25816: https://www.python.org/downloads/ not working for 3.5.1 for wi http://bugs.python.org/issue25816 closed by brett.cannon #25818: asyncio: If protocol_factory raises an error, the connection c http://bugs.python.org/issue25818 closed by ebarry #25819: print "Hi" in python 3 exception handling doesn't work http://bugs.python.org/issue25819 closed by ebarry #25820: Clean up run_gdb() calls http://bugs.python.org/issue25820 closed by martin.panter #25826: imaplib can't process lines after starttls http://bugs.python.org/issue25826 closed by David Wahlund #25831: dbm.gnu leaks file descriptors on .reorganize() http://bugs.python.org/issue25831 closed by ischwabacher #25832: Document weird behavior of `finally` when it has `break` in it http://bugs.python.org/issue25832 closed by r.david.murray #25835: httplib uses print for debugging http://bugs.python.org/issue25835 closed by r.david.murray #25837: Errors when Installing Anaconda 3 http://bugs.python.org/issue25837 closed by r.david.murray #25839: negative zero components are ignored in complex number literal http://bugs.python.org/issue25839 closed by mark.dickinson #25840: Allow `False` to be passed to `filter` http://bugs.python.org/issue25840 closed by r.david.murray

1 0

A function for Find-Replace in lists
by טל ח 09 Dec '15

09 Dec '15

3 2

Do windows 10 users, like windows 7 users need to install a SP before installing Python will work?
by Laura Creighton 09 Dec '15

09 Dec '15

As webmaster, I am dealing with 3 unhappy would-be python users who have windows 10. Right now their first problem is that when they click on the big yellow button here: https://www.python.org/downloads/ instead of getting a download of 3.5.1 they get a redirect to https://www.python.org/downloads/windows/ I've tried them on both the Download Windows x86 web-based installer and Download Windows x86-64 web-based installer but still no go, they get the Modify/Repair/Uninstall screen like: http://www2.openend.se/~lac/5796.2.png I do not know how to help them now. Laura

4 8

Request for pronouncement on PEP 493 (HTTPS verification backport guidance)
by Nick Coghlan 08 Dec '15

08 Dec '15

Hi folks, I have a confession to make - I dropped the ball on the HTTPS verification backport proposals in PEP 493, and let the upstream and downstream approval processes get out of sequence. As a result, the RHEL 7.2 beta released back in September incorporates the HTTPS verification feature backport based on the current PEP 493 draft, even though that hasn't formally been pronounced as an Active recommendation by python-dev yet. Accordingly, I'm belatedly submitting it for pronouncement now: https://www.python.org/dev/peps/pep-0493/ There's currently no BDFL-Delegate assigned, so if Guido doesn't want to handle it, we'll need to address that question first. Our last discussion back in July seemed to show that folks either didn't care about the question (because they're using unmodified upstream versions so the PEP didn't affect them), or else thought the approach described in the PEP was reasonable, so I'm hoping the consequences of my mistake won't be too severe. Regards, Nick. P.S. I'm aware that this looks like presenting a fait accompli at a point where it's too late to realistically say "No", but the truth is that preparation for the Python in Education miniconf at PyCon Australia ramped up immediately after the July discussion, and then I personally got confused as to the scope of what was being included in 7.2 (I mistakenly thought it was just PEP 466 for now, with 476+493 being deferred to a later release, but it's actually the whole package of 466+476+493). That's my fault for trying to keep track of too many things at once (and thus failing at some of them), not anyone else's. ================================ PEP: 493 Title: HTTPS verification recommendations for Python 2.7 redistributors Version: $Revision$ Last-Modified: $Date$ Author: Nick Coghlan <ncoghlan(a)gmail.com>, Robert Kuska <rkuska(a)redhat.com>, Marc-André Lemburg <mal(a)lemburg.com> Status: Draft Type: Informational Content-Type: text/x-rst Created: 10-May-2015 Post-History: 06-Jul-2015 Abstract ======== PEP 476 updated Python's default handling of HTTPS certificates to be appropriate for communication over the public internet. The Python 2.7 long term maintenance series was judged to be in scope for this change, with the new behaviour introduced in the Python 2.7.9 maintenance release. This PEP provides recommendations to downstream redistributors wishing to provide a smoother migration experience when helping their users to manage this change in Python's default behaviour. Rationale ========= PEP 476 changed Python's default behaviour to better match the needs and expectations of developers operating over the public internet, a category which appears to include most new Python developers. It is the position of the authors of this PEP that this was a correct decision. However, it is also the case that this change *does* cause problems for infrastructure administrators operating private intranets that rely on self-signed certificates, or otherwise encounter problems with the new default certificate verification settings. The long term answer for such environments is to update their internal certificate management to at least match the standards set by the public internet, but in the meantime, it is desirable to offer these administrators a way to continue receiving maintenance updates to the Python 2.7 series, without having to gate that on upgrades to their certificate management infrastructure. PEP 476 did attempt to address this question, by covering how to revert the new settings process wide by monkeypatching the ``ssl`` module to restore the old behaviour. Unfortunately, the ``sitecustomize.py`` based technique proposed to allow system administrators to disable the feature by default in their Standard Operating Environment definition has been determined to be insufficient in at least some cases. The specific case of interest to the authors of this PEP is the one where a Linux distributor aims to provide their users with a `smoother migration path <https://bugzilla.redhat.com/show_bug.cgi?id=1173041>`__ than the standard one provided by consuming upstream CPython 2.7 releases directly, but other potential challenges have also been pointed out with updating embedded Python runtimes and other user level installations of Python. Rather than allowing a plethora of mutually incompatibile migration techniques to bloom, this PEP proposes two alternative approaches that redistributors may take when addressing these problems. Redistributors may choose to implement one, both, or neither of these approaches based on their assessment of the needs of their particular userbase. These designs are being proposed as a recommendation for redistributors, rather than as new upstream features, as they are needed purely to support legacy environments migrating from older versions of Python 2.7. Neither approach is being proposed as an upstream Python 2.7 feature, nor as a feature in any version of Python 3 (whether published directly by the Python Software Foundation or by a redistributor). Requirements for capability detection ===================================== As these recommendations are intended to cover backports to earlier Python versions, the Python version number cannot be used as a reliable means for detecting them. Instead, the recommendations are defined to allow the presence or absence of the feature to be determined using the following technique:: python -c "import ssl; ssl._relevant_attribute" This will fail with `AttributeError` (and hence a non-zero return code) if the relevant capability is not available. The marker attributes are prefixed with an underscore to indicate the implementation dependent nature of these capabilities - not all Python distributions will offer them, only those that are providing a multi-stage migration process from the legacy HTTPS handling to the new default behaviour. Recommendation for an environment variable based security downgrade =================================================================== Some redistributors may wish to provide a per-application option to disable certificate verification in selected applications that run on or embed CPython without needing to modify the application itself. In these cases, a configuration mechanism is needed that provides: * an opt-out model that allows certificate verification to be selectively turned off for particular applications after upgrading to a version of Python that verifies certificates by default * the ability for all users to configure this setting on a per-application basis, rather than on a per-system, or per-Python-installation basis This approach may be used for any redistributor provided version of Python 2.7, including those that advertise themselves as providing Python 2.7.9 or later. Required marker attribute ------------------------- The required marker attribute on the ``ssl`` module when implementing this recommendation is:: _https_verify_envvar = 'PYTHONHTTPSVERIFY' This not only makes it straightforward to detect the presence (or absence) of the capability, it also makes it possible to programmatically determine the relevant environment variable name. Recommended modifications to the Python standard library -------------------------------------------------------- The recommended approach to providing a per-application configuration setting for HTTPS certificate verification that doesn't require modifications to the application itself is to: * modify the ``ssl`` module to read the ``PYTHONHTTPSVERIFY`` environment variable when the module is first imported into a Python process * set the ``ssl._create_default_https_context`` function to be an alias for ``ssl._create_unverified_context`` if this environment variable is present and set to ``'0'`` * otherwise, set the ``ssl._create_default_https_context`` function to be an alias for ``ssl.create_default_context`` as usual Example implementation ---------------------- :: _https_verify_envvar = 'PYTHONHTTPSVERIFY' def _get_https_context_factory(): config_setting = os.environ.get(_https_verify_envvar) if config_setting == '0': return _create_unverified_context return create_default_context _create_default_https_context = _get_https_context_factory() Security Considerations ----------------------- Relative to an unmodified version of CPython 2.7.9 or later, this approach does introduce a new downgrade attack against the default security settings that potentially allows a sufficiently determined attacker to revert Python to the vulnerable configuration used in CPython 2.7.8 and earlier releases. However, such an attack requires the ability to modify the execution environment of a Python process prior to the import of the ``ssl`` module, and any attacker with such access would already be able to modify the behaviour of the underlying OpenSSL implementation. Recommendation for backporting to earlier Python versions ========================================================= Some redistributors, most notably Linux distributions, may choose to backport the PEP 476 HTTPS verification changes to modified Python versions based on earlier Python 2 maintenance releases. In these cases, a configuration mechanism is needed that provides: * an opt-in model that allows the decision to enable HTTPS certificate verification to be made independently of the decision to upgrade to the Python version where the feature was first backported * the ability for system administrators to set the default behaviour of Python applications and scripts run directly in the system Python installation * the ability for the redistributor to consider changing the default behaviour of *new* installations at some point in the future without impacting existing installations that have been explicitly configured to skip verifying HTTPS certificates by default This approach should not be used for any Python installation that advertises itself as providing Python 2.7.9 or later, as most Python users will have the reasonable expectation that all such environments will validate HTTPS certificates by default. Required marker attribute ------------------------- The required marker attribute on the ``ssl`` module when implementing this recommendation is:: _cert_verification_config = '<path to configuration file>' This not only makes it straightforward to detect the presence (or absence) of the capability, it also makes it possible to programmatically determine the relevant configuration file name. Recommended modifications to the Python standard library -------------------------------------------------------- The recommended approach to backporting the PEP 476 modifications to an earlier point release is to implement the following changes relative to the default PEP 476 behaviour implemented in Python 2.7.9+: * modify the ``ssl`` module to read a system wide configuration file when the module is first imported into a Python process * define a platform default behaviour (either verifying or not verifying HTTPS certificates) to be used if this configuration file is not present * support selection between the following three modes of operation: * ensure HTTPS certificate verification is enabled * ensure HTTPS certificate verification is disabled * delegate the decision to the redistributor providing this Python version * set the ``ssl._create_default_https_context`` function to be an alias for either ``ssl.create_default_context`` or ``ssl._create_unverified_context`` based on the given configuration setting. Recommended file location ------------------------- This approach is currently only defined for \*nix system Python installations. The recommended configuration file name is ``/etc/python/cert-verification.cfg``. The ``.cfg`` filename extension is recommended for consistency with the ``pyvenv.cfg`` used by the ``venv`` module in Python 3's standard library. Recommended file format ----------------------- The configuration file should use a ConfigParser ini-style format with a single section named ``[https]`` containing one required setting ``verify``. Permitted values for ``verify`` are: * ``enable``: ensure HTTPS certificate verification is enabled by default * ``disable``: ensure HTTPS certificate verification is disabled by default * ``platform_default``: delegate the decision to the redistributor providing this particular Python version If the ``[https]`` section or the ``verify`` setting are missing, or if the ``verify`` setting is set to an unknown value, it should be treated as if the configuration file is not present. Example implementation ---------------------- :: _cert_verification_config = '/etc/python/cert-verification.cfg' def _get_https_context_factory(): # Check for a system-wide override of the default behaviour context_factories = { 'enable': create_default_context, 'disable': _create_unverified_context, 'platform_default': _create_unverified_context, # For now :) } import ConfigParser config = ConfigParser.RawConfigParser() config.read(_cert_verification_config) try: verify_mode = config.get('https', 'verify') except (ConfigParser.NoSectionError, ConfigParser.NoOptionError): verify_mode = 'platform_default' default_factory = context_factories.get('platform_default') return context_factories.get(verify_mode, default_factory) _create_default_https_context = _get_https_context_factory() Security Considerations ----------------------- The specific recommendations for the backporting case are designed to work for privileged, security sensitive processes, even those being run in the following locked down configuration: * run from a locked down administrator controlled directory rather than a normal user directory (preventing ``sys.path[0]`` based privilege escalation attacks) * run using the ``-E`` switch (preventing ``PYTHON*`` environment variable based privilege escalation attacks) * run using the ``-s`` switch (preventing user site directory based privilege escalation attacks) * run using the ``-S`` switch (preventing ``sitecustomize`` based privilege escalation attacks) The intent is that the *only* reason HTTPS verification should be getting turned off system wide when using this approach is because: * an end user is running a redistributor provided version of CPython rather than running upstream CPython directly * that redistributor has decided to provide a smoother migration path to verifying HTTPS certificates by default than that being provided by the upstream project * either the redistributor or the local infrastructure administrator has determined that it is appropriate to override the default upstream behaviour (at least for the time being) Using an administrator controlled configuration file rather than an environment variable has the essential feature of providing a smoother migration path, even for applications being run with the ``-E`` switch. Combining the recommendations ============================= If a redistributor chooses to implement both recommendations, then the environment variable should take precedence over the system-wide configuration setting. This allows the setting to be changed for a given user, virtual environment or application, regardless of the system-wide default behaviour. In this case, if the ``PYTHONHTTPSVERIFY`` environment variable is defined, and set to anything *other* than ``'0'``, then HTTPS certificate verification should be enabled. Example implementation ---------------------- :: _https_verify_envvar = 'PYTHONHTTPSVERIFY' _cert_verification_config = '/etc/python/cert-verification.cfg' def _get_https_context_factory(): # Check for am environmental override of the default behaviour config_setting = os.environ.get(_https_verify_envvar) if config_setting is not None: if config_setting == '0': return _create_unverified_context return create_default_context # Check for a system-wide override of the default behaviour context_factories = { 'enable': create_default_context, 'disable': _create_unverified_context, 'platform_default': _create_unverified_context, # For now :) } import ConfigParser config = ConfigParser.RawConfigParser() config.read(_cert_verification_config) try: verify_mode = config.get('https', 'verify') except (ConfigParser.NoSectionError, ConfigParser.NoOptionError): verify_mode = 'platform_default' default_factory = context_factories.get('platform_default') return context_factories.get(verify_mode, default_factory) _create_default_https_context = _get_https_context_factory() Copyright ========= This document has been placed into the public domain. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

16 56

Third milestone of FAT Python
by Victor Stinner 07 Dec '15

07 Dec '15

Hi, I implemented 3 new optimizations in FAT Python: loop unrolling, constant folding and copy builtin functions to constants. In the previous thread, Terry Reedy asked me if the test suite is complete enough to ensure that FAT Python doesn't break the Python semantic. I can now say that the second milestone didn't optimize enough code to see bugs, new optimizations helped me to find a *lot* of bugs which are now all fixed. The full Python test suite pass with all optimizations enabled. Only two tests are skipped in FAT Python: test_dynamic and mock tests of test_unittest. test_dynamic checks that it's possible to replace builtin functions in a function and then use the replaced builtins from the same function. FAT Python currently doesn't support the specific case of this test. The mock tests of test_unittest does something similar, I'm more concerned by these failures. This email is an updated version on my previous blog article: https://haypo.github.io/fat-python-status-nov26-2015.html Since I wrote this blog article, I implemented the constant folding optimization and I fixed the two major bugs mentioned in the article (line number and exec(code, dict)). Documentation ============= I combined the documentation of (my) various optimizations projects into a single documentation: http://faster-cpython.readthedocs.org/ The FAT Python project has its own page: http://faster-cpython.readthedocs.org/fat_python.html Constant folding ================ This optimization propagates constant values of variables. Example: def func() x = 1 y = x return y Constant folding: def func() x = 1 y = 1 return 1 This optimization alone is not really exciting. It will more used later when the optimizer will implement peephole optimizations (ex: a+b) and remove dead code. For example, it will be possible to remove code specific to a platform (ex: 'if sys.platform.startswith("freebsd"): ...'). Later, removal of unused local variables will be implemented to simplify the code even more. The previous example will be simplified to: def func(): return 1 Loop unrolling optimization =========================== The optimization generates assignement statements (for the loop index variable) and duplicates the loop body to reduce the cost of loops. Example: def func(): for i in range(2): print(i) Loop unrolled: def func(): i = 0 print(i) i = 1 print(i) If the iterator uses the builtin range function, two guards are required on builtins and globals namespaces. The optimization also handles tuple iterator (ex: "for i in (1, 2, 3): ..."). No guard is needed in this case (the code is always optimized). Loop unrolling combines well with constant folding. The previous example is simplified to: def func(): i = 0 print(0) i = 1 print(1) Again, with a future removal of unused local variables optimization, the previous example will be simplified to: def func(): print(0) print(1) Copy builtins to constants optimization ======================================= This optimization is currently disabled by default. (Well, in practice, it's enabled by the site module to test it and detect code which doesn't work with it.) The LOAD_GLOBAL instruction is used to load a builtin function. The instruction requires two dictionary lookup: one in the globals namespace (which almost always fail) and then in the builtins namespace. It's rare to replace builtins, so the idea here is to replace the dynamic LOAD_GLOBAL instruction with a static LOAD_CONST instruction which loads the function from a C array, a fast O(1) access. It is not possible to inject a builtin function during the compilation. Python code objects are serialized by the marshal module which only support simple types like integers, strings and tuples, not functions. The trick is to modify the constants at runtime when the module is loaded. I added a new patch_constants() method to functions. Example: def log(message): print(message) This function is specialized to: def log(message): 'LOAD_GLOBAL print'(message) log.patch_constants({'LOAD_GLOBAL print': print}) The specialized bytecode uses two guards on builtins and globals namespaces to disable the optimization if the builtin function is replaced. This optimization doesn't support the case when builtins are modified while the function is executed. I think that it will be safer to disable the optimization by default. Later, we can enhance the optimization to enable it if the function cannot modify builtins and if it only calls funcions which cannot modify builtins. I bet that the optimization will be safe with these additional checks. Changes on builtin guards ========================= When a guard is used on a builtin function, the specialization of the function is now ignored if the builtin was replaced or if a function with the name already exists in the globals namespace. At the end of the Python finalization (after the site module is imported), the fat module keeps a private copy of builtins. When a builtin guard is used, the current builtin function is simply compared to the old copy old builtins. The assumption here is that builtin functions are not replaced during Python initialization. By the way, I started to document FAT PYthon limitations and effects on the Python semantic: http://faster-cpython.readthedocs.org/fat_python.html#limitations-and-pytho… Lot of enhancements of the AST optimizer ======================================== New optimizations helped to find bugs in the AST optimizer. Many fixes and various enhancements were done in the AST optimizer. The optimizer was optimized, copy.deepcopy() is no more used to duplicate a full tree. The new NodeTransformer class only duplicates modified nodes. The optimizer understands now much better Python namespaces (globals, locals, non locals, etc.). It is now able to optimize a function without guards: it's used to unroll a loop using a tuple as iterator. Versionned dictionary ===================== In the previous milestone of FAT Python, the versionned dictionary was a new type inherited from the builtin dict type which added a __version__ read-only (global "version" of dictionary, incremented at each change), a getversion(key) method (version of a one key) and it added support for weak references. I done my best to make FAT Python changes optional to leave CPython completly unchanged to not hit performances when the FAT mode is not used. But I had two major technical issues. The first one is that using a different structure for dictionary entries would make the dict code more complex and maybe even slower (which is not acceptable). The second one is that it was no more possible to call exec(code, globals, locals) in FAT mode where globals or locals were a dict. The code needed to be modified using something like: globals = fat.verdict() if __fat__ or {} It required to import the fat module and modify all code calling exec(). I removed the fat.verdict type and added the __version__ property to the builtin dict type. It's incremented at each change. The getversion() method and the support for weak reference is removed. Python has already special code to handle reference cycles of dictionaries, there is no need to support weak references. Guards now use strong references to namespaces. Victor

5 6

Wrong change log link for 3.5.1
by Andrei Fokau 07 Dec '15

07 Dec '15

Hi, The Changelog link on https://www.python.org/downloads/release/python-351/ has wrong object id. It should be https://docs.python.org/3.5/whatsnew/changelog.html#python-3-5-1-final Thanks, Andrei -- Andrei Fokau 5 Monkeys Agency AB Stadsgården 10, 12tr SE-116 45 Stockholm tel: 08-5000 66 53 mob: +46 76 3060 888 skype:andrei.fokau andrei(a)5monkeys.se

2 2

[RELEASED] Python 3.5.1 and 3.4.4rc1 are now available
by Larry Hastings 07 Dec '15

07 Dec '15

On behalf of the Python development community and the Python 3.4 and 3.5 release teams, I'm pleased to announce the simultaneous availability of Python 3.5.1 and Python 3.4.4rc1. As point releases, both have many incremental improvements over their predecessor releases. You can find Python 3.5.1 here: https://www.python.org/downloads/release/python-351/ And you can find Python 3.4.4rc1 here: https://www.python.org/downloads/release/python-344rc1/ Python 2.7.11 shipped today too, so it's a Python release-day hat trick! Happy computing, //arry/

2 1