Mailman 3 June 2017 - Python-Dev

RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7
by Victor Stinner 10 Jun '17

10 Jun '17

Hi, I wrote a PEP based on the previous thread "Backport ssl.MemoryBIO on Python 2.7?". Thanks for Cory Benfield, Alex Gaynor and Nick Coghlan who helped me to write it! HTML version: https://www.python.org/dev/peps/pep-0546/ Inline verison below. Victor PEP: 546 Title: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7 Version: $Revision$ Last-Modified: $Date$ Author: Victor Stinner <victor.stinner(a)gmail.com>, Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 30-May-2017 Abstract ======== Backport the ssl.MemoryBIO and ssl.SSLObject classes from Python 3 to Python 2.7 to enhance the overall security of Python 2.7. Rationale ========= While Python 2.7 is getting closer to its end-of-support date (scheduled for 2020), it is still used on production systems and the Python community is still responsible for its security. This PEP will help facilitate the future adoption of :pep:`543` across all supported Python versions, which will improve security for both Python 2 and Python 3 users. This PEP does NOT propose a general exception for backporting new features to Python 2.7 - every new feature proposed for backporting will still need to be justified independently. In particular, it will need to be explained why relying on an independently updated backport on the Python Package Index instead is not an acceptable solution. PEP 543 ------- :pep:`543` defines a new TLS API for Python which would enhance Python security by giving Python applications access to the native TLS implementations on Windows and macOS, instead of using OpenSSL. A side effect is that it gives access to the system trust store and certificates installed locally by system administrators, enabling Python applications to use "company certificates" without having to modify each application and so to correctly validate TLS certificates (instead of having to ignore or bypass TLS certificate validation). For practical reasons, Cory Benfield would like to first implement an I/O-less class similar to ssl.MemoryBIO and ssl.SSLObject for :pep:`543`, and to provide a second class based on the first one to use sockets or file descriptors. This design would help to structure the code to support more backends and simplify testing and auditing, as well as implementation. Later, optimized classes using directly sockets or file descriptors may be added for performance. While :pep:`543` defines an API, the PEP would only make sense if it comes with at least one complete and good implementation. The first implementation would ideally be based on the ``ssl`` module of the Python standard library, as this is shipped to all users by default and can be used as a fallback implementation in the absence of anything more targetted. If this backport is not performed, the only baseline implementation that could be used would be pyOpenSSL. This is problematic, however, because of the interaction with pip, which is shipped with CPython on all supported versions. requests, pip and ensurepip --------------------------- There are plans afoot to look at moving Requests to a more event-loop-y model, and doing so basically mandates a MemoryBIO. In the absence of a Python 2.7 backport, Requests is required to basically use the same solution that Twisted currently does: namely, a mandatory dependency on `pyOpenSSL <https://pypi.python.org/pypi/pyOpenSSL>`_. The `pip <https://pip.pypa.io/>`_ program has to embed all its dependencies for practical reasons: namely, that it cannot rely on any other installation method being present. Since pip depends on requests, it means that it would have to embed a copy of pyOpenSSL. That would imply substantial usability pain to install pip. Currently, pip doesn't support embedding C extensions which must be compiled on each platform and so require a C compiler. Since Python 2.7.9, Python embeds a copy of pip both for default installation and for use in virtual environments via the new ``ensurepip`` module. If pip ends up bundling PyOpenSSL, then CPython will end up bundling PyOpenSSL. Only backporting ``ssl.MemoryBIO`` and ``ssl.SSLObject`` would avoid the need to embed pyOpenSSL, and would fix the bootstrap issue (python -> ensurepip -> pip -> requests -> MemoryBIO). Changes ======= Add ``MemoryBIO`` and ``SSLObject`` classes to the ``ssl`` module of Python 2.7. The code will be backported and adapted from the master branch (Python 3). The backport also significantly reduced the size of the Python 2/Python 3 difference of the ``_ssl`` module, which make maintenance easier. Links ===== * :pep:`543` * `[backport] ssl.MemoryBIO <https://bugs.python.org/issue22559>`_: Implementation of this PEP written by Alex Gaynor (first version written at October 2014) * :pep:`466` Discussions =========== * `[Python-Dev] Backport ssl.MemoryBIO on Python 2.7? <https://mail.python.org/pipermail/python-dev/2017-May/147981.html>`_ (May 2017) Copyright ========= This document has been placed in the public domain.

16 105

2017 Python Language Summit coverage finale
by Jake Edge 10 Jun '17

10 Jun '17

Hola python-dev, I (finally!) finished up the coverage from the language summit ... The starting point is the overview article, here: https://lwn.net/Articles/723251/ which should now be free for anyone to see (and the first seven articles too). LWN subscribers can see the articles right away, but one week after they are published in the weekly edition, they become freely available for everyone. Until then, though, feel free to share the SubscriberLinks I am posting here. I have been asked about our policy on appropriate places to share SubscriberLinks; blogs, tweets, social media, mailing lists, etc. are all perfectly fine with us. The new articles are: Classes and types in the Python typing module https://lwn.net/Articles/724639/ or https://lwn.net/SubscriberLink/724639/b055ef642e22abfc/ Status of mypy and type checking https://lwn.net/Articles/724740/ or https://lwn.net/SubscriberLink/724740/5d0b0158880097cf/ Language summit lightning talks https://lwn.net/Articles/723823/ or https://lwn.net/SubscriberLink/723823/695e5554c4d68f1b/ Thanks again to Larry and Barry for inviting me! jake -- Jake Edge - LWN - jake(a)lwn.net - http://lwn.net

3 3

Extracting python bytecode from a linux core dump?
by CompuTinker 09 Jun '17

09 Jun '17

I hope this is the proper place for internals questions... I have a core file (produced via the gcore command) of a linux python2.6 process. I need to extract the byte code and de-compile it. I looked at https://wiki.python.org/moin/DebuggingWithGdb and related pages. However, these all seem to require either a running process and/or a binary with debugging symbols. I'm thinking that the compiled bytecode is likely in an array or contiguous set of memory within the python executable's image and that there's probably a way to pull it out with gdb. Unsurprisingly, the pyc 0xd1f20d0a magic number isn't kept in memory. So, how do I find the memory holding the compiled byte-code ?

3 2

Summary of Python tracker Issues
by Python tracker 09 Jun '17

09 Jun '17

ACTIVITY SUMMARY (2017-06-02 - 2017-06-09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 6035 (+10) closed 36335 (+51) total 42370 (+61) Open issues with patches: 2369 Issues opened (40) ================== #30555: _io._WindowsConsoleIO breaks in the face of fd redirection http://bugs.python.org/issue30555 opened by Segev Finer #30556: asyncio.wait very slow with FIRST_COMPLETED http://bugs.python.org/issue30556 opened by samuelcolvin #30560: Py_SetFatalErrorAbortFunc: Allow embedding program to handle f http://bugs.python.org/issue30560 opened by thomas.perl #30561: sync-up gammavariate and expovariate code http://bugs.python.org/issue30561 opened by leodema #30563: multiprocessing module with pool object issue http://bugs.python.org/issue30563 opened by Julien Verger #30565: PEP 538: default to skipping warning for implicit locale coerc http://bugs.python.org/issue30565 opened by ncoghlan #30566: punycode codec raises IndexError in decode_generalized_number( http://bugs.python.org/issue30566 opened by Vikram Hegde #30569: Tutorial section 2.1 has *nix example at 3.7, but Windows at 3 http://bugs.python.org/issue30569 opened by csabella #30570: issubclass segfaults on objects with weird __getattr__ http://bugs.python.org/issue30570 opened by Daniel Lepage #30571: Add integer formatting code for fixed-width signed arithmetic http://bugs.python.org/issue30571 opened by rhettinger #30576: http.server should support HTTP compression (gzip) http://bugs.python.org/issue30576 opened by quentel #30578: Misleading example in sys.set_coroutine_wrapper docs http://bugs.python.org/issue30578 opened by ncoghlan #30579: Allow traceback objects to be instantiated/mutated/annotated http://bugs.python.org/issue30579 opened by njs #30581: os.cpu_count() returns wrong number of processors on system wi http://bugs.python.org/issue30581 opened by robbuckley #30585: [security][3.3] Backport smtplib fix for TLS stripping vulnera http://bugs.python.org/issue30585 opened by haypo #30586: Encode to EBCDIC doesn't take into account conversion table ir http://bugs.python.org/issue30586 opened by Vladimir Filippov #30587: Mock with spec object does not ensure method call signatures http://bugs.python.org/issue30587 opened by cbelu #30588: Missing documentation for codecs.escape_decode http://bugs.python.org/issue30588 opened by mdartiailh #30589: With forkserver, Process.exitcode does not get signal number http://bugs.python.org/issue30589 opened by pitrou #30593: sqlite3 executescript does not respect isolation_level? http://bugs.python.org/issue30593 opened by Noah Levitt #30594: Refcounting mistake in _ssl.c http://bugs.python.org/issue30594 opened by njs #30595: test_queue_feeder_donot_stop_onexc() of test_multiprocessing_s http://bugs.python.org/issue30595 opened by haypo #30596: Add close() to multiprocessing.Process http://bugs.python.org/issue30596 opened by pitrou #30597: Show expected input in custom "print" error message http://bugs.python.org/issue30597 opened by ncoghlan #30598: Py_NewInterpreter() leaks a reference on warnoptions in _PySys http://bugs.python.org/issue30598 opened by haypo #30599: os.register_at_fork(): allow to unregister callbacks -- test_t http://bugs.python.org/issue30599 opened by haypo #30600: Error message incorrect when index is called with keyword argu http://bugs.python.org/issue30600 opened by SylvainDe #30602: [Windows] os.spawn*() tests of test_os leak references on Wind http://bugs.python.org/issue30602 opened by haypo #30603: textwrap: declining indent level has no test case http://bugs.python.org/issue30603 opened by jonathaneunice #30604: co_extra_freefuncs is stored thread locally and can lead to cr http://bugs.python.org/issue30604 opened by dino.viehland #30605: re.compile fails when compiling bytes under `-bb` mode http://bugs.python.org/issue30605 opened by Roy Williams #30607: Extract documentation theme into a separate package http://bugs.python.org/issue30607 opened by Jon Wayne Parrott #30608: argparse calculates string widths incorrectly http://bugs.python.org/issue30608 opened by Vanessa McHale #30609: Python 3.6.1 fails to generate 256 colors on Cygwin based 64-b http://bugs.python.org/issue30609 opened by rigordo #30610: Python's libexpat vulnerable to CVE-2016-0718 http://bugs.python.org/issue30610 opened by Duy Phan Thanh #30611: Windows HTML Help always opens maximized http://bugs.python.org/issue30611 opened by Christian.Ullrich #30612: Unusual Windows registry path syntax http://bugs.python.org/issue30612 opened by Christian.Ullrich #30613: Distutils register command creates non-standard multipart data http://bugs.python.org/issue30613 opened by Kaeptm Blaubaer #30614: [EASY][2.7] testInitNonExistentFile() of test_bz2 leaks refere http://bugs.python.org/issue30614 opened by haypo #30615: [EASY][2.7] test_recursive_repr() of test_xml_etree_c leaks re http://bugs.python.org/issue30615 opened by haypo Most recent 15 issues with no replies (15) ========================================== #30615: [EASY][2.7] test_recursive_repr() of test_xml_etree_c leaks re http://bugs.python.org/issue30615 #30614: [EASY][2.7] testInitNonExistentFile() of test_bz2 leaks refere http://bugs.python.org/issue30614 #30613: Distutils register command creates non-standard multipart data http://bugs.python.org/issue30613 #30612: Unusual Windows registry path syntax http://bugs.python.org/issue30612 #30607: Extract documentation theme into a separate package http://bugs.python.org/issue30607 #30603: textwrap: declining indent level has no test case http://bugs.python.org/issue30603 #30593: sqlite3 executescript does not respect isolation_level? http://bugs.python.org/issue30593 #30589: With forkserver, Process.exitcode does not get signal number http://bugs.python.org/issue30589 #30587: Mock with spec object does not ensure method call signatures http://bugs.python.org/issue30587 #30578: Misleading example in sys.set_coroutine_wrapper docs http://bugs.python.org/issue30578 #30571: Add integer formatting code for fixed-width signed arithmetic http://bugs.python.org/issue30571 #30569: Tutorial section 2.1 has *nix example at 3.7, but Windows at 3 http://bugs.python.org/issue30569 #30565: PEP 538: default to skipping warning for implicit locale coerc http://bugs.python.org/issue30565 #30563: multiprocessing module with pool object issue http://bugs.python.org/issue30563 #30561: sync-up gammavariate and expovariate code http://bugs.python.org/issue30561 Most recent 15 issues waiting for review (15) ============================================= #30605: re.compile fails when compiling bytes under `-bb` mode http://bugs.python.org/issue30605 #30604: co_extra_freefuncs is stored thread locally and can lead to cr http://bugs.python.org/issue30604 #30589: With forkserver, Process.exitcode does not get signal number http://bugs.python.org/issue30589 #30560: Py_SetFatalErrorAbortFunc: Allow embedding program to handle f http://bugs.python.org/issue30560 #30519: Add daemon argument to Timer http://bugs.python.org/issue30519 #30512: CAN Socket support for NetBSD http://bugs.python.org/issue30512 #30509: Optimize calling type slots http://bugs.python.org/issue30509 #30502: Fix buffer handling of OBJ_obj2txt http://bugs.python.org/issue30502 #30501: Produce optimized code for boolean conditions http://bugs.python.org/issue30501 #30495: IDLE: modernize textview module http://bugs.python.org/issue30495 #30487: DOC: automatically create a venv and install Sphinx when runni http://bugs.python.org/issue30487 #30485: Element.findall(path, dict) doesn't insert null namespace http://bugs.python.org/issue30485 #30466: Tutorial doesn't explain the use of classes http://bugs.python.org/issue30466 #30465: FormattedValue expressions have wrong lineno and col_offset in http://bugs.python.org/issue30465 #30455: Generate C code from token.py and not vice versa http://bugs.python.org/issue30455 Top 10 most discussed issues (10) ================================= #25910: Fixing links in documentation http://bugs.python.org/issue25910 12 msgs #27321: Email parser creates a message object that can't be flattened http://bugs.python.org/issue27321 9 msgs #30542: test_files() of test_tools.test_unparse.DirectoryTestCase leak http://bugs.python.org/issue30542 8 msgs #30417: [buildbot] Disable the cpu resources on slow buildbots http://bugs.python.org/issue30417 7 msgs #30597: Show expected input in custom "print" error message http://bugs.python.org/issue30597 7 msgs #26103: Contradiction in definition of "data descriptor" between (dott http://bugs.python.org/issue26103 6 msgs #29464: Specialize FASTCALL for functions with positional-only paramet http://bugs.python.org/issue29464 6 msgs #30038: Race condition in how trip_signal writes to wakeup fd http://bugs.python.org/issue30038 6 msgs #30599: os.register_at_fork(): allow to unregister callbacks -- test_t http://bugs.python.org/issue30599 6 msgs #30610: Python's libexpat vulnerable to CVE-2016-0718 http://bugs.python.org/issue30610 6 msgs Issues closed (50) ================== #19180: some RFC references could be updated http://bugs.python.org/issue19180 closed by ncoghlan #21783: smtpd.py does not allow multiple helo/ehlo commands http://bugs.python.org/issue21783 closed by barry #24755: asyncio.wrap_future undocumented http://bugs.python.org/issue24755 closed by Mariatta #24899: Add an os.path <=> pathlib equivalence table in pathlib docs http://bugs.python.org/issue24899 closed by brett.cannon #25324: Importing tokenize modifies token http://bugs.python.org/issue25324 closed by serhiy.storchaka #25444: Py Launch Icon http://bugs.python.org/issue25444 closed by eryksun #29596: Unfinished sentence in howto/clinic.rst http://bugs.python.org/issue29596 closed by Mariatta #29822: inspect.isabstract does not work on abstract base classes duri http://bugs.python.org/issue29822 closed by serhiy.storchaka #30052: URL Quoting page links to function Bytes instead of defintion http://bugs.python.org/issue30052 closed by Mariatta #30095: HTMLCalendar allow custom classes http://bugs.python.org/issue30095 closed by doerwalter #30177: pathlib.resolve(strict=False) only includes first child http://bugs.python.org/issue30177 closed by steve.dower #30418: test_communicate_epipe() of test_subprocess fails randomly on http://bugs.python.org/issue30418 closed by haypo #30463: Add __slots__ to ABC convenience class http://bugs.python.org/issue30463 closed by serhiy.storchaka #30486: Allow setting cell value http://bugs.python.org/issue30486 closed by serhiy.storchaka #30489: Add CmdLineTest to standard library http://bugs.python.org/issue30489 closed by terry.reedy #30518: Import type aliases from another module http://bugs.python.org/issue30518 closed by Paragape #30520: loggers can't be pickled http://bugs.python.org/issue30520 closed by vinay.sajip #30526: Allow setting line_buffering on existing TextIOWrapper http://bugs.python.org/issue30526 closed by pitrou #30529: Incorrect error messages for invalid whitespaces in f-string s http://bugs.python.org/issue30529 closed by serhiy.storchaka #30530: Descriptors HowTo: Example on function.__get__ needs update http://bugs.python.org/issue30530 closed by Mariatta #30534: error message for incorrect call degraded in 3.7 http://bugs.python.org/issue30534 closed by serhiy.storchaka #30536: [EASY] SubinterpThreadingTests.test_threads_join_2() of test_t http://bugs.python.org/issue30536 closed by haypo #30537: Using PyNumber_AsSsize_t in itertools.islice http://bugs.python.org/issue30537 closed by rhettinger #30538: Functional Programming HOWTO describes one argument itertools. http://bugs.python.org/issue30538 closed by rhettinger #30540: regrtest: add --matchfile option http://bugs.python.org/issue30540 closed by haypo #30544: _io._WindowsConsoleIO.write raises the wrong error when WriteC http://bugs.python.org/issue30544 closed by steve.dower #30546: [EASY][Windows] test_uname_win32_ARCHITEW6432() of test_platfo http://bugs.python.org/issue30546 closed by haypo #30547: [Windows] SubinterpreterTest.test_callbacks_leak() of test_ate http://bugs.python.org/issue30547 closed by haypo #30554: Inaccessible attribute characters_written on OSError instances http://bugs.python.org/issue30554 closed by xiang.zhang #30557: faulthandler does not correctly filter fatal exceptions on Win http://bugs.python.org/issue30557 closed by steve.dower #30558: [Suggestion] Improve documentation for set() API http://bugs.python.org/issue30558 closed by martin.panter #30559: Bugs in Web app http://bugs.python.org/issue30559 closed by mark.dickinson #30562: SSL socket does not respect SO_RCVTIME0 timeouts http://bugs.python.org/issue30562 closed by christian.heimes #30564: Base64 decoding gives incorrect outputs. http://bugs.python.org/issue30564 closed by r.david.murray #30567: Leak in sys.getwindowsversion http://bugs.python.org/issue30567 closed by serhiy.storchaka #30568: README Formatting http://bugs.python.org/issue30568 closed by Mariatta #30572: pip is broken http://bugs.python.org/issue30572 closed by Dingo64 #30573: How to install cmake? http://bugs.python.org/issue30573 closed by Dingo64 #30574: Document why embedding the interpreter can sometimes crash. http://bugs.python.org/issue30574 closed by r.david.murray #30575: Python interpreter crashes on macOS http://bugs.python.org/issue30575 closed by Yibo Wang #30577: Multidimensional comprehensions cannot access class variables http://bugs.python.org/issue30577 closed by zach.ware #30580: WSGI examples raise AttributeError: __exit__ http://bugs.python.org/issue30580 closed by r.david.murray #30582: Incorrect propagation in logging.wheb creating lighters in a c http://bugs.python.org/issue30582 closed by vinay.sajip #30583: docs mention datetuil presumably it should be dateutil http://bugs.python.org/issue30583 closed by Mariatta #30584: test_os fails on non-English (Russian) Windows http://bugs.python.org/issue30584 closed by denis-osipov #30590: str.format no longer accepts unpacked defaultdict for default http://bugs.python.org/issue30590 closed by serhiy.storchaka #30591: textwrap: placeholder backtracking special case lacks test cov http://bugs.python.org/issue30591 closed by serhiy.storchaka #30592: Bad error message 'bool()() takes no keyword arguments' http://bugs.python.org/issue30592 closed by serhiy.storchaka #30601: [Windows] test_winconsoleio leaks references http://bugs.python.org/issue30601 closed by haypo #30606: Spam http://bugs.python.org/issue30606 closed by berker.peksag

1 0

Support for RFC 6920: Naming Things with Hashes?
by Sylvain Bellemare 08 Jun '17

08 Jun '17

Hi, I hope this is the right place to post this kind of question. If not I apologize. I was simply wondering if anyone had been looking into supporting RFC 6920 ( https://tools.ietf.org/html/rfc6920). For a simple example of what this is about see https://tools.ietf.org/html/rfc6920#section-8 Thanks! Sylvain

2 2

Buildbot report, june 2017
by Victor Stinner 08 Jun '17

08 Jun '17

Hi, With the help of Zachary Ware, we succeed to set up a mailing list getting email notifications when a buildbot fails whereas it previously succeeded (green/success or orange/warnings): https://mail.python.org/mm3/mailman3/lists/buildbot-status.python.org/ Please subscribe if you want to stay tuned. I started to create bug reports for each buildbot failures on 2.7, 3.5, 3.6 and master (future 3.7) branches. I try to reply to buildbot emails with a reference to the issue (I create a new issue if needed). The good news is that these issues are being slowly fixed one by one! In general, buildbots are now stable and at least able to catch the obvious regressions. Zachary Ware added two new "Refleaks" buildbot slaves, on Windows 8 and Gentoo, to catch reference leaks. We are fixing leaks. It seems like some of them are recent. Previously, we add a server ran by Antoine Pitrou, but Antoine didn't update its configuration for Git (when CPython migrated from Mercurial). I like the idea of an integrated "Refleaks" server in our buildbot farm. Information on CPython buildbots: https://www.python.org/dev/buildbot/ Victor

1 0

PEP 7 and braces { .... } on if
by Victor Stinner 07 Jun '17

07 Jun '17

Hi, I have a question on the CPython coding code for C code, the PEP 7: https://www.python.org/dev/peps/pep-0007/ """ Code structure: (...); braces are strongly preferred but may be omitted where C permits, and they should be formatted as shown: if (mro != NULL) { ... } else { ... } """ This section was updated after the creation of CPython in 1991, so as you may expect, a lot of existing C code doesn't respect this coding style. I'm trying to slowly add "missing" braces and indent as the example *when I have to modify existing code*. The problem is the "strongly preferred" and "omitted where C permits" part. I would like to make the PEP 7 more explicit on that part since in each review, I disagree with Serhiy who wants to omit them. Example: if (func == NULL) return NULL; versus if (func == NULL) { return NULL; } I now prefer the version with braces. It's more verbose, but it prevents bugs when the code is modified later for whatever reasons. IMHO it also makes the code easily to read and to understand. So I would suggest to modify the PEP 7 to *always* require braces for if. I would also suggest to require braces on "for(...) { ... }" and "while(...) { ... }". But only if the code has to be modified, not only to update the coding style. Changes which are pure coding style changes should be rejected to avoid conflicts with other pending pull requests and "code churn". Victor

14 24

The untuned tunable parameter ARENA_SIZE
by Larry Hastings 06 Jun '17

06 Jun '17

When CPython's small block allocator was first merged in late February 2001, it allocated memory in gigantic chunks it called "arenas". These arenas were a massive 256 KILOBYTES apiece. This tunable parameter has not been touched in the intervening 16 years. Yet CPython's memory consumption continues to grow. By the time a current "trunk" build of CPython reaches the REPL prompt it's already allocated 16 arenas. I propose we make the arena size larger. By how much? I asked Victor to run some benchmarks with arenas of 1mb, 2mb, and 4mb. The results with 1mb and 2mb were mixed, but his benchmarks with a 4mb arena size showed measurable (>5%) speedups on ten benchmarks and no slowdowns. What would be the result of making the arena size 4mb? * CPython could no longer run on a computer where at startup it could not allocate at least one 4mb continguous block of memory. * CPython programs would die slightly sooner in out-of-memory conditions. * CPython programs would use more memory. How much? Hard to say. It depends on their allocation strategy. I suspect most of the time it would be < 3mb additional memory. But for pathological allocation strategies the difference could be significant. (e.g: lots of allocs, followed by lots of frees, but the occasional object lives forever, which means that the arena it's in can never be freed. If 1 out of ever 16 256k arenas is kept alive this way, and the objects are spaced out precisely such that now it's 1 for every 4mb arena, max memory use would be the same but later stable memory use would hypothetically be 16x current) * Many programs would be slightly faster now and then, simply because we call malloc() 1/16 as often. What say you? Vote for your favorite color of bikeshed now! //arry/

11 50

Extremely slow test modules
by Antoine Pitrou 03 Jun '17

03 Jun '17

Hi, Is there a reason some of our tests are excruciatingly slow in `-uall` mode? `test_multiprocessing_spawn` is understandable (after all, it will spawn a new executable for each subprocess), but other tests leave me baffled: - test_tools: 7 min 41 sec - test_tokenize: 6 min 23 sec - test_datetime: 6 min 3 sec - test_lib2to3: 5 min 25 sec [excerpt from recent Travis CI logs] Why does datetime, 2to3 or tokenize testing take so long? And do we have so many tools that it should take 7 minutes to run all of them? I must admit, I don't understand how we got to such a point. Regards Antoine.

2 4

PEP 484 update proposal: annotating decorated declarations
by Guido van Rossum 02 Jun '17

02 Jun '17

There's a PR to the peps proposal here: https://github.com/python/peps/pull/242 The full text of the current proposal is below. The motivation for this is that for complex decorators, even if the type checker can figure out what's going on (by taking the signature of the decorator into account), it's sometimes helpful to the human reader of the code to be reminded of the type after applying the decorators (or a stack thereof). Much discussion can be found in the PR. Note that we ended up having `Callable` in the type because there's no rule that says a decorator returns a function type (e.g. `property` doesn't). This is a small thing but I'd like to run it by a larger audience than the core mypy devs who have commented so far. There was a brief discussion on python-ideas (my original <https://mail.python.org/pipermail/python-ideas/2017-April/045548.html>, favorable reply <https://mail.python.org/pipermail/python-ideas/2017-May/045550.html> by Nick, my response <https://mail.python.org/pipermail/python-ideas/2017-May/045557.html>). Credit for the proposal goes to Naomi Seyfer, with discussion by Ivan Levkivskyi and Jukka Lehtosalo. If there's no further debate here I'll merge it into the PEP and an implementation will hopefully appear in the next version of the typing module (also hopefully to be included in CPython 3.6.2 and 3.5.4). Here's the proposed text (wordsmithing suggestions in the PR please): +Decorators +---------- + +Decorators can modify the types of the functions or classes they +decorate. Use the ``decorated_type`` decorator to declare the type of +the resulting item after all other decorators have been applied:: + + from typing import ContextManager, Iterator, decorated_type + from contextlib import contextmanager + + class DatabaseSession: ... + + @decorated_type(Callable[[str], ContextManager[DatabaseSession]]) + @contextmanager + def session(url: str) -> Iterator[DatabaseSession]: + s = DatabaseSession(url) + try: + yield s + finally: + s.close() + +The argument of ``decorated_type`` is a type annotation on the name +being declared (``session``, in the example above). If you have +multiple decorators, ``decorated_type`` must be topmost. The +``decorated_type`` decorator is invalid on a function declaration that +is also decorated with ``overload``, but you can annotate the +implementation of the overload series with ``decorated_type``. + -- --Guido van Rossum (python.org/~guido <http://python.org/%7Eguido>)

8 16