Mailman 3 July 2017 - Python-Dev

Reminder: 3.5.4rc1 will be tagged next Saturday, July 22 2017
by Larry Hastings July 12, 2017

July 12, 2017

Just a quick reminder. I'll be tagging 3.5.4rc1 next Saturday, July 22. 3.5.4 final will be the last release of 3.5.4 that accepts bugfixes; after that, the 3.5 branch will transition to security-fixes-only mode. If you have bugfixes you want to ship with 3.5.4, please get them committed in the next nine days. Happy hacking, //arry/

1 0

Improve test coverage for standard library
by Artem Muterko July 10, 2017

July 10, 2017

Good day, I've noticed that test coverage of standard library tools can be improved and wanted to ask which module is better to start with? I'm kindly asking to point to the module which would be good to start writing tests for and also easy to review for other contributors. Best regards, Artem Muterko

2 1

Re: [Python-Dev] Pure pickle bechmark.
by Victor Stinner July 10, 2017

July 10, 2017

(Oops, I didn't notice that we started to talk off the list, let's discuss that in python-dev please.) I don't see the point of optimizing "pickle/unpickle pure python" benchmark on Python 3. This benchmark doesn't make sense on Python 3, since I don't know anyone using the pure Python pickle. The C accelerator is now used by default. I already proposed to remove this benchmark: https://mail.python.org/pipermail/speed/2017-April/000554.html *but* Antoine Pitrou mentionned that the cloudpickle project uses it. Maybe we should try to understand what's wrong with _pickle (C module) for cloudpickle? Victor 2017-07-10 2:10 GMT+02:00 Bhavishya <bhavishyagopesh(a)gmail.com>: > I was working on the two regressed benchmarks (i.e. pickle/unpickle > pure-python), and as it was a case with other benchmarks....that performance > is affected by import ...I thought that could be a case with pickle.py > too. And thus tried adding the above patch to Lib/pickle.py to measure the > initial import time. > > I haven't tried it for any practical use-case. > > > On Mon, Jul 10, 2017 at 5:27 AM, Victor Stinner <victor.stinner(a)gmail.com> > wrote: >> >> Sorry, I don't understand the direct link between the import time of 4 >> modules and the pickle module. Can you please elaborate? >> >> What are you trying to optimize? >> >> What is your use case? >> >> Victor

2 1

Pure pickle bechmark.
by Bhavishya July 9, 2017

July 9, 2017

Hello, 1).I was going through the code of *python pickle* to search any optimization possibility.But the only thing that I found very alarming was again the import time(I tried with lazy-import but it didn't helped much.) I found py3 to be ~45 times slower on* initial imports(very raw measure..using "time." ) *as compared to py2 on an usual example. py3-> ./python -c ' favorite_color = { "lion": "yellow", "kitty": "red" } pickle.dump( favorite_color, open( "save.p", "wb" ) )' 0.009715557098388672(time taken to do initial imports...measured using *time.time()* ) py2-> ./python -c ' favorite_color = { "lion": "yellow", "kitty": "red" } pickle.dump( favorite_color, open( "save.p", "wb" ) )' 0.000236034393311(time taken to do initial imports...measured using *time.time()* ) Do you have any thought/ideas on improving this? Thank You.

5 6

[RELEASE] Python 3.6.2rc2 is now available for testing
by Ned Deily July 8, 2017

July 8, 2017

On behalf of the Python development community and the Python 3.6 release team, I would like to announce the availability of Python 3.6.2rc2. 3.6.2rc2 is the second release candidate for Python 3.6.2, the next maintenance release of Python 3.6. 3.6.2rc2 includes fixes for three security-related issues resolved since the previous release candidate; see the change log (link below). While 3.6.2rc2 is a preview release and, thus, not intended for production environments, we encourage you to explore it and provide feedback via the Python bug tracker (https://bugs.python.org). Please see "What’s New In Python 3.6" for more information: https://docs.python.org/3.6/whatsnew/3.6.html You can find Python 3.6.2rc2 here: https://www.python.org/downloads/release/python-362rc2/ and its change log here: https://docs.python.org/3.6/whatsnew/changelog.html#python-3-6-2-release-ca… 3.6.2 is now planned for final release on 2017-07-17 with the next maintenance release expected to follow in about 3 months. More information about the 3.6 release schedule can be found here: https://www.python.org/dev/peps/pep-0494/ -- Ned Deily nad(a)python.org -- []

1 0

Summary of Python tracker Issues
by Python tracker July 7, 2017

July 7, 2017

ACTIVITY SUMMARY (2017-06-30 - 2017-07-07) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 6021 (+15) closed 36604 (+39) total 42625 (+54) Open issues with patches: 2344 Issues opened (48) ================== #23644: g++ module compile fails with ‘_Atomic’ does not name a ty http://bugs.python.org/issue23644 reopened by haypo #27945: Various segfaults with dict http://bugs.python.org/issue27945 reopened by ned.deily #30819: Linking with 'ld -b' fails with 64-bit using Itanium HP compil http://bugs.python.org/issue30819 opened by Robert Boehne #30820: email.contentmanager.raw_data_manager fails to create multipar http://bugs.python.org/issue30820 opened by elenril #30821: unittest.mock.Mocks with specs aren't aware of default argumen http://bugs.python.org/issue30821 opened by Max Rothman #30822: Python implementation of datetime module is not being tested c http://bugs.python.org/issue30822 opened by musically_ut #30823: os.startfile("") craches Python 2.7, 3.4 in Windows 7 32 bit i http://bugs.python.org/issue30823 opened by mikeee #30824: Add mimetype for extension .json http://bugs.python.org/issue30824 opened by quentel #30825: csv.Sniffer does not detect lineterminator http://bugs.python.org/issue30825 opened by vmax #30826: More details in reference 'Looping through a list in Python an http://bugs.python.org/issue30826 opened by the_darklord #30828: Out of bounds write in _asyncio_Future_remove_done_callback http://bugs.python.org/issue30828 opened by Ned Williamson #30830: HTTPHandlerTest of test_logging leaks a "dangling" thread on A http://bugs.python.org/issue30830 opened by haypo #30831: Inconsistent or wrong documentation around Asynchronous Contex http://bugs.python.org/issue30831 opened by dmiyakawa #30833: UnloadUserprofile displays the error "The handle is invalid" http://bugs.python.org/issue30833 opened by d doe #30834: Warning -- files was modified by test_import, After: ['@test http://bugs.python.org/issue30834 opened by haypo #30835: AttributeError when parsing multipart email with invalid non-d http://bugs.python.org/issue30835 opened by Andrew Donnellan #30836: test_c_locale_coercion fails on AIX http://bugs.python.org/issue30836 opened by haypo #30837: Mac OS High Sierra Beta - Python Crash http://bugs.python.org/issue30837 opened by ayesjm #30839: Larger and/or configurable _MAX_LENGTH for unittest messages http://bugs.python.org/issue30839 opened by maarten-treewalker #30840: Contrary to documentation, relative imports cannot pass throug http://bugs.python.org/issue30840 opened by Malcolm Smith #30841: A shadowing variable naming emitted for Python-ast.c http://bugs.python.org/issue30841 opened by OswinC #30842: pyenv activate for bash and tcsh http://bugs.python.org/issue30842 opened by PyAcrisel #30844: selectors: Add urgent data to read event http://bugs.python.org/issue30844 opened by pklanke #30845: [3.5] test_first_completed_some_already_completed() of test_co http://bugs.python.org/issue30845 opened by haypo #30846: [3.6] test_rapid_restart() of test_multiprocessing_fork fails http://bugs.python.org/issue30846 opened by haypo #30847: asyncio: selector_events: add_urgent() for urgent data to read http://bugs.python.org/issue30847 opened by pklanke #30848: test_multiprocessing_forkserver hangs on AMD64 FreeBSD CURRENT http://bugs.python.org/issue30848 opened by haypo #30849: test_stress_delivery_dependent() of test_signal randomly fails http://bugs.python.org/issue30849 opened by haypo #30850: [2.7] bsddb3: test01_basic_replication() of test_bsddb3 fails http://bugs.python.org/issue30850 opened by haypo #30851: IDLE: configdialog -- fix tkinter Variables http://bugs.python.org/issue30851 opened by terry.reedy #30852: _PyObject_GC_UNTRACK corruption when call a lambda function wi http://bugs.python.org/issue30852 opened by 盛茂家 #30853: IDLE: configdialog -- factor out Variable subclass http://bugs.python.org/issue30853 opened by terry.reedy #30856: unittest.TestResult.addSubTest should be called immediately af http://bugs.python.org/issue30856 opened by sir-sigurd #30857: test_bsddb3 hangs longer than 37 minutes on x86 Tiger 2.7 http://bugs.python.org/issue30857 opened by haypo #30858: Keyword can't be an expression? http://bugs.python.org/issue30858 opened by veky #30859: Can't install Python for Windows 3.6.1 on multiple profiles http://bugs.python.org/issue30859 opened by Joe Jacobs #30860: Consolidate stateful C globals under a single struct. http://bugs.python.org/issue30860 opened by eric.snow #30861: StreamReader does not return reamaing and ready data buffer be http://bugs.python.org/issue30861 opened by pfreixes #30863: Rewrite PyUnicode_AsWideChar() and PyUnicode_AsWideCharString( http://bugs.python.org/issue30863 opened by serhiy.storchaka #30864: Compile failure for linux socket CAN support http://bugs.python.org/issue30864 opened by Riccardo Magliocchetti #30865: python cannot import module located on a "VOLUME" directory http://bugs.python.org/issue30865 opened by apre #30866: Add _testcapi.stack_pointer() to measure the C stack consumpti http://bugs.python.org/issue30866 opened by haypo #30867: Add necessary macro that insure `HAVE_OPENSSL_VERIFY_PARAM` to http://bugs.python.org/issue30867 opened by signal1587 #30868: IDLE: Improve configuration tests with mock Save. http://bugs.python.org/issue30868 opened by terry.reedy #30869: regrtest: Add .idlerc to saved_test_environment http://bugs.python.org/issue30869 opened by louielu #30870: IDLE: configdialog/fonts: change font when select by key up/do http://bugs.python.org/issue30870 opened by louielu #30871: Add a "python info" command somewhere to dump versions of all http://bugs.python.org/issue30871 opened by haypo #30872: Update curses docs to Python 3 http://bugs.python.org/issue30872 opened by serhiy.storchaka Most recent 15 issues with no replies (15) ========================================== #30872: Update curses docs to Python 3 http://bugs.python.org/issue30872 #30870: IDLE: configdialog/fonts: change font when select by key up/do http://bugs.python.org/issue30870 #30868: IDLE: Improve configuration tests with mock Save. http://bugs.python.org/issue30868 #30867: Add necessary macro that insure `HAVE_OPENSSL_VERIFY_PARAM` to http://bugs.python.org/issue30867 #30864: Compile failure for linux socket CAN support http://bugs.python.org/issue30864 #30863: Rewrite PyUnicode_AsWideChar() and PyUnicode_AsWideCharString( http://bugs.python.org/issue30863 #30856: unittest.TestResult.addSubTest should be called immediately af http://bugs.python.org/issue30856 #30853: IDLE: configdialog -- factor out Variable subclass http://bugs.python.org/issue30853 #30852: _PyObject_GC_UNTRACK corruption when call a lambda function wi http://bugs.python.org/issue30852 #30851: IDLE: configdialog -- fix tkinter Variables http://bugs.python.org/issue30851 #30846: [3.6] test_rapid_restart() of test_multiprocessing_fork fails http://bugs.python.org/issue30846 #30842: pyenv activate for bash and tcsh http://bugs.python.org/issue30842 #30841: A shadowing variable naming emitted for Python-ast.c http://bugs.python.org/issue30841 #30833: UnloadUserprofile displays the error "The handle is invalid" http://bugs.python.org/issue30833 #30831: Inconsistent or wrong documentation around Asynchronous Contex http://bugs.python.org/issue30831 Most recent 15 issues waiting for review (15) ============================================= #30872: Update curses docs to Python 3 http://bugs.python.org/issue30872 #30867: Add necessary macro that insure `HAVE_OPENSSL_VERIFY_PARAM` to http://bugs.python.org/issue30867 #30863: Rewrite PyUnicode_AsWideChar() and PyUnicode_AsWideCharString( http://bugs.python.org/issue30863 #30860: Consolidate stateful C globals under a single struct. http://bugs.python.org/issue30860 #30828: Out of bounds write in _asyncio_Future_remove_done_callback http://bugs.python.org/issue30828 #30817: Abort in PyErr_PrintEx() when no memory http://bugs.python.org/issue30817 #30814: Import dotted name as alias breaks with concurrency http://bugs.python.org/issue30814 #30808: Use _Py_atomic API for concurrency-sensitive signal state http://bugs.python.org/issue30808 #30747: _Py_atomic_* not actually atomic on Windows with MSVC http://bugs.python.org/issue30747 #30714: test_ssl fails with openssl 1.1.0f http://bugs.python.org/issue30714 #30711: getaddrinfo invalid port number http://bugs.python.org/issue30711 #30710: getaddrinfo raises OverflowError http://bugs.python.org/issue30710 #30696: infinite loop in PyRun_InteractiveLoopFlags() http://bugs.python.org/issue30696 #30695: add a nomemory_allocator to the _testcapi module http://bugs.python.org/issue30695 #30693: tarfile add uses random order http://bugs.python.org/issue30693 Top 10 most discussed issues (10) ================================= #30822: Python implementation of datetime module is not being tested c http://bugs.python.org/issue30822 29 msgs #30302: Improve .__repr__ implementation for datetime.timedelta http://bugs.python.org/issue30302 14 msgs #30861: StreamReader does not return reamaing and ready data buffer be http://bugs.python.org/issue30861 12 msgs #29796: [2.7] test_weakref hangs on Python 2.7 on Windows http://bugs.python.org/issue29796 9 msgs #23644: g++ module compile fails with ‘_Atomic’ does not name a ty http://bugs.python.org/issue23644 8 msgs #29854: Segfault when readline history is more then 2 * history size http://bugs.python.org/issue29854 8 msgs #30779: IDLE: configdialog -- factor out Changes class http://bugs.python.org/issue30779 8 msgs #30844: selectors: Add urgent data to read event http://bugs.python.org/issue30844 8 msgs #30847: asyncio: selector_events: add_urgent() for urgent data to read http://bugs.python.org/issue30847 8 msgs #30814: Import dotted name as alias breaks with concurrency http://bugs.python.org/issue30814 7 msgs Issues closed (40) ================== #6691: Support for nested classes and function for pyclbr http://bugs.python.org/issue6691 closed by terry.reedy #19325: _osx_support imports many modules http://bugs.python.org/issue19325 closed by haypo #20042: Python Launcher, Windows, fails on scripts w/ non-latin names http://bugs.python.org/issue20042 closed by terry.reedy #20669: OpenBSD: socket.recvmsg tests fail with OSError: [Errno 40] Me http://bugs.python.org/issue20669 closed by haypo #29293: Missing parameter "n" on multiprocessing.Condition.notify() http://bugs.python.org/issue29293 closed by pitrou #30259: Test somehow that generated files are up to date: run make reg http://bugs.python.org/issue30259 closed by haypo #30315: test_ftplib.TestTLS_FTPClass: "[Errno 54] Connection reset by http://bugs.python.org/issue30315 closed by haypo #30319: Change socket.close() to ignore ECONNRESET http://bugs.python.org/issue30319 closed by pitrou #30328: test_ssl.test_connect_with_context(): ConnectionResetError on http://bugs.python.org/issue30328 closed by haypo #30330: test_socket.test_idna(): socket.gaierror: [Errno 11001] getadd http://bugs.python.org/issue30330 closed by haypo #30351: [2.7] regrtest hangs on Python 2.7 (test_threading?) http://bugs.python.org/issue30351 closed by haypo #30371: test_long_lines() fails randomly on AMD64 Windows7 SP1 3.x http://bugs.python.org/issue30371 closed by haypo #30441: os.environ raises RuntimeError: dictionary changed size during http://bugs.python.org/issue30441 closed by serhiy.storchaka #30448: test_subprocess creates a core dump on FreeBSD http://bugs.python.org/issue30448 closed by haypo #30532: email.policy.SMTP.fold() mangles long headers http://bugs.python.org/issue30532 closed by r.david.murray #30543: test_timeout fails on AMD64 FreeBSD CURRENT Debug 3.x: Connect http://bugs.python.org/issue30543 closed by haypo #30623: python-nightly import numpy fails since recently http://bugs.python.org/issue30623 closed by ned.deily #30649: test_utime_current_old() of test_os fails randomy on x86 Windo http://bugs.python.org/issue30649 closed by haypo #30651: test_poplib.test_stls_context() access violation on x86 Window http://bugs.python.org/issue30651 closed by haypo #30652: test_threading_not_handled() of test_socketserver hangs random http://bugs.python.org/issue30652 closed by haypo #30703: Non-reentrant signal handler (test_multiprocessing_forkserver http://bugs.python.org/issue30703 closed by pitrou #30726: [Windows] Warnings in elementtree due to new expat http://bugs.python.org/issue30726 closed by haypo #30739: pypi ssl errors [CERTIFICATE_VERIFY_FAILED] http://bugs.python.org/issue30739 closed by ned.deily #30741: https://www.pypi-mirrors.org/ error 503 http://bugs.python.org/issue30741 closed by ned.deily #30758: regrtest hangs sometimes on the master branch (test_pydoc? tes http://bugs.python.org/issue30758 closed by haypo #30759: [2.7] Fix python2 -m test --list-cases test_multibytecodec_sup http://bugs.python.org/issue30759 closed by haypo #30777: IDLE: configdialog -- add docstrings and improve comments http://bugs.python.org/issue30777 closed by terry.reedy #30789: Redesign PyCodeObject.co_extras to use a single memory block, http://bugs.python.org/issue30789 closed by serhiy.storchaka #30791: tkinter.Tk() adds suffix to window class name when launching m http://bugs.python.org/issue30791 closed by serhiy.storchaka #30795: OS X failures in test_site http://bugs.python.org/issue30795 closed by ned.deily #30804: bolen-dmg-3.x build-installer.py failed http://bugs.python.org/issue30804 closed by haypo #30818: Warning -- asyncore.socket_map was modified by test_ftplib on http://bugs.python.org/issue30818 closed by haypo #30827: Tweak order of links in https://www.python.org/downloads/sourc http://bugs.python.org/issue30827 closed by berker.peksag #30829: 'Cannot serialize socket object' after ssl.wrap_socket http://bugs.python.org/issue30829 closed by pitrou #30832: Remove own implementation for thread-local storage http://bugs.python.org/issue30832 closed by haypo #30838: re \w does not match some valid Unicode characters http://bugs.python.org/issue30838 closed by davidism #30843: [2.7] Lib/test/bisect.py conflicts with Lib/bisect.py when run http://bugs.python.org/issue30843 closed by haypo #30854: Compile error on Python/ceval.c without threads http://bugs.python.org/issue30854 closed by haypo #30855: [3.5] test_tk: test_use() of test_tkinter.test_widgets randoml http://bugs.python.org/issue30855 closed by haypo #30862: parent logger should also check the level http://bugs.python.org/issue30862 closed by vinay.sajip

1 0

64 bit units in PyLong
by Siyuan Ren July 5, 2017

July 5, 2017

The current PyLong implementation represents arbitrary precision integers in units of 15 or 30 bits. I presume the purpose is to avoid overflow in addition , subtraction and multiplication. But compilers these days offer intrinsics that allow one to access the overflow flag, and to obtain the result of 64 bit multiplication as a 128 bit number. Or at least on x86-64, which is the dominant platform. Any reason why it is not done? If it is only because no one bothers, I may be able to do it.

6 5

Star the CPython GitHub project if you like Python!
by Victor Stinner July 4, 2017

July 4, 2017

Hi, GitHub has a showcase page of hosted programming languages: https://github.com/showcases/programming-languages Python is only #11 with 8,539 stars, behind PHP and Ruby! Hey, you should "like" ("star"?) the CPython project if you like Python! https://github.com/python/cpython/ Click on "Star" at the top right. Thank you! Victor

4 4

New work-in-progress bisection tool for the Python test suite (in regrtest)
by Victor Stinner July 4, 2017

July 4, 2017

Hi, Last weeks, I worked on a new tool to bisect failing tests because it's painful to bisect manually reference leaks (I remove as much code as possible until the code is small enough to be reviewable manually). See the bisect_test.py script attached to this issue: http://bugs.python.org/issue29512 With the help of Louie Lu, I added new --list-cases option to "python -m test", so you can now list all test cases and write it into a text file: ./python -m test --list-cases test_os > tests I also added a new --matchfile option, to filter tests using a text file which contains one pattern per line: ./python -m test --matchfile=tests test_os fnmatch is used to match test names, so "*" joker character can be used in test names. My bisection tool takes a text file with the --matchfile format (one pattern per line) and creates a random subset of tests with half of the tests. If tests still fail, use the subset. Otherwise, create a new random subset. Loop until the subset contains a single test (configurable threshold, -n command line option). The long term plan is to integrate the bisection feature directly into regrtest. Right now, my script is hardcoded to bisect reference leak bugs, but it should be easy to modify it to bisect other test issues like test creating files without removing it ("ENV_CHANGED" failure in regrtest). For example, a core file is dumped when running test_subprocess on FreeBSD buildbots: http://bugs.python.org/issue30448 But I'm unable to reproduce the issue on my FreeBSD. It would be nice to be able to automate the bisection on the buildbot directly. --list-cases and --matchfile options are now available in 2.7, 3.5, 3.6 and master (3.7) branches. TODO: doctest tests are only partially supported, see: http://bugs.python.org/issue30683 Victor

8 20

Buildbot report (almost July)
by Victor Stinner July 4, 2017

July 4, 2017

Hi, Our buildbots are now even more stable than in my previous buildbot report. Many random failures have been fixed, even if there are still many rare random failures (most of them are related to multiprocessing). Search for issues created by "haypo" (me!) with the Component "Tests" to see all remaining issues. I'm trying to open an issue for *each* buildbot failure! So yeah, we have a long list of 44 open issues! Correct me if I'm wrong, but, for the first time, *all reference leaks* have been fixed on *all branches* (2.7, 3.5, 3.6 and master), on *Linux and Windows*! Before, we mostly focused on the master branch (called "default" in Mercurial) on Linux. I also started to fix a few "memory block" leaks, most (or all?) of them should also be fixed (on all branches, on Linux and Windows). A new policy has been decided (on python-committers): if a commit breaks too many buildbots and cannot fixed easily nor quickly, the commit will be reverted, just to keep our buildbots "green". It doesn't mean that the commit will be rejected forever. It's a practical solution to give more time to write a proper fix, take time to review it, and not have to be stressed to fix buildbots "ASAP". Moreover, keeping green buildbots all the time allows to catch regressions more quickly, which ease debug in many ways. You have be warned! Now I will not hesitate to revert your change if you break my little buildbots ;-) I mostly care of Linux, Windows, macOS and FreeBSD (10 and CURRENT). Breaking other platforms is less important, since other platforms already have issues, but not enough developers interested to fix them. Obviously, I'm interested in keeping more buildbots green, if someone shows up and help me to fix remaining issues! Victor

4 8