Mailman 3 July 2017 - Python-Dev

Am I allowed to use C++-style // comments?
by Devin Jeanpierre July 26, 2017

July 26, 2017

https://www.python.org/dev/peps/pep-0007/ says two things: > Python versions greater than or equal to 3.6 use C89 with several select C99 features: > [...] > C++-style line comments and also: > Never use C++ style // one-line comments. Which is it? -- Devin

2 2

RELEASED] Python 3.4.7rc1 and Python 3.5.4rc1 are now available
by Larry Hastings July 25, 2017

July 25, 2017

On behalf of the Python development community and the Python 3.4 and Python 3.5 release teams, I'm relieved to announce the availability of Python 3.4.7rc1 and Python 3.5.4rc1. Python 3.4 is now in "security fixes only" mode. This is the final stage of support for Python 3.4. Python 3.4 now only receives security fixes, not bug fixes, and Python 3.4 releases are source code only--no more official binary installers will be produced. Python 3.5.4 will be the final 3.5 release in "bug fix" mode. After 3.5.4 is released, Python 3.5 will also move into "security fixes mode". Both these releases are "release candidates". They should not be considered the final releases, although the final releases should contain only minor differences. Python users are encouraged to test with these releases and report any problems they encounter. You can find Python 3.4.7rc1 here: https://www.python.org/downloads/release/python-347rc1/ And you can find Python 3.5.4rc1 here: https://www.python.org/downloads/release/python-354rc1/ Python 3.4.7 final and Python 3.5.4 final are both scheduled for release on August 6th, 2017. Happy Pythoning, //arry/

2 1

Program runs in 12s on Python 2.7, but 5s on Python 3.5 -- why so much difference?
by Ben Hoyt July 24, 2017

July 24, 2017

Hi folks, (Not entirely sure this is the right place for this question, but hopefully it's of interest to several folks.) A few days ago I posted a note in response to Victor Stinner's articles on his CPython contributions, noting that I wrote a program that ran in 11.7 seconds on Python 2.7, but only takes 5.1 seconds on Python 3.5 (on my 2.5 GHz macOS i7), more than 2x as fast. Obviously this is a Good Thing, but I'm curious as to why there's so much difference. The program is a pentomino puzzle solver, and it works via code generation, generating a ton of nested "if" statements, so I believe it's exercising the Python bytecode interpreter heavily. Obviously there have been some big optimizations to make this happen, but I'm curious what the main improvements are that are causing this much difference. There's a writeup about my program here, with benchmarks at the bottom: http://benhoyt.com/writings/python-pentomino/ This is the generated Python code that's being exercised: https://github.com/benhoyt/python-pentomino/blob/master/generated_solve.py For reference, on Python 3.6 it runs in 4.6 seconds (same on Python 3.7 alpha). This smallish increase from Python 3.5 to Python 3.6 was more expected to me due to the bytecode changing to wordcode in 3.6. I tried using cProfile on both Python versions, but that didn't say much, because the functions being called aren't taking the majority of the time. How does one benchmark at a lower level, or otherwise explain what's going on here? Thanks, Ben

6 8

Is Windows XP still supported on Python 2.7?
by Victor Stinner July 24, 2017

July 24, 2017

Hi, We have a Windows XP buildbot for Python 2.7, run by David Bolen: http://buildbot.python.org/all/builders/x86%20Windows%20XP%202.7/ test_bsddb3 fails randomly on this buildbot: http://bugs.python.org/issue30778 But Windows XP clearly reached its end-of-life, Microsoft doesn't support it anymore. So my question is if it makes sense to spend time on it? We have a rule for new x.y.0 released, but not if a Microsoft Windows support expires during the lifetime of a Python release (2.7 here): https://www.python.org/dev/peps/pep-0011/#microsoft-windows Firefox made great efforts to support Windows XP last years, but they decided to drop support last March with Firefox 52, last release supporting XP and Visa: https://support.mozilla.org/en-US/kb/end-support-windows-xp-and-vista Victor

4 5

dictionaries in Dataframe column
by Katherine Bobrovnik July 24, 2017

July 24, 2017

Hello guys! I've stuck at this: I have pandas Dataframe with a lot of columns. One column contains dictionaries with emoji like {'count': 1, 'name': 'fire'}. My goal is to sort rows of this Dataframe by the number from 'count'. Like first row will be with {'count': 49, 'name': '+1'}, the last - {'count': 1, 'name': 'fire'}, for instance. Help, please :) Best regards, Kateryna

2 1

Python 3.5.4rc1 and 3.4.7rc1 slipping by a day, to July 24 2017
by Larry Hastings July 24, 2017

July 24, 2017

Release engineering for 3.5.4rc1 and 3.4.7rc1 took a lot longer than expected, because this is the first release using "blurb", and it turned out there was a lot of work left to do and a couple dark corners yet to stumble over. 3.5.4rc1 and 3.4.7rc1 will be released Monday, July 24, 2017. The release dates for 3.5.4 final and 3.4.7 final are not expected to change. Sorry about that, //arry/

1 0

startup time repeated? why not daemon
by Jim J. Jewett July 23, 2017

July 23, 2017

I agree that startup time is a problem, but I wonder if some of the pain could be mitigated by using a persistent process. For example, in https://mail.python.org/pipermail/python-dev/2017-July/148664.html Ben Hoyt mentions that the Google Cloud SDK (CLI) team has found it "especially problematic for shell tab completion helpers, because every time you press tab the shell has to load your Python program" Decades ago, I learned to set my editor to vi instead of emacs for similar reasons -- but there was also an emacsclient option that simply opened a new window from an already running emacs process. tab completion seems like the exactly the sort of thing that should be sent to an existing process instead of creating a new one. Is it too hard to create a daemon server? Is the communication and context switch slower than a new startup? Is the pattern just not well-enough advertised? -jJ

13 16

Re: [Python-Dev] Need help to fix urllib(.parse) vulnerabilities
by Victor Stinner July 22, 2017

July 22, 2017

Le 22 juil. 2017 8:04 AM, "Serhiy Storchaka" <storchaka(a)gmail.com> a écrit : I think the only reliable way of fixing the vulnerability is rejecting or escaping (as specified in RFC 2640) CR and LF inside sent lines. Adding the support of RFC 2640 is a new feature and can be added only in 3.7. And this feature should be optional since not all servers support RFC 2640. https://github.com/python/cpython/pull/1214 does the right thing. In that case, I suggest to reject newlines in ftplib, and maybe add an opt-in option to escape newlines. Java just rejected newlines, no? Or does Java allows to escape them? Victor

2 3

Need help to fix urllib(.parse) vulnerabilities
by Victor Stinner July 22, 2017

July 22, 2017

Hi, Recently, two security vulnerabilities were reported in the urllib module: https://bugs.python.org/issue30500 http://python-security.readthedocs.io/vuln/bpo-30500_urllib_connects_to_a_w… => already fixed in Python 3.6.2 https://bugs.python.org/issue29606 http://python-security.readthedocs.io/vuln/urllib_ftp_protocol_stream_injec… => not fixed yet I also proposed a more general protection: "Reject newline character (U+000A) in URLs in urllib.parse": http://bugs.python.org/issue30713 The problem with the urllib module is how we handle invalid URL. Right now, we return the URL unmodified if we cannot parse it. Should we raise an exception if an URL contains a newline for example? It's very hard to harden the urllib module without the backward compatibility. That's why it took 3 weeks to fix "urllib connects to a wrong host": find how to fix the vulnerability without brekaing the backward compatibility. Another proposed approach is to reject invalid data earlier or later, but not in urllib... So if you understand URLs, HTTP, etc. : please join these issues to help us to fix them! Victor

5 5

Summary of Python tracker Issues
by Python tracker July 21, 2017

July 21, 2017

ACTIVITY SUMMARY (2017-07-14 - 2017-07-21) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 6058 (+16) closed 36679 (+38) total 42737 (+54) Open issues with patches: 2343 Issues opened (37) ================== #19896: Exposing "q" and "Q" to multiprocessing.sharedctypes http://bugs.python.org/issue19896 reopened by haypo #30450: Pull Windows dependencies from GitHub rather than svn.python.o http://bugs.python.org/issue30450 reopened by steve.dower #30931: Race condition in asyncore may access the wrong dispatcher http://bugs.python.org/issue30931 opened by walkhour #30934: Document how to run coverage for repository idlelib files. http://bugs.python.org/issue30934 opened by terry.reedy #30935: document the new behavior of get_event_loop() in Python 3.6 http://bugs.python.org/issue30935 opened by chris.jerdonek #30937: csv module examples miss newline='' when opening files http://bugs.python.org/issue30937 opened by Pavel #30938: pdb lacks debugger command to list and show all user-defined v http://bugs.python.org/issue30938 opened by David Rieger #30939: Sphinx 1.6.3 deprecation warning for sphinx.util.compat.Direct http://bugs.python.org/issue30939 opened by ned.deily #30940: Documentation for round() is incorrect. http://bugs.python.org/issue30940 opened by George K #30944: Python 32 bit install fails on Windows - BitDefender false pos http://bugs.python.org/issue30944 opened by Arie van Wingerden #30945: loop.create_server does not detect if the interface is IPv6 en http://bugs.python.org/issue30945 opened by cecton #30947: Update embeded copy of libexpat to 2.2.2 http://bugs.python.org/issue30947 opened by haypo #30949: Provide assertion functions in unittest.mock http://bugs.python.org/issue30949 opened by odd_bloke #30950: Convert round() to Arument Clinic http://bugs.python.org/issue30950 opened by serhiy.storchaka #30951: Documentation error in inspect module http://bugs.python.org/issue30951 opened by jalexvig #30952: include Math extension in SQlite http://bugs.python.org/issue30952 opened by Big Stone #30953: Fatal python error when jumping into except clause http://bugs.python.org/issue30953 opened by ppperry #30956: ftplib behaves oddly if socket timeout is greater than the def http://bugs.python.org/issue30956 opened by arloclarke #30959: Constructor signature is duplicated in the help of namedtuples http://bugs.python.org/issue30959 opened by serhiy.storchaka #30962: Add caching to logging.Logger.isEnabledFor() http://bugs.python.org/issue30962 opened by aviso #30963: xxlimited.c XxoObject_Check should be XxoObject_CheckExact http://bugs.python.org/issue30963 opened by Jim.Jewett #30964: Mention ensurepip in package installation docs http://bugs.python.org/issue30964 opened by ncoghlan #30966: multiprocessing.queues.SimpleQueue leaks 2 fds http://bugs.python.org/issue30966 opened by arigo #30967: Crash in PyThread_ReInitTLS() in the child process after os.fo http://bugs.python.org/issue30967 opened by Thomas Mortensson #30969: Docs should say that `x is z or x == z` is used for `x in y` i http://bugs.python.org/issue30969 opened by ztane #30971: Improve code readability of json.tool http://bugs.python.org/issue30971 opened by dhimmel #30972: Event loop incorrectly inherited in child processes. http://bugs.python.org/issue30972 opened by Elvis.Pranskevichus #30974: Update os.samefile docstring to match documentation http://bugs.python.org/issue30974 opened by eMPee584 #30975: multiprocessing.Condition.notify_all() blocks indefinitely if http://bugs.python.org/issue30975 opened by mickp #30977: reduce uuid.UUID() memory footprint http://bugs.python.org/issue30977 opened by wbolster #30978: str.format_map() silences exceptions in __getitem__ http://bugs.python.org/issue30978 opened by Akuli #30979: the winapi fails to run shortcuts (because considers a shortcu http://bugs.python.org/issue30979 opened by Bernát Gábor #30980: Calling asyncore.file_wrapper.close twice may close unrelated http://bugs.python.org/issue30980 opened by Nir Soffer #30981: IDLE: Add config dialog font page tests http://bugs.python.org/issue30981 opened by terry.reedy #30982: AMD64 Windows8.1 Refleaks 3.x: compilation error, cannot open http://bugs.python.org/issue30982 opened by haypo #30983: eval frame rename in pep 0523 broke gdb's python extension http://bugs.python.org/issue30983 opened by bcap #30984: traceback.print_exc return value documentation http://bugs.python.org/issue30984 opened by Jelle Zijlstra Most recent 15 issues with no replies (15) ========================================== #30984: traceback.print_exc return value documentation http://bugs.python.org/issue30984 #30983: eval frame rename in pep 0523 broke gdb's python extension http://bugs.python.org/issue30983 #30980: Calling asyncore.file_wrapper.close twice may close unrelated http://bugs.python.org/issue30980 #30972: Event loop incorrectly inherited in child processes. http://bugs.python.org/issue30972 #30963: xxlimited.c XxoObject_Check should be XxoObject_CheckExact http://bugs.python.org/issue30963 #30952: include Math extension in SQlite http://bugs.python.org/issue30952 #30950: Convert round() to Arument Clinic http://bugs.python.org/issue30950 #30938: pdb lacks debugger command to list and show all user-defined v http://bugs.python.org/issue30938 #30937: csv module examples miss newline='' when opening files http://bugs.python.org/issue30937 #30935: document the new behavior of get_event_loop() in Python 3.6 http://bugs.python.org/issue30935 #30926: KeyError with cgitb inspecting exception in generator expressi http://bugs.python.org/issue30926 #30925: RPM build lacks ability to include other files similar to doc_ http://bugs.python.org/issue30925 #30924: RPM build doc_files needs files separated into separate lines http://bugs.python.org/issue30924 #30915: distutils sometimes assumes wrong C compiler http://bugs.python.org/issue30915 #30912: python 3 git master fails to find libffi and build _ctypes on http://bugs.python.org/issue30912 Most recent 15 issues waiting for review (15) ============================================= #30978: str.format_map() silences exceptions in __getitem__ http://bugs.python.org/issue30978 #30971: Improve code readability of json.tool http://bugs.python.org/issue30971 #30950: Convert round() to Arument Clinic http://bugs.python.org/issue30950 #30945: loop.create_server does not detect if the interface is IPv6 en http://bugs.python.org/issue30945 #30939: Sphinx 1.6.3 deprecation warning for sphinx.util.compat.Direct http://bugs.python.org/issue30939 #30937: csv module examples miss newline='' when opening files http://bugs.python.org/issue30937 #30919: Shared Array Memory Allocation Regression http://bugs.python.org/issue30919 #30891: importlib: _find_and_load() race condition on sys.modules[name http://bugs.python.org/issue30891 #30877: possibe typo in json/scanner.py http://bugs.python.org/issue30877 #30876: SystemError on importing module from unloaded package http://bugs.python.org/issue30876 #30872: Update curses docs to Python 3 http://bugs.python.org/issue30872 #30867: Add necessary macro that insure `HAVE_OPENSSL_VERIFY_PARAM` to http://bugs.python.org/issue30867 #30863: Rewrite PyUnicode_AsWideChar() and PyUnicode_AsWideCharString( http://bugs.python.org/issue30863 #30860: Consolidate stateful C globals under a single struct. http://bugs.python.org/issue30860 #30828: Out of bounds write in _asyncio_Future_remove_done_callback http://bugs.python.org/issue30828 Top 10 most discussed issues (10) ================================= #28638: Optimize namedtuple creation http://bugs.python.org/issue28638 33 msgs #30450: Pull Windows dependencies from GitHub rather than svn.python.o http://bugs.python.org/issue30450 29 msgs #30940: Documentation for round() is incorrect. http://bugs.python.org/issue30940 11 msgs #30822: Python implementation of datetime module is not being tested c http://bugs.python.org/issue30822 10 msgs #18558: Iterable glossary entry needs clarification http://bugs.python.org/issue18558 9 msgs #30919: Shared Array Memory Allocation Regression http://bugs.python.org/issue30919 9 msgs #30934: Document how to run coverage for repository idlelib files. http://bugs.python.org/issue30934 9 msgs #19896: Exposing "q" and "Q" to multiprocessing.sharedctypes http://bugs.python.org/issue19896 8 msgs #25988: collections.abc.Indexable http://bugs.python.org/issue25988 8 msgs #30981: IDLE: Add config dialog font page tests http://bugs.python.org/issue30981 7 msgs Issues closed (37) ================== #11230: "Full unicode import system" not in 3.2 http://bugs.python.org/issue11230 closed by serhiy.storchaka #23267: multiprocessing pool.py doesn't close inqueue and outqueue pip http://bugs.python.org/issue23267 closed by pitrou #26781: os.walk max_depth http://bugs.python.org/issue26781 closed by rhettinger #28523: Idlelib.configdialog: use 'color' insteadof 'colour' http://bugs.python.org/issue28523 closed by terry.reedy #30466: Tutorial doesn't explain the use of classes http://bugs.python.org/issue30466 closed by Mariatta #30490: Allow pass an exception to the Event.set method http://bugs.python.org/issue30490 closed by pfreixes #30585: [security][3.3] Backport smtplib fix for TLS stripping vulnera http://bugs.python.org/issue30585 closed by ned.deily #30694: Update embedded copy of expat to 2.2.1 http://bugs.python.org/issue30694 closed by ned.deily #30730: [security] Injecting environment variable in subprocess on Win http://bugs.python.org/issue30730 closed by serhiy.storchaka #30794: Add multiprocessing.Process.kill() http://bugs.python.org/issue30794 closed by pitrou #30797: ./pyconfig.h:1438:0: warning: "_GNU_SOURCE" redefined [enabled http://bugs.python.org/issue30797 closed by ned.deily #30808: Use _Py_atomic API for concurrency-sensitive signal state http://bugs.python.org/issue30808 closed by pitrou #30836: test_c_locale_coercion fails on AIX http://bugs.python.org/issue30836 closed by ncoghlan #30870: IDLE: configdialog/fonts: change font when select by key up/do http://bugs.python.org/issue30870 closed by terry.reedy #30883: test_urllib2net failed on s390x Debian 3.6: ftp.debian.org err http://bugs.python.org/issue30883 closed by berker.peksag #30917: IDLE: Add idlelib.config.IdleConf unittest http://bugs.python.org/issue30917 closed by terry.reedy #30920: Sequence Matcher from diff lib is not implementing longest com http://bugs.python.org/issue30920 closed by terry.reedy #30929: AttributeErrors after import in multithreaded environment http://bugs.python.org/issue30929 closed by larry #30932: Identity comparison ("is") fails for floats in Python3 but wor http://bugs.python.org/issue30932 closed by steven.daprano #30933: Python not IEEE 754 float compliant for various zero operation http://bugs.python.org/issue30933 closed by r.david.murray #30936: json module ref leaks detected by test_json http://bugs.python.org/issue30936 closed by serhiy.storchaka #30941: Missing line in example program http://bugs.python.org/issue30941 closed by r.david.murray #30942: Implement lwalk(levelwalk) function on os.py http://bugs.python.org/issue30942 closed by r.david.murray #30943: printf-style Bytes Formatting sometimes do not worked. http://bugs.python.org/issue30943 closed by r.david.murray #30946: readline module has obsolete code http://bugs.python.org/issue30946 closed by pitrou #30948: Docs for __subclasses__(): Add hint that Python imports are la http://bugs.python.org/issue30948 closed by r.david.murray #30954: backport unittest.TestCase.assertLogs to 2.7? http://bugs.python.org/issue30954 closed by r.david.murray #30955: \\N in f-string causes next { to be literal if not escaped http://bugs.python.org/issue30955 closed by ned.deily #30957: pathlib: Path and PurePath cannot be subclassed http://bugs.python.org/issue30957 closed by serhiy.storchaka #30958: Scripts folder is empty http://bugs.python.org/issue30958 closed by ned.deily #30960: Python script is failing to run http://bugs.python.org/issue30960 closed by serhiy.storchaka #30961: Py_Decref a borrowed reference in _tracemalloc http://bugs.python.org/issue30961 closed by xiang.zhang #30965: Unexpected behavior of operator "in" http://bugs.python.org/issue30965 closed by ammar2 #30968: test_get_font (idlelib.idle_test.test_config.IdleConfTest) fai http://bugs.python.org/issue30968 closed by terry.reedy #30970: return-value of filecmp.dircmp.report_* http://bugs.python.org/issue30970 closed by r.david.murray #30973: Regular expression "hangs" interpreter http://bugs.python.org/issue30973 closed by r.david.murray #30976: multiprocessing.Process.is_alive can show True for dead proces http://bugs.python.org/issue30976 closed by mickp

1 0