Mailman 3 June 2016 - Python-Dev

[RELEASED] Python 3.4.5rc1 and Python 3.5.2rc1 are now available
by Larry Hastings 13 Jun '16

13 Jun '16

On behalf of the Python development community and the Python 3.4 and Python 3.5 release teams, I'm pleased to announce the availability of Python 3.4.5rc1 and Python 3.5.2rc1. Python 3.4 is now in "security fixes only" mode. This is the final stage of support for Python 3.4. All changes made to Python 3.4 since Python 3.4.4 should be security fixes only; conventional bug fixes are not accepted. Also, Python 3.4.5rc1 and all future releases of Python 3.4 will only be released as source code--no official binary installers will be produced. Python 3.5 is still in active "bug fix" mode. Python 3.5.2rc1 contains many incremental improvements over Python 3.5.1. Both these releases are "release candidates". They should not be considered the final releases, although the final releases should contain only minor differences. Python users are encouraged to test with these releases and report any problems they encounter. You can find Python 3.4.5rc1 here: https://www.python.org/downloads/release/python-345rc1/ And you can find Python 3.5.2rc1 here: https://www.python.org/downloads/release/python-352rc1/ Python 3.4.5 final and Python 3.5.2 final are both scheduled for release on June 26th, 2016. Happy Pythoneering, //arry/

1 0

Reminder: 3.6.0a2 snapshot 2016-06-13 12:00 UTC
by Ned Deily 13 Jun '16

13 Jun '16

Just a quick reminder that the next alpha snapshot for the 3.6 release cycle is coming up in a couple of days. This is the second of four alphas we have planned. Alpha 2 follows the development sprints at the PyCon US 2016 in Portland. Thanks to all of you who were able to be there and contribute! And to all of you who continue to contribue from afar. While there are still plenty of proposed patches awaiting review, nearly 300 commits have been pushed to the default branch (for 3.6.0) in the four weeks since alpha 1. As a reminder, alpha releases are intended to make it easier for the wider community to test the current state of new features and bug fixes for an upcoming Python release as a whole and for us to test the release process. During the alpha phase, features may be added, modified, or deleted up until the start of the beta phase. Alpha users beware! Also note that Larry has announced plans to do a 3.5.2 release candidate sometime this weekend and Benjamin plans to do a 2.7.12 release candidate. So get important maintenance release fixes in ASAP. Looking ahead, the next alpha release, 3.6.0a3, will follow in about a month on 2016-07-11. 2016-06-13 ~12:00 UTC: code snapshot for 3.6.0 alpha 1 now to 2016-09-07: Alpha phase (unrestricted feature development) 2016-09-07: 3.6.0 feature code freeze, 3.7.0 feature development begins 2016-09-07 to 2016-12-04: 3.6.0 beta phase (bug and regression fixes, no new features) 2016-12-04 3.6.0 release candidate 1 (3.6.0 code freeze) 2016-12-16 3.6.0 release (3.6.0rc1 plus, if necessary, any dire emergency fixes) --Ned P.S. Just to be clear, this upcoming alpha snapshot will *not* contain a resolution for 3.6.0 of the current on-going discussions about the behavior of os.urandom(), the secrets module, and friends (Issue26839, Issue27288, et al). I think the focus should be on getting 3.5.2 settled and then we can decide on and implement any changes for 3.6.0 in an upcoming alpha prior to beta 1. https://www.python.org/dev/peps/pep-0494/ -- Ned Deily nad(a)python.org -- []

2 1

New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512
by Christian Heimes 12 Jun '16

12 Jun '16

Hi everybody, I have three hashing-related patches for Python 3.6 that are waiting for review. Altogether the three patches add ten new hash algorithms to the hashlib module: SHA3 (224, 256, 384, 512), SHAKE (SHA3 XOF 128, 256), BLAKE2 (blake2b, blake2s) and truncated SHA512 (224, 256). SHA-3 / SHAKE: https://bugs.python.org/issue16113 BLAKE2: https://bugs.python.org/issue26798 SHA512/224 / SHA512/256: https://bugs.python.org/issue26834 I like to push the patches during the sprints at PyCon. Please assist with reviews. Regards, Christian

13 34

BDFL ruling request: should we block forever waiting for high-quality random bits?
by Larry Hastings 12 Jun '16

12 Jun '16

A problem has surfaced just this week in 3.5.1. Obviously this is a good time to fix it for 3.5.2. But there's a big argument over what is "broken" and what is an appropriate "fix". As 3.5 Release Manager, I can put my foot down and make rulings, and AFAIK the only way to overrule me is with the BDFL. In two of three cases I've put my foot down. In the third I'm pretty sure I'm right, but IIUC literally everyone with a stated opinion else disagrees with me. So I thought it best I escalate it. Note that 3.5.2 is going to wait until the issue is settled and any changes to behavior are written and checked in. (Blanket disclaimer for the below: in some places I'm trying to communicate other's people positions. I apologize if I misrepresented yours; please reply and correct my mistake. Also, sorry for the length of this email. But feel even sorrier for me: this debate has already eaten two days this week.) BACKGROUND For 3.5 os.urandom() was changed: instead of reading from /dev/urandom, it uses the new system call getrandom() where available. This is a new system call on Linux (which has already been cloned by Solaris). getrandom(), as CPython uses it, reads from the same PRNG that /dev/urandom gets its bits from. But because it's a system call you don't have to mess around with file handles. Also it always works in chrooted environments. Sounds like a fine idea. Also for 3.5, several other places where CPython internally needs random bits were switched from reading from /dev/urandom to calling getrandom(). The two that I know of: choosing the seed for hash randomization, and initializing the default Mersenne Twister for the random module. There's one subtle but important difference between /dev/urandom and getrandom(). At startup, Linux seeds the urandom PRNG from the entropy pool. If the entropy pool is uninitialized, what happens? CPython's calls to getrandom() will block until the entropy pool is initialized, which is usually just a few seconds (or less) after startup. But /dev/urandom *guarantees* that reads will *always* work. If the entropy pool hasn't been initialized, it pulls numbers from the PRNG before it's been properly seeded. What this results in depends on various aspects of the configuration (do you have ECC RAM? how long was the machine powered down? does the system have a correct realtime clock?). In extreme circumstances this may mean the "random" numbers are shockingly predictable! Under normal circumstances this minor difference is irrelevant. After all, when would the entropy pool ever be uninitialized? THE PROBLEM Issue #26839: http://bugs.python.org/issue26839 (warning, the issue is now astonishingly long, and exhausting to read, and various bits of it are factually wrong) A user reports that when starting CPython soon after startup on a fresh virtual machine, the process would hang for a long time. Someone on the issue reported observed delays of over 90 seconds. Later we found out: it wasn't 90 seconds before CPython became usable, these 90 seconds delays were before systemd timed out and simply killed the process. It's not clear what the upper bound on the delay might be. The issue author had already identified the cause: CPython was blocking on getrandom() in order to initialize hash randomization. On this fresh virtual machine the entropy pool started out uninitialized. And since the only thing running on the machine was CPython, and since CPython was blocked on initialization, the entropy pool was initializing very, very slowly. Other posters to the thread pointed out that the same thing would happen in "import random", if your code could get that far. The constructor for the Random() object would seed the Mersenne Twister, which would call getrandom() and block. Naturally, callers to os.urandom() could also block for an unbounded period for the same reason. MY RULINGS SO FAR 1) The change in 3.5 that means "import random" may block for an unbounded period of time on Linux due to the switch to getrandom() must be backed out or amended so that it never blocks. I *think* everyone agrees with this. The Mersenne Twister is not a CPRNG, so seeding it with crypto-quality bits isn't necessary. And unbounded delays are bad. 2) The change in 3.5 that means hash randomization initialization may block for an unbounded period of time on Linux due to the switch to getrandom() must be backed out or amended so that it never blocks. I believe most people agree with me. The cryptography experts disagree. IIUC both Alex Gaynor and Christian Heimes feel the blocking is preferable to non-random hash "randomization". Yes, the bad random data means the hashing will be predictable. Neither choice is exactly what you want. But most people feel it's simply unreasonable that in extreme corner cases CPython can block for an unbounded amount of time before running user code. OS.URANDOM() Here's where it gets complicated--and where everyone else thinks I'm wrong. os.urandom() is currently the best place for a Python programmer to get high-quality random bits. The one-line summary for os.urandom() reads: "Return a string of n random bytes suitable for cryptographic use." On 3.4 and before, on Linux, os.urandom() would never block, but if the entropy pool was uninitialized it could return very-very-poor-quality random bits. On 3.5.0 and 3.5.1, on Linux, when using the getrandom() call, it will instead block for an apparently unbounded period before returning high-quality random bits. The question: is this new behavior preferable, or should we return to the old behavior? Since I'm the one writing this email, let me make the case for my position: I think that os.urandom() should never block on Linux. Why? 1) Functions in the os module that look like OS functions should behave predictably like thin wrappers over those OS functions. Most of the time this is exactly what they are. In some cases they're more sophisticated; examples include os.popen(), os.scandir(), and the byzantine os.utime(). There are also some functions provided by the os module that don't resemble any native functionality, but these have unique names that don't look like anything provided by the OS. This makes the behavior of the Python function easy to reason about: it always behaves like your local OS function. Python provides os.stat() and it behaves like the local stat(). So if you want to know how any os module function behaves, just read your local man page. Therefore, os.urandom() should behave exactly like a thin shell around reading the local /dev/urandom. On Linux, /dev/urandom guarantees that it will never block. This means it has undesirable behavior if read immediately after a fresh boot. But this guarantee is so strong that Theodore Ts'o couldn't break it to fix the undesirable behavior. Instead he added the getrandom() system call. But he left /dev/urandom alone. Therefore, on Linux, os.urandom() should behave the same way, and also never block. 2) It's unfair to change the semantics of a well-established function to such a radical degree. os.urandom() has been in Python since at least 2.6--I was too lazy to go back any further. From 2.6 to 3.4, it behaved exactly like /dev/urandom, which meant that on Linux it would never block. As of 3.5, on Linux, it might now block for an unbounded period of time. Any code that calls os.urandom() has had its behavior radically changed in this extreme corner case. 3) os.urandom() doesn't actually guarantee it's suitable for cryptography. The documentation for os.urandom() has contained this sentence, untouched, since 2.6: The returned data should be unpredictable enough for cryptographic applications, though its exact quality depends on the OS implementation. On a Unix-like system this will query /dev/urandom, and on Windows it will use CryptGenRandom(). Of course, version 3.5 added this: On Linux 3.17 and newer, the getrandom() syscall is now used when available. But the waffling about its suitability for cryptography remains unchanged. So, while it's undesirable that os.urandom() might return shockingly poor quality random bits, it is *permissible* according to the documentation. 4) This really is a rare corner-case we're talking about. I just want to re-state: this case on Linux where /dev/urandom returns totally predictable bytes, and getrandom() will block, only happens when the entropy pool for urandom is uninitialized. Although it has been seen in the field, it's extremely rare. 99.99999%+ of the time, reading /dev/urandom and calling getrandom() will both return the exact same high-quality random bits without blocking. 5) This corner-case behavior is fixable externally to CPython. I don't really understand the subject, but apparently it's entirely reasonable to expect sysadmins to directly manage the entropy pools of virtual machines. They should be able to spin up their VMs with a pre-filled entropy pool. So it should be possible to ensure that os.urandom() always returns the high-quality random bits we wanted, even on freshly-booted VMs. 6) Guido and Tim Peters already decided once that os.urandom() should behave like /dev/urandom. Issue #25003: http://bugs.python.org/issue25003 In 2.7.10, os.urandom() was changed to call getentropy() instead of reading /dev/urandom when getentropy() was available. getentropy() was "stunningly slow" on Solaris, on the order of 300x slower than reading /dev/urandom. Guido and Tim both participated in the discussion on the issue; Guido also apparently discussed it via email with Theo De Raadt. While it's not quite apples-to-apples, I think this establishes some precedent that os.urandom() should * behave like /dev/urandom, and * be fast. -- On the other side is... everybody else. I've already spent an enormous amount of time researching and writing and re-writing this email. Rather than try (and fail) to accurately present the other sides of this debate, I'm just going to end the email here and let the other participants reply and voice their views. Bottom line: Guido, in this extreme corner case on Linux, should os.urandom() return bad random data like it used to, or should it block forever like it does in 3.5.0 and 3.5.1? //arry/

35 176

Cutoff time for patches for upcoming releases
by Terry Reedy 11 Jun '16

11 Jun '16

A question for each of the three release managers: when is the earliest that you might tag your release and cutoff submission of further patches for the release? 2.7.12 ('6-12')? 3.5.2 ('6-12')? 3.6.0a2 ('6-13')? -- Terry Jan Reedy

2 1

Summary of Python tracker Issues
by Python tracker 11 Jun '16

11 Jun '16

ACTIVITY SUMMARY (2016-06-03 - 2016-06-10) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 5553 (+16) closed 33491 (+75) total 39044 (+91) Open issues with patches: 2424 Issues opened (69) ================== #16484: pydoc generates invalid docs.python.org link for xml.etree.Ele http://bugs.python.org/issue16484 reopened by martin.panter #26243: zlib.compress level as keyword argument http://bugs.python.org/issue26243 reopened by serhiy.storchaka #26839: Python 3.5 running on Linux kernel 3.17+ can block at startup http://bugs.python.org/issue26839 reopened by haypo #27186: add os.fspath() http://bugs.python.org/issue27186 reopened by brett.cannon #27197: mock.patch interactions with "from" imports http://bugs.python.org/issue27197 opened by clarkbreyman #27198: Adding an assertClose() method to unittest.TestCase http://bugs.python.org/issue27198 opened by ChrisBarker #27199: TarFile expose copyfileobj bufsize to improve throughput http://bugs.python.org/issue27199 opened by fried #27200: make doctest in CPython has failures http://bugs.python.org/issue27200 opened by Jelle Zijlstra #27201: expose the ABI name as a config variable http://bugs.python.org/issue27201 opened by doko #27204: Failing doctests in Doc/howto/ http://bugs.python.org/issue27204 opened by Jelle Zijlstra #27205: Failing doctests in Library/collections.rst http://bugs.python.org/issue27205 opened by Jelle Zijlstra #27206: Failing doctests in Doc/tutorial/ http://bugs.python.org/issue27206 opened by Jelle Zijlstra #27207: Failing doctests in Doc/whatsnew/3.2.rst http://bugs.python.org/issue27207 opened by Jelle Zijlstra #27208: Failing doctests in Library/traceback.rst http://bugs.python.org/issue27208 opened by Jelle Zijlstra #27209: Failing doctests in Library/email.*.rst http://bugs.python.org/issue27209 opened by Jelle Zijlstra #27210: Failing doctests due to environmental dependencies in Lib/*lib http://bugs.python.org/issue27210 opened by Jelle Zijlstra #27212: Doc for itertools, 'islice()' implementation have unwanted beh http://bugs.python.org/issue27212 opened by alex0307 #27213: Rework CALL_FUNCTION* opcodes http://bugs.python.org/issue27213 opened by serhiy.storchaka #27214: a potential future bug and an optimization that mostly undermi http://bugs.python.org/issue27214 opened by Oren Milman #27218: improve tracing performance with f_trace set to Py_None http://bugs.python.org/issue27218 opened by xdegaye #27219: turtle.fillcolor doesn't accept a tuple of floats http://bugs.python.org/issue27219 opened by Jelle Zijlstra #27220: Add a pure Python version of 'collections.defaultdict' http://bugs.python.org/issue27220 opened by ebarry #27221: multiprocessing documentation is outdated regarding method pic http://bugs.python.org/issue27221 opened by memeplex #27222: redundant checks and a weird use of goto statements in long_rs http://bugs.python.org/issue27222 opened by Oren Milman #27223: _ready_ready and _write_ready should respect _conn_lost http://bugs.python.org/issue27223 opened by lukasz.langa #27226: distutils: unable to compile both .opt-1.pyc and .opt2.pyc sim http://bugs.python.org/issue27226 opened by mgorny #27227: argparse fails to parse [] when using choices and nargs='*' http://bugs.python.org/issue27227 opened by evan_ #27231: Support the fspath protocol in the posixpath module http://bugs.python.org/issue27231 opened by Jelle Zijlstra #27232: os.fspath() should not use repr() on error http://bugs.python.org/issue27232 opened by Jelle Zijlstra #27233: Missing documentation for PyOS_FSPath http://bugs.python.org/issue27233 opened by Jelle Zijlstra #27235: Heap overflow occurred due to the int overflow (Python-2.7.11/ http://bugs.python.org/issue27235 opened by madness #27238: Bare except: usages in turtle.py http://bugs.python.org/issue27238 opened by Jelle Zijlstra #27240: 'UnstructuredTokenList' object has no attribute '_fold_as_ew' http://bugs.python.org/issue27240 opened by touilleMan #27241: Catch exceptions raised in pstats add (repl) http://bugs.python.org/issue27241 opened by llllllllll #27242: Make the docs for NotImplemented & NotImplementedError unambig http://bugs.python.org/issue27242 opened by ebarry #27243: __aiter__ should return async iterator instead of awaitable http://bugs.python.org/issue27243 opened by yselivanov #27244: print(';;') fails in pdb with SyntaxError http://bugs.python.org/issue27244 opened by cjw296 #27245: IDLE: Fix deletion of custom themes and key bindings http://bugs.python.org/issue27245 opened by terry.reedy #27248: Possible refleaks in PyType_Ready in error condition http://bugs.python.org/issue27248 opened by xiang.zhang #27250: Add os.urandom_block() http://bugs.python.org/issue27250 opened by haypo #27252: Make dict views copyable http://bugs.python.org/issue27252 opened by serhiy.storchaka #27253: More efficient deepcopying of Mapping http://bugs.python.org/issue27253 opened by serhiy.storchaka #27254: heap overflow in Tkinter module http://bugs.python.org/issue27254 opened by Emin Ghuliev #27255: More opcode predictions http://bugs.python.org/issue27255 opened by serhiy.storchaka #27256: header indentation destroyed http://bugs.python.org/issue27256 opened by frispete #27257: get_addresses results in traceback with an addrspec with an em http://bugs.python.org/issue27257 opened by frispete #27258: Exception in BytesGenerator.flatten http://bugs.python.org/issue27258 opened by frispete #27259: Possible missing deprecation warnings? http://bugs.python.org/issue27259 opened by mark #27260: Missing equality check for super objects http://bugs.python.org/issue27260 opened by Jelle Zijlstra #27261: io.BytesIO.truncate does not work as advertised http://bugs.python.org/issue27261 opened by justus.winter #27262: IDLE: move Aqua context menu code to maxosx http://bugs.python.org/issue27262 opened by terry.reedy #27263: IDLE sets the HOME environment variable breaking scripts http://bugs.python.org/issue27263 opened by Jarrod Petz #27266: Always use getrandom() in os.random() on Linux and add block=F http://bugs.python.org/issue27266 opened by haypo #27268: Incorrect error message on float('') http://bugs.python.org/issue27268 opened by Drekin #27269: ipaddress: Wrong behavior with ::ffff:1.2.3.4 style IPs http://bugs.python.org/issue27269 opened by ThiefMaster #27270: 'parentheses-equality' warnings when building with clang and c http://bugs.python.org/issue27270 opened by xdegaye #27272: random.Random should not read 2500 bytes from urandom http://bugs.python.org/issue27272 opened by haypo #27273: subprocess.run(cmd, input='text') should pass universal_newlin http://bugs.python.org/issue27273 opened by akira #27274: [ctypes] Allow from_pointer creation http://bugs.python.org/issue27274 opened by memeplex #27275: KeyError thrown by optimised collections.OrderedDict.popitem() http://bugs.python.org/issue27275 opened by kaniini #27277: Fatal Python error: Segmentation fault in test_exceptions http://bugs.python.org/issue27277 opened by Rohit Mediratta #27278: py_getrandom() uses an int for syscall() result http://bugs.python.org/issue27278 opened by haypo #27279: Add random.cryptorandom() and random.pseudorandom, deprecate o http://bugs.python.org/issue27279 opened by lemburg #27281: unpickling an xmlrpc.client.Fault raises TypeError http://bugs.python.org/issue27281 opened by Uri Okrent #27282: Raise BlockingIOError in os.urandom if kernel is not ready http://bugs.python.org/issue27282 opened by ncoghlan #27283: Add a "What's New" entry for PEP 519 http://bugs.python.org/issue27283 opened by brett.cannon #27285: Deprecate pyvenv in favor of python3 -m venv http://bugs.python.org/issue27285 opened by stevepiercy #27286: str object got multiple values for keyword argument http://bugs.python.org/issue27286 opened by martin.panter #27287: SIGSEGV when calling os.forkpty() http://bugs.python.org/issue27287 opened by Alexander Haensch Most recent 15 issues with no replies (15) ========================================== #27287: SIGSEGV when calling os.forkpty() http://bugs.python.org/issue27287 #27283: Add a "What's New" entry for PEP 519 http://bugs.python.org/issue27283 #27273: subprocess.run(cmd, input='text') should pass universal_newlin http://bugs.python.org/issue27273 #27269: ipaddress: Wrong behavior with ::ffff:1.2.3.4 style IPs http://bugs.python.org/issue27269 #27259: Possible missing deprecation warnings? http://bugs.python.org/issue27259 #27258: Exception in BytesGenerator.flatten http://bugs.python.org/issue27258 #27248: Possible refleaks in PyType_Ready in error condition http://bugs.python.org/issue27248 #27241: Catch exceptions raised in pstats add (repl) http://bugs.python.org/issue27241 #27240: 'UnstructuredTokenList' object has no attribute '_fold_as_ew' http://bugs.python.org/issue27240 #27227: argparse fails to parse [] when using choices and nargs='*' http://bugs.python.org/issue27227 #27223: _ready_ready and _write_ready should respect _conn_lost http://bugs.python.org/issue27223 #27222: redundant checks and a weird use of goto statements in long_rs http://bugs.python.org/issue27222 #27220: Add a pure Python version of 'collections.defaultdict' http://bugs.python.org/issue27220 #27218: improve tracing performance with f_trace set to Py_None http://bugs.python.org/issue27218 #27214: a potential future bug and an optimization that mostly undermi http://bugs.python.org/issue27214 Most recent 15 issues waiting for review (15) ============================================= #27286: str object got multiple values for keyword argument http://bugs.python.org/issue27286 #27281: unpickling an xmlrpc.client.Fault raises TypeError http://bugs.python.org/issue27281 #27273: subprocess.run(cmd, input='text') should pass universal_newlin http://bugs.python.org/issue27273 #27270: 'parentheses-equality' warnings when building with clang and c http://bugs.python.org/issue27270 #27266: Always use getrandom() in os.random() on Linux and add block=F http://bugs.python.org/issue27266 #27262: IDLE: move Aqua context menu code to maxosx http://bugs.python.org/issue27262 #27255: More opcode predictions http://bugs.python.org/issue27255 #27253: More efficient deepcopying of Mapping http://bugs.python.org/issue27253 #27252: Make dict views copyable http://bugs.python.org/issue27252 #27248: Possible refleaks in PyType_Ready in error condition http://bugs.python.org/issue27248 #27245: IDLE: Fix deletion of custom themes and key bindings http://bugs.python.org/issue27245 #27243: __aiter__ should return async iterator instead of awaitable http://bugs.python.org/issue27243 #27242: Make the docs for NotImplemented & NotImplementedError unambig http://bugs.python.org/issue27242 #27241: Catch exceptions raised in pstats add (repl) http://bugs.python.org/issue27241 #27238: Bare except: usages in turtle.py http://bugs.python.org/issue27238 Top 10 most discussed issues (10) ================================= #26839: Python 3.5 running on Linux kernel 3.17+ can block at startup http://bugs.python.org/issue26839 144 msgs #27266: Always use getrandom() in os.random() on Linux and add block=F http://bugs.python.org/issue27266 61 msgs #27186: add os.fspath() http://bugs.python.org/issue27186 18 msgs #27243: __aiter__ should return async iterator instead of awaitable http://bugs.python.org/issue27243 18 msgs #27272: random.Random should not read 2500 bytes from urandom http://bugs.python.org/issue27272 18 msgs #5124: IDLE - pasting text doesn't delete selection http://bugs.python.org/issue5124 16 msgs #27198: Adding an assertClose() method to unittest.TestCase http://bugs.python.org/issue27198 13 msgs #27250: Add os.urandom_block() http://bugs.python.org/issue27250 13 msgs #23401: Add pickle support of Mapping views http://bugs.python.org/issue23401 12 msgs #25548: Show the address in the repr for class objects http://bugs.python.org/issue25548 11 msgs Issues closed (71) ================== #8491: Need readline command and keybinding information http://bugs.python.org/issue8491 closed by martin.panter #12962: TitledHelpFormatter and IndentedHelpFormatter are not document http://bugs.python.org/issue12962 closed by berker.peksag #13771: HTTPSConnection __init__ super implementation causes recursion http://bugs.python.org/issue13771 closed by berker.peksag #15476: Index "code object" and link to code object definition http://bugs.python.org/issue15476 closed by martin.panter #17888: docs: more information on documentation team http://bugs.python.org/issue17888 closed by berker.peksag #18027: distutils should access stat_result timestamps via .st_*time a http://bugs.python.org/issue18027 closed by berker.peksag #18117: Missing symlink:Current after Mac OS X 3.3.2 package installat http://bugs.python.org/issue18117 closed by ned.deily #19234: socket.fileno() documentation http://bugs.python.org/issue19234 closed by Jelle Zijlstra #19611: inspect.getcallargs doesn't properly interpret set comprehensi http://bugs.python.org/issue19611 closed by ncoghlan #20041: TypeError when f_trace is None and tracing. http://bugs.python.org/issue20041 closed by serhiy.storchaka #20567: test_idle causes test_ttk_guionly 'can't invoke "event" comman http://bugs.python.org/issue20567 closed by terry.reedy #21272: use _sysconfigdata to itinialize distutils.sysconfig http://bugs.python.org/issue21272 closed by doko #21277: don't try to link _ctypes with a ffi_convenience library http://bugs.python.org/issue21277 closed by doko #21313: Py_GetVersion() is broken when using mqueue and a long patch n http://bugs.python.org/issue21313 closed by martin.panter #21916: Create unit tests for turtle textonly http://bugs.python.org/issue21916 closed by serhiy.storchaka #22797: urllib.request.urlopen documentation falsely guarantees that a http://bugs.python.org/issue22797 closed by r.david.murray #23264: Add pickle support of dict views http://bugs.python.org/issue23264 closed by serhiy.storchaka #24617: os.makedirs()'s [mode] not correct http://bugs.python.org/issue24617 closed by martin.panter #24810: UX mode for IDLE targeted to 'new learners' http://bugs.python.org/issue24810 closed by terry.reedy #25738: http.server doesn't handle RESET CONTENT status correctly http://bugs.python.org/issue25738 closed by martin.panter #25941: Add 'How to Review a Patch' section to devguide http://bugs.python.org/issue25941 closed by ned.deily #26014: Guide users to the newer package install instructions http://bugs.python.org/issue26014 closed by ned.deily #26305: Make Argument Clinic to generate PEP 7 conforming code http://bugs.python.org/issue26305 closed by serhiy.storchaka #26372: Popen.communicate not ignoring BrokenPipeError http://bugs.python.org/issue26372 closed by gregory.p.smith #26437: asyncio create_server() not always accepts the 'port' paramete http://bugs.python.org/issue26437 closed by berker.peksag #26448: dis.findlabels ignores EXTENDED_ARG http://bugs.python.org/issue26448 closed by serhiy.storchaka #26809: Add __all__ list to the string module http://bugs.python.org/issue26809 closed by python-dev #26884: android: cross-compilation of extension module links to the wr http://bugs.python.org/issue26884 closed by doko #26983: float() can return not exact float instance http://bugs.python.org/issue26983 closed by serhiy.storchaka #27052: Python2.7.11+ as in Debian testing and Ubuntu 16.04 LTS crashe http://bugs.python.org/issue27052 closed by doko #27066: SystemError if custom opener returns -1 http://bugs.python.org/issue27066 closed by barry #27072: random.getrandbits is limited to 2**31-1 bits on 64-bit Window http://bugs.python.org/issue27072 closed by rhettinger #27073: redundant checks in long_add and long_sub http://bugs.python.org/issue27073 closed by serhiy.storchaka #27105: cgi.__all__ is incomplete http://bugs.python.org/issue27105 closed by martin.panter #27107: mailbox.__all__ list is incomplete http://bugs.python.org/issue27107 closed by martin.panter #27108: mimetypes.__all__ list is incomplete http://bugs.python.org/issue27108 closed by martin.panter #27109: plistlib.__all__ list is incomplete http://bugs.python.org/issue27109 closed by martin.panter #27110: smtpd.__all__ list is incomplete http://bugs.python.org/issue27110 closed by martin.panter #27127: Never have GET_ITER not followed by FOR_ITER http://bugs.python.org/issue27127 closed by Demur Rumed #27136: sock_connect fails for bluetooth (and probably others) http://bugs.python.org/issue27136 closed by yselivanov #27156: IDLE: remove unused code http://bugs.python.org/issue27156 closed by terry.reedy #27164: zlib can't decompress DEFLATE using shared dictionary http://bugs.python.org/issue27164 closed by martin.panter #27167: subprocess reports signal as negative exit status, not documen http://bugs.python.org/issue27167 closed by gregory.p.smith #27187: Relax __all__ location requirement in PEP 8 http://bugs.python.org/issue27187 closed by python-dev #27196: Eliminate 'ThemeChanged' warning when running IDLE tests http://bugs.python.org/issue27196 closed by terry.reedy #27202: make doctest fails on 2.7 release notes http://bugs.python.org/issue27202 closed by orsenthil #27203: Failing doctests in Doc/faq/programming.rst http://bugs.python.org/issue27203 closed by orsenthil #27211: Heap corruption via Python 2.7.11 IOBase readline() http://bugs.python.org/issue27211 closed by python-dev #27215: Docstrings of Sequence and MutableSequence seems not right http://bugs.python.org/issue27215 closed by rhettinger #27216: Fix capitalisation of "Python runtime" in os.path.islink descr http://bugs.python.org/issue27216 closed by ned.deily #27217: IDLE 3.5.1 not using Tk 8.6 http://bugs.python.org/issue27217 closed by ned.deily #27224: IDLE: editor versus grep line number differ http://bugs.python.org/issue27224 closed by terry.reedy #27225: Potential refleak in type_new when setting __new__ fails http://bugs.python.org/issue27225 closed by serhiy.storchaka #27228: just for clearing: os.path.normpath("file://a") returns "file: http://bugs.python.org/issue27228 closed by georg.brandl #27229: In tree cross-build fails copying Include/graminit.h to itsel http://bugs.python.org/issue27229 closed by martin.panter #27230: Calculation involving mpmath gives wrong result with Python 3. http://bugs.python.org/issue27230 closed by ned.deily #27234: tuple - single value with comma is assigned as type tuple http://bugs.python.org/issue27234 closed by steven.daprano #27236: Add CHAINED_COMPARE_OP opcode http://bugs.python.org/issue27236 closed by serhiy.storchaka #27237: Kafka Python Consumer Messages gets truncated http://bugs.python.org/issue27237 closed by ned.deily #27239: Make idlelib.macosx self-contained. http://bugs.python.org/issue27239 closed by terry.reedy #27246: Keyboard Shortcuts Crash Idle http://bugs.python.org/issue27246 closed by ebarry #27247: telnetlib AttributeError: 'error' object has no attribute 'err http://bugs.python.org/issue27247 closed by berker.peksag #27249: Add os.urandom_info http://bugs.python.org/issue27249 closed by haypo #27251: TypeError in logging.HTTPHandler.emit; possible python 2 to 3 http://bugs.python.org/issue27251 closed by vinay.sajip #27264: python 3.4 vs. 3.5 strftime same locale different output on Wi http://bugs.python.org/issue27264 closed by eryksun #27265: Hash of different, specific Decimals created from str is the s http://bugs.python.org/issue27265 closed by mark.dickinson #27267: memory leak in _ssl.c, function load_cert_chain http://bugs.python.org/issue27267 closed by python-dev #27271: asyncio lost udp packets http://bugs.python.org/issue27271 closed by gvanrossum #27276: FileFinder.find_spec() incompatible with finder specification http://bugs.python.org/issue27276 closed by paulmar #27280: Paste fail in ipaddress documentation http://bugs.python.org/issue27280 closed by berker.peksag #27284: Spam http://bugs.python.org/issue27284 closed by eryksun

1 0

PEP: Ordered Class Definition Namespace
by Eric Snow 10 Jun '16

10 Jun '16

Hi all, Following discussion a few years back (and rough approval from Guido [1]), I started work on using OrderedDict for the class definition namespace by default. The bulk of the effort lay in implementing OrderedDict in C, which I got landed just in time for 3.5. The remaining work was quite minimal and the actual change is quite small. My intention was to land the patch soon, having gone through code review during PyCon. However, Nick pointed out to me the benefit of having a concrete point of reference for the change, as well as making sure it isn't a problem for other implementations. So in that spirit, here's a PEP for the change. Feedback is welcome, particularly from from other implementors. -eric [1] https://mail.python.org/pipermail/python-ideas/2013-February/019704.html ================================================== PEP: XXX Title: Ordered Class Definition Namespace Version: $Revision$ Last-Modified: $Date$ Author: Eric Snow <ericsnowcurrently(a)gmail.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 4-Jun-2016 Python-Version: 3.6 Post-History: 7-Jun-2016 Abstract ======== This PEP changes the default class definition namespace to ``OrderedDict``. Furthermore, the order in which the attributes are defined in each class body will now be preserved in ``type.__definition_order__``. This allows introspection of the original definition order, e.g. by class decorators. Note: just to be clear, this PEP is *not* about changing ``type.__dict__`` to ``OrderedDict``. Motivation ========== Currently the namespace used during execution of a class body defaults to dict. If the metaclass defines ``__prepare__()`` then the result of calling it is used. Thus, before this PEP, if you needed your class definition namespace to be ``OrderedDict`` you had to use a metaclass. Metaclasses introduce an extra level of complexity to code and in some cases (e.g. conflicts) are a problem. So reducing the need for them is worth doing when the opportunity presents itself. Given that we now have a C implementation of ``OrderedDict`` and that ``OrderedDict`` is the common use case for ``__prepare__()``, we have such an opportunity by defaulting to ``OrderedDict``. The usefulness of ``OrderedDict``-by-default is greatly increased if the definition order is directly introspectable on classes afterward, particularly by code that is independent of the original class definition. One of the original motivating use cases for this PEP is generic class decorators that make use of the definition order. Changing the default class definition namespace has been discussed a number of times, including on the mailing lists and in PEP 422 and PEP 487 (see the References section below). Specification ============= * the default class *definition* namespace is now ``OrderdDict`` * the order in which class attributes are defined is preserved in the new ``__definition_order__`` attribute on each class * "dunder" attributes (e.g. ``__init__``, ``__module__``) are ignored * ``__definition_order__`` is a tuple * ``__definition_order__`` is a read-only attribute * ``__definition_order__`` is always set: * if ``__definition_order__`` is defined in the class body then it is used * types that do not have a class definition (e.g. builtins) have their ``__definition_order__`` set to ``None`` * types for which `__prepare__()`` returned something other than ``OrderedDict`` (or a subclass) have their ``__definition_order__`` set to ``None`` The following code demonstrates roughly equivalent semantics:: class Meta(type): def __prepare__(cls, *args, **kwargs): return OrderedDict() class Spam(metaclass=Meta): ham = None eggs = 5 __definition_order__ = tuple(k for k in locals() if (!k.startswith('__') or !k.endswith('__'))) Note that [pep487_] proposes a similar solution, albeit as part of a broader proposal. Compatibility ============= This PEP does not break backward compatibility, except in the case that someone relies *strictly* on dicts as the class definition namespace. This shouldn't be a problem. Changes ============= In addition to the class syntax, the following expose the new behavior: * builtins.__build_class__ * types.prepare_class * types.new_class Other Python Implementations ============================ Pending feedback, the impact on Python implementations is expected to be minimal. If a Python implementation cannot support switching to `OrderedDict``-by-default then it can always set ``__definition_order__`` to ``None``. Implementation ============== The implementation is found in the tracker. [impl_] Alternatives ============ type.__dict__ as OrderedDict ---------------------------- Instead of storing the definition order in ``__definition_order__``, the now-ordered definition namespace could be copied into a new ``OrderedDict``. This would mostly provide the same semantics. However, using ``OrderedDict`` for ``type,__dict__`` would obscure the relationship with the definition namespace, making it less useful. Additionally, doing this would require significant changes to the semantics of the concrete dict C-API. A "namespace" Keyword Arg for Class Definition ---------------------------------------------- PEP 422 introduced a new "namespace" keyword arg to class definitions that effectively replaces the need to ``__prepare__()``. [pep422_] However, the proposal was withdrawn in favor of the simpler PEP 487. References ========== .. [impl] issue #24254 (https://bugs.python.org/issue24254) .. [pep422] PEP 422 (https://www.python.org/dev/peps/pep-0422/#order-preserving-classes) .. [pep487] PEP 487 (https://www.python.org/dev/peps/pep-0487/#defining-arbitrary-namespaces) .. [orig] original discussion (https://mail.python.org/pipermail/python-ideas/2013-February/019690.html) .. [followup1] follow-up 1 (https://mail.python.org/pipermail/python-dev/2013-June/127103.html) .. [followup2] follow-up 2 (https://mail.python.org/pipermail/python-dev/2015-May/140137.html) Copyright =========== This document has been placed in the public domain.

8 23

Re: [Python-Dev] BDFL ruling request: should we block foreverwaiting for high-quality random bits?
by Larry Hastings 10 Jun '16

10 Jun '16

On 06/09/2016 05:00 PM, Steve Dower wrote: > If the pattern is really going to be the hasattr check you posted > earlier, can we just do it for people and save them writing code that > won't work on different OSs? No. That's what got us into this mess in the first place. 3.5.0 and 3.5.1 *already* changed to the new behavior, and it resulted in the situation where CPython blocked forever at startup in these certain edge cases. os.urandom() has been around for more than a decade, we can't unilaterally change its semantics now. os.urandom() in 3.5 has to go back to how it behaved on Linux in 3.4. And if I were release manager for 3.6, I'd say "it has to stay that way in 3.6 too". However, Guido's already said "don't add os.getrandom() in 3.5", so the debate is somewhat irrelevant. //arry/

1 0

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?
by Colm Buckley 09 Jun '16

09 Jun '16

Larry Hastings wrote: On 3.4 and before, on Linux, os.urandom() would never block, but if the > entropy pool was uninitialized it could return very-very-poor-quality > random bits. On 3.5.0 and 3.5.1, on Linux, when using the getrandom() > call, it will instead block for an apparently unbounded period before > returning high-quality random bits. Just a point of information here. Ted Ts'o commented on the quality of the pre-initialization bits; it's not a given that they're "very very poor quality". Even before the per-boot entropy pool is initialized, the kernel has a few sources of randomness available to it - viz: interrupt timings, RDRAND (on x86) and a little per-machine data (uname -a). If RDRAND is trusted, this is enough to provide quite significant entropy, however that's not much help to all the ARM devices out there. The most pressing issue from my perspective is the hash randomization initialization; as there is currently nothing a script author can do to influence its behavior (except setting PYTHONHASHSEED before invocation, which might not be an option). It should be possible, at least conceptually, for Python to be used to implement /sbin/init. This isn't currently the case on Linux with Python 3.5.1 and Linux 3.17+ For what it's worth, I do agree with Larry that os.urandom() should hew as closely as possible to the OS-specific urandom implementation. Adding an optional "blocking" boolean flag might be a useful addition for 3.6. Colm -- Colm Buckley / colm(a)tuatha.org / +353 87 2469146

1 0

Round 2 of the Python Language Summit coverage at LWN
by Jake Edge 09 Jun '16

09 Jun '16

Howdy python-dev, The second batch of articles from the Python Language Summit is now available. The starting point is here: https://lwn.net/Articles/688969/ (or here for non-subscribers: https://lwn.net/SubscriberLink/688969/91cbeeaf32807914/ for the next few hours anyway, it will be open to all after that using either link.) I have added five more sessions since last week's three, still six more to go, which should all be done by next week (and I'll post here again). Python's GitHub migration and workflow changes: https://lwn.net/Articles/689937/ https://lwn.net/SubscriberLink/689937/1fd56367a74206bf/ The state of mypy: https://lwn.net/Articles/690081/ https://lwn.net/SubscriberLink/690081/5c35679cafe42d1b/ An introduction to pytype: https://lwn.net/Articles/690150/ https://lwn.net/SubscriberLink/690150/660acde532afb8a3/ PyCharm and type hints: https://lwn.net/Articles/690186/ https://lwn.net/SubscriberLink/690186/848c447551204ffe/ Python 3.6 and 3.7 release cycles: https://lwn.net/Articles/690404/ https://lwn.net/SubscriberLink/690404/73cfb918fa21d27c/ The articles will be freely available (without using the SubscriberLink) to the world at large in a week (and the next batch the week after that) ... until then, feel free to share the SubscriberLinks. Hopefully I have captured things reasonably well. If there are corrections or clarifications needed, though, I recommend posting them as comments on the article. enjoy! jake -- Jake Edge - LWN - jake(a)lwn.net - http://lwn.net

2 1