Mailman 3 July 2014 - Python-Dev

Summary of Python tracker Issues
by Python tracker 25 Jul '14

25 Jul '14

ACTIVITY SUMMARY (2014-07-18 - 2014-07-25) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 4591 ( +2) closed 29248 (+60) total 33839 (+62) Open issues with patches: 2160 Issues opened (42) ================== #19884: Importing readline produces erroneous output http://bugs.python.org/issue19884 reopened by haypo #22010: Idle: better management of Shell window output http://bugs.python.org/issue22010 opened by terry.reedy #22011: test_os extended attribute setxattr tests can fail with ENOSPC http://bugs.python.org/issue22011 opened by Hibou57 #22012: struct.unpack('?', '\x02') returns (False,) on Mac OSX http://bugs.python.org/issue22012 opened by wayedt #22013: Add at least minimal support for thread groups http://bugs.python.org/issue22013 opened by rhettinger #22014: Add summary table for OS exception <-> errno mapping http://bugs.python.org/issue22014 opened by ncoghlan #22016: Add a new 'surrogatereplace' output only error handler http://bugs.python.org/issue22016 opened by ncoghlan #22018: Add a new signal.set_wakeup_socket() function http://bugs.python.org/issue22018 opened by haypo #22021: shutil.make_archive() root_dir do not work http://bugs.python.org/issue22021 opened by DemoHT #22023: PyUnicode_FromFormat is broken on python 2 http://bugs.python.org/issue22023 opened by alex #22024: Add to shutil the ability to wait until files are definitely d http://bugs.python.org/issue22024 opened by zach.ware #22025: webbrowser.get(command_line) does not support Windows-style pa http://bugs.python.org/issue22025 opened by dan.oreilly #22027: RFC 6531 (SMTPUTF8) support in smtplib http://bugs.python.org/issue22027 opened by zvyn #22028: Python 3.4.1 Installer ended prematurely (Windows msi) http://bugs.python.org/issue22028 opened by DieInSente #22029: argparse - CSS white-space: like control for individual text b http://bugs.python.org/issue22029 opened by paul.j3 #22033: Subclass friendly reprs http://bugs.python.org/issue22033 opened by serhiy.storchaka #22034: posixpath.join() and bytearray http://bugs.python.org/issue22034 opened by serhiy.storchaka #22035: Fatal error in dbm.gdbm http://bugs.python.org/issue22035 opened by serhiy.storchaka #22038: Implement atomic operations on non-x86 platforms http://bugs.python.org/issue22038 opened by Vitor.de.Lima #22039: PyObject_SetAttr doesn't mention value = NULL http://bugs.python.org/issue22039 opened by pitrou #22041: http POST request with python 3.3 through web proxy http://bugs.python.org/issue22041 opened by AlexMJ #22042: signal.set_wakeup_fd(fd): set the fd to non-blocking mode http://bugs.python.org/issue22042 opened by haypo #22043: Use a monotonic clock to compute timeouts http://bugs.python.org/issue22043 opened by haypo #22044: Premature Py_DECREF while generating a TypeError in call_tzinf http://bugs.python.org/issue22044 opened by Knio #22045: Python make issue http://bugs.python.org/issue22045 opened by skerr #22046: ZipFile.read() should mention that it might throw NotImplement http://bugs.python.org/issue22046 opened by detly #22047: argparse improperly prints mutually exclusive options when the http://bugs.python.org/issue22047 opened by Sam.Kerr #22049: argparse: type=<callable> doesn't honor nargs > 1 http://bugs.python.org/issue22049 opened by Chris.Bruner #22051: Turtledemo: stop reloading demos http://bugs.python.org/issue22051 opened by terry.reedy #22052: Comparison operators called in reverse order for subclasses wi http://bugs.python.org/issue22052 opened by mark.dickinson #22054: Add os.get_blocking() and os.set_blocking() functions http://bugs.python.org/issue22054 opened by haypo #22057: The doc say all globals are copied on eval(), but only __built http://bugs.python.org/issue22057 opened by amishne #22058: datetime.datetime() should accept a datetime.date as init para http://bugs.python.org/issue22058 opened by facundobatista #22059: incorrect type conversion from str to bytes in asynchat module http://bugs.python.org/issue22059 opened by hoxily #22060: Clean up ctypes.test, use unittest test discovery http://bugs.python.org/issue22060 opened by zach.ware #22062: Fix pathlib.Path.(r)glob doc glitches. http://bugs.python.org/issue22062 opened by terry.reedy #22063: asyncio: sock_xxx() methods of event loops should make the soc http://bugs.python.org/issue22063 opened by haypo #22064: Misleading message from 2to3 when skipping optional fixers http://bugs.python.org/issue22064 opened by ncoghlan #22065: Update turtledemo menu creation http://bugs.python.org/issue22065 opened by terry.reedy #22066: subprocess.communicate() does not receive full output from the http://bugs.python.org/issue22066 opened by juj #22067: time_test fails after strptime() http://bugs.python.org/issue22067 opened by serhiy.storchaka #22068: test_gc fails after test_idle http://bugs.python.org/issue22068 opened by serhiy.storchaka Most recent 15 issues with no replies (15) ========================================== #22067: time_test fails after strptime() http://bugs.python.org/issue22067 #22066: subprocess.communicate() does not receive full output from the http://bugs.python.org/issue22066 #22064: Misleading message from 2to3 when skipping optional fixers http://bugs.python.org/issue22064 #22060: Clean up ctypes.test, use unittest test discovery http://bugs.python.org/issue22060 #22057: The doc say all globals are copied on eval(), but only __built http://bugs.python.org/issue22057 #22051: Turtledemo: stop reloading demos http://bugs.python.org/issue22051 #22046: ZipFile.read() should mention that it might throw NotImplement http://bugs.python.org/issue22046 #22045: Python make issue http://bugs.python.org/issue22045 #22039: PyObject_SetAttr doesn't mention value = NULL http://bugs.python.org/issue22039 #22035: Fatal error in dbm.gdbm http://bugs.python.org/issue22035 #22034: posixpath.join() and bytearray http://bugs.python.org/issue22034 #22033: Subclass friendly reprs http://bugs.python.org/issue22033 #22027: RFC 6531 (SMTPUTF8) support in smtplib http://bugs.python.org/issue22027 #22024: Add to shutil the ability to wait until files are definitely d http://bugs.python.org/issue22024 #22016: Add a new 'surrogatereplace' output only error handler http://bugs.python.org/issue22016 Most recent 15 issues waiting for review (15) ============================================= #22068: test_gc fails after test_idle http://bugs.python.org/issue22068 #22065: Update turtledemo menu creation http://bugs.python.org/issue22065 #22060: Clean up ctypes.test, use unittest test discovery http://bugs.python.org/issue22060 #22054: Add os.get_blocking() and os.set_blocking() functions http://bugs.python.org/issue22054 #22051: Turtledemo: stop reloading demos http://bugs.python.org/issue22051 #22044: Premature Py_DECREF while generating a TypeError in call_tzinf http://bugs.python.org/issue22044 #22043: Use a monotonic clock to compute timeouts http://bugs.python.org/issue22043 #22042: signal.set_wakeup_fd(fd): set the fd to non-blocking mode http://bugs.python.org/issue22042 #22041: http POST request with python 3.3 through web proxy http://bugs.python.org/issue22041 #22038: Implement atomic operations on non-x86 platforms http://bugs.python.org/issue22038 #22035: Fatal error in dbm.gdbm http://bugs.python.org/issue22035 #22034: posixpath.join() and bytearray http://bugs.python.org/issue22034 #22033: Subclass friendly reprs http://bugs.python.org/issue22033 #22029: argparse - CSS white-space: like control for individual text b http://bugs.python.org/issue22029 #22027: RFC 6531 (SMTPUTF8) support in smtplib http://bugs.python.org/issue22027 Top 10 most discussed issues (10) ================================= #22018: Add a new signal.set_wakeup_socket() function http://bugs.python.org/issue22018 35 msgs #22003: BytesIO copy-on-write http://bugs.python.org/issue22003 18 msgs #21933: Allow the user to change font sizes with the text pane of turt http://bugs.python.org/issue21933 16 msgs #22012: struct.unpack('?', '\x02') returns (False,) on Mac OSX http://bugs.python.org/issue22012 10 msgs #1602: windows console doesn't print or input Unicode http://bugs.python.org/issue1602 9 msgs #22041: http POST request with python 3.3 through web proxy http://bugs.python.org/issue22041 8 msgs #22058: datetime.datetime() should accept a datetime.date as init para http://bugs.python.org/issue22058 8 msgs #18643: add a fallback socketpair() implementation in test.support http://bugs.python.org/issue18643 7 msgs #19884: Importing readline produces erroneous output http://bugs.python.org/issue19884 7 msgs #22013: Add at least minimal support for thread groups http://bugs.python.org/issue22013 7 msgs Issues closed (60) ================== #1049450: Solaris: EINTR exception in select/socket calls in telnetlib http://bugs.python.org/issue1049450 closed by haypo #4350: Remove dead code from Tkinter.py http://bugs.python.org/issue4350 closed by serhiy.storchaka #5718: Problem compiling ffi part of build on AIX 5.3. http://bugs.python.org/issue5718 closed by skrah #6167: Tkinter.Scrollbar: the activate method needs to return a value http://bugs.python.org/issue6167 closed by serhiy.storchaka #11266: asyncore does not handle EINTR in recv, send, connect, accept, http://bugs.python.org/issue11266 closed by haypo #11945: Adopt and document consistent semantics for handling NaN value http://bugs.python.org/issue11945 closed by rhettinger #12184: socketserver.ForkingMixin collect_children routine needs to co http://bugs.python.org/issue12184 closed by neologix #12801: C realpath not used by os.path.realpath http://bugs.python.org/issue12801 closed by haypo #15275: isinstance is called a more times that needed in ntpath http://bugs.python.org/issue15275 closed by serhiy.storchaka #15759: "make suspicious" doesn't display instructions in case of fail http://bugs.python.org/issue15759 closed by serhiy.storchaka #15982: asyncore.dispatcher does not handle windows socket error code http://bugs.python.org/issue15982 closed by haypo #16133: asyncore.dispatcher.recv doesn't handle EAGAIN / EWOULDBLOCK http://bugs.python.org/issue16133 closed by haypo #16494: Add a method on importlib.SourceLoader for creating bytecode f http://bugs.python.org/issue16494 closed by brett.cannon #16547: IDLE raises an exception in tkinter after fresh file's text ha http://bugs.python.org/issue16547 closed by serhiy.storchaka #17210: documentation of PyUnicode_Format() states wrong argument type http://bugs.python.org/issue17210 closed by python-dev #17391: _cursesmodule Fails to Build on GCC 2.95 (static) http://bugs.python.org/issue17391 closed by neologix #17709: http://docs.python.org/2.7/objects.inv doesn't support :func:` http://bugs.python.org/issue17709 closed by asvetlov #18093: Move main functions to a separate Programs directory http://bugs.python.org/issue18093 closed by ncoghlan #18132: buttons in turtledemo disappear on small screens http://bugs.python.org/issue18132 closed by terry.reedy #18168: plistlib output self-sorted dictionary http://bugs.python.org/issue18168 closed by serhiy.storchaka #18392: Doc: PyObject_Malloc() is not documented http://bugs.python.org/issue18392 closed by zach.ware #18436: Add mapping of symbol to function to operator module http://bugs.python.org/issue18436 closed by zach.ware #19629: support.rmtree fails on symlinks under Windows http://bugs.python.org/issue19629 closed by berker.peksag #21035: Python's HTTP server implementations hangs after 16.343 reques http://bugs.python.org/issue21035 closed by neologix #21500: Make use of the "load_tests" protocol in test_importlib packag http://bugs.python.org/issue21500 closed by zach.ware #21566: make use of the new default socket.listen() backlog argument http://bugs.python.org/issue21566 closed by neologix #21597: Allow turtledemo code pane to get wider. http://bugs.python.org/issue21597 closed by terry.reedy #21645: asyncio: Race condition in signal handling on FreeBSD http://bugs.python.org/issue21645 closed by haypo #21665: 2.7.7 ttk widgets not themed http://bugs.python.org/issue21665 closed by python-dev #21772: platform.uname() not EINTR safe http://bugs.python.org/issue21772 closed by neologix #21813: Enhance doc of os.stat_result http://bugs.python.org/issue21813 closed by haypo #21868: Tbuffer in turtle allows negative size http://bugs.python.org/issue21868 closed by rhettinger #21882: turtledemo modules imported by test___all__ cause side effects http://bugs.python.org/issue21882 closed by terry.reedy #21888: plistlib.FMT_BINARY behavior doesn't send required dict parame http://bugs.python.org/issue21888 closed by serhiy.storchaka #21901: test_selectors.PollSelectorTestCase.test_above_fd_setsize repo http://bugs.python.org/issue21901 closed by neologix #21947: `Dis` module doesn't know how to disassemble generators http://bugs.python.org/issue21947 closed by ncoghlan #21976: Fix test_ssl.py to handle LibreSSL versioning appropriately http://bugs.python.org/issue21976 closed by pitrou #21989: Missing (optional) argument `start` and `end` in documentation http://bugs.python.org/issue21989 closed by r.david.murray #22002: Make full use of test discovery in test subpackages http://bugs.python.org/issue22002 closed by python-dev #22006: thread module documentation erroneously(?) states not all buil http://bugs.python.org/issue22006 closed by mark.dickinson #22007: sys.stdout.write on Python 2.7 is not EINTR safe http://bugs.python.org/issue22007 closed by neologix #22008: Symtable's syntax warning should contain the word "because" http://bugs.python.org/issue22008 closed by python-dev #22009: pdb.set_trace() crashes with UnicodeDecodeError when binary da http://bugs.python.org/issue22009 closed by ned.deily #22015: C signal handler doesn't save/restore errno http://bugs.python.org/issue22015 closed by haypo #22017: Bad reference counting in the _warnings module http://bugs.python.org/issue22017 closed by python-dev #22019: ntpath.join() error with Chinese character Path http://bugs.python.org/issue22019 closed by ezio.melotti #22020: tutorial 9.10. Generators statement error http://bugs.python.org/issue22020 closed by ezio.melotti #22022: test_pathlib: shutil.rmtree() sporadic failures on Windows http://bugs.python.org/issue22022 closed by zach.ware #22026: 2.7.8 ttk Button text display problem http://bugs.python.org/issue22026 closed by zach.ware #22030: Use calloc in set resizing http://bugs.python.org/issue22030 closed by rhettinger #22031: Hexadecimal id in reprs http://bugs.python.org/issue22031 closed by serhiy.storchaka #22032: Use __qualname__ together with __module__ http://bugs.python.org/issue22032 closed by serhiy.storchaka #22036: Obsolete reference to stringobject in comment http://bugs.python.org/issue22036 closed by python-dev #22037: Poor grammar in asyncio TCP echo client example http://bugs.python.org/issue22037 closed by asvetlov #22040: Add a "force" parameter to shutil.rmtree http://bugs.python.org/issue22040 closed by r.david.murray #22048: Add weighted random choice to random package http://bugs.python.org/issue22048 closed by mark.dickinson #22050: argparse: read nargs > 1 options from file doesn't work http://bugs.python.org/issue22050 closed by r.david.murray #22053: turtledemo: clean up start and stop, fix warning http://bugs.python.org/issue22053 closed by terry.reedy #22055: Incomplete sentence in asyncio BaseEventLoop doc http://bugs.python.org/issue22055 closed by asvetlov #22061: Restore deleted tkinter functions with deprecaton dummies. http://bugs.python.org/issue22061 closed by serhiy.storchaka

1 0

Does Zip Importer have to be Special?
by Phil Thompson 25 Jul '14

25 Jul '14

I have an importer for use in applications that embed an interpreter that does a similar job to the Zip importer (except that the storage is a C data structure rather than a .zip file). Just like the Zip importer I need to import my importer and add it to sys.path_hooks. However the earliest opportunity I have to do this is after the Py_Initialize() call returns - but this is too late because some parts of the standard library have already needed to be imported. My current workaround is to include a modified version of _bootstrap.py as a frozen module that has the necessary steps added to the end of its _install() function. The Zip importer doesn't have this problem because it gets special treatment - the call to its equivalent code is hard-coded and happens exactly when needed. What would help is a table of functions that were called where _PyImportZip_Init() is currently called. By default the only entry in the table would be _PyImportZip_Init. There would be a way of modifying the table, either like how PyImport_FrozenModules is handled or how Inittab is handled. ...or if there is a better solution that I have missed that doesn't require a modified _bootstrap.py. Thanks, Phil

3 7

PEP 471 "scandir" accepted
by Victor Stinner 24 Jul '14

24 Jul '14

Hi, I asked privately Guido van Rossum if I can be the BDFL-delegate for the PEP 471 and he agreed. I accept the latest version of the PEP: http://legacy.python.org/dev/peps/pep-0471/ I consider that the PEP 471 "scandir" was discussed enough to collect all possible options (variations of the API) and that main flaws have been detected. Ben Hoyt modified his PEP to list all these options, and for each option gives advantages and drawbacks. Great job Ben :-) Thanks all developers who contributed to the threads on the python-dev mailing list! The new version of the PEP has an optional "follow_symlinks" parameter which is True by default. IMO this API fits better the common case, list the content of a single directory, and it's now simple to not follow symlinks to implement a recursive function like os.walk(). The PEP also explicitly mentions that os.walk() will be modified to benefit of the new os.scandir() function. I'm happy because the final API is very close to os.path functions and pathlib.Path methods. Python stays consistent, which is a great power of this language! The PEP is accepted. It's time to review the implementation ;-) The current code can be found at: https://github.com/benhoyt/scandir (I don't think that Ben already updated his implementation for the latest version of the PEP.) Victor

6 11

[PEP466] SSLSockets, and sockets, _socketobjects oh my!
by Alex Gaynor 24 Jul '14

24 Jul '14

Hi all, I've been happily working on the SSL module backports for Python2 (pursuant to PEP466), and I've hit something of a snag: In python3, the SSLSocket keeps a weak reference to the underlying socket, rather than a strong reference, as Python2 uses. Unfortunately, due to the way sockets work in Python2, this doesn't work: On Python2, _socketobject composes around _real_socket from the _socket module, whereas on Python3, it subclasses _socket.socket. Since you now have a Python- level class, you can weak reference it. The question is: a) Should we backport weak referencing _socket.sockets (changing the structure of the module seems overly invasive, albeit completely backwards compatible)? b) Does anyone know why weak references are used in the first place? The commit message just alludes to fixing a leak with no reference to an issue. Anyone who's interested in the state of the branch can see it at: github.com/alex/cpython on the backport-ssl branch. Note that many many tests are still failing, and you'll need to apply the patch from http://bugs.python.org/issue22023 to get it to work. Thanks, Alex PS: Any help in getting http://bugs.python.org/issue22023 landed which be very much appreciated.

3 7

Remaining decisions on PEP 471 -- os.scandir()
by Ben Hoyt 22 Jul '14

22 Jul '14

Hi folks, Thanks Victor, Nick, Ethan, and others for continued discussion on the scandir PEP 471 (most recent thread starts at https://mail.python.org/pipermail/python-dev/2014-July/135377.html). Just an aside ... I was reminded again recently why scandir() matters: a scandir user emailed me the other day, saying "I used scandir to dump the contents of a network dir in under 15 seconds. 13 root dirs, 60,000 files in the structure. This will replace some old VBA code embedded in a spreadsheet that was taking 15-20 minutes to do the exact same thing." I asked if he could run scandir's benchmark.py on his directory tree, and here's what it printed out: C:\Python34\scandir-master>benchmark.py "\\my\network\directory" Using fast C version of scandir Priming the system's cache... Benchmarking walks on \\my\network\directory, repeat 1/3... Benchmarking walks on \\my\network\directory, repeat 2/3... Benchmarking walks on \\my\network\directory, repeat 3/3... os.walk took 8739.851s, scandir.walk took 129.500s -- 67.5x as fast That's right -- os.walk() with scandir was almost 70x as fast as the current version! Admittedly this is a network file system, but that's still a real and important use case. It really pays not to throw away information the OS gives you for free. :-) On the recent python-dev thread, Victor especially made some well thought out suggestions. It seems to me there's general agreement that the basic API in PEP 471 is good (with Ethan not a fan at first, but it seems he's on board after further discussion :-). That said, I think there's basically one thing remaining to decide: whether or not to have DirEntry.is_dir() and .is_file() follow symlinks by default. I think Victor made a pretty good case that: (a) following links is usually what you want (b) that's the precedent set by the similar functions os.path.isdir() and pathlib.Path.is_dir(), so to do otherwise would be confusing (c) with the non-link-following version, if you wanted to follow links you'd have to say something like "if (entry.is_symlink() and os.path.isdir(entry.full_name)) or entry.is_dir()" instead of just "if entry.is_dir()" (d) it's error prone to have to do (c), as I found out recently when I had a bug in my implementation of os.walk() with scandir -- I had a bug due to getting this exact test wrong If we go with Victor's link-following .is_dir() and .is_file(), then we probably need to add his suggestion of a follow_symlinks=False parameter (defaults to True). Either that or you have to say "stat.S_ISDIR(entry.lstat().st_mode)" instead, which is a little bit less nice. As a KISS enthusiast, I admit I'm still somewhat partial to the DirEntry methods just returning (non-link following) info about the *directory entry* itself. However, I can definitely see the error-proneness of that, and the advantages given the points above. So I guess I'm on the fence. Given the above arguments for symlink-following is_dir()/is_file() methods (have I missed any, Victor?), what do others think? I'd be very keen to come to a consensus on this, so that I can make some final updates to the PEP and see about getting it accepted and/or implemented. :-) -Ben

10 28

subprocess research - max limit for piped output
by anatoly techtonik 20 Jul '14

20 Jul '14

I am trying to figure out what is maximum size for piped input in subprocess.check_output() I've got limitation of about 500Mb after which Python exits with MemoryError without any additional details. I have only 2.76Gb memory used out of 8Gb, so what limit do I hit? 1. subprocess output read buffer 2. Python limit on size of variable 3. some OS limit on output pipes Testcase attached. C:\discovery\interface\subprocess>py dead.py Testing size: 520Mb ..truncating to 545259520 .. Traceback (most recent call last): File "dead.py", line 66, in <module> backticks(r'type largefile') File "dead.py", line 36, in backticks output = subprocess.check_output(command, shell=True) File "C:\Python27\lib\subprocess.py", line 567, in check_output output, unused_err = process.communicate() File "C:\Python27\lib\subprocess.py", line 791, in communicate stdout = _eintr_retry_call(self.stdout.read) File "C:\Python27\lib\subprocess.py", line 476, in _eintr_retry_call return func(*args) MemoryError The process tried to write to a nonexistent pipe. -- anatoly t.

1 0

Summary of Python tracker Issues
by Python tracker 18 Jul '14

18 Jul '14

ACTIVITY SUMMARY (2014-07-11 - 2014-07-18) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 4589 ( +1) closed 29188 (+47) total 33777 (+48) Open issues with patches: 2154 Issues opened (36) ================== #21044: tarfile does not handle file .name being an int http://bugs.python.org/issue21044 reopened by zach.ware #21946: 'python -u' yields trailing carriage return '\r' (Python2 for http://bugs.python.org/issue21946 reopened by haypo #21950: import sqlite3 not running after configure --prefix=/alt/path; http://bugs.python.org/issue21950 reopened by r.david.murray #21958: Allow python 2.7 to compile with Visual Studio 2013 http://bugs.python.org/issue21958 opened by Zachary.Turner #21960: Better path handling in Idle find in files http://bugs.python.org/issue21960 opened by terry.reedy #21961: Add What's New for Idle. http://bugs.python.org/issue21961 opened by terry.reedy #21962: No timeout for asyncio.Event.wait() or asyncio.Condition.wait( http://bugs.python.org/issue21962 opened by ajaborsk #21963: 2.7.8 backport of Issue1856 (avoid daemon thread problems at s http://bugs.python.org/issue21963 opened by ned.deily #21964: inconsistency in list-generator comprehension with yield(-from http://bugs.python.org/issue21964 opened by hakril #21965: Add support for Memory BIO to _ssl http://bugs.python.org/issue21965 opened by geertj #21967: Interpreter crash upon accessing frame.f_restricted of a frame http://bugs.python.org/issue21967 opened by anselm.kruis #21969: WindowsPath constructor does not check for invalid characters http://bugs.python.org/issue21969 opened by Antony.Lee #21970: Broken code for handling file://host in urllib.request.FileHan http://bugs.python.org/issue21970 opened by vadmium #21971: Index and update turtledemo doc. http://bugs.python.org/issue21971 opened by terry.reedy #21972: Bugs in the lexer and parser documentation http://bugs.python.org/issue21972 opened by François-René.Rideau #21973: Idle should not quit on corrupted user config files http://bugs.python.org/issue21973 opened by Tomk #21975: Using pickled/unpickled sqlite3.Row results in segfault rather http://bugs.python.org/issue21975 opened by Elizacat #21976: Fix test_ssl.py to handle LibreSSL versioning appropriately http://bugs.python.org/issue21976 opened by worr #21980: Implement `logging.LogRecord.__repr__` http://bugs.python.org/issue21980 opened by cool-RR #21983: segfault in ctypes.cast http://bugs.python.org/issue21983 opened by Anthony.LaTorre #21986: Pickleability of code objects is inconsistent http://bugs.python.org/issue21986 opened by ppperry #21987: TarFile.getmember on directory requires trailing slash iff ove http://bugs.python.org/issue21987 opened by moloney #21989: Missing (optional) argument `start` and `end` in documentation http://bugs.python.org/issue21989 opened by SylvainDe #21990: saxutils defines an inner class where a normal one would do http://bugs.python.org/issue21990 opened by alex #21991: The new email API should use MappingProxyType instead of retur http://bugs.python.org/issue21991 opened by r.david.murray #21992: New AST node Else() should be introduced http://bugs.python.org/issue21992 opened by Igor.Bronshteyn #21995: Idle: pseudofiles have no buffer attribute. http://bugs.python.org/issue21995 opened by terry.reedy #21996: gettarinfo method does not handle files without text string na http://bugs.python.org/issue21996 opened by vadmium #21997: Pdb.set_trace debugging does not end correctly in IDLE http://bugs.python.org/issue21997 opened by ppperry #21998: asyncio: a new self-pipe should be created in the child proces http://bugs.python.org/issue21998 opened by haypo #21999: shlex: bug in posix more handling of empty strings http://bugs.python.org/issue21999 opened by isoschiz #22000: cross type comparisons clarification http://bugs.python.org/issue22000 opened by Jim.Jewett #22001: containers "same" does not always mean "__eq__". http://bugs.python.org/issue22001 opened by Jim.Jewett #22002: Make full use of test discovery in test subpackages http://bugs.python.org/issue22002 opened by zach.ware #22003: BytesIO copy-on-write http://bugs.python.org/issue22003 opened by dw #22005: datetime.__setstate__ fails decoding python2 pickle http://bugs.python.org/issue22005 opened by eddygeek Most recent 15 issues with no replies (15) ========================================== #22005: datetime.__setstate__ fails decoding python2 pickle http://bugs.python.org/issue22005 #22000: cross type comparisons clarification http://bugs.python.org/issue22000 #21999: shlex: bug in posix more handling of empty strings http://bugs.python.org/issue21999 #21998: asyncio: a new self-pipe should be created in the child proces http://bugs.python.org/issue21998 #21997: Pdb.set_trace debugging does not end correctly in IDLE http://bugs.python.org/issue21997 #21996: gettarinfo method does not handle files without text string na http://bugs.python.org/issue21996 #21995: Idle: pseudofiles have no buffer attribute. http://bugs.python.org/issue21995 #21992: New AST node Else() should be introduced http://bugs.python.org/issue21992 #21991: The new email API should use MappingProxyType instead of retur http://bugs.python.org/issue21991 #21990: saxutils defines an inner class where a normal one would do http://bugs.python.org/issue21990 #21989: Missing (optional) argument `start` and `end` in documentation http://bugs.python.org/issue21989 #21971: Index and update turtledemo doc. http://bugs.python.org/issue21971 #21967: Interpreter crash upon accessing frame.f_restricted of a frame http://bugs.python.org/issue21967 #21965: Add support for Memory BIO to _ssl http://bugs.python.org/issue21965 #21960: Better path handling in Idle find in files http://bugs.python.org/issue21960 Most recent 15 issues waiting for review (15) ============================================= #22003: BytesIO copy-on-write http://bugs.python.org/issue22003 #22002: Make full use of test discovery in test subpackages http://bugs.python.org/issue22002 #21999: shlex: bug in posix more handling of empty strings http://bugs.python.org/issue21999 #21990: saxutils defines an inner class where a normal one would do http://bugs.python.org/issue21990 #21989: Missing (optional) argument `start` and `end` in documentation http://bugs.python.org/issue21989 #21986: Pickleability of code objects is inconsistent http://bugs.python.org/issue21986 #21976: Fix test_ssl.py to handle LibreSSL versioning appropriately http://bugs.python.org/issue21976 #21975: Using pickled/unpickled sqlite3.Row results in segfault rather http://bugs.python.org/issue21975 #21965: Add support for Memory BIO to _ssl http://bugs.python.org/issue21965 #21958: Allow python 2.7 to compile with Visual Studio 2013 http://bugs.python.org/issue21958 #21955: ceval.c: implement fast path for integers with a single digit http://bugs.python.org/issue21955 #21951: tcl test change crashes AIX http://bugs.python.org/issue21951 #21947: `Dis` module doesn't know how to disassemble generators http://bugs.python.org/issue21947 #21944: Allow copying of CodecInfo objects http://bugs.python.org/issue21944 #21941: Clean up turtle TPen class http://bugs.python.org/issue21941 Top 10 most discussed issues (10) ================================= #21645: asyncio: Race condition in signal handling on FreeBSD http://bugs.python.org/issue21645 16 msgs #15443: datetime module has no support for nanoseconds http://bugs.python.org/issue15443 14 msgs #21815: imaplib truncates some untagged responses http://bugs.python.org/issue21815 14 msgs #21935: Implement AUTH command in smtpd. http://bugs.python.org/issue21935 11 msgs #21955: ceval.c: implement fast path for integers with a single digit http://bugs.python.org/issue21955 10 msgs #21975: Using pickled/unpickled sqlite3.Row results in segfault rather http://bugs.python.org/issue21975 9 msgs #21986: Pickleability of code objects is inconsistent http://bugs.python.org/issue21986 9 msgs #21927: BOM appears in stdin when using Powershell http://bugs.python.org/issue21927 8 msgs #1598: unexpected response in imaplib http://bugs.python.org/issue1598 7 msgs #18320: python installation is broken if prefix is overridden on an in http://bugs.python.org/issue18320 7 msgs Issues closed (43) ================== #8849: python.exe problem with cvxopt http://bugs.python.org/issue8849 closed by r.david.murray #9390: Error in sys.excepthook on windows when redirecting output of http://bugs.python.org/issue9390 closed by zach.ware #14714: PEP 414 tokenizing hook does not preserve tabs http://bugs.python.org/issue14714 closed by aronacher #15962: Windows STDIN/STDOUT Redirection is actually FIXED http://bugs.python.org/issue15962 closed by terry.reedy #16178: atexit._run_exitfuncs should be a public API http://bugs.python.org/issue16178 closed by rhettinger #16237: bdist_rpm SPEC files created with distutils may be distro spec http://bugs.python.org/issue16237 closed by ncoghlan #16382: Better warnings exception for bad category http://bugs.python.org/issue16382 closed by berker.peksag #16859: tarfile.TarInfo.fromtarfile does not check read() return value http://bugs.python.org/issue16859 closed by lars.gustaebel #16895: Batch file to mimic 'make' on Windows http://bugs.python.org/issue16895 closed by zach.ware #17308: Dialog.py crashes when putty Window resized http://bugs.python.org/issue17308 closed by berker.peksag #18144: FD leak in urllib2 http://bugs.python.org/issue18144 closed by serhiy.storchaka #18974: Use argparse in the diff script http://bugs.python.org/issue18974 closed by serhiy.storchaka #19076: Pdb.do_break calls error with obsolete file kwarg http://bugs.python.org/issue19076 closed by berker.peksag #19355: Initial modernization of OpenWatcom support http://bugs.python.org/issue19355 closed by Jeffrey.Armstrong #20451: os.exec* mangles argv on windows (splits on spaces, etc) http://bugs.python.org/issue20451 closed by rhettinger #21059: idle_test.test_warning failure http://bugs.python.org/issue21059 closed by zach.ware #21163: asyncio doesn't warn if a task is destroyed during its executi http://bugs.python.org/issue21163 closed by haypo #21247: test_asyncio: test_subprocess_send_signal hangs on Fedora buil http://bugs.python.org/issue21247 closed by haypo #21323: CGI HTTP server not running scripts from subdirectories http://bugs.python.org/issue21323 closed by ned.deily #21599: Argument transport in attach and detach method in Server class http://bugs.python.org/issue21599 closed by haypo #21655: Write Unit Test for Vec2 and TNavigator class in the Turtle Mo http://bugs.python.org/issue21655 closed by Lita.Cho #21765: Idle: make 3.x HyperParser work with non-ascii identifiers. http://bugs.python.org/issue21765 closed by terry.reedy #21899: Futures are not marked as completed http://bugs.python.org/issue21899 closed by Sebastian.Kreft.Deezer #21906: Tools\Scripts\md5sum.py doesn't work in Python 3.x http://bugs.python.org/issue21906 closed by berker.peksag #21913: threading.Condition.wait() is not interruptible in Python 2.7 http://bugs.python.org/issue21913 closed by neologix #21918: Convert test_tools to directory http://bugs.python.org/issue21918 closed by zach.ware #21953: pythonrun.c does not check std streams the same as fileio.c http://bugs.python.org/issue21953 closed by steve.dower #21957: ASCII Formfeed (FF) & ASCII Vertical Tab (VT) Have Hexadecimal http://bugs.python.org/issue21957 closed by ned.deily #21959: msi product code for 2.7.8150 not in Tools/msi/uuids.py http://bugs.python.org/issue21959 closed by r.david.murray #21966: InteractiveConsole does not support -q option http://bugs.python.org/issue21966 closed by belopolsky #21968: 'abort' object is not callable http://bugs.python.org/issue21968 closed by Apple Grew #21974: Typo in "Set" in PEP 289 http://bugs.python.org/issue21974 closed by rhettinger #21977: In the re's token example OP and SKIP regexes can be improved http://bugs.python.org/issue21977 closed by rhettinger #21978: Support index access on OrderedDict views (e.g. o.keys()[7]) http://bugs.python.org/issue21978 closed by rhettinger #21979: SyntaxError not raised on "0xaor 1" http://bugs.python.org/issue21979 closed by mark.dickinson #21981: Idle problem http://bugs.python.org/issue21981 closed by eric.smith #21982: Idle configDialog: fix regression and add minimal unittest http://bugs.python.org/issue21982 closed by terry.reedy #21984: list(itertools.repeat(1)) causes the system to hang http://bugs.python.org/issue21984 closed by rhettinger #21985: test_asyncio prints some junk http://bugs.python.org/issue21985 closed by haypo #21988: Decrease iterating overhead in timeit http://bugs.python.org/issue21988 closed by gvanrossum #21993: counterintuitive behavior of list.index with boolean values http://bugs.python.org/issue21993 closed by ezio.melotti #21994: Syntax error in the ssl module documentation http://bugs.python.org/issue21994 closed by berker.peksag #22004: io documentation refers to newline as newlines http://bugs.python.org/issue22004 closed by python-dev

1 0

cStringIO vs io.BytesIO
by Mikhail Korobov 17 Jul '14

17 Jul '14

Hi, cStringIO was removed from Python 3. It seems the suggested replacement is io.BytesIO. But there is a problem: cStringIO.StringIO(b'data') didn't copy the data while io.BytesIO(b'data') makes a copy (even if the data is not modified later). This means io.BytesIO is not suited well to cases when you want to get a readonly file-like interface for existing byte strings. Isn't it one of the main io.BytesIO use cases? Wrapping bytes in cStringIO.StringIO used to be almost free, but this is not true for io.BytesIO. So making code 3.x compatible by ditching cStringIO can cause a serious performance/memory regressions. One can change the code to build the data using BytesIO (without creating bytes objects in the first place), but that is not always possible or convenient. I believe this problem affects tornado ( https://github.com/tornadoweb/tornado/issues/1110), Scrapy (this is how I became aware of this issue), NLTK (anecdotical evidence - I tried to port some hairy NLTK module to io.BytesIO, it became many times slower) and maybe pretty much every IO-related project ported to Python 3.x (django - check <https://github.com/django/django/blob/fff7b507ef2f85bb47abd2ee32982682d7822…>, werkzeug and frameworks based on it - check <https://github.com/mitsuhiko/werkzeug/blob/976b63cadf3d5482aa975df053fa458f…>, requests - check <https://github.com/kennethreitz/requests/blob/6b21e5c8f0c8fafda661d80f4555c…> - they all wrap user data to BytesIO, and this may cause slowdowns and up to 2x memory usage in Python 3.x). Do you know if there a workaround? Maybe there is some stdlib part that I'm missing, or a module on PyPI? It is not that hard to write an own wrapper that won't do copies (or to port [c]StringIO to 3.x), but I wonder if there is an existing solution or plans to fix it in Python itself - this BytesIO use case looks quite important.

4 5

Another case for frozendict
by Jason R. Coombs 16 Jul '14

16 Jul '14

I repeatedly run into situations where a frozendict would be useful, and every time I do, I go searching and find the (unfortunately rejected) PEP-416. I'd just like to share another case where having a frozendict in the stdlib would be useful to me. I was interacting with a database and had a list of results from 206 queries: >>> res = [db.cases.remove({'_id': doc['_id']}) for doc in fives] >>> len(res) 206 I can see that the results are the same for the first two queries. >>> res[0] {'n': 1, 'err': None, 'ok': 1.0} >>> res[1] {'n': 1, 'err': None, 'ok': 1.0} So I'd like to test to see if that's the case, so I try to construct a 'set' on the results, which in theory would give me a list of unique results: >>> set(res) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unhashable type: 'dict' I can't do that because dict is unhashable. That's reasonable, and if I had a frozen dict, I could easily work around this limitation and accomplish what I need. >>> set(map(frozendict, res)) Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'frozendict' is not defined PEP-416 mentions a MappingProxyType, but that's no help. >>> res_ex = list(map(types.MappingProxyType, res)) >>> set(res_ex) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unhashable type: 'mappingproxy' I can achieve what I need by constructing a set on the 'items' of the dict. >>> set(tuple(doc.items()) for doc in res) {(('n', 1), ('err', None), ('ok', 1.0))} But that syntax would be nicer if the result had the same representation as the input (mapping instead of tuple of pairs). A frozendict would have readily enabled the desirable behavior. Although hashability is mentioned in the PEP under constraints, there are many use-cases that fall out of the ability to hash a dict, such as the one described above, which are not mentioned at all in use-cases for the PEP. If there's ever any interest in reviving that PEP, I'm in favor of its implementation.

11 15

== on object tests identity in 3.x
by Andreas Maier 16 Jul '14

16 Jul '14

Am 2014-07-07 23:11, schrieb Jan Kaliszewski: > 07.07.2014 18:11, Andreas Maier wrote: >> Am 07.07.2014 17:58, schrieb Xavier Morel: >>> On 2014-07-07, at 13:22 , Andreas Maier <andreas.r.maier(a)gmx.de> wrote: >>> >>>> While discussing Python issue #12067 >>>> (http://bugs.python.org/issue12067#msg222442), I learned that >>>> Python 3.4 implements '==' and '!=' on the object type such that if >>>> no special equality test operations are implemented in derived >>>> classes, there is a default implementation that tests for identity >>>> (as opposed to equality of the values). > [...] >>>> IMHO, that default implementation contradicts the definition that >>>> '==' and '!=' test for equality of the values of an object. > [...] >>>> To me, a sensible default implementation for == on object would be >>>> (in Python): >>>> >>>> if v is w: >>>> return True; >>>> elif type(v) != type(w): >>>> return False >>>> else: >>>> raise ValueError("Equality cannot be determined in default >>>> implementation") >>> >>> Why would comparing two objects of different types return False >> >> Because I think (but I'm not sure) that the type should play a role >> for comparison of values. But maybe that does not embrace duck typing >> sufficiently, and the type should be ignored by default for comparing >> object values. >> >>> but comparing two objects of the same type raise an error? >> >> That I'm sure of: Because the default implementation (after having >> exhausted all possibilities of calling __eq__ and friends) has no way >> to find out whether the values(!!) of the objects are equal. > > IMHO, in Python context, "value" is a very vague term. Quite often we > can read it as the very basic (but not the only one) notion of "what > makes objects being equal or not" -- and then saying that "objects are > compared by value" is a tautology. > > In other words, what object's "value" is -- is dependent on its > nature: e.g. the value of a list is what are the values of its > consecutive (indexed) items; the value of a set is based on values of > all its elements without notion of order or repetition; the value of a > number is a set of its abstract mathematical properties that determine > what makes objects being equal, greater, lesser, how particular > arithmetic operations work etc... > > I think, there is no universal notion of "the value of a Python > object". The notion of identity seems to be most generic (every > object has it, event if it does not have any other property) -- and > that's why by default it is used to define the most basic feature of > object's *value*, i.e. "what makes objects being equal or not" (== and > !=). Another possibility would be to raise TypeError but, as Ethan > Furman wrote, it would be impractical (e.g. key-type-heterogenic dicts > or sets would be practically impossible to work with). On the other > hand, the notion of sorting order (< > <= >=) is a much more > specialized object property. On the universal notion of a value in Python: In both 2.x and 3.x, it reads (in 3.1. Objects, values and types): - "Every object has an identity, a type and a value." - "An object's /identity/ never changes once it has been created; .... The /value/ of some objects can change. Objects whose value can change are said to be /mutable/; objects whose value is unchangeable once they are created are called /immutable/." These are clear indications that there is an intention to have separate concepts of identity and value in Python. If an instance of type object can exist but does not have a universal notion of value, it should not allow operations that need a value. I do not really buy into the arguments that try to show how identity and value are somehow the same. They are not, not even in Python. The argument I can absolutely buy into is that the implementation cannot be changed within a major release. So the real question is how we document it. I'll try to summarize in a separate posting. Andy

13 34