Mailman 3 May 2013 - Python-Dev

[RELEASED] Python 2.7.5
by Benjamin Peterson 20 May '13

20 May '13

It is my greatest pleasure to announce the release of Python 2.7.5. 2.7.5 is the latest maintenance release in the Python 2.7 series. You may be surprised to hear from me so soon, as Python 2.7.4 was released slightly more than a month ago. As it turns out, 2.7.4 had several regressions and incompatibilities with 2.7.3. Among them were regressions in the zipfile, gzip, and logging modules. 2.7.5 fixes these. In addition, a data file for testing in the 2.7.4 tarballs and binaries aroused the suspicion of some virus checkers. The 2.7.5 release removes this file to resolve that issue. For details, see the Misc/NEWS file in the distribution or view it at http://hg.python.org/cpython/file/ab05e7dd2788/Misc/NEWS Downloads are at http://python.org/download/releases/2.7.5/ As always, please report bugs to http://bugs.python.org/ (Thank you to those who reported these bugs in 2.7.4.) This is a production release. Happy May, Benjamin Peterson 2.7 Release Manager (on behalf of all of Python 2.7's contributors)

8 15

Async subprocesses on Windows with tulip
by Richard Oudkerk 19 May '13

19 May '13

Attached is a pretty trivial example of asynchronous interaction with a python subprocess using tulip on Windows. It does not use transports or protocols -- instead sock_recv() and sock_sendall() are used inside tasks. I am not sure what the plan is for dealing with subprocesses currently. Shall I just add this to the examples folder for now? -- Richard

2 2

HAVE_FSTAT?
by Antoine Pitrou 19 May '13

19 May '13

Hello, Some pieces of code are still guarded by: #ifdef HAVE_FSTAT ... #endif I would expect all systems to have fstat() these days. It's pretty basic POSIX, and even Windows has had it for ages. Shouldn't we simply make those code blocks unconditional? It would avoid having to maintain unused fallback paths. Regards Antoine.

7 10

Re: [Python-Dev] cpython: Use PY_FORMAT_SIZE_T because Visual Studio does not understand %zd format.
by Serhiy Storchaka 18 May '13

18 May '13

18.05.13 19:37, richard.oudkerk написав(ла): > http://hg.python.org/cpython/rev/0648e7fe7a72 > changeset: 83829:0648e7fe7a72 > user: Richard Oudkerk <shibturn(a)gmail.com> > date: Sat May 18 17:35:19 2013 +0100 > summary: > Use PY_FORMAT_SIZE_T because Visual Studio does not understand %zd format. See also DEBUG_PRINT_FORMAT_SPEC() in Python/formatter_unicode.c, _PyDebugAllocatorStats() in Objects/obmalloc.c, and kqueue_event_repr() in Modules/selectmodule.c.

1 1

Re: [Python-Dev] cpython: Undo the deprecation of _asdict().
by Serhiy Storchaka 18 May '13

18 May '13

18.05.13 10:06, raymond.hettinger написав(ла): > http://hg.python.org/cpython/rev/1b760f926846 > changeset: 83823:1b760f926846 > user: Raymond Hettinger <python(a)rcn.com> > date: Sat May 18 00:05:20 2013 -0700 > summary: > Undo the deprecation of _asdict(). Why?

1 0

[RELEASED] Python 3.2.5 and Python 3.3.2
by Georg Brandl 18 May '13

18 May '13

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On behalf of the Python development team, I am pleased to announce the releases of Python 3.2.5 and 3.3.2. The releases fix a few regressions in 3.2.4 and 3.3.1 in the zipfile, gzip and xml.sax modules. Details can be found in the changelogs: http://hg.python.org/cpython/file/v3.2.5/Misc/NEWS and http://hg.python.org/cpython/file/v3.3.2/Misc/NEWS To download Python 3.2.5 or Python 3.3.2, visit: http://www.python.org/download/releases/3.2.5/ or http://www.python.org/download/releases/3.3.2/ respectively. As always, please report bugs to http://bugs.python.org/ (Thank you to those who reported these regressions.) Enjoy! - -- Georg Brandl, Release Manager georg at python.org (on behalf of the entire python-dev team and all contributors) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) iEYEARECAAYFAlGUbJ4ACgkQN9GcIYhpnLDH8ACdEM4k7bobLJsFmCb49zuwQR3W EjgAoIWAOFNhJNdTAWEGSWqFWUP20wrb =YnPr -----END PGP SIGNATURE-----

7 6

Mysterious Python pyc file corruption problems
by Barry Warsaw 18 May '13

18 May '13

I am looking into a particularly vexing Python problem on Ubuntu that manifests in several different ways. I think the problem is the same one described in http://bugs.python.org/issue13146 and I sent a message on the subject to the ubuntu-devel list: https://lists.ubuntu.com/archives/ubuntu-devel/2013-May/037129.html I don't know what's causing the problem and have no way to reproduce it, but all the clues point to corrupt pyc files in Pythons < 3.3. The common way this manifests is a traceback on an import statement. The actual error can be a "ValueError: bad marshal data (unknown type code)" such as in http://pad.lv/1010077 or an "EOFError: EOF read where not expected" as in http://pad.lv/1060842. We have many more instances of both of these. Since both error messages come from marshal.c when trying to read the pyc for a module being imported, I suspect that something is causing the pyc files to get partially overwritten or corrupted. The workaround is always to essentially blow away the .pyc file and re-create it. (Various different techniques can be used, but they all boil down to the same thing.) Another commonality is that this bug -- so far -- has not been observed in any Python 3.3 code, only 3.2 and earlier, including 2.7 and 2.6. This strengthens my hypothesis, since importlib in Python 3.3 included an atomic rename of the .pyc file whereas older Pythons only do an exclusive open on the pyc files, but do *not* do an atomic rename AFAICT. This leads me to hypothesize that the bug is due to an as yet unidentified race condition during installation of Python source code on Ubuntu, which is normally when we automatically byte compile the source to .pyc files. This can happen at package installation/upgrade time, or during a fresh install. In each of these cases there *should* be only one process attempting to write the .pyc, but my guess is that for some reason, multiple processes are trying to do this, triggering a truncation or other bogus content of .pyc files. Even in Python < 3.3, it should not be possible to corrupt a .pyc when only a single process is involved, due to the import lock and/or GIL. The exclusive open of the .pyc file is clearly not enough of a protection in a multiprocess situation, since the bug has already been identified in Python on buildbots during test_multiprocessing. See http://bugs.python.org/issue13146 I think the list of errors we've seen is too extensive to chalk up to a hardware bug, and I think the systems involved are modern enough to not be subject to file system data loss. There could be a missing fsync somewhere though that might be involved. I think it's doubtful that buggy remote file systems (e.g. NFSv2) are involved. I could be wrong about any of that. I have not succeeded in writing a standalone reproducer using Python 2.7. So, the mystery is: what process on Ubuntu is exploiting holes in the exclusive open and causing this problem? Issue 13146 is closed because the fix was applied to Python 3.3 (see above), but it was not backported to earlier versions. I think it would not be that difficult to backport it, and I would be willing to do so for Python 2.7 and 3.2. We might include 2.6 in that, but only in Ubuntu since I can't see how this bug could be exploited as a security vulnerability. Thoughts? -Barry

16 36

Summary of Python tracker Issues
by Python tracker 17 May '13

17 May '13

ACTIVITY SUMMARY (2013-05-10 - 2013-05-17) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 3966 ( +3) closed 25805 (+47) total 29771 (+50) Open issues with patches: 1776 Issues opened (32) ================== #17487: wave.Wave_read.getparams should be more user friendly http://bugs.python.org/issue17487 reopened by Claudiu.Popa #17807: Generator cleanup without tp_del http://bugs.python.org/issue17807 reopened by pitrou #17905: Add check for locale.h http://bugs.python.org/issue17905 reopened by pitrou #17951: TypeError during gdb backtracing http://bugs.python.org/issue17951 opened by Catalin.Patulea #17953: sys.modules cannot be reassigned http://bugs.python.org/issue17953 opened by Valentin.Lorentz #17955: Minor updates to Functional HOWTO http://bugs.python.org/issue17955 opened by akuchling #17956: add ScheduledExecutor http://bugs.python.org/issue17956 opened by neologix #17957: remove outdated (and unexcellent) paragraph in whatsnew http://bugs.python.org/issue17957 opened by tshepang #17960: Clarify the required behaviour of locals() http://bugs.python.org/issue17960 opened by ncoghlan #17961: Use enum names as values in enum.Enum convenience API http://bugs.python.org/issue17961 opened by ncoghlan #17963: Deprecate the frame hack for implicitly getting module details http://bugs.python.org/issue17963 opened by ncoghlan #17967: urllib2.open failed to access a url when a perent directory of http://bugs.python.org/issue17967 opened by foxkiller #17969: multiprocessing crash on exit http://bugs.python.org/issue17969 opened by kristjan.jonsson #17970: Mutlithread XML parsing cause segfault http://bugs.python.org/issue17970 opened by mrDoctorWho0.. #17972: inspect module docs omits many functions http://bugs.python.org/issue17972 opened by s7v7nislands(a)gmail.com #17974: Migrate unittest to argparse http://bugs.python.org/issue17974 opened by pitrou #17975: libpython3.so conflicts between $VERSIONs http://bugs.python.org/issue17975 opened by prlw1 #17976: file.write doesn't raise IOError when it should http://bugs.python.org/issue17976 opened by jasujm #17978: Python crashes if Py_Initialize/Py_Finalize are called multipl http://bugs.python.org/issue17978 opened by romuloceccon #17979: Cannot build 2.7 with --enable-unicode=no http://bugs.python.org/issue17979 opened by amaury.forgeotdarc #17980: CVE-2013-2099 ssl.match_hostname() trips over crafted wildcard http://bugs.python.org/issue17980 opened by fweimer #17984: io and _pyio modules require the _io module http://bugs.python.org/issue17984 opened by serhiy.storchaka #17985: multiprocessing Queue.qsize() and Queue.empty() with different http://bugs.python.org/issue17985 opened by aod #17986: Alternative async subprocesses (pep 3145) http://bugs.python.org/issue17986 opened by sbt #17987: test.support.captured_stderr, captured_stdin not documented http://bugs.python.org/issue17987 opened by fdrake #17988: ElementTree.Element != ElementTree._ElementInterface http://bugs.python.org/issue17988 opened by jwilk #17989: ElementTree.Element broken attribute setting http://bugs.python.org/issue17989 opened by jwilk #17991: ctypes.c_char gives a misleading error when passed a one-chara http://bugs.python.org/issue17991 opened by Steven.Barker #17994: Change necessary in platform.py to support IronPython http://bugs.python.org/issue17994 opened by icordasc #17996: socket module should expose AF_LINK http://bugs.python.org/issue17996 opened by giampaolo.rodola #17997: ssl.match_hostname(): sub string wildcard should not match IDN http://bugs.python.org/issue17997 opened by christian.heimes #17998: internal error in regular expression engine http://bugs.python.org/issue17998 opened by jdemeyer Most recent 15 issues with no replies (15) ========================================== #17998: internal error in regular expression engine http://bugs.python.org/issue17998 #17997: ssl.match_hostname(): sub string wildcard should not match IDN http://bugs.python.org/issue17997 #17996: socket module should expose AF_LINK http://bugs.python.org/issue17996 #17994: Change necessary in platform.py to support IronPython http://bugs.python.org/issue17994 #17991: ctypes.c_char gives a misleading error when passed a one-chara http://bugs.python.org/issue17991 #17987: test.support.captured_stderr, captured_stdin not documented http://bugs.python.org/issue17987 #17986: Alternative async subprocesses (pep 3145) http://bugs.python.org/issue17986 #17975: libpython3.so conflicts between $VERSIONs http://bugs.python.org/issue17975 #17942: IDLE Debugger: names, values misaligned http://bugs.python.org/issue17942 #17933: test_ftp failure / ftplib error formatting issue http://bugs.python.org/issue17933 #17924: Deprecate stat.S_IF* integer constants http://bugs.python.org/issue17924 #17923: test glob with trailing slash fail http://bugs.python.org/issue17923 #17916: Provide dis.Bytecode based equivalent of dis.distb http://bugs.python.org/issue17916 #17909: Autodetecting JSON encoding http://bugs.python.org/issue17909 #17902: Document that _elementtree C API cannot use custom TreeBuilder http://bugs.python.org/issue17902 Most recent 15 issues waiting for review (15) ============================================= #17988: ElementTree.Element != ElementTree._ElementInterface http://bugs.python.org/issue17988 #17980: CVE-2013-2099 ssl.match_hostname() trips over crafted wildcard http://bugs.python.org/issue17980 #17979: Cannot build 2.7 with --enable-unicode=no http://bugs.python.org/issue17979 #17978: Python crashes if Py_Initialize/Py_Finalize are called multipl http://bugs.python.org/issue17978 #17976: file.write doesn't raise IOError when it should http://bugs.python.org/issue17976 #17974: Migrate unittest to argparse http://bugs.python.org/issue17974 #17956: add ScheduledExecutor http://bugs.python.org/issue17956 #17951: TypeError during gdb backtracing http://bugs.python.org/issue17951 #17947: Code, test, and doc review for PEP-0435 Enum http://bugs.python.org/issue17947 #17945: tkinter/Python 3.3.0: peer_create doesn't instantiate Text http://bugs.python.org/issue17945 #17944: Refactor test_zipfile http://bugs.python.org/issue17944 #17941: namedtuple should support fully qualified name for more portab http://bugs.python.org/issue17941 #17940: extra code in argparse.py http://bugs.python.org/issue17940 #17937: Collect garbage harder at shutdown http://bugs.python.org/issue17937 #17936: O(n**2) behaviour when adding/removing classes http://bugs.python.org/issue17936 Top 10 most discussed issues (10) ================================= #17914: add os.cpu_count() http://bugs.python.org/issue17914 36 msgs #17980: CVE-2013-2099 ssl.match_hostname() trips over crafted wildcard http://bugs.python.org/issue17980 31 msgs #17947: Code, test, and doc review for PEP-0435 Enum http://bugs.python.org/issue17947 27 msgs #15392: Create a unittest framework for IDLE http://bugs.python.org/issue15392 16 msgs #17961: Use enum names as values in enum.Enum convenience API http://bugs.python.org/issue17961 15 msgs #17936: O(n**2) behaviour when adding/removing classes http://bugs.python.org/issue17936 14 msgs #17969: multiprocessing crash on exit http://bugs.python.org/issue17969 13 msgs #17976: file.write doesn't raise IOError when it should http://bugs.python.org/issue17976 10 msgs #8604: Adding an atomic FS write API http://bugs.python.org/issue8604 9 msgs #17974: Migrate unittest to argparse http://bugs.python.org/issue17974 9 msgs Issues closed (41) ================== #6208: path separator output ignores shell's path separator: / instea http://bugs.python.org/issue6208 closed by terry.reedy #14596: struct.unpack memory leak http://bugs.python.org/issue14596 closed by pitrou #17237: m68k aligns on 16bit boundaries. http://bugs.python.org/issue17237 closed by pitrou #17468: Generator memory leak http://bugs.python.org/issue17468 closed by pitrou #17547: "checking whether gcc supports ParseTuple __format__... " erro http://bugs.python.org/issue17547 closed by python-dev #17563: Excessive resizing of dicts when used as a cache http://bugs.python.org/issue17563 closed by rhettinger #17606: xml.sax.saxutils.XMLGenerator doesn't support byte strings http://bugs.python.org/issue17606 closed by serhiy.storchaka #17732: distutils.cfg Can Break venv http://bugs.python.org/issue17732 closed by georg.brandl #17742: Add _PyBytesWriter API http://bugs.python.org/issue17742 closed by haypo #17754: test_ctypes assumes LANG=C LC_ALL=C http://bugs.python.org/issue17754 closed by doko #17843: Lib/test/testbz2_bigmem.bz2 trigger virus warnings http://bugs.python.org/issue17843 closed by georg.brandl #17895: TemporaryFile name returns an integer in python3 http://bugs.python.org/issue17895 closed by terry.reedy #17906: JSON should accept lone surrogates http://bugs.python.org/issue17906 closed by serhiy.storchaka #17915: Encoding error with sax and codecs http://bugs.python.org/issue17915 closed by georg.brandl #17920: Documentation: "complete ordering" should be "total ordering" http://bugs.python.org/issue17920 closed by rhettinger #17927: Argument copied into cell still referenced by frame http://bugs.python.org/issue17927 closed by benjamin.peterson #17943: AttributeError: 'long' object has no attribute 'release' in Qu http://bugs.python.org/issue17943 closed by georg.brandl #17948: HTTPS and sending a big file size hangs. http://bugs.python.org/issue17948 closed by pitrou #17949: operator documentation mixup http://bugs.python.org/issue17949 closed by ezio.melotti #17950: Dynamic classes contain non-breakable reference cycles http://bugs.python.org/issue17950 closed by gvanrossum #17952: editors-and-tools section of devguide does not appear to be ac http://bugs.python.org/issue17952 closed by ned.deily #17954: Support creation of extensible enums through metaclass subclas http://bugs.python.org/issue17954 closed by ncoghlan #17958: int(math.log(2**i, 2)) http://bugs.python.org/issue17958 closed by mark.dickinson #17959: Alternate approach to aliasing for PEP 435 http://bugs.python.org/issue17959 closed by ncoghlan #17962: Broken OpenSSL version in Windows builds http://bugs.python.org/issue17962 closed by python-dev #17964: os.sysconf(): return type of the C function sysconf() is long, http://bugs.python.org/issue17964 closed by haypo #17965: argparse does not dest.replace('-', '_') for positionals http://bugs.python.org/issue17965 closed by r.david.murray #17966: Lack of consistency in PEP 8 -- Style Guide for Python Code http://bugs.python.org/issue17966 closed by gvanrossum #17968: memory leak in listxattr() http://bugs.python.org/issue17968 closed by pitrou #17971: Weird interaction between Komodo Python debugger C module & Py http://bugs.python.org/issue17971 closed by benjamin.peterson #17973: '+=' on a list inside tuple both succeeds and raises an except http://bugs.python.org/issue17973 closed by ronaldoussoren #17977: urllib.request.urlopen() cadefault argument is documented with http://bugs.python.org/issue17977 closed by barry #17981: SysLogHandler closes connection before using it http://bugs.python.org/issue17981 closed by vinay.sajip #17982: Syntax Error in IDLE3 not in IDLE http://bugs.python.org/issue17982 closed by terry.reedy #17983: global __class__ statement in class declaration http://bugs.python.org/issue17983 closed by python-dev #17990: 2.7 builds can fail due to unconditional inclusion of include http://bugs.python.org/issue17990 closed by benjamin.peterson #17992: test_asynchat hangs http://bugs.python.org/issue17992 closed by neologix #17993: Missed comma causes unintentional implicit string literal conc http://bugs.python.org/issue17993 closed by serhiy.storchaka #17995: report，中　高　层　管　理　技　能158766 http://bugs.python.org/issue17995 closed by fdrake #995907: memory leak with threads and enhancement of the timer class http://bugs.python.org/issue995907 closed by neologix #1662581: the re module can perform poorly: O(2**n) versus O(n**2) http://bugs.python.org/issue1662581 closed by gregory.p.smith

1 0

Why is nb_inplace_add copied to sq_inplace_concat?
by Matt Newell 17 May '13

17 May '13

I have encountered what I believe to be a bug but I'm sure there is some reason things are done as they are and I am hoping someone can shed some light or confirm it is indeed a bug. As a bit of background I have a c++ class that I use sip to generate python bindings. The potential python bug manifests itself as: >>> rl = RecordList() >>> rl += [] >>> rl NotImplemented The bindings fill in nb_inplace_add which appears to be implemented properly, returning a new reference to Py_NotImplemented if the right hand argument is not as expected. Where things appear to go wrong is that PyNumber_InPlaceAdd, after getting a NotImplemented return value from nb_inplace_add, then attempts to call sq_inplace_concat. From reading the code it appears that sq_inplace_concat is not supposed to return NotImplemented, instead it should set an exception and return null if the right hand arg is not supported. In my case sq_inplace_concat ends up being the same function as nb_inplace_add, which results in the buggy behavior. When I figured this out I tried to find out why sq_inplace_concat was set to the same function as nb_inplace_add, and ended up having to set a watchpoint in gdb which finally gave me the answer that python itself is setting sq_inplace_concat during type creation in the various functions in typeobject.c. Stack trace is below. I don't really understand what the fixup_slot_dispatchers function is doing, but it does seem like there must be a bug either in what it's doing, or in PyNumber_InPlaceAdd's handling of a NotImplemented return value from sq_inplace_concat. Thanks, Matt Python 2.7.3 (default, Jan 2 2013, 13:56:14) [GCC 4.7.2] on linux2 Stack trace where a watch on sq->sq_inplace_concat reveals the change: Hardware watchpoint 5: *(binaryfunc *) 0xcf6f88 Old value = (binaryfunc) 0 New value = (binaryfunc) 0x7ffff4d41c78 <slot_RecordList___iadd__(PyObject*, PyObject*)> #0 update_one_slot.25588 (type=type@entry=0xcf6c70, p=0x86ba90) at ../Objects/typeobject.c:6203 #1 0x00000000004b96d0 in fixup_slot_dispatchers (type=0xcf6c70) at ../Objects/typeobject.c:6299 #2 type_new.part.40 (kwds=<optimized out>, args=0x0, metatype=<optimized out>) at ../Objects/typeobject.c:2464 #3 type_new.25999 (metatype=<optimized out>, args=0x0, kwds=<optimized out>) at ../Objects/typeobject.c:2048 #4 0x0000000000463c08 in type_call.25547 (type=0x7ffff65953a0, args=('RecordList', (<sip.wrappertype at remote 0x7ffff5201080>,), {'__module__': 'blur.Stone'}), kwds=0x0) at ../Objects/typeobject.c:721 #5 0x00000000004644eb in PyObject_Call (func=<type at remote 0x7ffff65953a0>, arg=<optimized out>, kw=<optimized out>) at ../Objects/abstract.c:2529

2 3

Re: [Python-Dev] [Python-checkins] cpython: fix compilation on Windows
by Terry Reedy 17 May '13

17 May '13

On 5/16/2013 4:17 PM, victor.stinner wrote: > summary: fix compilation on Windows That fixed my problem with compiling 3.4, 32 bit, Win 7. Thanks. But I cannot compile 3.3 python_d since May 6. In fact, there are more errors now than 8 hours ago. 7 things failed to build instead of 5 (3 is normal for me, given the lack of some dependencies). I believe the following is new. Red error box with .../p33/PCBuild/make_versioninfo_d.exe is not a valid Win32 application. The VS gui output box has "Please verify that you have sufficient rights to run this command." Some more errors: 10>..\PC\pylauncher.rc(16): error RC2104: undefined keyword or key name: FIELD3 10> 9> symtable.c 9>..\Python\symtable.c(1245): error C2143: syntax error : missing ';' before 'type' 9>..\Python\symtable.c(1246): error C2065: 'cur' : undeclared identifier 9>..\Python\symtable.c(1248): error C2065: 'cur' : undeclared identifier 9>..\Python\symtable.c(1253): error C2065: 'cur' : undeclared identifier 23> Traceback (most recent call last): 23> File "build_ssl.py", line 253, in <module> 23> main() 23> File "build_ssl.py", line 187, in main 23> os.chdir(ssl_dir) 23> FileNotFoundError: [WinError 2] The system cannot find the file specified: '..\\..\\openssl-1.0.1e' Earlier, about 4 other files had several warnings. I do no see the warnings now because they compiled then and have not changed. Errors are more urgent, but should warnings be ignored? Terry

1 0