Mailman 3 May 2010 - Python-Dev

urllib2/urllib incompatibility?
by Vinay Sajip 24 May '10

24 May '10

> Not top-posting, but gmane seems too finicky in this area. I encountered what seems like an incompatibility between urllib and urllib2 in the way they handle file:// URLs, is this a bug? I had a look on the bug tracker and via Google to see if there were prior reports, but perhaps my search-fu is deficient. Here's a console session to illustrate: vinay@eta-karmic:/tmp$ echo Hello, world! >hello.txt vinay@eta-karmic:/tmp$ cat hello.txt Hello, world! vinay@eta-karmic:/tmp$ python Python 2.6.4 (r264:75706, Dec 7 2009, 18:45:15) [GCC 4.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import urllib,urllib2 >>> s = 'file:////tmp/hello.txt' >>> f1 = urllib.urlopen(s) >>> f1.read() 'Hello, world!\n' >>> f2 = urllib2.urlopen(s) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python2.6/urllib2.py", line 124, in urlopen return _opener.open(url, data, timeout) File "/usr/lib/python2.6/urllib2.py", line 389, in open response = self._open(req, data) File "/usr/lib/python2.6/urllib2.py", line 407, in _open '_open', req) File "/usr/lib/python2.6/urllib2.py", line 367, in _call_chain result = func(*args) File "/usr/lib/python2.6/urllib2.py", line 1240, in file_open return self.parent.open(req) File "/usr/lib/python2.6/urllib2.py", line 389, in open response = self._open(req, data) File "/usr/lib/python2.6/urllib2.py", line 407, in _open '_open', req) File "/usr/lib/python2.6/urllib2.py", line 367, in _call_chain result = func(*args) File "/usr/lib/python2.6/urllib2.py", line 1287, in ftp_open raise URLError('ftp error: no host given') urllib2.URLError: <urlopen error ftp error: no host given> >>> Anyone seen this before? Should I file an issue on the tracker? If I've missed something obvious, sorry for the noise, but please do tell! Regards, Vinay Sajip

2 2

Ask a question for a script about re.findall Modlue
by Lu, Hongchao (UI Health Care) 22 May '10

22 May '10

Hi All, I am working on a script to use re.findall, But the result seems wield. -------------------------------------------------------------------- import re p=re.compile("\S+",re.M) pirfile=""" >DL;mm9| >DL;petMar1| >DL;cavPor3| >DL;mm9|: """ for m in p.findall(pirfile,re.M): print m Results: | >DL;petMar1| >DL;cavPor3| >DL;mm9| The first result is missing some characters, Could you explain why? Any wrong with the script? I tried Python2.4/2.6/2.7, the result is same. Thank you. Hongchao Lu

3 2

Adding a new C API function in 2.6
by Antoine Pitrou 21 May '10

21 May '10

Hello, I would like to check that it's possible to a new C API function in the 2.6 branch, on the basis that it would help solve what seems to be reported as a security problem by several vendors (including Linux distributions) -- see http://bugs.python.org/issue5753 for a thorough discussion. The change is rather minimal at the code level; it adds a new function PySys_SetArgvEx which has an additional flag telling it whether to update sys.path or not. The existing PySys_SetArgv function unconditionally updates sys.path, which can allow shadowing of stdlib or third-party library modules by an attacker. Thank you Antoine.

5 6

Summary of Python tracker Issues
by Python tracker 21 May '10

21 May '10

ACTIVITY SUMMARY (2010-05-14 - 2010-05-21) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2693 open (+46) / 17908 closed (+24) / 20601 total (+70) Open issues with patches: 1098 Average duration of open issues: 719 days. Median duration of open issues: 497 days. Open Issues Breakdown open 2672 (+46) languishing 12 ( +0) pending 8 ( +0) Issues Created Or Reopened (73) _______________________________ heapq change breaking compatibility 2010-05-17 CLOSED http://bugs.python.org/issue3051 reopened exarkun patch subprocess.Popen.__del__ causes AttributeError (os module == N 2010-05-14 CLOSED http://bugs.python.org/issue5099 reopened haypo patch, needs review Buggy _strerror in asyncore 2010-05-18 CLOSED http://bugs.python.org/issue8573 reopened merwok easy multiprocessing needs option to eschew fork() under Linux 2010-05-14 http://bugs.python.org/issue8713 created brandon-rhodes Delayed signals in the REPL on OpenBSD (possibly libpthread re 2010-05-14 http://bugs.python.org/issue8714 created skrah Create PyUnicode_EncodeFSDefault() function 2010-05-14 CLOSED http://bugs.python.org/issue8715 created haypo patch test_tk fails on OS X if run from buildbot slave daemon -- cra 2010-05-14 http://bugs.python.org/issue8716 created janssen patch Subprocess.py broken in trunk 2010-05-14 CLOSED http://bugs.python.org/issue8717 created pakal patch subprocess: NameError: name 'WaitForSingleObject' is not def 2010-05-14 CLOSED http://bugs.python.org/issue8718 created srid buildbot: segfault on FreeBSD (signal 11) 2010-05-14 http://bugs.python.org/issue8719 created haypo undo findsource regression/change 2010-05-14 http://bugs.python.org/issue8720 created hpk patch urlparse.urlsplit regression in 2.7 2010-05-15 CLOSED http://bugs.python.org/issue8721 created srid Documentation for __getattr__ 2010-05-15 http://bugs.python.org/issue8722 created Paul.Davis IDLE won't start import os error 2010-05-15 CLOSED http://bugs.python.org/issue8723 created gromij bind_and_activate parameter is missed from directive 2010-05-15 http://bugs.python.org/issue8724 created naoki Python3: use ASCII for the file system encoding on initfsencod 2010-05-15 http://bugs.python.org/issue8725 created haypo patch test_capi failure 2010-05-15 CLOSED http://bugs.python.org/issue8726 created pitrou test_import failure 2010-05-15 CLOSED http://bugs.python.org/issue8727 created pitrou 2.7 regression in httplib.py: AttributeError: 'NoneType' objec 2010-05-15 http://bugs.python.org/issue8728 created srid The Mapping ABC's __eq__ method should return NotImplemented i 2010-05-15 http://bugs.python.org/issue8729 created stutzbach patch Spurious test failure in distutils 2010-05-16 http://bugs.python.org/issue8730 created ronaldoussoren BeOS for 2.7 - add to unsupported 2010-05-16 http://bugs.python.org/issue8731 created rpetrov patch Should urrllib2.urlopen send an Accept-Encoding header? 2010-05-16 http://bugs.python.org/issue8732 created dabrahams list type and UserList do not call super in __init__ and there 2010-05-16 http://bugs.python.org/issue8733 created alonho msvcrt get_osfhandle crash on bad FD 2010-05-16 http://bugs.python.org/issue8734 created pakal patch optparse: parse_args(values=...) does not set up default value 2010-05-16 http://bugs.python.org/issue8735 created john(a)nmt.edu *= is undocumented for lists 2010-05-16 CLOSED http://bugs.python.org/issue8736 created stutzbach easy ssl.RAND_add() should only accept bytes (not str) 2010-05-16 CLOSED http://bugs.python.org/issue8737 created haypo patch cPickle dumps(tuple) != dumps(loads(dumps(tuple))) 2010-05-17 http://bugs.python.org/issue8738 created Alberto.Planas.Domínguez Update to smtpd.py to RFC 5321 2010-05-17 http://bugs.python.org/issue8739 created alfmel patch infinite recursion with setfilesystemencoding and pdb 2010-05-17 CLOSED http://bugs.python.org/issue8740 created ccomb 2.7 regression in tarfile: IOError: link could not be created 2010-05-17 http://bugs.python.org/issue8741 created srid broken asdl link in Parser/asdl.py 2010-05-17 CLOSED http://bugs.python.org/issue8742 created pyfex set() operators don't work with collections.Set instances 2010-05-17 http://bugs.python.org/issue8743 created stutzbach Maybe typo in doc 2010-05-18 CLOSED http://bugs.python.org/issue8744 created naoki zipimport is a bit slow 2010-05-18 http://bugs.python.org/issue8745 created Goplat patch *chflags detection broken on FreeBSD 9-CURRENT 2010-05-18 http://bugs.python.org/issue8746 created yaneurabeya Autoconf tests in python not portably correct 2010-05-18 http://bugs.python.org/issue8747 created yaneurabeya integer-to-complex comparisons give incorrect results 2010-05-18 http://bugs.python.org/issue8748 created mark.dickinson patch Cruft in object.c: PyObject_GenericGetAttr 2010-05-18 http://bugs.python.org/issue8749 created Yaniv.Aknin Many of MutableSet's methods assume that the other parameter i 2010-05-18 http://bugs.python.org/issue8750 created stutzbach patch, needs review Threading and KeyError: 51 2010-05-18 CLOSED http://bugs.python.org/issue8751 created Vignesh.K set_contains, etc. should check for collections.Set in additio 2010-05-18 CLOSED http://bugs.python.org/issue8752 created stutzbach Py_ReprEnter and Py_ReprLeave are undocumented 2010-05-18 http://bugs.python.org/issue8753 created stutzbach ImportError: quote bad module name in message 2010-05-18 http://bugs.python.org/issue8754 created tjreedy easy idle crash UnicodeDecodeError utf-8 codec 2010-05-18 http://bugs.python.org/issue8755 created andyharrington Multiprocessing and UUID bug on Mac OSX 2010-05-18 CLOSED http://bugs.python.org/issue8756 created Gavin.Roy Race condition when checking for set in set 2010-05-18 http://bugs.python.org/issue8757 created stutzbach BUILD_LIST followed by BINARY_SUBSCR can be optimized to a BUI 2010-05-18 http://bugs.python.org/issue8758 created alex 2.7: wrong user site directory on Linux 2010-05-18 CLOSED http://bugs.python.org/issue8759 created srid Python 2.6.5 fails to build on AIX 5.3 2010-05-18 CLOSED http://bugs.python.org/issue8760 created oirraza PyUnicode_CompareWithASCIIString name is not mangled (UCS2, UC 2010-05-18 http://bugs.python.org/issue8761 created haypo patch default value in constructor not unique across objects 2010-05-19 CLOSED http://bugs.python.org/issue8762 created bolt py3K bdist_msi wrongly installs itself in ALL python versions 2010-05-19 http://bugs.python.org/issue8763 created pakal logging RotatingFileHandler cause too long to fininsh 2010-05-19 CLOSED http://bugs.python.org/issue8764 created davy.zhang Tests unwillingly writing unicocde to raw streams 2010-05-19 http://bugs.python.org/issue8765 created pakal patch Segmentation fault with empty "encodings" subdirectory of dire 2010-05-19 CLOSED http://bugs.python.org/issue8766 created Arfrever patch Configure: Cannot disable unicode 2010-05-19 CLOSED http://bugs.python.org/issue8767 created redcomet The checkempty_symmetric_difference test is never called 2010-05-19 http://bugs.python.org/issue8768 created stutzbach patch Straightforward usage of email package fails to round-trip 2010-05-19 http://bugs.python.org/issue8769 created akuchling patch Make 'python -m sysconfig' print something useful 2010-05-19 http://bugs.python.org/issue8770 created srid Socket freezing under load issue on Mac. 2010-05-19 http://bugs.python.org/issue8771 created amcgregor sysconfig: _get_default_scheme can be made public? 2010-05-20 http://bugs.python.org/issue8772 created srid mailbox module is needlessly executable 2010-05-20 http://bugs.python.org/issue8773 created meatballhat tabnanny improperly handles non-ascii source files 2010-05-20 CLOSED http://bugs.python.org/issue8774 created meatballhat patch Use locale encoding to decode sys.argv, not the file system en 2010-05-20 http://bugs.python.org/issue8775 created haypo Bytes version of sys.argv 2010-05-20 http://bugs.python.org/issue8776 created haypo Add threading.Barrier 2010-05-20 http://bugs.python.org/issue8777 created krisvale patch, patch typo in docs for symtable.SymbolTable.has_import_star 2010-05-20 CLOSED http://bugs.python.org/issue8778 created terrence patch utf8 codec fails to parse a character 2010-05-20 CLOSED http://bugs.python.org/issue8779 created Roman.Gershman py3k: child process don't inherit stdout / stdout on Windows 2010-05-21 http://bugs.python.org/issue8780 created haypo patch 32-bit wchar_t doesn't need to be unsigned to be usable (I thi 2010-05-21 http://bugs.python.org/issue8781 created stutzbach inspect.getsource returns invalid source for non-newline-endin 2010-05-21 http://bugs.python.org/issue8782 created hpk patch Issues Now Closed (66) ______________________ Removal of stale code in pyconfig.h 904 days http://bugs.python.org/issue1512 loewis Codepage unset in msilib.init_database() 855 days http://bugs.python.org/issue1855 loewis msilib.SetProperty(msilib.PID_CODEPAGE, '1252') raises 0x65d = 850 days http://bugs.python.org/issue1884 loewis add ftp-tls support to ftplib - RFC 4217 832 days http://bugs.python.org/issue2054 giampaolo.rodola patch Make sysmodule.c compatible with Bazaar 786 days http://bugs.python.org/issue2456 barry heapq change breaking compatibility 2 days http://bugs.python.org/issue3051 r.david.murray patch Py_FatalError causes infinite loop 633 days http://bugs.python.org/issue3605 haypo needs review PKG-INFO file should differentiate between authors and maintai 628 days http://bugs.python.org/issue3686 tarek urllib2 sends Basic auth across redirects 617 days http://bugs.python.org/issue3819 kiilerix patch, easy ssl module is missing SSL_OP_NO_SSLv2 499 days http://bugs.python.org/issue4870 pitrou patch subprocess.Popen.__del__ causes AttributeError (os module == N 0 days http://bugs.python.org/issue5099 haypo patch, needs review csv sniffer 454 days http://bugs.python.org/issue5332 r.david.murray patch tarfile normalizes arcname 364 days http://bugs.python.org/issue6054 srid 2to3 fails to fix test.test_support 295 days http://bugs.python.org/issue6583 pakal file_close() ignores return value of close_the_file 222 days http://bugs.python.org/issue7079 pitrou patch, needs review distutils makefile from python.org installer doesn't work on O 144 days http://bugs.python.org/issue7580 loewis buffered io seek() buggy 130 days http://bugs.python.org/issue7640 pitrou patch newgil backport 117 days http://bugs.python.org/issue7753 jcea patch, needs review relative import broken 99 days http://bugs.python.org/issue7902 brett.cannon patch, needs review _ssl: support surrogates in filenames, and bytes/bytearray fil 26 days http://bugs.python.org/issue8477 haypo patch subprocess: support bytes program name (POSIX) 25 days http://bugs.python.org/issue8513 haypo patch, needs review Expose SSL contexts 19 days http://bugs.python.org/issue8550 pitrou patch test_gdb: test_strings() fails with ASCII locale 22 days http://bugs.python.org/issue8559 haypo patch Buggy _strerror in asyncore 0 days http://bugs.python.org/issue8573 giampaolo.rodola easy test_warnings.CEnvironmentVariableTests.test_nonascii fails un 18 days http://bugs.python.org/issue8589 haypo patch Python3/POSIX: errors if file system encoding is None 11 days http://bugs.python.org/issue8610 haypo patch Doc bug for urllib.request._urlopener in Python 3.1+ 13 days http://bugs.python.org/issue8619 orsenthil tarfile doesn't support undecodable filename in PAX format 12 days http://bugs.python.org/issue8633 lars.gustaebel subprocess: canonicalize env to bytes on Unix (Python3) 12 days http://bugs.python.org/issue8640 haypo patch Failed compile in a non-ASCII path 11 days http://bugs.python.org/issue8663 haypo patch "make pycremoval" fails 8 days http://bugs.python.org/issue8665 pitrou distutils sdist is too laze w.r.t. recalculating MANIFEST 6 days http://bugs.python.org/issue8688 tarek patch, needs review sqlite3 parameter substitution breaks with multiple parameters 3 days http://bugs.python.org/issue8689 tjreedy Use divide-and-conquer for faster factorials 5 days http://bugs.python.org/issue8692 mark.dickinson patch, patch, needs review OpenID blunder 2 days http://bugs.python.org/issue8708 brett.cannon Document PyUnicode_DecodeFSDefault() and PyUnicode_DecodeFSDef 0 days http://bugs.python.org/issue8711 haypo patch Create PyUnicode_EncodeFSDefault() function 1 days http://bugs.python.org/issue8715 haypo patch Subprocess.py broken in trunk 0 days http://bugs.python.org/issue8717 haypo patch subprocess: NameError: name 'WaitForSingleObject' is not def 0 days http://bugs.python.org/issue8718 haypo urlparse.urlsplit regression in 2.7 5 days http://bugs.python.org/issue8721 r.david.murray IDLE won't start import os error 0 days http://bugs.python.org/issue8723 gromij test_capi failure 2 days http://bugs.python.org/issue8726 jyasskin test_import failure 3 days http://bugs.python.org/issue8727 barry *= is undocumented for lists 1 days http://bugs.python.org/issue8736 stutzbach easy ssl.RAND_add() should only accept bytes (not str) 0 days http://bugs.python.org/issue8737 haypo patch infinite recursion with setfilesystemencoding and pdb 2 days http://bugs.python.org/issue8740 ccomb broken asdl link in Parser/asdl.py 1 days http://bugs.python.org/issue8742 orsenthil Maybe typo in doc 0 days http://bugs.python.org/issue8744 orsenthil Threading and KeyError: 51 0 days http://bugs.python.org/issue8751 draghuram set_contains, etc. should check for collections.Set in additio 0 days http://bugs.python.org/issue8752 rhettinger Multiprocessing and UUID bug on Mac OSX 0 days http://bugs.python.org/issue8756 r.david.murray 2.7: wrong user site directory on Linux 2 days http://bugs.python.org/issue8759 r.david.murray Python 2.6.5 fails to build on AIX 5.3 3 days http://bugs.python.org/issue8760 amaury.forgeotdarc default value in constructor not unique across objects 0 days http://bugs.python.org/issue8762 ysj.ray logging RotatingFileHandler cause too long to fininsh 0 days http://bugs.python.org/issue8764 vinay.sajip Segmentation fault with empty "encodings" subdirectory of dire 1 days http://bugs.python.org/issue8766 haypo patch Configure: Cannot disable unicode 0 days http://bugs.python.org/issue8767 redcomet tabnanny improperly handles non-ascii source files 1 days http://bugs.python.org/issue8774 haypo patch typo in docs for symtable.SymbolTable.has_import_star 0 days http://bugs.python.org/issue8778 benjamin.peterson patch utf8 codec fails to parse a character 0 days http://bugs.python.org/issue8779 benjamin.peterson csv module cannot handle embedded \r 2174 days http://bugs.python.org/issue967934 r.david.murray ftplib: add support for MDTM command 2033 days http://bugs.python.org/issue1053369 giampaolo.rodola patch Bugs in _csv module - lineterminator 2004 days http://bugs.python.org/issue1072404 r.david.murray Missing trailing newline with comment raises SyntaxError 1860 days http://bugs.python.org/issue1184112 benjamin.peterson easy urllib.quote is too slow 1712 days http://bugs.python.org/issue1285086 flox patch Py_FileSystemDefaultEncoding can be non-canonical 1262 days http://bugs.python.org/issue1608805 haypo patch Top Issues Most Discussed (10) ______________________________ 26 Use divide-and-conquer for faster factorials 5 days closed http://bugs.python.org/issue8692 19 implement PEP 3118 struct changes 703 days open http://bugs.python.org/issue3132 16 integer-to-complex comparisons give incorrect results 3 days open http://bugs.python.org/issue8748 10 tabnanny improperly handles non-ascii source files 1 days closed http://bugs.python.org/issue8774 10 urlparse.urlsplit regression in 2.7 5 days closed http://bugs.python.org/issue8721 10 Check that _PyUnicode_AsString() result is not NULL 281 days open http://bugs.python.org/issue6697 9 Python 2.6.5 fails to build on AIX 5.3 3 days closed http://bugs.python.org/issue8760 9 Failed compile in a non-ASCII path 11 days closed http://bugs.python.org/issue8663 8 2.7: wrong user site directory on Linux 2 days closed http://bugs.python.org/issue8759 8 test_import failure 3 days closed http://bugs.python.org/issue8727

1 0

PEP 3148 ready for pronouncement
by Yannick Loitière 21 May '10

21 May '10

The description of Future.add_done_callback() does not specify whether the callbacks are done in any particular order (such as the order they were added) or not. I'm guessing they are not, but believe it should be documented either way. -- Yannick Loitière

1 0

Reasons behind misleading TypeError message when passing the wrong number of arguments to a method
by Giampaolo Rodolà 21 May '10

21 May '10

>>> class A: ... def echo(self, x): ... return x ... >>> a = A() >>> a.echo() Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: echo() takes exactly 2 arguments (1 given) >>> I bet my last 2 cents this has already been raised in past but I want to give it a try and revamp the subject anyway. Is there a reason why the error shouldn't be adjusted to state that *1* argument is actually required instead of 2? --- Giampaolo http://code.google.com/p/pyftpdlib http://code.google.com/p/psutil

13 19

bug or feature? fixing argparse's default help value for version actions
by Steven Bethard 20 May '10

20 May '10

Sorry I haven't had time to get around to the argparse issues. I should have time this weekend. I need a release manager call on one of the issues though. Two things I assume are fine to fix at this stage: * In the documentation, the '--version' example should either not use a shorthand, or should use the conventional '-V' * In the documentation, the difference between the argparse and optparse ways of specifying versions needs to be mentioned in the section on migrating from optparse. One thing I'm not sure about: * Right now, the default help value for the version action is None, which is pretty useless since it means that everyone has to type in the same help="show program's version number and exit" when they use the version action. Instead, the string "show program's version number and exit" should have been the default value for the help argument. To be explicit, right now everyone who adds a version to their argument parser has to write: parser.add_argument('-V', action='version', version='<the version>', help="show program's version number and exit") With the fixed default value you would only have to write: parser.add_argument('-V', action='version', version='<the version>') Can this be considered a bug, i.e. something that can be committed before the RC? Or does it need to be considered a feature, i.e. at this point can never be added to Python 2.7? Thanks, and sorry again for not having time for this until now, Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus

4 3

Incorrect length of collections.Counter objects / Multiplicity function
by Gustavo Narea 20 May '10

20 May '10

Hello, everyone. I've checked the new collections.Counter class and I think I've found a bug: > >>> from collections import Counter > >>> c1 = Counter([1, 2, 1, 3, 2]) > >>> c2 = Counter([1, 1, 2, 2, 3]) > >>> c3 = Counter([1, 1, 2, 3]) > >>> c1 == c2 and c3 not in (c1, c2) > True > >>> # Perfect, so far. But... There's always a "but": > ... > >>> len(c1) > 3 The length of a Counter is the amount of unique elements. But the length must be the cardinality, and the cardinality of a multiset is the total number of elements (including duplicates) [1] [2]. The source code mentions that the recipe on ActiveState [3] was one of the references, but that recipe has this right. Also, why is it indexed? The indexes of a multiset call to mind the position of its elements, but there's no such thing in sets. I think this is inconsistent with the built-in set. I would have implemented the multiplicity function as a method instead of the indexes: c1.get_multiplicity(element) # instead of c1[element] Is this the intended behavior? If so, I'd like to propose a proper multiset implementation for the standard library (preferably called "Multiset"; should I create a PEP?). If not, I can write a patch to fix it, although I'm afraid it'd be a backwards incompatible change. Cheers, [1] http://en.wikipedia.org/wiki/Multiset#Overview [2] http://preview.tinyurl.com/smalltalk-bag [3] http://code.activestate.com/recipes/259174/ -- Gustavo Narea <xri://=Gustavo>. | Tech blog: =Gustavo/(+blog)/tech ~ About me: =Gustavo/about |

3 4

Re: [Python-Dev] Fixing the GIL (with a BFS scheduler)
by David Beazley 19 May '10

19 May '10

> From: "Martin v. L?wis" <martin(a)v.loewis.de> > To: Dj Gilcrease <digitalxero(a)gmail.com> > Cc: python-dev(a)python.org > Subject: Re: [Python-Dev] Fixing the GIL (with a BFS scheduler) > Message-ID: <4BF385E3.9030903(a)v.loewis.de> > Content-Type: text/plain; charset=ISO-8859-1 > >> I think the new GIL should be given a year or so in the wild before >> you start trying to optimize theoretical issues you may run into. If >> in a year people come back and have some examples of where a proper >> scheduler would help improve speed on multi-core systems even more, >> then we can address the issue at that time. > > Exactly my feelings. > Although I don't agree that the problem of I/O convoying is merely some "theoretical issue", I would agree with a go-slow approach---after all, the new GIL hasn't even appeared in any actual release yet. It might be a good idea to prominently document the fact that the new GIL has some known performance issues (e.g., possible I/O convoying), but that feedback concerning the performance of real-world applications is desired. Cheers, Dave

5 5

pybuildbot.identify?
by Bill Janssen 19 May '10

19 May '10

The PPC buildbots are running pretty well, now that I've opened a few more ports, but I'd like to find this script "pybuildbot.identify" that they keep complaining about, and install it. I've poked around the Python sources, but haven't found it. Anyone know where to get it from? Thanks. Bill

3 5

Fixing the GIL (with a BFS scheduler)
by Nir Aides 19 May '10

19 May '10

Hi all, Here is a second (last?) attempt at getting traction on fixing the GIL (is it broken?) with BFS (WTF?). So don't be shy (don't be too rude either) since ignoring counts as down voting. Relevant Python issue: http://bugs.python.org/issue7946 *Bottom line first* I submitted an implementation of BFS ( http://ck.kolivas.org/patches/bfs/sched-BFS.txt) as a patch to the GIL, which to the extent I have tested it, behaves nicely on Windows XP, Windows 7, GNU/Linux with either CFS or O(1) schedulers, 1/2/4 cores, laptop, desktop and VirtualBox VM guest (some data below). The patch is still work in progress and requires work in terms of style, moving code where it belongs, test code, etc... nevertheless, Python core developers recommended I already (re)post to python-dev for discussion. *So is the GIL broken?* There seems to be some disagreement on that question among Python core developers (unless you all agree it is not broken :) ). Some developers maintain the effects described by David Beazley do not affect real world systems. Even I took the role of a devil's advocate in a previous discussion, but in fact I think that Python, being a general purpose language, is similar to the OS in that regard. It is used across many application domains, platforms, and development paradigms, just as OS schedulers are, and therefore accepting thread scheduling with such properties as a fact of life is not a good idea. I was first bitten by the original GIL last year while testing a system, and found David's research while looking for answers, and later had to work around that problem in another system. Here are other real world cases: 1) Zope people hit this back in 2002 and documented the problem with interesting insight: http://www.zope.org/Members/glpb/solaris/multiproc "I have directly observed a 30% penalty under MP constraints when the sys.setcheckinterval value was too low (and there was too much GIL thrashing)." http://www.zope.org/Members/glpb/solaris/report_ps "A machine that's going full-throttle isn't as bad, curiously enough -- because the other CPU's are busy doing real work, the GIL doesn't have as much opportunity to get shuffled between CPUs. On a MP box it's very important to set sys.setcheckinterval() up to a fairly large number, I recommend pystones / 50 or so." 2) Python mailing list - 2005 http://mail.python.org/pipermail/python-list/2005-August/336286.html "The app suffers from serious performance degradation (compared to pure c/C++) and high context switches that I suspect the GIL unlocking may be aggravating ?" 3) Python mailing list - 2008 http://mail.python.org/pipermail/python-list/2008-June/1143217.html "When I start the server, it sometimes eats up 100% of the CPU for a good minute or so... though none of the threads are CPU-intensive" 4) Twisted http://twistedmatrix.com/pipermail/twisted-python/2005-July/011048.html "When I run a CPU intensive method via threads.deferToThread it takes all the CPU away and renders the twisted process unresponsive." Admittedly, it is not easy to dig reports up in Google. Finally, I think David explained the relevance of this problem quite nicely: http://mail.python.org/pipermail/python-dev/2010-March/098416.html *What about the new GIL?* There is no real world experience with the new GIL since it is under development. What we have is David's analysis and a few benchmarks from the bug report. *Evolving the GIL into a scheduler* The problem addressed by the GIL has always been *scheduling* threads to the interpreter, not just controlling access to it. The patches by Antoine and David essentially evolve the GIL into a scheduler, however both cause thread starvation or high rate of context switching in some scenarios (see data below). *BFS* Enter BFS, a new scheduler designed by Con Kolivas, a Linux kernel hacker who is an expert in this field: http://ck.kolivas.org/patches/bfs/sched-BFS.txt "The goal of the Brain Fuck Scheduler, referred to as BFS from here on, is to completely do away with the complex designs of the past for the cpu process scheduler and instead implement one that is very simple in basic design. The main focus of BFS is to achieve excellent desktop interactivity and responsiveness without heuristics and tuning knobs that are difficult to understand, impossible to model and predict the effect of, and when tuned to one workload cause massive detriment to another." I submitted an implementation of BFS (bfs.patch) which on my machines gives comparable performance to gilinter2.patch (Antoine's) and seems to schedule threads more fairly, predictably, and with lower rate of context switching (see data below). There are however, some issues in bfs.patch: 1) It works on top of the OS scheduler, which means (for all GIL patches!): a) It does not control and is not aware of information such as OS thread preemption, CPU core to run on, etc... b) There may be hard to predict interaction between BFS and the underlying OS scheduler, which needs to be tested on each target platform. 2) It works best when TSC (http://en.wikipedia.org/wiki/Time_Stamp_Counter) is available and otherwise falls back to gettimeofday(). I expect the scheduler to misbehave to some degree or affect performance when TSC is not available and either of the following is true: a) if gettimeofday() is very expensive to read (impacts release/acquire overhead). b) if gettimeofday() has very low precision ~10ms. By design of BFS, once CPU load crosses a given threshold (about 8 CPU bound tasks which need the CPU at once), the scheduler falls back to FIFO behavior and latency goes up sharply. I have no data on how bfs.patch behaves on ARM, AMD, old CPU models, OSX, FreeBSD, Solaris, or mobile. The patch may require some tuning to work properly on those systems, so data is welcome (make sure TSC code in Include/cycle.h works on those systems before benching). All that said, to the extent I have tested it, bfs.patch behaves nicely on Windows XP, Windows 7, GNU/Linux with either CFS or O(1) schedulers, 1/2/4 cores, laptop, desktop and VirtualBox VM guest. *Data* Comparison of proposed patches running ccbench on Windows XP: http://bugs.python.org/issue7946#msg104899 Comparison of proposed patches running Florent's write.py test on Ubuntu Karmic: http://bugs.python.org/issue7946#msg105687 Comparison of old GIL, new GIL and BFS running ccbench on Ubuntu Karmic: http://bugs.python.org/issue7946#msg105874 Last comparison includes a run of old GIL with sys.setcheckinterval(2500) as Zope people do. IO latency shoots up to ~1000ms as result. *What can be done with it?* Here are some options: 1) Abandon it - no one is interested, yawn. 2) Take ideas and workarounds from its code and apply to other patches. 3) Include it in the interpreter as an auxiliary (turn on with a runtime switch) scheduler. 4) Adopt it as the Python scheduler. *Opinion?* Your opinion is needed (however, please submit code review comments which are not likely to interest other people, e.g. "why did you use volatile for X?", at the issue page: http://bugs.python.org/issue7946). Thanks, Nir

13 49

Re: [Python-Dev] Summing up
by David Beazley 19 May '10

19 May '10

Antoine, This is a pretty good summary that mirrors my thoughts on the GIL matter as well. In the big picture, I do think it's desirable for Python to address the multicore performance issue--namely to not have the performance needlessly thrashed in that environment. The original new GIL addressed this. The I/O convoy effect problem is more subtle. Personally, I think it's an issue that at least merits further study because trying to overlap I/O with computation is a known programming technique that might be useful for people using Python to do message passing, distributed computation, etc. As an example, the multiprocessing module uses threads as part of its queue implementation. Is it impacted by convoying? I honestly don't know. I agree that getting some more real-world experience would be useful. Cheers, Dave > From: Antoine Pitrou <solipsis(a)pitrou.net> > > Ok, this is a good opportunity to try to sum up, from my point of view. > > The main problem of the old GIL, which was evidenced in Dave's original > study (not this year's, but the previous one) *is* fixed unless someone > demonstrates otherwise. > > It should be noted that witnessing a slight performance degradation on > a multi-core machine is not enough to demonstrate such a thing. The > degradation could be caused by other factors, such as thread migration, > bad OS behaviour, or even locking peculiarities in your own > application, which are not related to the GIL. A good test is whether > performance improves if you play with sys.setswitchinterval(). > > > Dave's newer study regards another issue, which I must stress is also > present in the old GIL algorithm, and therefore must have affected, if > it is serious, real-world applications in 2.x. And indeed, the test I > recently added to ccbench evidences the huge drop in socket I/Os per > second when there's a background CPU thread; this test exercises the > same situation as Dave's demos, only with a less trivial CPU workload: > > == CPython 2.7b2+.0 (trunk:81274M) == > == x86_64 Linux on 'x86_64' == > > --- I/O bandwidth --- > > Background CPU task: Pi calculation (Python) > > CPU threads=0: 23034.5 packets/s. > CPU threads=1: 6.4 ( 0 %) > CPU threads=2: 15.7 ( 0 %) > CPU threads=3: 13.9 ( 0 %) > CPU threads=4: 20.8 ( 0 %) > > (note: I've just changed my desktop machine, so these figures are > different from what I've posted weeks or months ago) > > > Regardless of the fact that apparently noone reported it in real-world > conditions, we *could* decide that the issue needs fixing. If we > decide so, Nir's approach is the most rigorous one: it tries to fix > the problem thoroughly, rather than graft an additional heuristic. Nir > also has tested his patch on a variety of machines, more so than Dave > and I did with our own patches; he is obviously willing to go forward. > > Right now, there are two problems with Nir's proposal: > > - first, what Nick said: the difficulty of having reliable > high-precision cross-platform time sources, which are necessary for > the BFS algorithm. Ironically, timestamp counters have their own > problems on multi-core machines (they can go out of sync between > CPUs). gettimeofday() and clock_gettime() may be precise enough on > most Unices, though. > > - second, the BFS algorithm is not that well-studied, since AFAIK it > was refused for inclusion in the Linux kernel; someone in the > python-dev community would therefore have to make sense of, and > evaluate, its heuristic. > > I also don't consider my own patch a very satisfactory "solution", > although it has the reassuring quality of being simple and short (and > easy to revert!). > > > That said, most of us are programmers and we love to invent ways > of fixing technical issues. It sometimes leads us to consider some > things issues even when they are mostly theoretical. This is why I > am lukewarm on this. I think interested people should focus on > real-world testing (rather than Dave and I's synthetic tests) of the new > GIL, with or without the various patches, and share the results. > > Otherwise, Dj Gilcrease's suggestion of waiting for third-party reports > is also a very good one. > > Regards > > Antoine.

3 2

Python versions for Ubuntu 10.10 (Maverick Meerkat)
by Barry Warsaw 18 May '10

18 May '10

I just wanted to let the python-dev community know about some tracks we had at the recently concluded Ubuntu Developer Summit in Brussels. Among the several Python-related discussions, we talked about what versions of Python will be supported and default in the next version of Ubuntu (10.10, code name Maverick Meerkat, to be released in October). If you're interested in following and participating in this discussion, I've started a wiki page as the central place to collect information: https://wiki.ubuntu.com/MaverickMeerkat/TechnicalOverview/Python While we don't have consensus yet, and we have a lot of footwork to do before we can make a decision, my goal is to include Python 2.6, 2.7, and 3.2 (beta) in Ubuntu 10.10. I'd like to make Python 2.7 the default if possible, and we will need to get pre-approval to do stable release upgrades to Python 3.2 to track the move to its final release, which will happen after Ubuntu 10.10. is released. It seems that most discussions are happening on the debian-python mailing list right now. they-don't-call-it-maverick-for-nuthin'-ly y'rs, -Barry

1 0

Possible patch for functools partial - Interested?
by VanL 17 May '10

17 May '10

Howdy all - I have an app where I am using functools.partial to bundle up jobs to do, where a job is defined as a callable + args. In one case, I wanted to keep track of whether I had previously seen a job, so I started putting them into a set... only to find out that partials never test equal to each other: >>> import operator >>> from functools import partial >>> p1 = partial(operator.add) >>> p2 = partial(operator.add) >>> p1 == p2 False >>> seen = set();seen.add(p1) >>> p2 in seen False I created a subclass of functools.partial that provides appropriate __eq__ and __hash__ methods, so that this works as expected. I called the subclass a Job: >>> j1 = Job(operator.add) >>> j2 = Job(operator.add) >>> j1 == j2 True >>> seen = set();seen.add(j1) >>> j2 in seen True >>> j1 is j2 False While I was at it, I also added a nice repr. Would this group be interested in a patch, or is this not interesting? Thanks, Van

16 24

OS X buildbots and launchd
by Bill Janssen 16 May '10

16 May '10

I can find no evidence that the buildbot installation process given on the wiki will cause the buildbot slave to be restarted after a reboot of the machine. To accomplish this, you should also undertake the work described in http://buildbot.net/trac/wiki/UsingLaunchd On my Leopard slave, I created the file /Library/LaunchAgents/org.python.buildbot.slave.plist, and put in it this XML: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>StandardOutPath</key> <string>twistd.log</string> <key>StandardErrorPath</key> <string>twistd-err.log</string> <key>EnvironmentVariables</key> <dict> <key>PATH</key> <string>/usr/local/bin:/sbin:/usr/sbin:/bin:/usr/bin</string> </dict> <key>GroupName</key> <string>daemon</string> <key>KeepAlive</key> <dict> <key>SuccessfulExit</key> <false/> </dict> <key>Label</key> <string>org.python.buildbot.slave</string> <key>ProgramArguments</key> <array> <string>/usr/bin/twistd</string> <string>-no</string> <string>-y</string> <string>/Users/buildbot/buildarea/buildbot.tac</string> </array> <key>RunAtLoad</key> <true/> <key>UserName</key> <string>buildbot</string> <key>WorkingDirectory</key> <string>/Users/buildbot/buildarea</string> </dict> </plist> Note that I am using the system Python 2.5, and that I have configured the buildbot slave machine to login the "buildbot" account automatically upon boot, which login will cause this daemon to start, and will provide access to the GUI context for testing tk. I've configured it with a locking screensaver that starts immediately; let's see if that works OK. I've also put "dinsdale.python.org" and "svn.python.org" in /etc/hosts. Bill

1 2

what ports are needed for a buildbot?
by Bill Janssen 16 May '10

16 May '10

I'm running my buildbots in a captive firewalled subnet, and I have to enumerate all the ports that the buildbot is allowed to open to the outside world. Right now, I'm getting this error: ERROR: test_connect (test.test_smtpnet.SmtpSSLTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/buildbot/buildarea/3.x.parc-leopard-1/build/Lib/test/test_smtpnet.py", line 15, in test_connect server = smtplib.SMTP_SSL(self.testServer, self.remotePort) File "/Users/buildbot/buildarea/3.x.parc-leopard-1/build/Lib/smtplib.py", line 757, in __init__ SMTP.__init__(self, host, port, local_hostname, timeout) File "/Users/buildbot/buildarea/3.x.parc-leopard-1/build/Lib/smtplib.py", line 239, in __init__ (code, msg) = self.connect(host, port) File "/Users/buildbot/buildarea/3.x.parc-leopard-1/build/Lib/smtplib.py", line 295, in connect self.sock = self._get_socket(host, port, self.timeout) File "/Users/buildbot/buildarea/3.x.parc-leopard-1/build/Lib/smtplib.py", line 762, in _get_socket new_socket = socket.create_connection((host, port), timeout) File "/Users/buildbot/buildarea/3.x.parc-leopard-1/build/Lib/socket.py", line 321, in create_connection raise error(msg) socket.error: [Errno 61] Connection refused This make me think that the list of ports enumerated at http://wiki.python.org/moin/BuildBot is incomplete. We need a complete list. If you know of additional ports, please add them to the wiki page. Bill

1 0

buildbot svn exceptions again...
by Bill Janssen 16 May '10

16 May '10

parc-leopard-1 (and most of the other builders) are failing the svn checkout with the following error: svn: PROPFIND of '/projects/python/trunk': Could not resolve hostname `svn.python.org': Temporary failure in name resolution (http://svn.python.org) When I log into that machine as "buildbot" and do the svn checkout manually using the following command /usr/bin/svn checkout --non-interactive --no-auth-cache --revision HEAD http://svn.python.org/projects/python/trunk build it works fine. Bill

2 4

frozendict (was: Possible patch for functools partial...)
by Yaniv Aknin 16 May '10

16 May '10

> > > So I'm thinking either we make an > > immutable/hashable dict while we're at it, or store the keyword > > arguments as a tuple (which guarantees immutability), and only > > convert them back to a dict when you want to call the partial object > > (simpler, slower). > > I'd support an immutable dict. [...] I've set out to implement a frozendict (like frozenset) for use to store functools.keywords' attribute, and quickly realized I didn't think of an obvious flaw in that idea. A frozenset will only accept hashable members, but a frozendict can't afford this luxury for its values. I'm not sure how should I go about handling that, if at all. Should I implement a frozendict which will remain unhashable but support equality testing like a regular dict? A frozendict that is only hashable if all its values are hashable, like a tuple? Is the whole notion of a frozendict worthy, given this limitation? I'd be happy to hear python-dev's guidance on this. Cheers, - Yaniv

3 2

robots exclusion file on the buildbot pages?
by Antoine Pitrou 15 May '10

15 May '10

Hello, The buildbots are sometimes subject to a flood of "svn exception" errors. It has been conjectured that these errors are caused by Web crawlers pressing "force build" buttons without filling any of the fields (of course, the fact that we get such ugly errors in the buildbot results, rather than a clean error message when pressing the button, is a buildbot bug in itself). Couldn't we simply exclude all crawlers from the buildbot Web pages? Regards Antoine.

5 14

Python Web Developer @ Meebo (Perm in Mountain View, CA.)
by Kiko Griffin 15 May '10

15 May '10

Hello All, Meebo is looking for a bright, fun, dedicated software engineer who's interested in supporting a growing revenue team, loves a good challenge, and wants to learn about how advertising operations works from the inside out. In this role, you will be responsible for implementing and maintaining critical tools that support the ongoing adops infrastructure, ranging from content acquisition to ad inventory management. As a developer on the ads team, you'll be exposed to many business functions and you'll support and develop tools and systems to streamline the entire process. Our team is fun, friendly, and willing to work really hard to meet our goals. We love a great user experience, internal and external. *Responsibilities:* - Work with ads team to create and maintain mission critical support tools - Work closely with Sales and Marketing teams to elicit feedback from partners - Design and implement partner facing tools - Participate in product meetings to determine necessary systems - Scale and optimize with a growing revenue stream - Interface with both server and client side teams to leverage existing tools *What you will need:* - 3-5+ years of software engineering experience in Linux/Unix - BS, MS in Computer Science (or equivalent) - Strong expertise in Python, Perl, php, or other scripting languages - Knowledge of Django, PIG a plus - Database experience a plus - Ability to work with technical and non-technical teams, in an extremely fast-paced environment - Strong UI design sense with the ability to adapt, take critical feedback, and execute quickly on tasks *Big pluses:* - Experience working with large scale systems - Experience working with ad ops industry tools such as: Doubleclick, Atlas, 24/7, Salesforce, Dart Sales Manager, Solbright, Operative If at all interested OR know someone who might be interested please reply here Kiko(a)meebo-inc.com. Thanks, Kiko Griffin kiko(a)meebo-inc.com

2 1

configuring the buildbot to skip some tests?
by Bill Janssen 14 May '10

14 May '10

I've got parc-tiger-1 up and running again. It's failing on test_tk, which makes sense, because it's running as a background twisted process, and thus can't access the window server. I should configure that out. I'm looking for documentation on how to configure the build slave so that it skips this test. Bill

8 22

HEADS UP: Compilation risk with new GCC 4.5.0
by Jesus Cea 14 May '10

14 May '10

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 [Zlib-devel] HEADS UP: Apparent bad compilation under (just released) GCC 4.5.0 <http://mail.madler.net/pipermail/zlib-devel_madler.net/2010-May/002267.html> GCC Bugzilla Bug 40838 gcc shouldn't assume that the stack is aligned http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40838 Short history: new GCC 4.5.0 (released a month ago), when compiling with - -O3, is adding MMX/SSE instructions that requires stack aligned to 16 byte. This is wrong, since x86 ABI only requires stack aligned to 4 bytes. If you compile EVERYTHING with GCC 4.5.0, you are safe (I guess!), but if your environment has mixed compiled code (for instance, the OS libraries), you can possibly "core dump". If you have an old compiled Python and you update libs compiled with GCC 4.5.0, you can crash in the process. Psyco is showing the issue, but it is not the culprit. It only leaves - -correctly- the stack in not 16-byte alignment. But there are plenty of examples of crashes not related to python+psyco. Proposal: add "-fno-tree-vectorize" to compilation options for 2.7/3.2. Warm 2.3/2.4/2.5/2.6/3.0/3.1 users. Or warm users compiling with GCC 4.5.0. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea(a)jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQCVAwUBS+qpcZlgi5GaxT1NAQLAjgP/anZ5C2lED190++ffcluApF3ASF20iSnH t21ynHxrz2fgkPeOxeGRAqGEGCc3jZ0UuXchECcLLzFhI8WQS8K766EOgOKdwZLV WCt0ohWZ0FfsJZX4J5Y73x39uhjShbnl6SSek2uEvPi/tua/R4W/kVXrAZ0ZZR6/ vRqSUXpdolM= =5K+S -----END PGP SIGNATURE-----

6 8

Summary of Python tracker Issues
by Python tracker 14 May '10

14 May '10

ACTIVITY SUMMARY (2010-05-07 - 2010-05-14) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2678 open (+43) / 17853 closed (+23) / 20531 total (+66) Open issues with patches: 1090 Average duration of open issues: 722 days. Median duration of open issues: 495 days. Open Issues Breakdown open 2658 (+43) languishing 12 ( +0) pending 7 ( +0) Issues Created Or Reopened (72) _______________________________ __import__ with fromlist= 2010-05-08 http://bugs.python.org/issue2090 reopened brett.cannon patch SSL handshake fails after TCP connection in getpeername() 2010-05-07 CLOSED http://bugs.python.org/issue4171 reopened ddvoinikov A number tests "crash" if python is compiled --without-threads 2010-05-13 http://bugs.python.org/issue7449 reopened skrah patch, easy, needs review pep-0370 on osx duplicates existing functionality 2010-05-08 http://bugs.python.org/issue8084 reopened mark.dickinson patch, needs review, buildbot tiger buildbot: test_abspath_issue3426 failure (test_genericpa 2010-05-13 CLOSED http://bugs.python.org/issue8422 reopened flox buildbot PyUnicode_GetMax is undocumented 2010-05-07 http://bugs.python.org/issue8647 created stutzbach The UTF-7 codec functions are undocumented 2010-05-07 http://bugs.python.org/issue8648 created stutzbach Py_UNICODE_* functions are undocumented 2010-05-07 http://bugs.python.org/issue8649 created stutzbach zlibmodule.c isn't 64-bit clean 2010-05-07 http://bugs.python.org/issue8650 created pitrou "s#" and friends can silently truncate buffer length 2010-05-07 http://bugs.python.org/issue8651 created pitrou Minor improvements to the "Handling Exceptions" part of the tu 2010-05-07 http://bugs.python.org/issue8652 created marienz urlparse.urlparse/urlsplit doc missing 2010-05-07 http://bugs.python.org/issue8653 created dabrahams Improve ABI compatibility between UCS2 and UCS4 builds 2010-05-07 http://bugs.python.org/issue8654 created stutzbach patch Problem with getpeercert in the ssl module when retrieving cli 2010-05-07 http://bugs.python.org/issue8655 created Westly.Ward urllib2 mangles file://-scheme URLs 2010-05-08 CLOSED http://bugs.python.org/issue8656 created dabrahams urlparse.urlunsplit should be smarter about + 2010-05-08 CLOSED http://bugs.python.org/issue8657 created dabrahams patch urlparse.urlunsplit should be smarter about + and file urls 2010-05-08 CLOSED http://bugs.python.org/issue8658 created orsenthil patch ABS(x) == 0 can be simplified to x == 0 2010-05-08 CLOSED http://bugs.python.org/issue8659 created stutzbach patch, needs review py3k weakref documentation mentions the long built-in type 2010-05-08 CLOSED http://bugs.python.org/issue8660 created durban FAQ item 2.25 is unclear 2010-05-08 http://bugs.python.org/issue8661 created exarkun bytes join cannot join bytes 2010-05-08 CLOSED http://bugs.python.org/issue8662 created perey Failed compile in a non-ASCII path 2010-05-08 http://bugs.python.org/issue8663 created pitrou patch py_compile.compile() should consistently create parent directo 2010-05-08 CLOSED http://bugs.python.org/issue8664 created Arfrever patch "make pycremoval" fails 2010-05-08 http://bugs.python.org/issue8665 created pitrou Allow ConfigParser.get*() to take a default value 2010-05-08 http://bugs.python.org/issue8666 created Gumnos patch Link to PEP 3147 from importlib docs 2010-05-08 http://bugs.python.org/issue8667 created brett.cannon easy add a 'develop' command 2010-05-09 http://bugs.python.org/issue8668 created meatballhat lack of bdist_rpm module raises error on 'setup.py --help-comm 2010-05-09 http://bugs.python.org/issue8669 created meatballhat c_types.c_wchar should not assume that sizeof(wchar_t) == size 2010-05-09 http://bugs.python.org/issue8670 created stutzbach patch A small erorr on http://docs.python.org/library/re.html 2010-05-09 CLOSED http://bugs.python.org/issue8671 created bones7456 Error decompressing valid zlib data 2010-05-09 CLOSED http://bugs.python.org/issue8672 created matthew.brett patch configure script doesn't recognize 10.5 SDK correctly 2010-05-10 CLOSED http://bugs.python.org/issue8673 created ronaldoussoren audioop: incorrect integer overflow checks 2010-05-10 CLOSED http://bugs.python.org/issue8674 created thoger patch audioop module needs an int -> Py_ssize_t upgrade 2010-05-10 CLOSED http://bugs.python.org/issue8675 created mark.dickinson patch, easy Py3k Built-in Exceptions documentation mentions the raise stat 2010-05-10 CLOSED http://bugs.python.org/issue8676 created durban Modules needing PY_SSIZE_T_CLEAN 2010-05-10 http://bugs.python.org/issue8677 created pitrou crashers in rgbimg 2010-05-10 http://bugs.python.org/issue8678 created brett.cannon write a distutils to distutils2 converter 2010-05-10 http://bugs.python.org/issue8679 created tarek Add a sandbox in Distutils2 2010-05-10 http://bugs.python.org/issue8680 created tarek Make the zlib module emit more detailed error messages 2010-05-10 CLOSED http://bugs.python.org/issue8681 created pitrou patch _ssl.c uses PyWeakref_GetObject but doesn't incref result 2010-05-10 http://bugs.python.org/issue8682 created pitrou HPUX Segmentation fault in Modules/gcmodule.c -- if (!gc_list_ 2010-05-11 CLOSED http://bugs.python.org/issue8683 created srid improvements to sched.py 2010-05-11 http://bugs.python.org/issue8684 created josiahcarlson patch, patch, needs review set(range(100000)).difference(set()) is slow 2010-05-11 http://bugs.python.org/issue8685 created spiv patch "This isn't defined beyond that" phrase is not friendly to non 2010-05-11 http://bugs.python.org/issue8686 created naoki sched.py module doesn't have a test suite 2010-05-11 http://bugs.python.org/issue8687 created giampaolo.rodola patch distutils sdist is too laze w.r.t. recalculating MANIFEST 2010-05-11 http://bugs.python.org/issue8688 created ronaldoussoren sqlite3 parameter substitution breaks with multiple parameters 2010-05-11 http://bugs.python.org/issue8689 created azriphale multiprocessing.dummy.Queue does not expose same interface as 2010-05-11 http://bugs.python.org/issue8690 created Jason.Baker Doc: left alignment is not the default for numbers 2010-05-11 http://bugs.python.org/issue8691 created tjreedy Use divide-and-conquer for faster factorials 2010-05-11 http://bugs.python.org/issue8692 created stutzbach patch, patch, needs review Py_Initialize: can't initialize sys standard streams 2010-05-12 CLOSED http://bugs.python.org/issue8693 reopened mbnoimi python3 FAQ mentions unicode() 2010-05-12 http://bugs.python.org/issue8694 created ohnobinki Issue while installing Python 2.6.5 in IBM AIX 6.1 2010-05-12 http://bugs.python.org/issue8695 created senthil_l PEP 391: Adding documentation of logging.config.dictConfig 2010-05-12 CLOSED http://bugs.python.org/issue8696 created akuchling patch PEP 391: Adding documentation of logging.config.dictConfig 2010-05-12 CLOSED http://bugs.python.org/issue8697 created akuchling patch, needs review PEP 391: Adding documentation of logging.config.dictConfig 2010-05-12 CLOSED http://bugs.python.org/issue8698 created akuchling patch, needs review Equality and hashing for functools.partial 2010-05-12 http://bugs.python.org/issue8699 created durban patch strip() is removing an extra character if the strip pattern co 2010-05-12 CLOSED http://bugs.python.org/issue8700 created Balachander.Ganesan tarfile: first character of member names doubled 2010-05-12 CLOSED http://bugs.python.org/issue8701 created pooryorick difflib: unified_diff produces wrong patches (again) 2010-05-13 CLOSED http://bugs.python.org/issue8702 created techtonik easy Py3k PyList_Type documentation mentions types.ListType 2010-05-13 CLOSED http://bugs.python.org/issue8703 created durban cgitb sends a bogus HTTP header if the app crashes before fini 2010-05-13 http://bugs.python.org/issue8704 created stutzbach easy shutil.rmtree with empty filepath 2010-05-13 http://bugs.python.org/issue8705 created dbkoch accept keyword arguments on all base type methods and builtins 2010-05-13 http://bugs.python.org/issue8706 created gregory.p.smith Duplicated document in telnetlib. 2010-05-13 http://bugs.python.org/issue8707 created naoki OpenID blunder 2010-05-14 http://bugs.python.org/issue8708 created Jack.Diederich mention explicitly Windows support for os.devnull 2010-05-14 http://bugs.python.org/issue8709 created akira patch test_xpickle: compat tests: workaround for missing test_suppor 2010-05-14 http://bugs.python.org/issue8710 created skrah Document PyUnicode_DecodeFSDefault() and PyUnicode_DecodeFSDef 2010-05-14 http://bugs.python.org/issue8711 created haypo patch Skip libpthread related test failures on OpenBSD 2010-05-14 http://bugs.python.org/issue8712 created skrah patch, needs review bdist_deb - Debian packager 2010-05-07 CLOSED http://bugs.python.org/issue1054967 reopened merwok patch Issues Now Closed (54) ______________________ cmath is numerically unsound 920 days http://bugs.python.org/issue1381 bins patch test_uuid is warning about unreliable functions 904 days http://bugs.python.org/issue1481 skrah patch Add a factorial function 786 days http://bugs.python.org/issue2138 mark.dickinson Py_FatalError causes infinite loop 632 days http://bugs.python.org/issue3605 jyasskin needs review SSL handshake fails after TCP connection in getpeername() 1 days http://bugs.python.org/issue4171 pitrou {context,unified}_diff add spurious trailing whitespace if fro 490 days http://bugs.python.org/issue4898 r.david.murray patch Implement PEP 376 484 days http://bugs.python.org/issue4908 tarek patch subprocess.Popen.__del__ causes AttributeError (os module == N 469 days http://bugs.python.org/issue5099 brett.cannon patch, needs review test_smtplib fails on os x 10.6 220 days http://bugs.python.org/issue7040 giampaolo.rodola difflib should separate filename from timestamp with tab 137 days http://bugs.python.org/issue7585 techtonik patch unittest: allow an 'import_path' as an alternative to 'top_lev 103 days http://bugs.python.org/issue7780 michael.foord Provide unittest.TestCase.assertNotRegexpMatches 67 days http://bugs.python.org/issue8038 michael.foord patch Python 3.1.2rc1 doesn't compile using the 10.4 sdk on a 10.6 M 62 days http://bugs.python.org/issue8126 ronaldoussoren Putting a function in a unittest.TestSuite doesn't work 35 days http://bugs.python.org/issue8301 michael.foord python -m unittest -h and python -m unittest discover -h messa 37 days http://bugs.python.org/issue8303 michael.foord siginterrupt with flag=False is reset when signal received 30 days http://bugs.python.org/issue8354 exarkun patch, needs review tiger buildbot: test_abspath_issue3426 failure (test_genericpa 0 days http://bugs.python.org/issue8422 haypo buildbot openssl version detection doesn't work properly when using OSX 20 days http://bugs.python.org/issue8444 ronaldoussoren asyncore test suite 19 days http://bugs.python.org/issue8490 giampaolo.rodola patch, patch update to autoconf2.65 18 days http://bugs.python.org/issue8510 mark.dickinson patch, needs review Add fsencode() functions to os module 15 days http://bugs.python.org/issue8514 haypo patch unittest test discovery can fail when package under test is al 11 days http://bugs.python.org/issue8547 michael.foord zlib causes a SystemError when decompressing a chunk >1GB 8 days http://bugs.python.org/issue8571 pitrou patch Update/reorganize _winreg documentation 12 days http://bugs.python.org/issue8575 brian.curtin patch, patch, needs review test_urllib2.py test failures on Py3K Mac OS X 8 days http://bugs.python.org/issue8588 mark.dickinson enumerate() test cases do not cover optional start argument 2 days http://bugs.python.org/issue8636 benjamin.peterson patch json.loads description 3 days http://bugs.python.org/issue8642 georg.brandl patch timedelta.total_seconds needlessly inaccurate, especially for 2 days http://bugs.python.org/issue8644 mark.dickinson patch urllib2 mangles file://-scheme URLs 0 days http://bugs.python.org/issue8656 orsenthil urlparse.urlunsplit should be smarter about + 5 days http://bugs.python.org/issue8657 orsenthil patch urlparse.urlunsplit should be smarter about + and file urls 0 days http://bugs.python.org/issue8658 orsenthil patch ABS(x) == 0 can be simplified to x == 0 0 days http://bugs.python.org/issue8659 mark.dickinson patch, needs review py3k weakref documentation mentions the long built-in type 0 days http://bugs.python.org/issue8660 benjamin.peterson bytes join cannot join bytes 0 days http://bugs.python.org/issue8662 perey py_compile.compile() should consistently create parent directo 0 days http://bugs.python.org/issue8664 benjamin.peterson patch A small erorr on http://docs.python.org/library/re.html 0 days http://bugs.python.org/issue8671 eric.smith Error decompressing valid zlib data 2 days http://bugs.python.org/issue8672 pitrou patch configure script doesn't recognize 10.5 SDK correctly 3 days http://bugs.python.org/issue8673 ronaldoussoren audioop: incorrect integer overflow checks 1 days http://bugs.python.org/issue8674 mark.dickinson patch audioop module needs an int -> Py_ssize_t upgrade 1 days http://bugs.python.org/issue8675 mark.dickinson patch, easy Py3k Built-in Exceptions documentation mentions the raise stat 0 days http://bugs.python.org/issue8676 benjamin.peterson Make the zlib module emit more detailed error messages 1 days http://bugs.python.org/issue8681 pitrou patch HPUX Segmentation fault in Modules/gcmodule.c -- if (!gc_list_ 1 days http://bugs.python.org/issue8683 pitrou Py_Initialize: can't initialize sys standard streams 0 days http://bugs.python.org/issue8693 mbnoimi PEP 391: Adding documentation of logging.config.dictConfig 0 days http://bugs.python.org/issue8696 akuchling patch PEP 391: Adding documentation of logging.config.dictConfig 0 days http://bugs.python.org/issue8697 akuchling patch, needs review PEP 391: Adding documentation of logging.config.dictConfig 1 days http://bugs.python.org/issue8698 meatballhat patch, needs review strip() is removing an extra character if the strip pattern co 0 days http://bugs.python.org/issue8700 r.david.murray tarfile: first character of member names doubled 1 days http://bugs.python.org/issue8701 pooryorick difflib: unified_diff produces wrong patches (again) 0 days http://bugs.python.org/issue8702 mark.dickinson easy Py3k PyList_Type documentation mentions types.ListType 0 days http://bugs.python.org/issue8703 benjamin.peterson Carbon Event ReceiveNextEvent 2479 days http://bugs.python.org/issue779285 tjreedy bdist_deb - Debian packager 0 days http://bugs.python.org/issue1054967 tarek patch update urlparse to RFC 3986 1280 days http://bugs.python.org/issue1591035 orsenthil easy Top Issues Most Discussed (10) ______________________________ 45 Use divide-and-conquer for faster factorials 3 days open http://bugs.python.org/issue8692 25 Improve ABI compatibility between UCS2 and UCS4 builds 7 days open http://bugs.python.org/issue8654 14 Failed compile in a non-ASCII path 6 days open http://bugs.python.org/issue8663 11 let's equip ftplib.FTP with __enter__ and __exit__ 482 days open http://bugs.python.org/issue4972 10 add a distutils test command 39 days open http://bugs.python.org/issue8324 10 cookielib doesn't handle URLs with / in parameters 625 days open http://bugs.python.org/issue3704 9 Py_Initialize: can't initialize sys standard streams 0 days closed http://bugs.python.org/issue8693 9 timedelta.total_seconds needlessly inaccurate, especially for n 2 days closed http://bugs.python.org/issue8644 9 update to autoconf2.65 18 days closed http://bugs.python.org/issue8510 9 input() doesn't catch _PyUnicode_AsString() exception; io.Strin 47 days open http://bugs.python.org/issue8256

1 0

PEP 7 updated
by Antoine Pitrou 13 May '10

13 May '10

Hello, The untabification of C files didn't produce any noticeable problem on the buildbots. I've updated PEP 7 with the mention that all C files should be 4-space indented, and removed the obsolete wording about some files being indented with tabs. Regards Antoine.

8 9

Python on Windows with CoApp
by Trent Nelson 12 May '10

12 May '10

Howdy folks, Quick e-mail at 34,000ft (aren't wifi-enabled flights great?) to mention a new initiative that's been started by Microsoft called CoApp (Common Opensource Application Publishing Platform). The aim is simple: make open source software rock on Windows ;-) It's probably easiest to think of it as a Microsoft-endorsed-but-community-run open source distribution for Windows, akin to all the various package managers for Linux distributions and ports/packages for the *BSDs. There are specific user and developer experiences we'll be addressing -- like making it easy to install and use open source software, or use it within your own project (open source or not). CoApp will affect Python in one of two ways. Once there's a clear-cut specification for open source projects to follow, Python can either decide to follow it, or not. The same applies to all open source packages, actually. For those that follow it, great! If not, no problem -- the plan is to shallow-fork such projects via launchpad and the CoApp community will take responsibility for getting releases of open source projects into CoApp shape. It's in its infancy at the moment -- it took the chap (Garrett Serack) who's spearheading it at Microsoft about six months to get it all signed off by the lawyers and platform/server VPs. So, for those of you out there who are Windows-inclined, now's a perfect time to get involved to help shape the direction of CoApp going forward. The website/wiki is http://coapp.org/ and the launchpad project site is http://launchpad.net/coapp (which is where the mailing list is hosted). We're actually having a 'CoApp Development Summit' tomorrow and Friday in Seattle (that Microsoft's graciously sponsored). The event will be accessible via Live Meeting for those that are interested: http://coapp.org/Project_Planning/CoApp_Design_and_Development_Summit Regards, Trent.

1 0