Mailman 3 October 2009 - Python-Dev

language summit topic: issue tracker
by Brett Cannon 03 Nov '09

03 Nov '09

Another summit, another potential time to see if people want to change anything about the issue tracker. I would bring up: - Dropping Stage in favor of some keywords (e.g. 'needs unit test', 'needs docs') - Adding a freestyle text box to delineate which, if any, stdlib module is the cause of a bug and tie that into Misc/maintainers.rst; would potentially scale back the Component box -Brett

8 11

thanks to everyone cleaning up the tests
by Brett Cannon 03 Nov '09

03 Nov '09

Just wanted to publicly thank everyone who has been causing all the checkins to fix and stabilize the test suite (I think it's mostly Antoine and Mark, but I could be missing somebody; I'm under a deadline so only have marginal higher brain functionality). -Brett

4 3

A new way to configure logging
by Vinay Sajip 02 Nov '09

02 Nov '09

At present, configuration of Python's logging package can be done in one of two ways: 1. Create a ConfigParser-readable configuration file and use logging.config.fileConfig() to read and implement the configuration therein. 2. Use the logging API to programmatically configure logging using getLogger(), addHandler() etc. The first of these works for simple cases but does not cover all of the logging API (e.g. Filters). The second of these provides maximal control, but besides requiring users to write the configuration code, it fixes the configuration in Python code and does not facilitate changing it easily at runtime. In addition, the ConfigParser format appears to engender dislike (sometimes strong dislike) in some quarters. Though it was chosen because it was the only configuration format supported in the stdlib at that time, many people regard it (or perhaps just the particular schema chosen for logging's configuration) as 'crufty' or 'ugly', in some cases apparently on purely aesthetic grounds. Recent versions of Python of course support an additional battery-included format which can be used for configuration - namely, JSON. Other options, such as YAML, are also possible ways of configuring systems, Google App Engine-style, and PyYAML has matured nicely. There has also been talk on the django-dev mailing list about providing better support for using Python logging in Django. When it happens (as of course I hope it does) this has the consequence that many new users who use Django but are relatively inexperienced in Python (e.g. in PHP shops which are moving to Django) will become exposed to Python logging. As Django is configured using a Python module and use of ConfigParser-style files is not a common approach in that ecosystem, users will find either of the two approaches outlined above a particular pain point when configuring logging for their Django applications and websites, unless something is done to avoid it. All three of the contenders for the title of "commonly found configuration mechanism" - JSON, YAML and Python code - will be expressible, in Python, as Python dicts. So it seems to make sense to add, to logging.config, a new callable bound to "dictConfig" which will take a single dictionary argument and configure logging from that dictionary. An important facet of implementing such a scheme will be the format or schema which the dictionary has to adhere to. I have started working on what such a schema would look like, and if people here think it's a good idea to go ahead with this, I'll provide the details of the schema on a separate post which I'll also cross-post on comp.lang.python so that I can get feedback from there, too. In outline, the scheme I have in mind will look like this, in terms of the new public API: class DictConfigurator: def __init__(self, config): #config is a dict-like object (duck-typed) import copy self.config = copy.deepcopy(config) def configure(self): # actually do the configuration here using self.config dictConfigClass = DictConfigurator def dictConfig(config): dictConfigClass(config).configure() This allows easy replacement of DictConfigurator with a suitable subclass where needed. What's the general feeling here about this proposal? All comments and suggestions will be gratefully received. Regards, Vinay Sajip

6 16

Possible language summit topic: buildbots
by Mark Dickinson 02 Nov '09

02 Nov '09

Would it be worth spending some time discussing the buildbot situation at the PyCon 2010 language summit? In the past, I've found the buildbots to be an incredibly valuable resource; especially when working with aspects of Python or C that tend to vary significantly from platform to platform (for me, this usually means floating-point, and platform math libraries, but there are surely many other things it applies to). But more recently there seem to have been some difficulties keeping a reasonable number of buildbots up and running. A secondary problem is that it can be awkward to debug some of the more obscure test failures on buildbots without having direct access to the machine. From conversations on IRC, I don't think I'm alone in wanting to find ways to make the buildbots more useful. So the question is: how best to invest time and possibly money to improve the buildbot situation (and as a result, I hope, improve the quality of Python)? What could be done to make maintenance of build slaves easier? Or to encourage interested third parties to donate hardware and time? Are there good alternatives to Buildbot that might make a difference? What do other projects do? These are probably the wrong questions; I'm hoping that a discussion would help produce the right questions, and possibly some answers. Mark

15 71

"Buildbot" category on the tracker
by Antoine Pitrou 31 Oct '09

31 Oct '09

Hello, What do you think of creating a "buildbot" category in the tracker? There are often problems on specific buildbots which would be nice to track, but there's nowhere to do so. Regards Antoine.

5 17

Add const to python API - issue 6952
by Barry Scott 31 Oct '09

31 Oct '09

http://bugs.python.org/issue6952 Martin v. Löwis suggested that solutions to this issue should be discussed here. My goal is to avoid compiler warning and the need to cast to remove const when calling the python API. For example I see compiler messages like this on Mac OS X Snow Leopard g++ reports: example.cxx:633: warning: deprecated conversion from string constant to ‘char*’ The patch I developed for comment only adds const to the input parameters and used casts to allow output parameters to stay without the const. This is because adding the const to the input params will not break existing code, but adding const to output parameters may well require code changes for users of the Python API. What is the best approach to this problem that will be acceptable? Barry

3 6

Summary of Python tracker Issues
by Python tracker 30 Oct '09

30 Oct '09

ACTIVITY SUMMARY (10/23/09 - 10/30/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2483 open (+31) / 16582 closed (+19) / 19065 total (+50) Open issues with patches: 989 Average duration of open issues: 675 days. Median duration of open issues: 430 days. Open Issues Breakdown open 2447 (+31) pending 35 ( +0) Issues Created Or Reopened (53) _______________________________ distutils and IronPython compatibility 10/27/09 CLOSED http://bugs.python.org/issue7071 reopened michael.foord 26backport Popen blocks program from another thread 10/23/09 CLOSED http://bugs.python.org/issue7193 reopened dgriff1 test_thread is flaky 10/23/09 CLOSED http://bugs.python.org/issue7194 created pitrou Value error 'path is on drive c: start on drive d:' in os.path.r 10/23/09 http://bugs.python.org/issue7195 created jaraco Clarify str.split() behavior 10/24/09 http://bugs.python.org/issue7196 created gagenellina patch test_multiprocessing crashes under Windows when run in verbose m 10/24/09 http://bugs.python.org/issue7197 created pitrou patch csv.writer 10/24/09 CLOSED http://bugs.python.org/issue7198 created zacktu Doc: Logging level order seems inconsistent 10/24/09 CLOSED http://bugs.python.org/issue7199 created tjreedy multiprocessing deadlock on Mac OS X when queue collected before 10/24/09 http://bugs.python.org/issue7200 created bquinlan double Endian problem and more on arm 10/24/09 http://bugs.python.org/issue7201 created mancausoft "python setup.py MYCOMMAND --verbose" does not yield an unrecogn 10/25/09 http://bugs.python.org/issue7202 created zooko fixer for map(None, ...) needs to consider multi-argument case 10/25/09 CLOSED http://bugs.python.org/issue7203 created georg.brandl Strange shebang line in test_pep263 10/25/09 CLOSED http://bugs.python.org/issue7204 created wiz bz2file deadlock 10/25/09 CLOSED http://bugs.python.org/issue7205 created rbcollins patch 64 bit python fails on Windows 7 10/25/09 http://bugs.python.org/issue7206 created richo test_telnetlib fails on OS X 10.6 10/26/09 CLOSED http://bugs.python.org/issue7207 created mark.dickinson patch, needs review Getpass echo's password to screen on 2.6, but not on 2.5 or 3.1 10/26/09 http://bugs.python.org/issue7208 created pajs(a)fodder.org.uk patch Prevents uint_t from being used on QNX 10/26/09 http://bugs.python.org/issue7209 created kraai patch Proposed Syntax Checks in Test Suite 10/26/09 http://bugs.python.org/issue7210 created ChuckRhode patch select module - kevent ident field 64 bit issue 10/26/09 http://bugs.python.org/issue7211 created mbroughton patch Retrieve an arbitrary element from a set without removing it 10/26/09 http://bugs.python.org/issue7212 created wrichert patch Popen.subprocess change close_fds default to True 10/26/09 http://bugs.python.org/issue7213 created milko.krachounov TreeBuilder.end(tag) differs between cElementTree and ElementTre 10/26/09 http://bugs.python.org/issue7214 created merrellb TreeBuilder.end(tag) differs between cElementTree and ElementTre 10/26/09 CLOSED http://bugs.python.org/issue7215 created merrellb low performance of zipfile readline() 10/27/09 http://bugs.python.org/issue7216 created volker_siepmann IDLE Subprocess Startup Error 10/27/09 http://bugs.python.org/issue7217 created irdb test_site failure under Windows 10/27/09 CLOSED http://bugs.python.org/issue7218 created pitrou Unhelpful error message when a distutils package install fails d 10/27/09 http://bugs.python.org/issue7219 created michael.foord 26backport python -m no longer executes packages directly 10/27/09 CLOSED http://bugs.python.org/issue7220 created Martijn DispatcherWithSendTests_UsePoll with test_asyncore does nothing 10/27/09 http://bugs.python.org/issue7221 created pitrou thread reaping is imperfect 10/27/09 http://bugs.python.org/issue7222 created pitrou patch Simplify and improve Thread.join() 10/27/09 CLOSED http://bugs.python.org/issue7223 created pitrou patch One obvious way to do interning 10/28/09 http://bugs.python.org/issue7224 created belopolsky patch fwrite() compiler warnings 10/28/09 http://bugs.python.org/issue7225 created benjamin.peterson IDLE right-clicks don't work on Mac OS 10.5 10/28/09 http://bugs.python.org/issue7226 created MLModel Shell Support for installation of Python Packages (.EGG) 10/28/09 CLOSED http://bugs.python.org/issue7227 created djlyon %lld for PyErr_Format (Modules/_io/bufferedio.c) 10/28/09 http://bugs.python.org/issue7228 created ocean-city Manual entry for time.daylight can be misleading 10/28/09 http://bugs.python.org/issue7229 created napik test_hotshot fails on solaris 10/28/09 http://bugs.python.org/issue7230 created csernazs Windows installer does not add \Scripts folder to the path 10/28/09 http://bugs.python.org/issue7231 created sorin Support of 'with' statement fo TarFile class 10/29/09 http://bugs.python.org/issue7232 created jaime.buelta patch decimal.py: two rotate() issues 10/29/09 CLOSED http://bugs.python.org/issue7233 created skrah Complete your registration to Python tracker -- keyRVZuTYMju0Lif 10/29/09 CLOSED http://bugs.python.org/issue7234 created kaka IO libary have some error 10/29/09 CLOSED http://bugs.python.org/issue7235 created kaka the reply's additional "Re:" is ok 10/29/09 CLOSED http://bugs.python.org/issue7236 created kaka Syntax error with not 10/29/09 CLOSED http://bugs.python.org/issue7237 created LambertDW frame.f_lineno doesn't get updated after local trace function as 10/29/09 http://bugs.python.org/issue7238 created eggy Error when running a code 10/29/09 CLOSED http://bugs.python.org/issue7239 created MNilson subprocess.Popen.stdout.flush fails os OS-X 10.6.1 10/30/09 http://bugs.python.org/issue7240 created petegibson tkinter fails to import 10/30/09 CLOSED http://bugs.python.org/issue7241 reopened python.noob Forking in a thread raises RuntimeError 10/30/09 http://bugs.python.org/issue7242 created csernazs mac binary download link for 2.6.4 broken 10/30/09 http://bugs.python.org/issue7243 created beard 0.0 and -0.0 identified, with surprising results 10/24/09 http://bugs.python.org/issue1678380 reopened mark.dickinson patch Issues Now Closed (47) ______________________ Option to ignore or substitute ~/.pydistutils.cfg 769 days http://bugs.python.org/issue1180 tarek patch, easy Extension module build fails for MinGW: missing vcvarsall.bat 550 days http://bugs.python.org/issue2698 tarek py3k warn on use of frame.f_exc* 486 days http://bugs.python.org/issue3223 benjamin.peterson easy 'dictionary changed size' error in test_multiprocessing 433 days http://bugs.python.org/issue3578 pitrou tarfile keeps excessive dir structure in compressed files 307 days http://bugs.python.org/issue4750 lars.gustaebel patch tarfile: path problem in arcname under windows 226 days http://bugs.python.org/issue5500 lars.gustaebel Incorrect DST transition 217 days http://bugs.python.org/issue5582 acummings Fix reference leak in io.StringIO 138 days http://bugs.python.org/issue6242 pitrou patch zipfile: Invalid argument when opening zero-sized files 102 days http://bugs.python.org/issue6511 skelker PEP 314 inconsistency (authors/author/maintainer) 28 days http://bugs.python.org/issue6992 tarek utf-8 encoding error 25 days http://bugs.python.org/issue7045 benjamin.peterson 'g'/'G' format docs need a little more explanation 24 days http://bugs.python.org/issue7051 mark.dickinson Automatic test___all__ 23 days http://bugs.python.org/issue7055 pitrou patch Add some test execution environment protection to regrtest 23 days http://bugs.python.org/issue7058 r.david.murray patch archive_util.make_archive doesn't restore the cwd if an error is 19 days http://bugs.python.org/issue7066 tarek distutils and IronPython compatibility 0 days http://bugs.python.org/issue7071 r.david.murray 26backport struct help in the interpreter does not explain about the fmt op 20 days http://bugs.python.org/issue7078 mark.dickinson patch, easy Patch for get_filename in email.message when content-disposition 19 days http://bugs.python.org/issue7082 r.david.murray patch printing a list releases the GIL carelessly 19 days http://bugs.python.org/issue7084 pitrou patch Decimal.is_normal should return True even for numbers with expon 17 days http://bugs.python.org/issue7099 mark.dickinson Extension module build fails for MinGW: missing vcvarsall.bat 12 days http://bugs.python.org/issue7131 tarek Fix Download Current Documentation link 10 days http://bugs.python.org/issue7157 georg.brandl zipfile leaves a file handle open if file is zero size 9 days http://bugs.python.org/issue7169 r.david.murray sum() doesn't work for lists. 6 days http://bugs.python.org/issue7176 ezio.melotti Popen blocks program from another thread 0 days http://bugs.python.org/issue7193 dgriff1 test_thread is flaky 5 days http://bugs.python.org/issue7194 ned.deily csv.writer 0 days http://bugs.python.org/issue7198 zacktu Doc: Logging level order seems inconsistent 4 days http://bugs.python.org/issue7199 vinay.sajip fixer for map(None, ...) needs to consider multi-argument case 2 days http://bugs.python.org/issue7203 segfaulthunter Strange shebang line in test_pep263 0 days http://bugs.python.org/issue7204 georg.brandl bz2file deadlock 2 days http://bugs.python.org/issue7205 pitrou patch test_telnetlib fails on OS X 10.6 1 days http://bugs.python.org/issue7207 mark.dickinson patch, needs review TreeBuilder.end(tag) differs between cElementTree and ElementTre 0 days http://bugs.python.org/issue7215 r.david.murray test_site failure under Windows 0 days http://bugs.python.org/issue7218 tarek python -m no longer executes packages directly 0 days http://bugs.python.org/issue7220 benjamin.peterson Simplify and improve Thread.join() 0 days http://bugs.python.org/issue7223 pitrou patch Shell Support for installation of Python Packages (.EGG) 0 days http://bugs.python.org/issue7227 tarek decimal.py: two rotate() issues 0 days http://bugs.python.org/issue7233 mark.dickinson Complete your registration to Python tracker -- keyRVZuTYMju0Lif 0 days http://bugs.python.org/issue7234 eric.smith IO libary have some error 0 days http://bugs.python.org/issue7235 kaka the reply's additional "Re:" is ok 0 days http://bugs.python.org/issue7236 eric.smith Syntax error with not 0 days http://bugs.python.org/issue7237 benjamin.peterson Error when running a code 0 days http://bugs.python.org/issue7239 MNilson tkinter fails to import 0 days http://bugs.python.org/issue7241 python.noob bdist_deb - Debian packager 251 days http://bugs.python.org/issue1054967 tarek patch long int bitwise ops speedup (patch included) 1773 days http://bugs.python.org/issue1087418 mark.dickinson patch Thread shutdown exception in Thread.notify() 630 days http://bugs.python.org/issue1722344 pitrou patch Top Issues Most Discussed (10) ______________________________ 19 Backport py3k float repr to trunk 17 days open http://bugs.python.org/issue7117 12 %lld for PyErr_Format (Modules/_io/bufferedio.c) 2 days open http://bugs.python.org/issue7228 11 unify pydistutils.cfg and distutils.cfg and use .local 10 days open http://bugs.python.org/issue7175 10 64 bit python fails on Windows 7 5 days open http://bugs.python.org/issue7206 10 long int bitwise ops speedup (patch included) 1773 days closed http://bugs.python.org/issue1087418 9 select module - kevent ident field 64 bit issue 4 days open http://bugs.python.org/issue7211 9 bz2file deadlock 2 days closed http://bugs.python.org/issue7205 9 Python 2.5 64 bit compile fails on Solaris 10/gcc 4.1.1 1029 days open http://bugs.python.org/issue1628484 7 Proposed Syntax Checks in Test Suite 4 days open http://bugs.python.org/issue7210 7 fixer for map(None, ...) needs to consider multi-argument case 2 days closed http://bugs.python.org/issue7203

1 0

yappi - any thoughts?
by Sümer Cip 30 Oct '09

30 Oct '09

Hi there, yappi(Yet Another Python Profiler) is a multithreaded profiler for 2.x series(do not know if it will work on 3k, not tested yet). I have done my best to make it efficient as possible.(details are under the technical bullet in the website). It is aimed for long-running programs, to attach/detach profiler and retrieve stats on the fly. The current cProfile module needs substantial amount of work to accomplish this task and also have some problems with recursive functions, basically that is why I have written a profiler from scratch. yappi is currently(v0.2) modifying the profilefunc variable fo the ThreadState object to profile an exsiting thread. The newly created threads are catched via threading.setprofile() call(from VM to our extension). And for the deleted threads, we rely on the recycling of the ThreadState objects.(Please see the website for details.) I have tested it under a game server (100k players per-day) for about a month for any issues and now releasing it. Please see: http://code.google.com/p/yappi/ Any thoughts/comments/ideas, anything?:) -- Sumer Cip

2 1

ssl module
by Bruno Harbulot 30 Oct '09

30 Oct '09

Hello, I would like to ask a few questions and suggestions regarding the ssl module (in Python 2.6). (I gather from [1] that there is some effort going on to enhance the ssl API, but I'm not sure if this is the right place to discuss it.) Like other Python users, I was a bit surprised by the lack of verification of httplib/urllib2 (hence I started to write a small library a while back, only published today [2]), but the following points are not HTTP-specific. 1. Hostname checking. From what I gather by reading the archives on this list, the issue of hostname checking seems controversial [3]. It seems widely admitted by browser communities nowadays to check that the hostname the CN field of the subject DN or the DNS entries of subjectAltName. I'd feel more comfortable if this was the default behaviour of the client in Python's SSL module, although having a mechanism to override this would be useful indeed. It's more or less a basic security requirement to check the identity of the server before doing anything else. 2. Cipher suite selection. It's useful to restrict the list of cipher suites that can be used, not just for speed (as mentioned in [1]), but also because some cipher suites may be considered insecure by some institutions. This would be a good feature to have indeed. 3. Full chain of certificates. The PyOpenSSL module is able to take a callback function that verifies each certificate in the chain (using depth). According to the documentation, the ssl module only exposes the first certificate in the chain (no CA). In some applications, it is useful to verify certain policies according to attributes further up in the chain. I'd like to suggest having an "SSLSocket.getpeercerts(binary_form=False)" (plural) that returns a list of certificates in the verification chain. Is there a place where the status of the ssl module is summarized, or a better place to discuss this? I could try to provide contributions or further details if appropriate. Best wishes, Bruno. [1] http://mail.python.org/pipermail/python-dev/2009-September/091636.html [2] http://code.google.com/p/python-httpclient [2] http://bugs.python.org/issue1589

3 2

MSDN subscribers: Using Visual Studio?
by Steve Holden 29 Oct '09

29 Oct '09

I just wondered, with the recent flood of new MSDN subscriptions loosed on the developer community, how many people have installed the required version of Visual Studio and built Python for Windows from source? Not being that familiar with the process myself I was hoping for some advice from the inexperienced who have overcome any hurdles there might be. I have a pristine virtual machine just waiting for the right pieces ... Also what other uses have you found for the licenses? It would be good to get some information about how useful the licenses have been, and how they have helped people to improve Python's quality or ease of distribution (if they have). I'm sure Microsoft would also appreciate some positive feedback to their generosity, and I'll undertake to provide that if this message elicits much by way of reply. Since I'm pretty much too busy to follow the dev list right now I'd appreciate direct Cc's. regards Steve PS: If any further core developers need licenses, I plan to apply to Microsoft again in the new year. I'll be sending out a message then, I don't intend to keep a waiting list. -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ Watch PyCon on video now! http://pycon.blip.tv/

4 3