Mailman 3 August 2008 - Python-Dev

Stable / unstable buildbots
by Antoine Pitrou Aug. 28, 2008

Aug. 28, 2008

Hello everyone, What is the rationale behind the distinction between "stable" and "unstable" buildbots? I ask that because the OpenBSD buildbot has failed compiling 3.0 for quite some time, but since that buildbot was in the "unstable" bunch, it was not discovered until someone filed a bug report for it (see http://bugs.python.org/issue3696). Regards Antoine.

3 4

Reference leak in thread._local
by Ben.Cottrell＠nominum.com Aug. 27, 2008

Aug. 27, 2008

I noticed that thread._local can leak references if objects are being stored inside the thread._local object whose destructors might release the GIL. The way this happens is that in Modules/threadmodule.c, in the _ldict() function, it does things like this: Py_CLEAR(self->dict); Py_INCREF(ldict); self->dict = ldict; If the Py_CLEAR ends up taking away the last reference to an object contained in the dict, and a thread context switch occurs during that object's deallocation, then self->dict might not be NULL on return from Py_CLEAR; another thread might have run, accessed something in the same thread._local object, and caused self->dict to be set to something else (and Py_INCREF'ed). So when we blindly do the assignment into self->dict, we may be overwriting a valid reference, and not properly Py_DECREFing it. The recent change (revision 64601 to threadmodule.c) did not address context switches during the Py_CLEAR call; only context switches during tp_init. The attached patch (against trunk) is my first attempt at fixing this. It detects if self->dict has been set to something else after the Py_CLEAR, and retries the Py_CLEAR (because _ldict really only cares about installing the proper value of self->dict for the currently running thread). However, I am still uncomfortable about the fact that local_getattro and local_setattro discard the value returned from _ldict, and instead hand off control to the PyObject_Generic layer and trust that by the time self->dict is actually used, it still has the correct value for the current thread. Would it be better to, say, inline a copy of the PyObject_Generic* functions inside local_getattro/local_setattro, and force the operations to be done on the actual dict returned by _ldict()? Thanks, ~Ben

2 1

subprocess insufficiently platform-independent?
by Guido van Rossum Aug. 27, 2008

Aug. 27, 2008

Several people at Google seem to have independently discovered that despite all of the platform-independent goodness in subprocess.py, you still need to be platform aware. One of my colleagues summarized it like this: """ Given a straightforward command list like: cmd = ['svn', 'ls', 'http://rietveld.googlecode.com/svn/trunk'] You apparently cannot pass this list to any subprocess function (subprocess.call() or otherwise) with a set of arguments that allow it to "just work" on both Windows and non-Windows systems. If you call: subprocess.call(cmd, shell=False) Then it works on Linux, but fails on Windows because it does not perform the Windows %PATHEXT% search that allows it to find that "svn.exe" is the actual executable to be invoked. If you call: subprocess.call(cmd, shell=True) Then it works on Windows (it finds the "svn.exe" executable), but it fails on Linux because it *only* executes the first argument in the list ("svn") and does not pass the remaining arguments in the list to the "svn" invocation. This forces you to code platform-dependent behavior in how you call subprocess.call() when using a list. For example, you can make the shell= argument depend on the platform you're running on, like: subprocess.call(cmd, shell=(sys.platform=='win32')) Or you could vary cmd[0] in your list based on the platform: if sys.platform == 'win32': svn = 'svn.exe' else: svn = 'svn' cmd = [svn, 'ls', 'http://rietveld.googlecode.com/svn/trunk'] But either way, this seems clearly broken (or at least sub-optimal) for a module that's supposed to abstract platform-specific execution issues from the user. """ -- --Guido van Rossum (home page: http://www.python.org/~guido/)

11 14

the explicit self
by Kilian Klimek Aug. 27, 2008

Aug. 27, 2008

Hello, i know this has been discusses very much, i'm sorry, but i can't help it. In a nutshell, the proposal is as follows: 1. Self remains explicit (like it is now). 2. if a class is a subclass of a special class, e.g. named 'selfless', the self parameter is not required and a special variable, named 'this' is provided. For example: class Foo (selfless): def __init__ (x, y): this.x = x ... A patch for 3.0b3 implementing this can be found at http://www-lehre.inf.uos.de/~kklimek/misc/python_slp_8.diff regards, Kilian Klimek

8 8

Things to Know About Super
by michele.simionato＠gmail.com Aug. 26, 2008

Aug. 26, 2008

Recently I have opened a blog on Artima and I am publishing a few Python-related essays I had in store. In particular a trilogy of papers about "super". From the foreword: """ In 2004 I decided to write a comprehensive paper documenting ``super`` pitfalls and traps, with the goal of publishing it on the Python web site, just as I had published my essay on multiple inheritance and the `Method Resolution Order`_. With time the paper grew longer and longer but I never had the feeling that I had covered everything I needed to say: moreover I have a full time job, so I never had the time to fully revise the paper as a whole. As a consequence, four years have passed and the paper is still in draft status. This is a pity, since it documents issues that people encounter and that regularly come out on the Python newsgroups and forums. Keeping the draft sitting on my hard disk is doing a disservice to the community. Still, I lack to time to finish it properly. To come out from the impasse, I decided to split the long paper in a series of short blog posts, which I do have the time to review properly. Moreover people are free to post comments and corrections in case I am making mistakes (speaking about ``super`` this is always possible). Once I finish the series, I may integrate the corrections, put it together again and possibly publish it as whole on the Python website. In other words, in order to finish the task, I am trying the strategies of *divide et conquer* and *release early, release often*. We will see how it goes. """ It seems the strategy worked since I have finished the paper, by splitting it in three post. Actually, it it not really finished because it does not say anything about the new super in Python 3.0, but that could do in a separate essay. Here I am asking for feedback/corrections. Moreover if people think it is worthy idea, I can re-assemble the paper again, donate it to the PSF and publish it on the Python website. Michele Simionato

6 11

Mercurial mirrors
by Antoine Pitrou Aug. 26, 2008

Aug. 26, 2008

Hello, Thanks to Neil Schemenauer, we now have some Mercurial mirrors hosted at http://code.python.org/hg/ Here are the URLs for each of the available repositories: - http://code.python.org/hg/trunk/ - http://code.python.org/hg/branches/py3k/ - http://code.python.org/hg/branches/release2.5-maint/ For Mercurial beginners, each of the URLs above references both a human-readable Web interface to browse the repositories (view changesets, subscribe to an RSS feed, annotate source files...), and a machine-queryable address to "clone" the repositories for local, offline use. That is, if you run "hg clone http://code.python.org/hg/trunk/", you'll get a "trunk" subdirectory containing the whole trunk history in a local repository, as well as a working copy. You can then do, e.g. "hg annotate setup.py" at light speed. The Mercurial mirrors are sync'ed with the SVN repo every 5 minutes. They are read-only, that is you cannot push to them. (technical note: the mirrors are done using my own "hgsvn", but an alternate strategy would be to use the builtin "hg convert" instead) Regards Antoine.

4 6

current problems that should be fixed before release (2.6)
by Neal Norwitz Aug. 26, 2008

Aug. 26, 2008

Below are the problems I found that have not been fixed at r65995 on trunk (2.6). There will be a separate mail for the 3.0 problems. I've done the following: * built in debug and opt mode (gcc 4.1.2) fixing the important warnings * run all the tests in both modes * run all the tests (except test_logging and test_ssl) under valgrind 3.3.1 fixing the problems * run failmalloc (1.0) on startup for the first 4000 allocations fixing most problems * run pychecker 0.8.18 over the stdlib (Lib/*.py, Lib/[bcdelhjmswx]*/*.py) Can someone run purify on windows over 2.6? Can someone (else) compare performance of 2.5, 2.6, and 3.0? pychecker reported problems: Lib/gzip.py:114: Using property (filename) in classic class GzipFile may not work Lib/io.py:1394: No module attribute (device_encoding) found - device_encoding access is protected, but I don't see where it can be set Lib/sched.py:122: Using property (queue) in classic class scheduler may not work Lib/tempfile.py:535: Using property (closed) in classic class SpooledTemporaryFile may not work Lib/tempfile.py:539: Using property (encoding) in classic class SpooledTemporaryFile may not work Lib/tempfile.py:553: Using property (mode) in classic class SpooledTemporaryFile may not work Lib/tempfile.py:557: Using property (name) in classic class SpooledTemporaryFile may not work Lib/tempfile.py:561: Using property (newlines) in classic class SpooledTemporaryFile may not work Lib/tempfile.py:580: Using property (softspace) in classic class SpooledTemporaryFile may not work Lib/multiprocessing/connection.py:95: No global (PipeListener) found Lib/multiprocessing/connection.py:132: No global (PipeClient) found - both of the Pipe classes are defined only for windows. I don't see a similar def for Unix. Lib/multiprocessing/connection.py:378: No global (AuthenticationError) found I haven't looked into the property complaint to see how bad it is. test_bsddb3 is failing on several 2.6 buildbots. test_bsddb3 leaks 80 references. Results of valgrind 3.3.1 on Ubuntu 6.10 amd64, gcc 4.1.2 (prerelease): Uninitialized memory reads: No problems in python code. Though it's possible there are errors in ctypes. I'm ignoring those reports due to a problem in dlopen. Wild memory writes: No problems in python code. Memory leaks: No major problems. However, it seems that there are a bunch of little leaks when forking a process. I haven't determined if these are real problems or not. There is one memory leak in ctypes (sorry no real useful information): 44 bytes in 1 blocks are definitely lost in loss record 43 of 191 malloc (vg_replace_malloc.c:207) resize (callproc.c:1733) It seems to always leak 44 bytes n

6 10

Fix python darwin build to look for non-framework Tk if not a framework build?
by Russell E. Owen Aug. 26, 2008

Aug. 26, 2008

Python's own setup.py file seems to only look for a Framework Tcl/Tk on darwin. This is a headache if one is trying to build a non-framework python that uses a non-framework tcl/tk. Any chance of getting this fixed? I'm willing to work on a patch if it has any chance of being accepted. Any hints on how to determine if it is a framework build from within setup.py would be most welcome! -- Russell

2 1

Confusing listreverseiterator Behavior
by Armin Ronacher Aug. 26, 2008

Aug. 26, 2008

Hi, I stumbled upon a confusing listreverseiterator behavior: >>> l = [1, 2, 3, 4] >>> i = iter(l) >>> ri = reversed(l) >>> len(i) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: object of type 'listiterator' has no len() >>> len(ri) 4 >>> ri.next() 4 >>> len(ri) 3 This is the only reverse iterator with that sort of behavior. Is that intentional if yes, why? I stumbled upon that when writing a templating engine that tried to lazily get the length of the sequence / iterator but failed doing so after the first iteration because the length of the reverse iterator changes every iteration. Regards, Armin

4 5

distutils small fix
by Tarek Ziadé Aug. 26, 2008

Aug. 26, 2008

Hello Marc-André and I are looking for one more reviewer for the patch http://bugs.python.org/issue2562 It makes it possible to use accented letters in distutils' metadata, like the author name, by switching those fields to Unicode. setuptools and PyPI are both fine with this, it is just a matter of making distutils writing the PKG_FILE correctly. Regards Tarek -- Tarek Ziadé | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/

1 0