Mailman 3 October 2006 - Python-Dev

DRAFT: python-dev summary for 2006-09-16 to 2006-09-30
by Steven Bethard Oct. 28, 2006

Oct. 28, 2006

Thanks to all of those who have already given me feedback on the last summary. Here's the next one (for the second half of September). I found the "OS X universal binaries" and "Finer-grained locking than the GIL" discussions particularly hard to follow, so I'd especially appreciate corrections on those. Thanks! ========= Summaries ========= --------------- Import features --------------- Fabio Zadrozny ran into the `previously reported relative import issues`_ where a ``from . import xxx`` always fails from a top-level module. This is because relative imports rely on the ``__name__`` of a module, so when it is just ``"__main__"``, they can't handle it properly. On the subject of imports, Guido said that one of the missing import features was to be able to say "*this* package lives *here*". Paul Moore whipped up a Python API to an import hook that could do this, but indicated that a full mechanism would need to pay more attention to the environment (e.g. PYTHONPATH and .pth files). There was also some discussion about trying to have a sort of per-module ``sys.path`` so that you could have multiple versions of the same module present, with different modules importing different versions. Phillip J. Eby suggested that this was probably not a very common need, and that implementing it would be quite difficult with things like C extensions only being able to be loaded once. In general, people seemed interested in a pure-Python implementation of the import mechanism so that they could play with some of these approaches. It looked like Brett Cannon would probably be working on that. .. _previously reported relative import issues: http://www.python.org/dev/summary/2006-06-16_2006-06-30/#relative-imports-a… Contributing thread: - `New relative import issue <http://mail.python.org/pipermail/python-dev/2006-September/068806.html>`__ ---------------------------- Python library documentation ---------------------------- A less-trolly-than-usual post from Xah Lee started a discussion about the Python documentation. Greg Ewing and others suggested following the documentation style of the Inside Macintosh series: first an "About this module" narrative explaining the concepts and how they fit together, followed by the extensive API reference. Most people agreed that simply extracting the documentation from the docstrings was a bad idea -- it lacks the high-level overview and gives equal importance to all functions, regardless of their use. Contributing thread: - `Python Doc problems <http://mail.python.org/pipermail/python-dev/2006-September/069023.html>`__ ----------------------- OS X universal binaries ----------------------- Jack Howarth asked about creating universal binaries for OS X that would support 32-bit or 64-bit on both PPC and x86. Ronald Oussoren pointed out that the 32-bit part of this was already supported, but indicated that adding 64-bit support simultaneously might be more difficult. Ronald seemed to think that modifications to pyconfig.h.in might solve the problem, though he was worried that this might cause distutils to detect some architecture features incorrectly. Contributing thread: - `python, lipo and the future? <http://mail.python.org/pipermail/python-dev/2006-September/068800.html>`__ ---------------------------------- Finer-grained locking than the GIL ---------------------------------- Martin Devera was looking into replacing the global interpreter lock (GIL) with finer-grained locking, tuned to minimize locking by assuming that most objects were used only by a single thread. For objects that were shared across multiple threads, this approach would allow non-blocking reads, but require all threads to "come home" before modifications could be made. Phillip J. Eby pointed out that most object accesses in Python are actually modifications too, due to reference counting, so it looked like Martin's proposal wouldn't work well with the current refcounting implementation of Python. After Martin v. Löwis found a bug in the locking algorithm, Martin Devera decided to take his idea back to the drawing board. Contributing thread: - `deja-vu .. python locking <http://mail.python.org/pipermail/python-dev/2006-September/068828.html>`__ --------------------------- OS X and ssize_t formatting --------------------------- The buildbots spotted an OS X error in the itertools module. After Jack Diederich fixed a bug where ``size_t`` had been used instead of ``ssize_t``, Neal Norwitz noticed some problems with ``%zd`` on OS X. Despite documentation to the contrary in both the man page and the C99 Standard, using that specifier on OS X treats a negative number as an unsigned number. Ronald Oussoren and others reported the bug to Apple. Contributing thread: - `test_itertools fails for trunk on x86 OS X machine <http://mail.python.org/pipermail/python-dev/2006-September/068898.html>`__ ------------------- itertools.flatten() ------------------- Michael Foord asked about including a flatten function that would take a sequence with sub-sequences nested to an arbitrary depth and create a simple non-nested sequence from that. People were strongly opposed to adding this as a builtin, but even as an itertools function, there was disagreement. How should strings, dicts and other arbitrary iterables be flattened? Since there wasn't one clear answer, it looked like the proposal didn't have much of a chance. Contributing thread: - `Suggestion for a new built-in - flatten <http://mail.python.org/pipermail/python-dev/2006-September/068941.html>`__ ------------------------------- Class definition syntax changes ------------------------------- Fabio Zadrozny noted that in Python 2.5, classes can now be declared as:: class C(): ... Some folks wanted the result to be a new-style class, but the presence or absence of ``()`` was deemed too subtle of a cue to make the new-style/old-style distinction. For the Python 2.X series, explicit subclassing of ``object`` will still be necessary. Contributing thread: - `Grammar change in classdef <http://mail.python.org/pipermail/python-dev/2006-September/068782.html>`__ ---------------------- Python 2.5 and GCC 4.2 ---------------------- Armin Rigo found some more signed integer overflows when using GCC 4.2 like the ones `reported earlier`_. Because Python 2.5 final was scheduled to be released in 24 hours, and it looked like there wouldn't be too many people affected these problems, they were deferred until 2.5.1. For the moment at least, the README indicates that GCC 4.1 and 4.2 shouldn't be used to compile Python. .. _reported earlier: http://www.python.org/dev/summary/2006-08-16_2006-08-31/#gcc-4-2-and-intege… Contributing threads: - `Before 2.5 - More signed integer overflows <http://mail.python.org/pipermail/python-dev/2006-September/068781.html>`__ - `GCC 4.x incompatibility <http://mail.python.org/pipermail/python-dev/2006-September/068879.html>`__ ---------------------------------- Discard method for dicts and lists ---------------------------------- Gustavo Niemeyer and Greg Ewing suggested adding ``dict.discard()`` and ``list.discard()`` along the lines of ``set.discard()``. Fred L. Drake, Jr. explained that ``dict.discard(foo)`` is currently supported with ``dict.pop(foo, None)``. There was more debate about the ``list`` version, but most people seemed to think that wrapping ``list.remove()`` with the appropriate if-statement or try-except was fine. Contributing threads: - `dict.discard <http://mail.python.org/pipermail/python-dev/2006-September/068884.html>`__ - `list.discard? (Re: dict.discard) <http://mail.python.org/pipermail/python-dev/2006-September/068912.html>`__ -------------------- weakref enhancements -------------------- tomer filiba offered some additions to the weakref module, weakattr_ and weakmethod_. Raymond Hettinger questioned how frequently these would be useful in the real world, but both tomer and Alex Martelli assured him that they had real-world use-cases for these. However, there didn't generally seem to be enough support for them to include them in the standard library. .. _weakattr: http://sebulba.wikispaces.com/recipe+weakattr .. _weakmethod: http://sebulba.wikispaces.com/recipe+weakmethod Contributing thread: - `weakref enhancements <http://mail.python.org/pipermail/python-dev/2006-September/069031.html>`__ ------------------------ AST structure guarantees ------------------------ Anthony Baxter asked that the AST structure get the same guarantees as the byte-code format, that is, that it would change as little as possible so that people who wanted to hack it wouldn't have to change their code for each release. Pretty much everyone agreed that this was a good idea. In a related thread, Sanghyeon Seo asked if the AST structure should become part of the Python specification so that other implementations like IronPython_ would use it as well. While most people felt like it would be good if the various specifications had similar AST representations, it seemed like mandating it as part of the implementation would lock things down too much. .. _IronPython: http://www.codeplex.com/Wiki/View.aspx?ProjectName=IronPython Contributing threads: - `IronPython and AST branch <http://mail.python.org/pipermail/python-dev/2006-September/068778.html>`__ - `IronPython and AST branch <http://mail.python.org/pipermail/python-dev/2006-September/068787.html>`__ - `AST structure and maintenance branches <http://mail.python.org/pipermail/python-dev/2006-September/068970.html>`__ ----------------------------- PEP 302: phase 2 import hooks ----------------------------- For his dissertation work, Brett Cannon needed to implement phase 2 of the `PEP 302`_ import hooks. He asked for feedback on whether it would be easier to do this within the current C code, or whether it would be better to rewrite the import mechanisms in Python first. Phillip J. Eby gave some advice on how to restructure things, and suggested that the C code was somewhat delicate and having a Python implementation around would be a Good Thing. Armin Rigo strongly recommended rewriting things in Python. .. _PEP 302: http://www.python.org/dev/peps/pep-0302/ Contributing thread: - `difficulty of implementing phase 2 of PEP 302 in Python source <http://mail.python.org/pipermail/python-dev/2006-September/069017.html>`__ ---------------------------------------------------- Testsuite fails on Windows if a space is in the path ---------------------------------------------------- Martin v. Löwis was trying to fix some bugs where spaces in Windows paths caused some of the testsuite to fail. For example, test_popen was getting an error because ``os.popen`` invoked:: cmd.exe /c "c:\Program Files\python25\python.exe" -c "import sys;print sys.version" which failed complaining that c:\Program is not a valid executable. Jean-Paul Calderon and Tim Peters explained that the ``cmd.exe`` part is necessary to force proper cmd.exe-style argument parsing and to allow environment variable substitution. After scrutinizing the MS quoting rules, it seemed like fixing this for Python 2.5 was too likely to introduce incompatibilities, so it was postponed to 2.6. Contributing thread: - `Testsuite fails on Windows if a space is in the path <http://mail.python.org/pipermail/python-dev/2006-September/068784.html>`__ ----------------------------------------- PEP 353: Backwards-compatibility #defines ----------------------------------------- David Abrahams suggested a modification to the suggested backwards-compatibility #define incantation of `PEP 353`_ so that the PY_SSIZE_T_MAX and PY_SSIZE_T_MIN would only ever get defined once. There was some discussion about whether or not this was absolutely necessary, but everyone agreed that the change was probably sensible regardless. .. _PEP 353: http://www.python.org/dev/peps/pep-0353/ Contributing thread: - `Pep 353: Py_ssize_t advice <http://mail.python.org/pipermail/python-dev/2006-September/068944.html>`__ ----------------- Bare-bones Python ----------------- Milan Krcmar asked about what he could drop from Python to make it small enough to fit on a platform with only 2 MiB of flash ROM and 16 MiB of RAM. Giovanni Bajo suggested dropping the CJK codecs (which account for about 800K), though he also noted that after that there weren't any really low-hanging fruit. Martin v. Löwis suggested that he might also get a gain out of dropping support for dynamic loading of extension modules, and linking all necessary modules statically. Gustavo Niemeyer pointed him to `Python for S60`_ and `Python for Maemo`_ which had to undergo similar stripping down. .. _Python for S60: http://opensource.nokia.com/projects/pythonfors60/ .. _Python for Maemo: http://pymaemo.sf.net Contributing thread: - `Minipython <http://mail.python.org/pipermail/python-dev/2006-September/068987.html>`__ ================ Deferred Threads ================ - `Removing __del__ <http://mail.python.org/pipermail/python-dev/2006-September/068875.html>`__ - `Caching float(0.0) <http://mail.python.org/pipermail/python-dev/2006-September/069049.html>`__ - `PEP 355 status <http://mail.python.org/pipermail/python-dev/2006-September/069059.html>`__ - `PEP 351 - do while <http://mail.python.org/pipermail/python-dev/2006-September/069088.html>`__ ================== Previous Summaries ================== - `Signals, threads, blocking C functions <http://mail.python.org/pipermail/python-dev/2006-September/068996.html>`__ =============== Skipped Threads =============== - `Thank you all <http://mail.python.org/pipermail/python-dev/2006-September/068780.html>`__ - `BRANCH FREEZE/IMMINENT RELEASE: Python 2.5 (final). 2006-09-19, 00:00UTC <http://mail.python.org/pipermail/python-dev/2006-September/068819.html>`__ - `RELEASED Python 2.5 (FINAL) <http://mail.python.org/pipermail/python-dev/2006-September/068851.html>`__ - `release25-maint branch - please keep frozen for a day or two more. <http://mail.python.org/pipermail/python-dev/2006-September/068852.html>`__ - `Download URL typo <http://mail.python.org/pipermail/python-dev/2006-September/068855.html>`__ - `Exceptions and slicing <http://mail.python.org/pipermail/python-dev/2006-September/068866.html>`__ - `Weekly Python Patch/Bug Summary <http://mail.python.org/pipermail/python-dev/2006-September/068872.html>`__ - `release25-maint is UNFROZEN <http://mail.python.org/pipermail/python-dev/2006-September/068878.html>`__ - `Small Py3k task: fix modulefinder.py <http://mail.python.org/pipermail/python-dev/2006-September/068883.html>`__ - `win32 - results from Lib/test - 2.5 release-maint <http://mail.python.org/pipermail/python-dev/2006-September/068888.html>`__ - `Weekly Python Patch/Bug Summary ** REVISED ** <http://mail.python.org/pipermail/python-dev/2006-September/068893.html>`__ - `[Python-checkins] release25-maint is UNFROZEN <http://mail.python.org/pipermail/python-dev/2006-September/068937.html>`__ - `Python network Programmign <http://mail.python.org/pipermail/python-dev/2006-September/068938.html>`__ - `Relative import bug? <http://mail.python.org/pipermail/python-dev/2006-September/068942.html>`__ - `GCC patch for catching errors in PyArg_ParseTuple <http://mail.python.org/pipermail/python-dev/2006-September/068958.html>`__ - `Typo.pl scan of Python 2.5 source code <http://mail.python.org/pipermail/python-dev/2006-September/068963.html>`__ - `Maybe we should have a C++ extension for testing... <http://mail.python.org/pipermail/python-dev/2006-September/068971.html>`__ - `Python 2.5 bug? Changes in behavior of traceback module <http://mail.python.org/pipermail/python-dev/2006-September/068975.html>`__ - `Need help with C - problem in sqlite3 module <http://mail.python.org/pipermail/python-dev/2006-September/068979.html>`__ - `PyErr_CheckSignals error return value <http://mail.python.org/pipermail/python-dev/2006-September/068994.html>`__ - `python-dev summary for 2006-08-01 to 2006-08-15 <http://mail.python.org/pipermail/python-dev/2006-September/069002.html>`__ - `2.4.4c1 October 11, 2.4.4 final October 18 <http://mail.python.org/pipermail/python-dev/2006-September/069003.html>`__ - `[SECUNIA] "buffer overrun in repr() for unicode strings" Potential Vulnerability (fwd) <http://mail.python.org/pipermail/python-dev/2006-September/069007.html>`__ - `List of candidate 2.4.4 bugs? <http://mail.python.org/pipermail/python-dev/2006-September/069008.html>`__ - `openssl - was: 2.4.4c1 October 11, 2.4.4 final October 18 <http://mail.python.org/pipermail/python-dev/2006-September/069013.html>`__ - `Collecting 2.4.4 fixes <http://mail.python.org/pipermail/python-dev/2006-September/069024.html>`__ - `os.unlink() closes file? <http://mail.python.org/pipermail/python-dev/2006-September/069052.html>`__ - `Tix not included in 2.5 for Windows <http://mail.python.org/pipermail/python-dev/2006-September/069070.html>`__ - `Possible semantic changes for PEP 352 in 2.6 <http://mail.python.org/pipermail/python-dev/2006-September/069091.html>`__

3 2

Re: [Python-Dev] Typo.pl scan of Python 2.5 source code
by Johnny Lee Oct. 28, 2006

Oct. 28, 2006

I grabbed the latest Python2.5 code via subversion and ran my typo script on it. Weeding out the obvious false positives and Neal's comments leaves about 129 typos. See http://www.geocities.com/typopl/typoscan.htm Should I enter the typos as bugs in the Python bug db? J > Date: Fri, 22 Sep 2006 21:51:38 -0700> From: nnorwitz(a)gmail.com> To: typo_pl(a)hotmail.com> Subject: Re: [Python-Dev] Typo.pl scan of Python 2.5 source code> CC: python-dev(a)python.org> > On 9/22/06, Johnny Lee <typo_pl(a)hotmail.com> wrote:> >> > Hello,> > My name is Johnny Lee. I have developed a *ahem* perl script which scans> > C/C++ source files for typos.> > Hi Johnny.> > Thanks for running your script, even if it is written in Perl and ran> on Windows. :-)> > > The Python 2.5 typos can be classified into 7 types.> >> > 2) realloc overwrite src if NULL, i.e. p = realloc(p, new_size);> > If realloc() fails, it will return NULL. If you assign the return value to> > the same variable you passed into realloc,> > then you've overwritten the variable and possibly leaked the memory that the> > variable pointed to.> > A bunch of these warnings were accurate and a bunch were not. There> were 2 reasons for the false positives. 1) The pointer was aliased,> thus not lost, 2) On failure, we exited (Parser/*.c)> > > 4) if ((X!=0) || (X!=1))> > These 2 cases occurred in binascii. I have no idea if the warning is> wright or the code is.> > > 6) XX;;> > Just being anal here. Two semicolons in a row. Second one is extraneous.> > I already checked in a fix for these on HEAD. Hard for even me to> screw up those fixes. :-)> > > 7) extraneous test for non-NULL ptr> > Several memory calls that free memory accept NULL ptrs.> > So testing for NULL before calling them is redundant and wastes code space.> > Now some codepaths may be time-critical, but probably not all, and smaller> > code usually helps.> > I ignored these as I'm not certain all the platforms we run on accept> free(NULL).> > Below is my categorization of the warnings except #7. Hopefully> someone will fix all the real problems in the first batch.> > Thanks again!> > n> --> > # Problems> Objects\fileobject.c (338): realloc overwrite src if NULL; 17:> file->f_setbuf=(char*)PyMem_Realloc(file->f_setbuf,bufsize)> Objects\fileobject.c (342): using PyMem_Realloc result w/no check> 30: setvbuf(file->f_fp, file->f_setbuf, type, bufsize);> [file->f_setbuf]> Objects\listobject.c (2619): using PyMem_MALLOC result w/no check> 30: garbage[i] = selfitems[cur]; [garbage]> Parser\myreadline.c (144): realloc overwrite src if NULL; 17:> p=(char*)PyMem_REALLOC(p,n+incr)> Modules\_csv.c (564): realloc overwrite src if NULL; 17:> self->field=PyMem_Realloc(self->field,self->field_size)> Modules\_localemodule.c (366): realloc overwrite src if NULL; 17:> buf=PyMem_Realloc(buf,n2)> Modules\_randommodule.c (290): realloc overwrite src if NULL; 17:> key=(unsigned#long*)PyMem_Realloc(key,bigger*sizeof(*key))> Modules\arraymodule.c (1675): realloc overwrite src if NULL; 17:> self->ob_item=(char*)PyMem_REALLOC(self->ob_item,itemsize*self->ob_size)> Modules\cPickle.c (536): realloc overwrite src if NULL; 17:> self->buf=(char*)realloc(self->buf,n)> Modules\cPickle.c (592): realloc overwrite src if NULL; 17:> self->buf=(char*)realloc(self->buf,bigger)> Modules\cPickle.c (4369): realloc overwrite src if NULL; 17:> self->marks=(int*)realloc(self->marks,s*sizeof(int))> Modules\cStringIO.c (344): realloc overwrite src if NULL; 17:> self->buf=(char*)realloc(self->buf,self->buf_size)> Modules\cStringIO.c (380): realloc overwrite src if NULL; 17:> oself->buf=(char*)realloc(oself->buf,oself->buf_size)> Modules\_ctypes\_ctypes.c (2209): using PyMem_Malloc result w/no> check 30: memset(obj->b_ptr, 0, dict->size); [obj->b_ptr]> Modules\_ctypes\callproc.c (1472): using PyMem_Malloc result w/no> check 30: strcpy(conversion_mode_encoding, coding);> [conversion_mode_encoding]> Modules\_ctypes\callproc.c (1478): using PyMem_Malloc result w/no> check 30: strcpy(conversion_mode_errors, mode);> [conversion_mode_errors]> Modules\_ctypes\stgdict.c (362): using PyMem_Malloc result w/no> check 30: memset(stgdict->ffi_type_pointer.elements, 0,> [stgdict->ffi_type_pointer.elements]> Modules\_ctypes\stgdict.c (376): using PyMem_Malloc result w/no> check 30: memset(stgdict->ffi_type_pointer.elements, 0,> [stgdict->ffi_type_pointer.elements]> > # No idea if the code or tool is right.> Modules\binascii.c (1161)> Modules\binascii.c (1231)> > # Platform specific files. I didn't review and won't fix without testing.> Python\thread_lwp.h (107): using malloc result w/no check 30:> lock->lock_locked = 0; [lock]> Python\thread_os2.h (141): using malloc result w/no check 30:> (long)sem)); [sem]> Python\thread_os2.h (155): using malloc result w/no check 30:> lock->is_set = 0; [lock]> Python\thread_pth.h (133): using malloc result w/no check 30:> memset((void *)lock, '\0', sizeof(pth_lock)); [lock]> Python\thread_solaris.h (48): using malloc result w/no check 30:> funcarg->func = func; [funcarg]> Python\thread_solaris.h (133): using malloc result w/no check 30:> if(mutex_init(lock,USYNC_THREAD,0)) [lock]> > # Who cares about these modules.> Modules\almodule.c:182> Modules\svmodule.c:547> > # Not a problem.> Parser\firstsets.c (76)> Parser\grammar.c (40)> Parser\grammar.c (59)> Parser\grammar.c (83)> Parser\grammar.c (102)> Parser\node.c (95)> Parser\pgen.c (52)> Parser\pgen.c (69)> Parser\pgen.c (126)> Parser\pgen.c (438)> Parser\pgen.c (462)> Parser\tokenizer.c (797)> Parser\tokenizer.c (869)> Modules\_bsddb.c (2633)> Modules\_csv.c (1069)> Modules\arraymodule.c (1871)> Modules\gcmodule.c (1363)> Modules\zlib\trees.c (375) _________________________________________________________________ Get the new Windows Live Messenger! http://get.live.com/messenger/overview

1 0

Re: [Python-Dev] [Python-checkins] r52482 - in python/branches/release25-maint: Lib/urllib.py Lib/urllib2.py Misc/NEWS
by Anthony Baxter Oct. 27, 2006

Oct. 27, 2006

On Saturday 28 October 2006 03:13, andrew.kuchling wrote: > 2.4 backport candidate, probably. FWIW, I'm not planning on doing any more "collect all the bugfixes" releases of 2.4. It's now in the same category as 2.3 - that is, only really serious bugs (in particular, security related bugs) will get a new release, and then only with the serious bugfixes applied. One active maintenance branch is quite enough to deal with, IMHO. -- Anthony Baxter <anthony(a)interlink.com.au> It's never too late to have a happy childhood.

1 0

Modulefinder
by Thomas Heller Oct. 27, 2006

Oct. 27, 2006

I have patched Lib/modulefinder.py to work with absolute and relative imports. It also is faster now, and has basic unittests in Lib/test/test_modulefinder.py. The work was done in a theller_modulefinder SVN branch. If nobody objects, I will merge this into trunk, and possibly also into release25-maint, when I have time. Thanks, Thomas

3 3

DRAFT: python-dev summary for 2006-09-01 to 2006-09-15
by Steven Bethard Oct. 27, 2006

Oct. 27, 2006

Here's the summary for the first half of September. As always, comments and corrections are greatly appreciated! ============= Announcements ============= ---------------------------- QOTF: Quote of the Fortnight ---------------------------- Through a cross-posting slip-up, Jean-Paul Calderone managed to provide us with some inspiring thoughts on mailing-list archives: One could just as easily ask why no one bothers to read mailing list archives to see if their question has been answered before. No one will ever know, it is just one of the mysteries of the universe. Contributing thread: - `[Twisted-Python] Newbie question <http://mail.python.org/pipermail/python-dev/2006-September/068682.html>`__ ------------------------- Monthly Arlington sprints ------------------------- Jeffrey Elkner has arranged for monthly Arlington Python sprints. See the `Arlington sprint wiki`_ for more details. .. _Arlington sprint wiki: http://wiki.python.org/moin/ArlingtonSprint Contributing thread: - `Arlington sprints to occur monthly <http://mail.python.org/pipermail/python-dev/2006-September/068688.html>`__ ========= Summaries ========= ----------------------------------------- Signals, threads and blocking C functions ----------------------------------------- Gustavo Carneiro explained a problem that pygtk was running into. Their main loop function, ``gtk_main()``, blocks forever. If there are threads in the program, they cannot receive signals because Python catches the signal and calls ``Py_AddPendingCall()``, relying on the main thread to call ``Py_MakePendingCalls()``. Since with pygtk, the main thread is blocked calling a C function, it has no way other than polling to decide when ``Py_MakePendingCalls()`` needs to be called. Gustavo was hoping for some sort of API so that his blocking thread could get notified when ``Py_AddPendingCall()`` had been called. There was a long discussion about the feasibility of this and other solutions to his problem. One of the main problems is that almost nothing can safely be done from a signal handler context, so some people felt like having Python invoke arbitrary third-party code was a bad idea. Gustavo was reasonably confident that he could write to a pipe within that context, which was all he needed to do to solve his problem, but Nick Maclaren explained in detail some of the problems, e.g. writing proper synchronization primitives that are signal-handler safe. Jan Kanis suggested that threads in a pygtk program should occasionally check the signal handler flags and calls PyGTK's callback to wake up the main thread. But Gustavo explained that things like the GnomeVFS library have their own thread pools and know nothing about Python so can't make such a callback. Adam Olsen that Python could create a single non-blocking pipe for all signals. When a signal was handled, the signal number would be written to that pipe as a single byte. Third-party libraries, like pygtk, could poll the appropriate file descriptor, waking up and handing control back to Python when a signal was received. There were some disadvantages to this approach, e.g. if there is a large burst of signals, some of them would be lost, but folks seemed to think that these kinds of things would not cause many real-world problems. Gustavo and Adam then worked out the code in a little more detail. The `Py_signal_pipe patch`_ was posted to SourceForge. .. _Py_signal_pipe patch: http://bugs.python.org/1564547 Contributing thread: - `Signals, threads, blocking C functions <http://mail.python.org/pipermail/python-dev/2006-September/068569.html>`__ ------------------------ API for str.rpartition() ------------------------ Raymond Hettinger pointed out that in cases where the separator was not found, ``str.rpartition()`` was putting the remainder of the string in the wrong spot, e.g. ``str.rpartition()`` worked like:: 'axbxc'.rpartition('x') == ('axb', 'x', 'c') 'axb'.rpartition('x') == ('a', 'x', 'b') 'a'.rpartition('x') == ('a', '', '') # should be ('', '', 'a') Thus code that used ``str.rpartition()`` in a loop or recursively would likely never terminate. Raymond checked in a fix for this, spawning an enormous discussion about how the three bits ``str.rpartition()`` returns should be named. There was widespread disagreement on which side was the "head" and which side was the "tail", and the only unambiguous one seemed to be "left, sep, right". Raymond and others were not as happy with this version because it was no longer suggestive of the use cases, but it looked like this might be the best compromise. Contributing threads: - `Problem withthe API for str.rpartition() <http://mail.python.org/pipermail/python-dev/2006-September/068565.html>`__ - `Fwd: Problem withthe API for str.rpartition() <http://mail.python.org/pipermail/python-dev/2006-September/068615.html>`__ --------------- Unicode Imports --------------- Kristján V. Jónsson submitted a `unicode import patch`_ that would allow unicode paths in sys.path and use the unicode file API on Windows. It got a definite "no" from the Python 2.5 release managers since it was already too late in the release process. Nonetheless there was a long discussion about whether or not it should be considered a bug or a feature. Martin v. Löwis explained that it was definitely a feature because it would break existing introspection tools expecting things like __file__ to be 8-bit strings (not unicode strings as they would be with the patch). .. _unicode import patch: http://bugs.python.org/1552880 Contributing thread: - `Unicode Imports <http://mail.python.org/pipermail/python-dev/2006-September/068686.html>`__ ------------------------- Exception and __unicode__ ------------------------- Marcin 'Qrczak' Kowalczyk reported a `TypeError from unicode()`_ when applied to an Exception class. Brett Cannon explained the source of this: BaseException defined a ``__unicode__`` descriptor which was complaining when it was handed a class, not an instance. The easiest solution seemed to be the best for Python 2.5: simply rip out the ``__unicode__`` method entirely. M.-A. Lemburg suggested that for Python 2.6 this should be fixed by introducing a tp_unicode slot. .. _TypeError from unicode(): http://bugs.python.org/1551432 Contributing thread: - `2.5 status <http://mail.python.org/pipermail/python-dev/2006-September/068607.html>`__ -------------------------- Slowdown in inspect module -------------------------- Fernando Perez reported an enormous slowdown in Python 2.5's inspect module. Nick Coghlan figured out that this was a result of ``inspect.findsource()`` calling ``os.path.abspath()`` and ``os.path.normpath()`` repeatedly on the module's file name. Nick provided a `patch to speed things up`_ by caching the absolute, normalized file names. .. _patch to speed things up: http://bugs.python.org/1553314 Contributing thread: - `inspect.py very slow under 2.5 <http://mail.python.org/pipermail/python-dev/2006-September/068656.html>`__ -------------------------------- Cross-platform float consistency -------------------------------- Andreas Raab asked about trying to minimize some of the cross-platform differences in floating-point calcuations, by using something like fdlibm_. Tim Peters pointed him to a `previous thread on this issue`_ and suggested that best route was probably to package a Python wrapper for fdlibm_ and see how much interest there was. .. _fdlibm: http://www.netlib.org/fdlibm/ .. _previous thread on this issue: http://mail.python.org/pipermail/python-list/2005-July/290164.html Contributing thread: - `Cross-platform math functions? <http://mail.python.org/pipermail/python-dev/2006-September/068599.html>`__ ----------------------------------- Refcounting and errors in functions ----------------------------------- Mihai Ibanescu pointed out that refcount status for functions that can fail is generally poorly documented. Greg Ewing explained that refcounting behavior should be independent of whether the call succeeds or fails, but it was clear that this was not always the case. Mihai promised to file a low-severity bug so that this problem wouldn't be lost. Contributing thread: - `Py_BuildValue and decref <http://mail.python.org/pipermail/python-dev/2006-September/068708.html>`__ ------------ Python 2.3.6 ------------ Barry Warsaw offered to push out a Python 2.3.6 if folks were interested in getting some bugfixes out to the platforms which were still running Python 2.3. After an underwhelming response, he retracted the offer. Contributing threads: - `Interest in a Python 2.3.6? <http://mail.python.org/pipermail/python-dev/2006-August/068520.html>`__ - `Interest in a Python 2.3.6? <http://mail.python.org/pipermail/python-dev/2006-September/068731.html>`__ - `Python 2.4.4 was: Interest in a Python 2.3.6? <http://mail.python.org/pipermail/python-dev/2006-September/068737.html>`__ ----------------------------------- Effbot Python library documentation ----------------------------------- Johann C. Rocholl asked about the status of http://effbot.org/lib/, Fredrik Lundh's alternative format and rendering for the Python library documentation. Fredrik indicated that due to the pushback from some folks on python-dev, they've been working mainly "under the radar" on this. (At least until some inconsiderate soul put them in the summary...) ;-) Contributing threads: - `That library reference, yet again <http://mail.python.org/pipermail/python-dev/2006-August/068556.html>`__ - `That library reference, yet again <http://mail.python.org/pipermail/python-dev/2006-September/068561.html>`__ ================ Deferred Threads ================ - `IronPython and AST branch <http://mail.python.org/pipermail/python-dev/2006-September/068778.html>`__ ================== Previous Summaries ================== - `Py2.5 issue: decimal context manager misimplemented, misdesigned, and misdocumented <http://mail.python.org/pipermail/python-dev/2006-September/068564.html>`__ - `Error while building 2.5rc1 pythoncore_pgo on VC8 <http://mail.python.org/pipermail/python-dev/2006-September/068582.html>`__ - `gcc 4.2 exposes signed integer overflows <http://mail.python.org/pipermail/python-dev/2006-September/068602.html>`__ - `no remaining issues blocking 2.5 release <http://mail.python.org/pipermail/python-dev/2006-September/068606.html>`__ - `new security doc using object-capabilities <http://mail.python.org/pipermail/python-dev/2006-September/068673.html>`__ =============== Skipped Threads =============== - `A test suite for unittest <http://mail.python.org/pipermail/python-dev/2006-September/068558.html>`__ - `Fwd: [Python-checkins] r51674 - python/trunk/Misc/Vim/vimrc <http://mail.python.org/pipermail/python-dev/2006-September/068562.html>`__ - `Weekly Python Patch/Bug Summary <http://mail.python.org/pipermail/python-dev/2006-September/068567.html>`__ - `Windows build slave down until Tuesday-ish <http://mail.python.org/pipermail/python-dev/2006-September/068573.html>`__ - `[Python-checkins] TRUNK IS UNFROZEN, available for 2.6 work if you are so inclined <http://mail.python.org/pipermail/python-dev/2006-September/068605.html>`__ - `Exception message for invalid with statement usage <http://mail.python.org/pipermail/python-dev/2006-September/068665.html>`__ - `buildbot breakage <http://mail.python.org/pipermail/python-dev/2006-September/068671.html>`__ - `Change in file() behavior in 2.5 <http://mail.python.org/pipermail/python-dev/2006-September/068678.html>`__ - `'with' bites Twisted <http://mail.python.org/pipermail/python-dev/2006-September/068681.html>`__ - `What windows tool chain do I need for python 2.5 extensions? <http://mail.python.org/pipermail/python-dev/2006-September/068709.html>`__ - `2.5c2 <http://mail.python.org/pipermail/python-dev/2006-September/068742.html>`__ - `_PyGILState_NoteThreadState should be static or not? <http://mail.python.org/pipermail/python-dev/2006-September/068743.html>`__ - `BRANCH FREEZE: release25-maint, 00:00UTC 12 September 2006 <http://mail.python.org/pipermail/python-dev/2006-September/068745.html>`__ - `datetime's strftime implementation: by design or bug <http://mail.python.org/pipermail/python-dev/2006-September/068747.html>`__ - `Subversion 1.4 <http://mail.python.org/pipermail/python-dev/2006-September/068761.html>`__ - `RELEASED Python 2.5 (release candidate 2) <http://mail.python.org/pipermail/python-dev/2006-September/068766.html>`__ - `Maybe we should have a C++ extension for testing... <http://mail.python.org/pipermail/python-dev/2006-September/068768.html>`__ - `.pyc file has different result for value "1.79769313486232e+308" than .py file <http://mail.python.org/pipermail/python-dev/2006-September/068769.html>`__ - `release is done, but release25-maint branch remains near-frozen <http://mail.python.org/pipermail/python-dev/2006-September/068773.html>`__ - `fun threading problem <http://mail.python.org/pipermail/python-dev/2006-September/068774.html>`__ - `Thank you all <http://mail.python.org/pipermail/python-dev/2006-September/068779.html>`__

2 1

Weekly Python Patch/Bug Summary
by Kurt B. Kaiser Oct. 27, 2006

Oct. 27, 2006

Patch / Bug Summary ___________________ Patches : 434 open ( +3) / 3430 closed ( +5) / 3864 total ( +8) Bugs : 929 open (+13) / 6285 closed (+12) / 7214 total (+25) RFE : 245 open ( +1) / 240 closed ( +0) / 485 total ( +1) New / Reopened Patches ______________________ various datetime methods fail in restricted mode (2006-10-17) http://python.org/sf/1578643 opened by lplatypus PyErr_Format corrections (2006-10-17) http://python.org/sf/1578999 opened by Martin v. Löwis posix.readlink doesn't use filesystemencoding (2006-10-19) http://python.org/sf/1580674 opened by Ronald Oussoren Duplicated declaration of PyCallable_Check (2006-10-20) CLOSED http://python.org/sf/1580872 opened by Matthias Klose Allow textwrap to preserve leading and trailing whitespace (2006-10-20) http://python.org/sf/1581073 opened by Dwayne Bailey tarfile.py: 100-char filenames are truncated (2006-10-24) CLOSED http://python.org/sf/1583506 opened by Lars Gustäbel tarfile.py: better use of TarInfo objects with longnames (2006-10-24) http://python.org/sf/1583880 opened by Lars Gustäbel Tix: subwidget names (bug #1472877) (2006-10-25) http://python.org/sf/1584712 opened by Matthias Kievernagel Patches Closed ______________ patch for building trunk with VC6 (2006-03-24) http://python.org/sf/1457736 closed by loewis a faster Modulefinder (2005-11-11) http://python.org/sf/1353872 closed by theller Duplicated declaration of PyCallable_Check (2006-10-20) http://python.org/sf/1580872 closed by loewis Exec stacks in python 2.5 (2006-09-18) http://python.org/sf/1560695 closed by loewis tarfile.py: 100-char filenames are truncated (2006-10-24) http://python.org/sf/1583506 closed by gbrandl New / Reopened Bugs ___________________ 2.4.4c1 will not build when cross compiling (2006-10-16) CLOSED http://python.org/sf/1578513 opened by smithj --disable-sunaudiodev --disable-tk does not work (2006-10-17) http://python.org/sf/1579029 opened by ThurnerRupert Segfault provoked by generators and exceptions (2006-10-17) http://python.org/sf/1579370 opened by Mike Klaas Use flush() before os.exevp() (2006-10-18) http://python.org/sf/1579477 opened by Thomas Guettler Wrong syntax for PyDateTime_IMPORT in documentation (2006-10-18) CLOSED http://python.org/sf/1579796 opened by David Faure not configured for tk (2006-10-18) http://python.org/sf/1579931 opened by Carl Wenrich glob.glob("c:\\[ ]\*) doesn't work (2006-10-19) http://python.org/sf/1580472 opened by Koblaid "make install" for Python 2.4.4 not working properly (2006-10-19) http://python.org/sf/1580563 opened by Andreas Jung Configure script does not work for RHEL 4 x86_64 (2006-10-19) http://python.org/sf/1580726 reopened by gbrandl Configure script does not work for RHEL 4 x86_64 (2006-10-19) http://python.org/sf/1580726 reopened by spotvt01 Configure script does not work for RHEL 4 x86_64 (2006-10-19) http://python.org/sf/1580726 opened by Chris httplib hangs reading too much data (2006-10-19) http://python.org/sf/1580738 opened by Dustin J. Mitchell Definition of a "character" is wrong (2006-10-20) http://python.org/sf/1581182 opened by Adam Olsen pickle protocol 2 failure on int subclass (2006-10-20) http://python.org/sf/1581183 opened by Anders J. Munch missing __enter__ + __getattr__ forwarding (2006-10-21) http://python.org/sf/1581357 opened by Hirokazu Yamamoto Text search gives bad count if called from variable trace (2006-10-20) http://python.org/sf/1581476 opened by Russell Owen test_sqlite fails on OSX G5 arch if test_ctypes is run (2006-10-21) http://python.org/sf/1581906 opened by Skip Montanaro email.header decode within word (2006-10-22) http://python.org/sf/1582282 opened by Tokio Kikuchi Python is dumping core after the test test_ctypes (2006-10-23) http://python.org/sf/1582742 opened by shashi Bulding source with VC6 fails due to missing files (2006-10-23) CLOSED http://python.org/sf/1582856 opened by Ulrich Hockenbrink class member inconsistancies (2006-10-23) CLOSED http://python.org/sf/1583060 opened by EricDaigno Different behavior when stepping through code w/ pdb (2006-10-24) http://python.org/sf/1583276 opened by John Ehresman tarfile incorrectly handles long filenames (2006-10-24) CLOSED http://python.org/sf/1583537 opened by Mike Looijmans yield+break stops tracing (2006-10-24) http://python.org/sf/1583862 opened by Lukas Lalinsky __str__ cannot be overridden on unicode-derived classes (2006-10-24) http://python.org/sf/1583863 opened by Mike K SSL "issuer" and "server" functions problems - security (2006-10-24) http://python.org/sf/1583946 opened by John Nagle remove() during iteration causes items to be skipped (2006-10-24) CLOSED http://python.org/sf/1584028 opened by Kevin Rabsatt os.tempnam fails on SUSE Linux to accept directory argument (2006-10-25) CLOSED http://python.org/sf/1584723 opened by Andreas Events in list return None not True on wait() (2006-10-26) CLOSED http://python.org/sf/1585135 opened by SpinMess Bugs Closed ___________ from_param and _as_parameter_ truncating 64-bit value (2006-10-12) http://python.org/sf/1575945 closed by theller 2.4.4c1 will not build when cross compiling (2006-10-16) http://python.org/sf/1578513 closed by loewis Error with callback function and as_parameter with NumPy ndp (2006-10-10) http://python.org/sf/1574584 closed by theller PyThreadState_Clear() docs incorrect (2003-04-17) http://python.org/sf/723205 deleted by theller Wrong syntax for PyDateTime_IMPORT in documentation (2006-10-18) http://python.org/sf/1579796 closed by akuchling Configure script does not work for RHEL 4 x86_64 (2006-10-19) http://python.org/sf/1580726 closed by loewis Example typo in section 4 of 'Installing Python Modules' (2006-10-12) http://python.org/sf/1576348 closed by akuchling Bulding source with VC6 fails due to missing files (2006-10-23) http://python.org/sf/1582856 closed by loewis class member inconsistancies (2006-10-23) http://python.org/sf/1583060 closed by gbrandl mac installer profile patch vs. .bash_login (2006-09-19) http://python.org/sf/1561243 closed by sf-robot idle in python 2.5c1 freezes on macos 10.3.9 (2006-08-18) http://python.org/sf/1542949 closed by sf-robot Launcher reset to factory button provides bad command-line (2006-10-03) http://python.org/sf/1570284 closed by sf-robot tarfile incorrectly handles long filenames (2006-10-24) http://python.org/sf/1583537 deleted by cdwave remove() during iteration causes items to be skipped (2006-10-24) http://python.org/sf/1584028 closed by rhettinger os.tempnam fails on SUSE Linux to accept directory argument (2006-10-25) http://python.org/sf/1584723 closed by gbrandl Events in list return None not True on wait() (2006-10-26) http://python.org/sf/1585135 closed by gbrandl New / Reopened RFE __________________ Add os.link() and os.symlink() support for Windows (2006-10-16) http://python.org/sf/1578269 opened by M.-A. Lemburg

1 0

Fwd: Re: ANN compiler2 : Produce bytecode from Python 2.5 AST
by Michael Spencer Oct. 25, 2006

Oct. 25, 2006

Martin v. Löwis wrote: > Georg Brandl schrieb: >> Perhaps you can bring up a discussion on python-dev about your improvements >> and how they could be integrated into the standard library... > > Let me second this. The compiler package is largely unmaintained and > was known to be broken (and perhaps still is). A replacement > implementation, especially if it comes with a new maintainer, would > be welcome. > > Regards, > Martin Hello python-dev. I use AST-based code inspection and manipulation, and I've been looking forward to using v2.5 ASTs for their increased accuracy, consistency and speed. However, there is as yet no Python-exposed mechanism for compiling v2.5 ASTs to bytecode. So to meet my own need and interest I've been implementing 'compiler2', similar in scope to the stblib compiler package, but generating code from Python 2.5 _ast.ASTs. The code has evolved considerably from the compiler package: in aggregate the changes amount to a re-write. More about the package and its status below. I'm introducing this project here to discuss whether and how these changes should be integrated with the stdlib. I believe there is a prima facie need to have a builtin/stdlib capability for compiling v2.5 ASTs from Python, and there is some advantage to having that be implemented in Python. There is also a case for deprecating the v2.4 ASTs to ease maintenance and reduce the confusion associated with two different AST formats. If there is interest, I'm willing make compiler2 stdlib-ready. I'm also open to alternative approaches, including doing nothing. compiler2 Objectives and Status =============================== My goal is to get compiler2 to produce identical output to __builtin__.compile (at least optionally), while also providing an accessible framework for AST-manipulation, experimental compiler optimizations and customization. compiler2 is not finished - there are some unresolved bugs, and open questions on interface design - but already it produces identical output to __builtin__.compile for all of the stdlib modules and their tests (except for the stackdepth attribute which is different in 12 cases). All but three stdlib modules pass their tests after being compiled using compiler2. More on goals, status, known issues etc... in the project readme.txt at: http://svn.brownspencer.com/pycompiler/branches/new_ast/readme.txt Code is available in Subversion at http://svn.brownspencer.com/pycompiler/branches/new_ast/ The main test script is test/test_compiler.py which compiles all the modules in /Lib and /Lib/test and compares the output with __builtin__.compile. Best regards Michael Spencer

1 0

__str__ bug?
by Mike Krell Oct. 25, 2006

Oct. 25, 2006

Is this a bug? If not, how do I override __str__ on a unicode derived class? class S(str): def __str__(self): return '__str__ overridden' class U(unicode): def __str__(self): return '__str__ overridden' def __unicode__(self): return u'__unicode__ overridden' s = S() u = U() print 's:', s print "str(s):", str(s) print 's substitued is "%s"\n' % s print 'u:', u print "str(u):", str(u) print 'u substitued is "%s"' % u ----------------------------------------------------- s: __str__ overridden str(s): __str__ overridden s substitued is "__str__ overridden" u: str(u): __str__ overridden u substitued is "" Results are identical for 2.4.2 and 2.5c2 (running under windows). Mike

3 3

Re: [Python-Dev] __str__ bug?
by RD6T-KJYM＠asahi-net.or.jp Oct. 25, 2006

Oct. 25, 2006

I believe you've overriden unicode.__str__ as you expect. class S(str): def __str__(self): return "S.__str__" class U(unicode): def __str__(self): return "U.__str__" print str(S()) print str(U()) This script prints: S.__str__ U.__str__ Regards, -- KAJIYAMA, Tamito <RD6T-KJYM(a)asahi-net.or.jp> >Is this a bug? If not, how do I override __str__ on a unicode derived class? > >class S(str): > def __str__(self): return '__str__ overridden' > >class U(unicode): > def __str__(self): return '__str__ overridden' > def __unicode__(self): return u'__unicode__ overridden' > >s = S() >u = U() > >print 's:', s >print "str(s):", str(s) >print 's substitued is "%s"\n' % s >print 'u:', u >print "str(u):", str(u) >print 'u substitued is "%s"' % u > >----------------------------------------------------- > >s: __str__ overridden >str(s): __str__ overridden >s substitued is "__str__ overridden" > >u: >str(u): __str__ overridden >u substitued is "" > >Results are identical for 2.4.2 and 2.5c2 (running under windows). > > Mike

3 2

Python 2.4.4 docs?
by A.M. Kuchling Oct. 25, 2006

Oct. 25, 2006

Does someone need to unpack the 2.4.4 docs in the right place so that http://www.python.org/doc/2.4.4/ works? --amk

2 1