Mailman 3 August 2004 - Python-Dev

refleak hunting season
by Michael Hudson 03 Aug '04

03 Aug '04

Last time round I found a bunch of reference counting bugs in Python just *after* a major release; I'm trying to do it all before 2.4.0 this time. I've hacked regrtest again (patch attached). Here's my notes file, though as is the case with such things writing stuff down tends to cause thoughts which cause experiments which cause the notes to be out of date more or less immediately... test_codeccallbacks leaked [2, 2, 2, 2] references registries test_copy leaked [82, 82, 82, 82] references copy_reg.dispatch_table -- now taken care of in regrtest.py test_descr leaked [2, 2, 2, 2] references resurrection, still test_descrtut leaked [1, 1, 1, 1] references exec 'print x' in defaultdict() fixed, need to check in (bit subtle though) test_distutils leaked [2, 2, 2, 2] references no idea test_gc leaked [18, 18, 18, 18] references trashcan test_hotshot leaked [111, 111, 111, 111] references there's been a patch on SF for this for about a year! test_minidom leaked [1, 1, 1, 1] references testPickledDocument, don't know more test_mutants leaked [74, -1123, 1092, -521] references randomized test test_pkg leaked [3, 3, 3, 3] references something to do with execfile? or tempfile.mkstemp? the leak is on the dummy key from dictobject.c!! very odd. see bug #808596 for the explanation. test_pkgimport leaked [6, 6, 6, 6] references no idea similar to the above? (but certainly not the same) I think this might be caused by interning like the above. test_sax leaked [1899, 1899, 1899, 1899] references no idea -- scary numbers, though! test_threadedtempfile leaked [0, 1, -21, 9] references randomized test? test_threading_local leaked [9, 0, 7, 0] references not sure what's going on here; not sure it's a leak, but it's definitely odd test_unicode leaked [7, 7, 7, 7] references the codec registry test_urllib2 leaked [120, -80, -70, 120] references CacheFTPHandler test_xrange leaked [1, 1, 1, 1] references fixed in CVS I'd really appreciate someone taking a look at test_sax. The numbers for test_gc and test_descr are artifacts and if someone sees how to fix them, I'd be grateful. test_threading_local behaves very strangely. test_distutils I haven't even begun to look at. Generally, we're in pretty good shape! (Or our tests really suck, your call ;-). Cheers, mwh -- I never disputed the Perl hacking skill of the Slashdot creators. My objections are to the editors' taste, the site's ugly visual design, and the Slashdot community's raging stupidity. -- http://www.cs.washington.edu/homes/klee/misc/slashdot.html#faq

5 8

Bug day coming up this Saturday
by A.M. Kuchling 03 Aug '04

03 Aug '04

The third Python Bug Day is coming up this Saturday, August 7 2004. Having more participation from people with CVS access would be very helpful. On the two previous bug days, there have been several non-developers contributing but only one or two people who can actually check in code or close bugs. You don't need to hang around all day; an hour or two is enough. This will be the last bug day before October. I won't be running a bug day in September because I'm getting married on September 19th and will need every spare weekend possible. If someone else wants to organize a September bug day, please do. (The tasks are pretty simple: updating the Wiki pages before the chosen date, and then staying on IRC for the entire day, to ensure that there's *someone* around to answer questions and commit things.) When? ===== This Saturday, August 7, 2004, from 9AM to 5PM EDT (1PM to 9PM GMT). You don't need to be around all day; feel free to stop by for a few hours and contribute. Where? ====== The #python-dev channel on irc.freenode.net. More information ================ For instructions and more information, see http://www.python.org/cgi-bin/moinmoin/PythonBugDay --amk

2 1

ConfigParser behavior change
by David Goodger 02 Aug '04

02 Aug '04

Could I get some feedback re <http://www.python.org/sf/997050>? I'd like to resolve this before 2.4a2 if possible. In summary, RawConfigParser.set() doesn't allow non-string arguments for 'value' any more. This breaks Docutils code. I specifically chose RawConfigParser for Docutils internal settings support because it *did* allow non-string values. An earlier bug report (<http://www.python.org/sf/810843>, "Support for non-string data in ConfigParser unclear/buggy") offered 4 options: (1) Cast the option to a string explicitly instead of assuming it is a string (2) Check that the option is a string (3) State in the documentation that the set() method only takes string values (4) State in the documentation that the raw parameter should be used when an option's value might not be of type string Options 2 & 3 were implemented in the "bug fix" (just a type check). I think that the behavior should not have been changed, rather that the documentation needed updating (i.e., choose option 4 instead). I volunteer to undo the change and update the documentation ASAP. The comment for bug 810843 says "This is documented behavior", but I couldn't find any such documentation pre-dating this change. Thanks! -- David Goodger <http://python.net/~goodger>

2 2

Proposed Python api for testing whether a module exists
by Jim Fulton 02 Aug '04

02 Aug '04

I propose to add the following function to the imp module: exists(module_name) Test whether the named module can be found. This method should be used when you want to perform alternate actions depending on whether a module is available. Returns True of the module can be found and False otherwise. A True return value does not guarentee that the module will be importable. The module will not be imported as a result of calling exists, but parent modules may be. If a dotted module name is given, then parent modules will be imported if they haven't already been imported. If there are no objections, I'll try to get this in before Wednesday night. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

5 6

Re: [Python-Dev] Proposed Python api for testing whether a module exists
by Edward Loper 02 Aug '04

02 Aug '04

> I propose to add the following function to the imp module: > > exists(module_name) > > Test whether the named module can be found. [...] > If a dotted module name is given, then parent modules will be > imported if they haven't already been imported. If a parent module doesn't exist, will it return False or raise an ImportError? -Edward

2 1

Re: [Python-Dev] Proposed Python api for testing whether a module exists
by Jim Fulton 02 Aug '04

02 Aug '04

Anthony Baxter wrote: > Jim Fulton wrote: > >> If a dotted module name is given, then parent modules will be >> imported if they haven't already been imported. > > > So you might end up spelling something like: > > if imp.exists('pkg') and imp.exists('pkg.subpkg') \ > and imp.exists('pkg.subpkg.module'): > > to avoid an importerror on the parent module? No. If pkg doesn't exist then exists will report False. Here's new wording: exists(module_name) Test whether the named module can be found. This method should be used when you want to perform alternate actions depending on whether a module is available. Returns True of the module can be found and False otherwise. A True return value does not guarentee that the module will be importable. The module will not be imported as a result of calling exists, but parent modules may be. If a dotted module name is given and parent modules exist, then the parent modules will be imported if they haven't already been imported. Thanks, Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

1 0

Safe to change a thread's interpreter?
by Phillip J. Eby 02 Aug '04

02 Aug '04

Recently I've been researching implementation strategies for adding Java classloader-like capabilities to Python. I was pleasantly surprised to find out that CPython already supports multiple interpreters via the C API, where each "interpreter" includes fresh versions of 'sys', '__builtin__', etc. The C API doc for PyInterpreter_New(), however, says: """It is possible to insert objects created in one sub-interpreter into a namespace of another sub-interpreter; this should be done with great care to avoid sharing user-defined functions, methods, instances or classes between sub-interpreters, since import operations executed by such objects may affect the wrong (sub-)interpreter's dictionary of loaded modules. (XXX This is a hard-to-fix bug that will be addressed in a future release.)""" It seems to me that the bug described could be fixed (or at least worked around) by having __import__ temporarily change the 'interp' field of the current thread state to point to the interpreter that the __import__ function lives in. Then, at the end of the __import__, reset the 'interp' field back to its original value. (Of course, it would also have to fix up the linked lists of the interpreters' thread states during each swap, but that shouldn't be too difficult.) My question is: does this make sense, or am I completely out in left field here? The only thing I can think of that this would affect is the 'threading' module, in that trying to get the current thread from there (during such an import) might see a foreign interpreter's thread as its own. But, I'm hard-pressed to think of any damage that could possibly cause. Indeed, it seems to me that Python itself doesn't really care how many interpreters or thread states there are running around, and that it only has the linked lists to support "advanced debuggers". Even if it's undesirable to fix the problem this way in the Python core, would it be acceptable to do so in an extension module? What I have in mind is to create an extension module that wraps Py_InterpreterState/Py_ThreadState objects up in a subclassable extension type, designed to ensure the integrity of Python as a whole, while still allowing various import-related methods to be overridden in order to implement Java-style classloader hierarchies. So, you might do something like: from interpreter import Interpreter # Run 'somescript in its own interpreter. it = Interpreter() exit_code = it.run_main("somescript.py") # Release resources without waiting for GC it.close() My thought here also is that performing operations such as running code in a given Interpreter would also operate by swapping the thread state's 'interp' field. Thus, exceptions in the child interpreter would be seamlessly carried through to the parent interpreter. In order to implement the full Java classloader model, it would also be necessary to be able to force imports *not* to use the Interpreter that the code doing the import came from. (i.e. the equivalent of using 'java.lang.Thread.setContextClassLoader()'). This can also probably be implemented via a thread-local variable in the 'interpreter' module. So... must a thread state always reference the same interpreter object? If not, then I think I see a way to safely implement access to multiple interpreters from within Python itself.

2 4

Re: [Python-Dev] 2.4a2, and @decorators
by Anthony Baxter 02 Aug '04

02 Aug '04

Mark Russell wrote: > Sorry - my mistake for not running the test suite under the debug > build. I've attached a fix. Anthony, can you check this in as well > please? I've added "run the test suite under a debug build" to my list of build tasks for the release process thingy. Patch applied. Anthony

1 0

Optik 1.5 in Python 2.4
by Greg Ward 01 Aug '04

01 Aug '04

Hope I'm not too late to get Optik 1.5 into Python 2.4. I release Optik 1.5a1 last weekend and so far haven't heard a peep of complaint. Sneaking it into Python 2.4a2 would help ferret out any compatibility bugs. (The changes are pretty small; the only reason I called it Optik 1.5 instead of 1.4.2 is because there are a bunch of small changes, and it's over a year since 1.4.1. ;-) The only catch is that I've updated the Optik reStructuredText docs, and haven't yet devised a clever way to convert them to LaTeX-for-the- Python-manual format (talking about optparse rather than Optik). (Code donations welcome. ;-) So: I'd like to checkin Optik 1.5a1 as optparse.py right now, and worry about the docs later. Hopefully before Python 2.4a2 goes out the door, but no promises. Objections? If I don't hear any soon, I'll do the deed tomorrow morning. Greg -- Greg Ward <gward(a)python.net> http://www.gerg.ca/ Everybody is going somewhere!! It's probably a garage sale or a disaster Movie!!

2 2

New developer
by "Martin v. Löwis" 01 Aug '04

01 Aug '04

I just granted Sean Reifschneider CVS write access, so he can directly edit the RPM spec files. Sean has been virtually the only author of that file for a long time. Regards, Martin

1 0