Mailman 3 November 2007 - Python-Dev

Statsvn output for /python/branches/py3k
by Joseph Armbruster 29 Nov '07

29 Nov '07

All, I was looking at statsvn today at work and gave it a test-run on a repo there. I wondered what it would look like for python3k. And... here are the results: http://www.joevial.com/statsvn/ Enjoy, Joseph Armbruster

4 3

Memory benchmarking?
by Titus Brown 29 Nov '07

29 Nov '07

Hi all, is there a good, or standard memory benchmarking system for Python? pybench doesn't return significantly different results when Python 2.6 is compiled with pymalloc and without pymalloc. Thinking on it, I'm not too surprised -- pybench probably benchmarks a lot of stuff -- but some guidance on how/whether to benchmark different memory allocation schemes would be welcome. thanks, --titus refs: http://code.google.com/p/google-highly-open-participation-psf/issues/detail… http://evanjones.ca/memoryallocator/ http://www.advogato.org/person/wingo/diary/225.html

2 1

Fwd: Google Highly Open Participation Contest
by Titus Brown 28 Nov '07

28 Nov '07

Dear Python-Dev-ers, about half of the tasks for the GHOP contest (described below) are currently on the stdlib, Python core, or Py3K: http://code.google.com/p/google-highly-open-participation-psf/issues/list?q… http://code.google.com/p/google-highly-open-participation-psf/issues/list?q… http://code.google.com/p/google-highly-open-participation-psf/issues/list?q… We have room for another twenty or so; if you can come up with testing tasks, documentation updates/review needs, or anything else that you can write down in a fairly detailed and specific way, we'd love to have them. Here are the new task guidelines: http://code.google.com/p/google-highly-open-participation-psf/wiki/NewTaskG… and I'm happy to write up the tasks if people send me good ideas. cheers, --titus ----- Forwarded message from Titus Brown <titus(a)caltech.edu> ----- From: Titus Brown <titus(a)caltech.edu> Subject: Google Highly Open Participation Contest Dear Python community, I'm happy to announce that the Python Software Foundation is part of a new Google Open Source program, the Highly Open Participation Contest. This contest is an effort by Google to engage pre-college students in open source programming: Google is offering prizes and awards for completing a variety of tasks for several organizations. You can read the official announcement of the PSF's involvement over on the Python Software Foundation blog, at http://pyfound.blogspot.com/ or read about the overall Google Highly Open Participation Contest on the GHOP page itself, http://code.google.com/opensource/ghop/2007-8 Python's project page is: http://code.google.com/p/google-highly-open-participation-psf/ There are several ways that the community can help; please check out the MentorPage, http://code.google.com/p/google-highly-open-participation-psf/wiki/MentorPa… for more information. We could use more mentors, and there's room for ~40 more tasks -- why not write one up on your own project? Also be sure to thank the people who made this possible, http://code.google.com/p/google-highly-open-participation-psf/wiki/Contribu… and, of course, Google! best, --titus ----- End forwarded message -----

2 1

Re: [Python-Dev] removing the new and types modules
by Nick Coghlan 28 Nov '07

28 Nov '07

Redirecting discussion from python-checkins to python-dev. 'new' has now been deprecated for 3.0, GvR suggested it would be nice to get rid of 'types' as well. Christian Heimes wrote: > Georg Brandl wrote: >> I've just looked, and the types you can't get trivially via builtin or >> type(singleton) are >> >> * module >> * function >> * generator >> * code >> * method >> * builtin-function-or-method >> * frame >> * traceback >> * dictproxy >> * getset- and member-descriptor >> >> Where would one put them? > > Python 3.0 has several more types that aren't exposed through types. For > example the views like dict_keys, dict_values and dict_items are not in > types. Most of those 'hard-to-get-at' types are tied pretty tightly to the interpreter internals, and would fit in with Guido's suggestion of a 'pyvm' module. Such a module would let us clean up the 'sys' namespace a little bit by moving some of the more arcane hackery to another module (like the recursion limit and the threading check interval). The only ones I would suggest putting elsewhere are the module type (exposing it through the imp module instead), and the new dict_keys/_values/_items types (exposing them through the collections module, as others have suggested). Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org

2 1

Re: [Python-Dev] SSL tests are failing...
by Facundo Batista 27 Nov '07

27 Nov '07

2007/11/27, Bill Janssen <janssen(a)parc.com>: > Which branch is this? The trunk, sorry. -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/

1 0

.pyc location?
by Andreas Raab 27 Nov '07

27 Nov '07

Hi - I'm having a few problems with our (embedded) Python interpreter trying to write .pyc files in the installation of our product (which actually sometimes works and sometimes doesn't depending on OS and paranoia level involved). This can give raise to a few very confusing situations and I'd like to be able to specify an explicit location for the .pyc files. Ideally, this would be a cache location where all .pyc files are created under an hashed (MD5 or similar) name of the original source file so that the cache location can be used for all of the compiled Python files. While googling this issue, I noticed that the status of PEP 304 says "withdrawn" but since most of the discussion around it seems to be a couple of years old, I thought I'd ask if anyone has done something in this area recently. If not, a pointer to the place which manages these mappings (or a starting point for further digging) would be greatly welcome. Cheers, - Andreas

6 6

SSL tests are failing...
by Facundo Batista 27 Nov '07

27 Nov '07

...in my machine with no Internet connection. I saw what's happening: there's a test in BasicTests that tries to access "svn.python.org". It's strange, because this test is an exact copy of the one in NetworkTest (but the latter is included only if the "network" resource is enabled). Seeing more in deep, I saw that Bill replaced in r58164, in this BasicTest, a lot of previous code (I think that from r57464) that actually tested it *locally*. Do you know if there's a good reason to this? I'm sure that making a network test if the resource is not available is not a good thing, and as the test is just repeated, it can be removed, but maybe this is part of a bigger mistake... that's why I'm asking. Thanks!! -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/

2 1

Build Notes for building trunk with Visual Studio 2008 Express Edition
by Paul Moore 26 Nov '07

26 Nov '07

I have just built the current trunk version of Python on Windows, using the new PCBuild9 directory, and Visual Studio 2008 Express Edition. Everything went extremely well. I include below my notes on what I did, for reference. To be honest, there's nothing in here that really warrants a change to the readme.txt - to all intents and purposes, the process "just works". OK, here are my notes: Install Visual C++ 2008 Express Edition. Only select Silverlight from the options (no documentation or SQL Server - Silverlight probably isn't actually needed either). I already had the Platform SDK installed, but did nothing to tell VS about it, or to integrate it. I doubt this is relevant. I am using Python trunk at revision 59132. (But I hit an issue fixed in 59136, so better to use that). Start VC. Open project PCBuild9\pcbuild.sln Message "Solution Folders are not supported in this version of Visual Studio. Solution folder 'Solution Items' will be displayed as unavailable." Select OK. Select the release build (Build > Configuration Manager) Right click pythoncore > Build make_buildinfo - succeeded 1 warning (unlink vs _unlink) make_versioninfo - succeeded pythoncore - I hit an error in ast.c so I needed to svn up (to 59136). Succeeded. Right click python > Build. Succeeded. Right click pythonw > Build. Succeeded. Right click _socket > Build. Succeeded. Right click _testcapi > Build. Succeeded. Right click pyexpat > Build. Succeeded. Right click select > Build. Succeeded. Right click unicodedata > Build. Succeeded. Right click winsound > Build. Succeeded. At this point, we have finished the modules documented as "build out of the box" in PCBuild9\readme.txt. The modules _tkinter, bz2, _bsddb, _sqlite3, _ssl are documented as having dependencies. See below. Modules _ctypes, _ctypes_test, _elementtree, _msi, w9xpopen are not mentioned in readme.txt. They all build without error. bz2 --- The include dir is a macro, and I can't work out how to override the default (which is bzip2-1.0.3). So I stuck to 1.0.3 and exported it from Python svn, as per the instructions. Built OK. _sqlite3 -------- Again, couldn't work out how to change the macro, so I stuck with the external from svn (sqlite-source-3.3.4). The pre-link step failed with an error about not finding TCL. I edited the prelink step to include a /DNO_TCL flag on the cl command. There may be a better approach - I don't know if not having TCL is an issue for Python's use of sqlite. _tkinter and _bsddb ------------------- The instructions suggest using VS 2003 to build the dependencies. I don't have VS 2003 and don't have the time at the moment to investigate further. _ssl ---- Christian has been making changes to allow this to build without Perl, so I gave it a try. I used openssl 0.9.8g, which I extracted to the build directory (I noticed afterwards that this is the same version as in Python svn, so I could have used the svn external!) I needed to download nasm (nasm.sf.net) version 2.00rc1, and rename nasm.exe to nasmw.exe and put it on my PATH. Build succeeded, no issues. Tests ----- Running the tests, all succeed except test_tcl and test_bsddb, which are because I didn't build those two extensions, and test_os. The test_os failure is just because it looks for C:\pagefile.sys and my pagefile is on D:\. (There's also a problem with test_socket_ssl hanging, but this is present in the standard 2.6 snapshot build. I've raised a separate bug report for this). Paul.

7 18

Possible error in Python 2.4 garbage collector
by Moritz Mühlenhoff 26 Nov '07

26 Nov '07

Hi, we're running into a problem with the Python interpreter, which we suspect triggers a bug in the garbage collector or a related part of the memory management. We are using Python 2.4.4 in the version from Debian 4.0. Our C application imports various Python plugins using PyNode_Compile(). This used to work very well (it's a feature we've been using for years now), but if we import more than 23 modules we're running a segmentation fault with the backtrace below: (gdb) bt #0 0xb7ee9cad in visit_decref (op=0xb7287ec0, data=0x0) at ../Modules/gcmodule.c:269 #1 0xb7e716d1 in list_traverse (o=0xb724e62c, visit=0xb7ee9ca0 <visit_decref>, arg=0x0) at ../Objects/listobject.c:2280 #2 0xb7eea729 in collect (generation=0) at ../Modules/gcmodule.c:294 #3 0xb7eeb0a3 in _PyObject_GC_Malloc (basicsize=20) at ../Modules/gcmodule.c:872 #4 0xb7eeb0eb in _PyObject_GC_NewVar (tp=0xb7f25720, nitems=2) at ../Modules/gcmodule.c:1283 #5 0xb7e913e4 in PyTuple_New (size=2) at ../Objects/tupleobject.c:68 #6 0xb7e91bb5 in PyTuple_Pack (n=2) at ../Objects/tupleobject.c:143 #7 0xb7ebfded in com_add (c=0xbfc5d3ac, list=0x0, dict=0xb7ee9ca0, v=0xb770fc20) at ../Python/compile.c:1375 #8 0xb7ec4b1e in com_addop_varname (c=0xbfc5d3ac, kind=0, name=0xb71f9218 "new") at ../Python/compile.c:1410 #9 0xb7ecea53 in com_atom (c=0xbfc5d3ac, n=0xb71f8ca0) at ../Python/compile.c:2213 #10 0xb7ecefe0 in com_power (c=0x70732f72, n=0xb71fb5c0) at ../Python/compile.c:2587 #11 0xb7ecf4c3 in com_term (c=0x70732f72, n=0xb71fb590) at ../Python/compile.c:2727 #12 0xb7ecf639 in com_shift_expr (c=0xbfc5d3ac, n=0xb71fb560) at ../Python/compile.c:2762 #13 0xb7ecf924 in com_xor_expr (c=0x70732f72, n=0xb71fb530) at ../Python/compile.c:2814 #14 0xb7ec5064 in com_comparison (c=0x70732f72, n=0xb71fb2a8) at ../Python/compile.c:2858 #15 0xb7ec5e5a in com_test (c=0xbfc5d3ac, n=0xb71f8714) at ../Python/compile.c:2986 #16 0xb7eca88a in com_if_stmt (c=0xbfc5d3ac, n=0xb71fb278) at ../Python/compile.c:3844 #17 0xb7ec9101 in com_node (c=0xbfc5d3ac, n=0xb71f4d2c) at ../Python/compile.c:4181 #18 0xb7eca926 in com_if_stmt (c=0xbfc5d3ac, n=0xb71f7398) at ../Python/compile.c:3848 #19 0xb7ec9101 in com_node (c=0xbfc5d3ac, n=0xb720a688) at ../Python/compile.c:4181 #20 0xb7ecd1af in com_try_except (c=0xbfc5d3ac, n=0xb71f5728) at ../Python/compile.c:3998 #21 0xb7ec9101 in com_node (c=0xbfc5d3ac, n=0xb722ecdc) at ../Python/compile.c:4181 #22 0xb7eca926 in com_if_stmt (c=0xbfc5d3ac, n=0xb71dcb60) at ../Python/compile.c:3848 #23 0xb7ec9101 in com_node (c=0xbfc5d3ac, n=0xb71d8318) at ../Python/compile.c:4181 #24 0xb7ecac18 in com_if_stmt (c=0xbfc5d3ac, n=0xb71d4110) at ../Python/compile.c:3855 #25 0xb7ec9101 in com_node (c=0xbfc5d3ac, n=0xb737c610) at ../Python/compile.c:4181 #26 0xb7ec6d4c in compile_node (c=0xbfc5d3ac, n=<value optimized out>) at ../Python/compile.c:4755 #27 0xb7ec7e80 in jcompile (n=0xb71d4098, filename=<value optimized out>, base=0xbfc5d72c, flags=0x0) at ../Python/compile.c:5006 #28 0xb7ec8776 in com_funcdef (c=0xbfc5d72c, n=0xb71d4098) at ../Python/compile.c:4942 #29 0xb7ec6c8e in compile_node (c=0xbfc5d72c, n=<value optimized out>) at ../Python/compile.c:4728 #30 0xb7ec7e80 in jcompile (n=0xb7341668, filename=<value optimized out>, base=0x0, flags=0x0) at ../Python/compile.c:5006 #31 0xb7ec86ca in PyNode_Compile (n=0xb71a58f0, filename=0x82d5aa0 "/usr/lib/univention-directory-listener/system/cyrus-sieve.py") at ../Python/compile.c:4913 #32 0x0804e5de in module_import (filename=0x82d5aa0 "/usr/lib/univention-directory-listener/system/cyrus-sieve.py") at handlers.c:98 #33 0x0804ec6e in handler_import (filename=0x82d5aa0 "/usr/lib/univention-directory-listener/system/cyrus-sieve.py") at handlers.c:184 #34 0x0804fb37 in handlers_load_path (path=0x8062130 "/usr/lib/univention-directory-listener/system") at handlers.c:472 #35 0x0804fbc0 in handlers_load_all_paths () at handlers.c:491 #36 0x0804fff0 in handlers_init () at handlers.c:568 #37 0x0804bfb9 in main (argc=16, argv=0xbfc5dce4) at main.c:489 (gdb) Given the function names I would suspect a bug in the garbage collector? I'm not familiar with the internals of Python's memory managent, so I can't really tell for sure. It could also be that some default internal limits of the memory manager are triggered. Shall I open a bug in the tracker for this does anyone suspect a different cause or even a workaround? Cheers, Moritz -- Moritz Muehlenhoff muehlenhoff(a)univention.de fon: +49 421 22 232- 0 Development Linux for Your Business fax: +49 421 22 232-99 Univention GmbH http://www.univention.de/ mobil: +49 175 22 999 23

2 2

Should we do away with unbound methods in Py3k?
by Guido van Rossum 25 Nov '07

25 Nov '07

I'm asking a Py3k question on python-dev because I'd like to have opinions from people who haven't thought about Py3k much yet. Consider the following example: class C: def foo(self): pass C.foo(42) This currently fails with this error message: TypeError: unbound method foo() must be called with C instance as first argument (got int instance instead) This message is called when isinstance(self, C) returns False, where self is the first argument passed to the unbound method. That's nice, but there is a cost associated with this: the expression "C.foo" can't just return the function object "foo", it has to wrap it in an unbound method object. In Py3k the cost of calling an unbound method object goes up, because the isinstance() check may be overloaded. This typically happens when the class C uses the special metaclass (abc.ABCMeta) used for virtual inheritance (see PEP 3119). in Py3k the I/O stream classes are perhaps the most common use case. Given that the error is of limited value and that otherwise the unbound method behaves exactly the same as the original function object, I'd like to see if there are strenuous objections against dropping unbound method objects altogether (or at least not using them in this case), so that explicit super calls (via the unbound method) may go a little faster. Also, it would make it easier to fix this issue: http://bugs.python.org/issue1109 To illustrate the cost of the isinstance() overloading, step through this simple program with Py3k: import abc, pdb class Base(metaclass=abc.ABCMeta): @abc.abstractmethod def foo(self): pass class C(Base): def foo(self): Base.foo(self) c = C() pdb.run("c.foo()") -- --Guido van Rossum (home page: http://www.python.org/~guido/)

12 27