Mailman 3 December 2008 - Python-Dev

Python 3 - Mac Installer?
by Mark Summerfield 02 Jan '09

02 Jan '09

Hi, Just wondered if/when there'd be a Mac installer for Python 3? Thanks! -- Mark Summerfield, Qtrac Ltd, www.qtrac.eu C++, Python, Qt, PyQt - training and consultancy "Programming in Python 3" - ISBN 0137129297

7 11

A wart which should have been repaired in 3.0?
by skip＠pobox.com 02 Jan '09

02 Jan '09

The doc for os.path.commonprefix states: Return the longest path prefix (taken character-by-character) that is a prefix of all paths in list. If list is empty, return the empty string (''). Note that this may return invalid paths because it works a character at a time. I remember encountering this in an earlier version of Python 2.x (maybe 2.2 or 2.3?) and "fixed" it to work by pathname components instead of by characters. That had to be reverted because it was a behavior change and broke code which used it for strings which didn't represent paths. After the reversion I then forgot about it. I just stumbled upon it again. It seems to me this would have been a good thing to fix in 3.0. Is this something which could change in 3.1 (or be deprecated in 3.1 with deletion in 3.2)? Skip

12 39

opcode dispatch optimization
by Antoine Pitrou 02 Jan '09

02 Jan '09

Hello, I would like to mention that I've written a patch which enables "threaded interpretation" on the ceval loop with gcc (*). On my computer (an Athlon X2 3600+), it is good for a 15-20% speedup of the interpreter on pystone and pybench. I also had the opportunity to test it on a Core2-derived CPU, where it doesn't make a difference (I conjecture it's because Core2 CPUs have hardware-based indirect branch optimizations). It will make no difference if the interpreter is compiled with something else than gcc (I tested on Windows). The additional complexity is very small. There's a separate script which is run to build the dispatch table (only if needed, that is if dis.py has been modified). In ceval.c, there are a couple of macros and some #ifdef's. That's all. It breaks no test in the regression suite. Could other people test and report their results here? (the patch is for py3k, btw). Also, what are you thoughts for/against integrating this patch in the standard interpreter? Regards Antoine. (*) please note: it has nothing to see with multithreading.

4 4

test_subprocess and sparc buildbots
by Nick Coghlan 01 Jan '09

01 Jan '09

Does anyone have local access to a sparc machine to try to track down the ongoing buildbot failures in test_subprocess? (I think the problem is specific to 3.x builds on sparc machines, but I haven't checked the buildbots all that closely - that assessment is just based on what I recall of the buildbot failure emails). Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia ---------------------------------------------------------------

4 6

Missing FAQ about Python3 and unicode
by Victor Stinner 30 Dec '08

30 Dec '08

Hi, Slowly, we get recurrent questions about Python3 and unicode. It's maybe time to start a FAQ? Here is an ugly draft to start it ;-) (1) Exit on undecodable command line arguments $ LANG=en_GB.UTF-8 python3.0 test.py $'\xff' Could not convert argument 2 to string$ Is it an expected behaviour? Yes! Example of the question: http://bugs.python.org/issue3023 (2) Undecodable filenames os.listdir(str)->str raises an exception on undecodable filenames. Solution: use os.listdir(bytes)->bytes. To display the filename to the user, use a function like: import sys def humanFilename(filename): encoding = sys.getfilesystemencoding() return filename.encode(encoding, "replace") See also http://bugs.python.org/issue3187 (3) Bytes environment variables Python 3.0 only supports decodable variables for os.environ. Undecodable variables are skipped for the creation of os.environ but original variables still exist at the C level. $ A=$(echo -e "\xff") B=c ./python Python 3.1a0 (py3k:67973M, Dec 31 2008, 00:51:49) >>> import os >>> os.environ.get('A'), os.environ.get('B') (None, 'c') >>> retcode=os.system('echo -n $A|hexdump -C') 00000000 ff |.| 00000001 >>> retcode=os.system('echo -n $B|hexdump -C') 00000000 63 |c| 00000001 Discussion to support bytes environment variables: http://mail.python.org/pipermail/python-dev/2008-December/083856.html -- Victor Stinner aka haypo http://www.haypocalc.com/blog/

2 1

extremely slow exit for program having huge (45G) dict (python 2.5.2)
by Mike Coleman 30 Dec '08

30 Dec '08

I have a program that creates a huge (45GB) defaultdict. (The keys are short strings, the values are short lists of pairs (string, int).) Nothing but possibly the strings and ints is shared. The program takes around 10 minutes to run, but longer than 20 minutes to exit (I gave up at that point). That is, after executing the final statement (a print), it is apparently spending a huge amount of time cleaning up before exiting. I haven't installed any exit handlers or anything like that, all files are already closed and stdout/stderr flushed, and there's nothing special going on. I have done 'gc.disable()' for performance (which is hideous without it)--I have no reason to think there are any loops. Currently I am working around this by doing an os._exit(), which is immediate, but this seems like a bit of hack. Is this something that needs fixing, or that has already been fixed? Mike

19 68

Commands for correctly merging to the python 3.0 maintenance branch
by Nick Coghlan 30 Dec '08

30 Dec '08

Getting the svnmerge-intergrated property right when merging trunk->py3k->release30 is a little tricky. The most concise set of instructions I have found which gets it right is to do the following in the 3.0 maintenance branch after committing to the py3k branch: svn update svnmerge merge -r <py3k-rev> svn revert . svnmerge -M -F <py3k-rev> <test change still works, etc> svn commit -F svnmerge-commit-message.txt Revert and property changes on "." and running that second svnmerge line is also useful if you do a "svn update" after the first svnmerge and get a conflict on the svnmerge-intregrated property. The -M option tells the utility to only make the property changes, while the -F tells it to go ahead despite the existence of local modification. Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia ---------------------------------------------------------------

2 2

WinCE port (issues #4075 #4051)
by Ulrich Eckhardt 30 Dec '08

30 Dec '08

Hi! I'm currently working again on the CE port, and since 2.6 and 3.0 are now out of the door, could you apply the patches in #4075 & #4051? Both patches are fairly isolated and easy to review and I'm pretty sure they won't cause any inconveniences. Note: this is far from everything that is necessary for Python to rock on CE, but these are prerequisites, as explained in both bugs' histories. thanks Uli

1 0

Call PyType_Ready on builtin types during interpreter startup?
by Nick Coghlan 28 Dec '08

28 Dec '08

Some strangeness was recently reported for the range() type in Py3k where instances are unhashable until an attribute is retrieved from the range type itself, and then they become hashable. [1] While there is definitely an associated bug in the range implementation (it doesn't block inheritance of the default object.__hash__ implementation), there's also the fact that when the interpreter *starts* the hash implementation hasn't been inherited yet, but it does get inherited later. It turns out that _PyBuiltin_Init doesn't call PyType_Ready on any of the builtin types - they're left to have it called implicitly when an operation using them needs tp_dict filled in. Such operations (which includes retrieving an attribute from the type object) will implicitly call PyType_Ready to populate tp_dict, which also has the side effect of inheriting slot implementations from base classes. Is there a specific reason for not fully initialising the builtin types? Or should we be calling PyType_Ready on each of them from _PyBuiltin_Init? Cheers, Nick. [1] http://bugs.python.org/issue4701 -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia ---------------------------------------------------------------

4 7

Use -M option on buildbots?
by Antoine Pitrou 28 Dec '08

28 Dec '08

Hi all, Could we use the -M option (with a suitable value depending on the amount of physical RAM) for regression tests on the buildbots? It would help avoid the kind of situation described in http://bugs.python.org/issue3700 cheers Antoine.

1 0