Mailman 3 June 2008 - Python-Dev

Another Proposal: Run GC less often
by none June 21, 2008

June 21, 2008

Instead of collecting objects after a fixed number of allocations (700) You could make it dynamic like this: # initial values min_allocated_memory = 0 max_allocated_memory = 0 next_gc_run = 1024 * 1024 def manage_memory(): allocated_memory = get_allocated_memory() min_allocated_memory = min(min_allocated_memory, allocated_memory) max_allocated_memory = max(max_allocated_memory, allocated_memory) if max_allocated_memory - min_allocated_memory > next_gc_run: # run the gc memory_freed, allocated_memory = run_gc() next_gc_run = max( allocated_memory * 1.5 - memory_freed, 1024 * 1024 ) min_allocated_memory = allocated_memory max_allocated_memory = allocated_memory manage_memory() should be called after every allocation and after a ref count of an object reaches 0 (memory is freed) Expected behaviours: => As less objects contain cyclic references as less often the GC will run (memory_freed is small) => As more objects contain cyclic references as more often the GC will run (memory_freed is large) => If memory utiliaziation grows fast (burst allocations) GC will run less often: next_gc_run = allocated_memory * 1.5 - memory_freed ... Of course the constants: 1.5 and 1024 * 1024 are only suggestions... - Ralf

2 1

RELEASED Python 2.6b1 and 3.0b1
by Barry Warsaw June 21, 2008

June 21, 2008

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On behalf of the Python development team and the Python community, I am happy to announce the first beta releases of Python 2.6 and Python 3.0. Please note that these are beta releases, and as such are not suitable for production environments. We continue to strive for a high degree of quality, and these releases are intended to freeze the feature set for Python 2.6 and 3.0. From now until the planned final releases in September 2008, we will be fixing known problems and stabilizing these new Python versions. You can help by downloading and testing them, providing feedback and hopefully helping to fix bugs. You can also use these releases to determine how changes in 2.6 and 3.0 might impact you. If you find things broken or incorrect, please submit bug reports at http://bugs.python.org For more information and downloadable distributions, see the Python 2.6 website: http://www.python.org/download/releases/2.6/ and the Python 3.0 web site: http://www.python.org/download/releases/3.0/ See PEP 361 for release schedule details: http://www.python.org/dev/peps/pep-0361/ Enjoy, - -Barry Barry Warsaw barry(a)python.org Python 2.6/3.0 Release Manager (on behalf of the entire python-dev team) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSFnIbnEjvBPtnXfVAQKCMgP+L9Vpk9vQgYw01KShtF9HyvCKE0guSy86 0Q/IRbkXGahXg910D4nVLY9NORaOSq/VP8Kx+fOU8egUDHvVF9MuKkY75ZHYoixR mieeV7BIpYHYKYhva48FVoF4p0+sUeNvfeOAP0nnV5Pi2icGqCpzizfSHZloXJjc Y8M4F5vVrOM= =Oo15 -----END PGP SIGNATURE-----

4 5

csv module and Unicode in 2.6 and 3.0
by skip＠pobox.com June 21, 2008

June 21, 2008

If I understand things correctly, the csv module in 3.0 supports Unicode, kind of for free because all strings are Unicode. I suspect the Unicode examples at the bottom of the 3.0 version of the module doc need to be removed. Does the 2.6 version support Unicode out of the box as well or are the Unicode examples still required there? Skip

1 0

Status of Issue 2331 - Backport parameter annotations
by David Pokorny June 21, 2008

June 21, 2008

Hi, I am curious if there is any news on this issue. My understanding is that since this is a new feature, nothing will be committed until after 2.6 comes out, but it would be *really nice* if I could use annotations in 2.x. [warning - mini rant] Reason is, I am using 3.0a for now for this project, but I am sorry to say that using parentheses for raising exceptions drives me up the wall, and I don't like using them for the former print statement either. I suspect this is partly due to the fact that I'm using 2.5 for work, so I don't have the option to mentally "switch over" to the Python 3 way. /rant Does anyone have an annotations patch floating around against the trunk / 2.6b, or failing that, does anyone have opinions on what my chances are / things to look out for if I fiddle around with the annotations patch from the py3k branch? (My understanding is that this is revision 53170). Cheers, David

5 6

Re: [Python-Dev] [Python-checkins] r64407 - python/trunk/Doc/library/multiprocessing.rst
by A.M. Kuchling June 20, 2008

June 20, 2008

On Thu, Jun 19, 2008 at 03:04:11PM -0500, Benjamin Peterson wrote: > I don't think the whole introduction had to go. I think it helped give > some idea of how multiprocessing works before jumping straight to the > API reference. I don't think overview material like that should be buried inside the documentation for one particular module; it should go in a chapter introduction. My plan is to move multiprocessing from the 'Optional OS Services' chapter to the 'Interprocess Communication' chapter, and then expand the introduction of the chapter to discuss the GIL and related issues. Presumably it was put in the 'Optional OS Services' chapter because that's where the threading and thread modules are. --amk

6 8

Re: [Python-Dev] [Python-checkins] r64424 - in python/trunk: Include/object.h Lib/test/test_sys.py Misc/NEWS Objects/intobject.c Objects/longobject.c Objects/typeobject.c Python/bltinmodule.c
by Eric Smith June 20, 2008

June 20, 2008

I thought there was a discussion of this earlier, and the idea was to leave the prior implementation, because that's how it's implemented in 3.0. bin() is a new feature in 2.6, so there's no particular need to make it work like hex() and oct(). Recall that in 3.0, __bin__, __oct__, and __hex__ don't exist. Instead, you use __index__ for integer conversions. That's how bin() worked in 2.6 until this checkin. But now that I look for it, I can't find the original discussion. raymond.hettinger wrote: > Author: raymond.hettinger > Date: Fri Jun 20 06:18:15 2008 > New Revision: 64424 > > Log: > Make bin() implementation parallel oct() and hex() so that int/long subclasses can override or so that other classes can support.

1 0

Summary of Python tracker Issues
by Python tracker June 20, 2008

June 20, 2008

ACTIVITY SUMMARY (06/13/08 - 06/20/08) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1917 open (+28) / 13057 closed (+19) / 14974 total (+47) Open issues with patches: 581 Average duration of open issues: 712 days. Median duration of open issues: 1479 days. Open Issues Breakdown open 1892 (+28) pending 25 ( +0) Issues Created Or Reopened (50) _______________________________ Implement __format__ for Decimal 06/20/08 http://bugs.python.org/issue2110 reopened benjamin.peterson patch Create the urllib package 06/18/08 http://bugs.python.org/issue2885 reopened brett.cannon Python 2.5.2 Windows Source Distribution missing Visual Studio 2 06/13/08 http://bugs.python.org/issue3105 created MarkE speedup some comparisons 06/13/08 http://bugs.python.org/issue3106 created pitrou patch memory leak in make test (in "test list"), 2.5.2 not 2.5.1, Linu 06/13/08 http://bugs.python.org/issue3107 created hushp1pt Implicit exception chaining (PEP 3134) 06/13/08 CLOSED http://bugs.python.org/issue3108 created pitrou patch test_multiprocessing seems fragile 06/14/08 http://bugs.python.org/issue3109 created pitrou Multiprocessing package build problem on Solaris 10 06/14/08 http://bugs.python.org/issue3110 created jnoller multiprocessing ppc Debian/ ia64 Ubuntu compilation error 06/14/08 http://bugs.python.org/issue3111 created jnoller implement PEP 3134 exception reporting 06/14/08 http://bugs.python.org/issue3112 created benjamin.peterson patch Document exception chaining 06/14/08 http://bugs.python.org/issue3113 created benjamin.peterson bus error on lib2to3 06/15/08 CLOSED http://bugs.python.org/issue3114 reopened gvanrossum patch os.listdir randomly fails on occasions when it shouldn't 06/14/08 http://bugs.python.org/issue3115 created philipspencer patch Fix quadratic behavior for marshal.dumps() when len>32Mb 06/15/08 CLOSED http://bugs.python.org/issue3116 created rhettinger patch, patch segfault with (None,) as argument in a def/lambda 06/15/08 CLOSED http://bugs.python.org/issue3117 created santoemma test_math fails on 64bit 06/16/08 CLOSED http://bugs.python.org/issue3118 created benjamin.peterson patch, 64bit pickle.py is limited by python's call stack 06/16/08 http://bugs.python.org/issue3119 created habnabit patch subprocess module truncates handles on AMD64 06/16/08 http://bugs.python.org/issue3120 created rupole test_urllibnet fails 06/16/08 CLOSED http://bugs.python.org/issue3121 created cartman sys.getsizeof() gives an AttributeError for _sre objects. 06/16/08 http://bugs.python.org/issue3122 created schuppenies patch, patch 2to3 fails 06/16/08 CLOSED http://bugs.python.org/issue3123 created djc test_multiprocessing segfaults on Windows 06/16/08 CLOSED http://bugs.python.org/issue3124 created theller test_multiprocessing causes test_ctypes to fail 06/16/08 http://bugs.python.org/issue3125 created theller patch Lib/logging/__init__.py assumes its stream has flush and close m 06/16/08 CLOSED http://bugs.python.org/issue3126 created bobf patch Segfault provoked by generators and exceptions 06/17/08 CLOSED http://bugs.python.org/issue3127 created amajorek Regex causes python to hang up? / loop infinite? 06/17/08 CLOSED http://bugs.python.org/issue3128 created computercrustie struct allows repeat spec. without a format specifier 06/17/08 http://bugs.python.org/issue3129 created carrus85 In some UCS4 builds, sizeof(Py_UNICODE) could end up being more 06/17/08 http://bugs.python.org/issue3130 created schuppenies 2to3 can't find fixes_dir 06/17/08 http://bugs.python.org/issue3131 created bhy implement PEP 3118 struct changes 06/17/08 http://bugs.python.org/issue3132 created benjamin.peterson CGIHTTPRequestHandler does not work on windows 06/18/08 CLOSED http://bugs.python.org/issue3133 created amaury.forgeotdarc patch shutil references undefined WindowsError symbol 06/18/08 http://bugs.python.org/issue3134 created dvitek patch inspect.getcallargs() 06/18/08 http://bugs.python.org/issue3135 created gsakkis [PATCH] logging.config.fileConfig() compulsivly disable all exis 06/19/08 CLOSED http://bugs.python.org/issue3136 created llucax patch Python doesn't handle SIGINT well if it arrives during interpret 06/19/08 http://bugs.python.org/issue3137 created gjb1002 Hang when calling get() on an empty queue in the queue module 06/19/08 CLOSED http://bugs.python.org/issue3138 created slash2314 print is not thread safe 06/19/08 http://bugs.python.org/issue3139 created amaury.forgeotdarc patch str.format("{0:n}") poss. bug with setlocale() 06/19/08 http://bugs.python.org/issue3140 created mark Linux build requires Mac module _gestalt 06/19/08 CLOSED http://bugs.python.org/issue3141 created mark urllib docs don't match the new modules 06/19/08 http://bugs.python.org/issue3142 created mark development docs waste a lot of horizontal space on left nav bar 06/19/08 http://bugs.python.org/issue3143 created forest popen / popen[234] inconsistent fd behavior 06/19/08 CLOSED http://bugs.python.org/issue3144 created justincappos help> modules os raises UnicodeDecodeError 06/19/08 CLOSED http://bugs.python.org/issue3145 created msyang Sphinx/LaTeX fails on Python 3.0b1 documentation 06/19/08 http://bugs.python.org/issue3146 created vmanis1 tests for sys.getsizeof fail on win64 06/19/08 http://bugs.python.org/issue3147 created amaury.forgeotdarc multiprocessing test hangs 06/20/08 CLOSED http://bugs.python.org/issue3148 created MrJean1 multiprocessing build fails on Solaris 10 06/20/08 http://bugs.python.org/issue3149 created MrJean1 multiprocessing module not being installed 06/20/08 CLOSED http://bugs.python.org/issue3150 created hdiogenes patch, easy elementtree serialization bug for weird namespace urls 06/20/08 http://bugs.python.org/issue3151 created xrotwang SimpleHTTPServer reports wrong content-length for text files 06/13/08 http://bugs.python.org/issue839496 reopened tjreedy patch, easy Issues Now Closed (34) ______________________ test_socket_ssl hanhs on Windows (deadlock) 209 days http://bugs.python.org/issue1489 amaury.forgeotdarc o(n*n) marshal.dumps performance for largish objects with patch 156 days http://bugs.python.org/issue1792 rhettinger patch, easy Add a factorial function 92 days http://bugs.python.org/issue2138 marketdickinson Make range __eq__ work 67 days http://bugs.python.org/issue2603 gvanrossum patch range: lean and mean 45 days http://bugs.python.org/issue2735 gvanrossum patch Add memory footprint query 32 days http://bugs.python.org/issue2898 schuppenies patch Bug in slice.indices() 22 days http://bugs.python.org/issue3004 marketdickinson patch 3.0a5 tarballs contain the code twice 20 days http://bugs.python.org/issue3009 barry getsizeof incorrect for Unicode strings 7 days http://bugs.python.org/issue3048 schuppenies tokenize.py improvements 6 days http://bugs.python.org/issue3078 rhettinger patch All 3.0 buildbots are red 7 days http://bugs.python.org/issue3089 barry sys.sizeof test fails with wide unicode 5 days http://bugs.python.org/issue3098 schuppenies patch Implicit exception chaining (PEP 3134) 1 days http://bugs.python.org/issue3108 benjamin.peterson patch bus error on lib2to3 1 days http://bugs.python.org/issue3114 benjamin.peterson patch Fix quadratic behavior for marshal.dumps() when len>32Mb 1 days http://bugs.python.org/issue3116 rhettinger patch, patch segfault with (None,) as argument in a def/lambda 0 days http://bugs.python.org/issue3117 georg.brandl test_math fails on 64bit 2 days http://bugs.python.org/issue3118 benjamin.peterson patch, 64bit test_urllibnet fails 3 days http://bugs.python.org/issue3121 amaury.forgeotdarc 2to3 fails 0 days http://bugs.python.org/issue3123 benjamin.peterson test_multiprocessing segfaults on Windows 0 days http://bugs.python.org/issue3124 amaury.forgeotdarc Lib/logging/__init__.py assumes its stream has flush and close m 1 days http://bugs.python.org/issue3126 vsajip patch Segfault provoked by generators and exceptions 0 days http://bugs.python.org/issue3127 amaury.forgeotdarc Regex causes python to hang up? / loop infinite? 0 days http://bugs.python.org/issue3128 computercrustie CGIHTTPRequestHandler does not work on windows 1 days http://bugs.python.org/issue3133 amaury.forgeotdarc patch [PATCH] logging.config.fileConfig() compulsivly disable all exis 1 days http://bugs.python.org/issue3136 llucax patch Hang when calling get() on an empty queue in the queue module 0 days http://bugs.python.org/issue3138 benjamin.peterson Linux build requires Mac module _gestalt 0 days http://bugs.python.org/issue3141 benjamin.peterson popen / popen[234] inconsistent fd behavior 0 days http://bugs.python.org/issue3144 amaury.forgeotdarc help> modules os raises UnicodeDecodeError 0 days http://bugs.python.org/issue3145 amaury.forgeotdarc multiprocessing test hangs 0 days http://bugs.python.org/issue3148 benjamin.peterson multiprocessing module not being installed 0 days http://bugs.python.org/issue3150 benjamin.peterson patch, easy Allow PyArg_ParseTupleAndKeywords to unpack tuples 786 days http://bugs.python.org/issue1474454 loewis patch Python 2.5b2 fails to build on Solaris 10 (Sun Compiler) 695 days http://bugs.python.org/issue1528620 akuchling Add Tkinter.Checkbutton get() and set(value) 307 days http://bugs.python.org/issue1774370 gpolo patch Top Issues Most Discussed (10) ______________________________ 17 test_multiprocessing causes test_ctypes to fail 4 days open http://bugs.python.org/issue3125 17 sys.sizeof test fails with wide unicode 5 days closed http://bugs.python.org/issue3098 13 multiprocessing ppc Debian/ ia64 Ubuntu compilation error 6 days open http://bugs.python.org/issue3111 12 bus error on lib2to3 1 days closed http://bugs.python.org/issue3114 11 memory leak in make test (in "test list"), 2.5.2 not 2.5.1, Lin 7 days open http://bugs.python.org/issue3107 8 test_multiprocessing hangs on OS X 10.5.3 8 days open http://bugs.python.org/issue3088 8 Bug in slice.indices() 22 days closed http://bugs.python.org/issue3004 7 Let bin() show floats 21 days open http://bugs.python.org/issue3008 7 robotparser.py fixes 304 days open http://bugs.python.org/issue1778443 6 Regex causes python to hang up? / loop infinite? 0 days closed http://bugs.python.org/issue3128

1 0

test_multiprocessing: test_listener_client flakiness
by Trent Nelson June 19, 2008

June 19, 2008

I gave my Windows buildbots a little bit of TLC last night. This little chestnut in test_multiprocessing.py around line 1346 is causing my buildbots to wedge more often than not: def test_listener_client(self): for family in self.connection.families: l = self.connection.Listener(family=family) p = self.Process(target=self._test, args=(l.address,)) p.set_daemon(True) p.start() conn = l.accept() self.assertEqual(conn.recv(), 'hello') p.join() l.close() The wedging will be a result of that accept() call. Not knowing anything about the module or the test suite, I can only assume that there's a race condition introduced between when the subprocess attempts to connect to the listener, versus when the l.accept() call is actually entered. (On the basis that a race condition would explain why sometimes it wedges and sometimes it doesn't.) Just FYI, the error in the buildbot log (http://www.python.org/dev/buildbot/all/x86%20W2k8%20trunk/builds/810/step-t…) when this occurs is as follows: test_multiprocessing command timed out: 1200 seconds without output SIGKILL failed to kill process using fake rc=-1 program finished with exit code -1 remoteFailed: [Failure instance: Traceback from remote host -- Traceback (most recent call last): Failure: buildbot.slave.commands.TimeoutError: SIGKILL failed to kill process ] (The fact it can't be killed cleanly is a bug in Twisted's signalProcess('KILL') method, which doesn't work against Python processes that have entered accept() calls on Windows (which present the 'wedged' behaviour and have to be forcibly killed with OpenProcess/TerminateProcess).) Trent.

7 18

sum() in math module not duplicated?
by A.M. Kuchling June 19, 2008

June 19, 2008

In the comments before the implementation of sum() in mathmodule.c, note 4 says: Note 4: A similar implementation is in Modules/cmathmodule.c. Be sure to update both when making changes. I can't find a sum implementation or similar code in cmathmodule.c. Did someone intend to port the sum function to cmath, or was it copied and then taken out? I'm wondering if this note should simply be removed. --amk

2 1

Opcode frequency
by Georg Brandl June 19, 2008

June 19, 2008

Hi, Maciej Fijalkowski did an opcode analysis for PyPy, it also shows the relative frequency of opcodes following a specifc one: http://codespeak.net/svn/user/fijal/opcodes.txt Might it make sense to add more PREDICT()ions based on this, e.g. for BUILD_SLICE -> BINARY_SUBSCR? Georg

5 6