Instead of collecting objects after a fixed number of allocations (700)
You could make it dynamic like this:
# initial values
min_allocated_memory = 0
max_allocated_memory = 0
next_gc_run = 1024 * 1024
allocated_memory = get_allocated_memory()
min_allocated_memory = min(min_allocated_memory, allocated_memory)
max_allocated_memory = max(max_allocated_memory, allocated_memory)
if max_allocated_memory - min_allocated_memory > next_gc_run:
# run the gc
memory_freed, allocated_memory = run_gc()
next_gc_run = max(
allocated_memory * 1.5 - memory_freed, 1024 * 1024
min_allocated_memory = allocated_memory
max_allocated_memory = allocated_memory
manage_memory() should be called after every allocation and after a ref
count of an object reaches 0 (memory is freed)
=> As less objects contain cyclic references as less often the GC will
run (memory_freed is small)
=> As more objects contain cyclic references as more often the GC will
run (memory_freed is large)
=> If memory utiliaziation grows fast (burst allocations) GC will run
less often: next_gc_run = allocated_memory * 1.5 - memory_freed
... Of course the constants: 1.5 and 1024 * 1024 are only suggestions...
-----BEGIN PGP SIGNED MESSAGE-----
On behalf of the Python development team and the Python community, I
am happy to announce the first beta releases of Python 2.6 and Python
Please note that these are beta releases, and as such are not suitable
for production environments. We continue to strive for a high degree
of quality, and these releases are intended to freeze the feature set
for Python 2.6 and 3.0.
From now until the planned final releases in September 2008, we will
be fixing known problems and stabilizing these new Python versions.
You can help by downloading and testing them, providing feedback and
hopefully helping to fix bugs. You can also use these releases to
determine how changes in 2.6 and 3.0 might impact you. If you find
things broken or incorrect, please submit bug reports at
For more information and downloadable distributions, see the Python
and the Python 3.0 web site:
See PEP 361 for release schedule details:
Python 2.6/3.0 Release Manager
(on behalf of the entire python-dev team)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)
-----END PGP SIGNATURE-----
If I understand things correctly, the csv module in 3.0 supports Unicode,
kind of for free because all strings are Unicode. I suspect the Unicode
examples at the bottom of the 3.0 version of the module doc need to be
Does the 2.6 version support Unicode out of the box as well or are the
Unicode examples still required there?
I am curious if there is any news on this issue. My understanding is
that since this is a new feature, nothing will be committed until
after 2.6 comes out, but it would be *really nice* if I could use
annotations in 2.x.
[warning - mini rant]
Reason is, I am using 3.0a for now for this project, but I am sorry to
say that using parentheses for raising exceptions drives me up the
wall, and I don't like using them for the former print statement
either. I suspect this is partly due to the fact that I'm using 2.5
for work, so I don't have the option to mentally "switch over" to the
Python 3 way.
Does anyone have an annotations patch floating around against the
trunk / 2.6b, or failing that, does anyone have opinions on what my
chances are / things to look out for if I fiddle around with the
annotations patch from the py3k branch? (My understanding is that this
is revision 53170).
On Thu, Jun 19, 2008 at 03:04:11PM -0500, Benjamin Peterson wrote:
> I don't think the whole introduction had to go. I think it helped give
> some idea of how multiprocessing works before jumping straight to the
> API reference.
I don't think overview material like that should be buried inside the
documentation for one particular module; it should go in a chapter
My plan is to move multiprocessing from the 'Optional OS Services'
chapter to the 'Interprocess Communication' chapter, and then expand
the introduction of the chapter to discuss the GIL and related issues.
Presumably it was put in the 'Optional OS Services' chapter because
that's where the threading and thread modules are.
I thought there was a discussion of this earlier, and the idea was to
leave the prior implementation, because that's how it's implemented in
3.0. bin() is a new feature in 2.6, so there's no particular need to
make it work like hex() and oct().
Recall that in 3.0, __bin__, __oct__, and __hex__ don't exist. Instead,
you use __index__ for integer conversions. That's how bin() worked in
2.6 until this checkin.
But now that I look for it, I can't find the original discussion.
> Author: raymond.hettinger
> Date: Fri Jun 20 06:18:15 2008
> New Revision: 64424
> Make bin() implementation parallel oct() and hex() so that int/long subclasses can override or so that other classes can support.
I gave my Windows buildbots a little bit of TLC last night. This little chestnut in test_multiprocessing.py around line 1346 is causing my buildbots to wedge more often than not:
for family in self.connection.families:
l = self.connection.Listener(family=family)
p = self.Process(target=self._test, args=(l.address,))
conn = l.accept()
The wedging will be a result of that accept() call. Not knowing anything about the module or the test suite, I can only assume that there's a race condition introduced between when the subprocess attempts to connect to the listener, versus when the l.accept() call is actually entered. (On the basis that a race condition would explain why sometimes it wedges and sometimes it doesn't.)
Just FYI, the error in the buildbot log (http://www.python.org/dev/buildbot/all/x86%20W2k8%20trunk/builds/810/step-t…) when this occurs is as follows:
command timed out: 1200 seconds without output
SIGKILL failed to kill process
using fake rc=-1
program finished with exit code -1
remoteFailed: [Failure instance: Traceback from remote host -- Traceback (most recent call last):
Failure: buildbot.slave.commands.TimeoutError: SIGKILL failed to kill process
(The fact it can't be killed cleanly is a bug in Twisted's signalProcess('KILL') method, which doesn't work against Python processes that have entered accept() calls on Windows (which present the 'wedged' behaviour and have to be forcibly killed with OpenProcess/TerminateProcess).)
In the comments before the implementation of sum()
in mathmodule.c, note 4 says:
Note 4: A similar implementation is in Modules/cmathmodule.c.
Be sure to update both when making changes.
I can't find a sum implementation or similar code in cmathmodule.c.
Did someone intend to port the sum function to cmath, or was it copied
and then taken out? I'm wondering if this note should simply be
Maciej Fijalkowski did an opcode analysis for PyPy,
it also shows the relative frequency of opcodes following a
Might it make sense to add more PREDICT()ions based
on this, e.g. for BUILD_SLICE -> BINARY_SUBSCR?