[Python-Dev] Python 2.3a1's mandatory use of cyclic GC causes existing applications to fail

[Robert Ledwith]

During the process of trying to upgrade my applications from Python 2.2.2
to Python 2.3.a1 I have discovered that applications which previously ran
to completion using Python 2.2.2 will no longer complete using Python 2.3a1
due to the increased memory needed for every Python object.  Using previous
versions of Python, I routinely used the --without-cycle-gc configuration
option, since the performance was better and the process size footprint was
smaller. Under Python 2.3.a1 this is no longer an option.  For my 32 bit
applications, this meant an increase of 8 bytes per Python object; for the
64 bit applications, the increase is 16 bytes per object.  For applications
which are small this increase has a negligible effect.  In fact, the
various optimizations that have been implemented has caused a mild net
improvement in run times.  However, large 32-bit applications which processed
the exact same input streams are now failing due to exceeding the maximum
process image size possible on the computers.  In addition, while the existing
64-bit applications run to completion, they typically require more CPU time due
to the larger footprint of the processes.

I believe my experiences highlight a potential issue for three classes of
Python programs:

	1. Programs written for embedded systems and tightly constrained
	   systems such as PDAs.  Previously running programs (especially
	   in-memory database based programs) may simply no longer fit
	   within the limited hardware. For these programs, migrating
	   to Python 2.3 will be impossible without rewriting the programs
	   to create fewer objects.
	   
	2. 32-bit Python programs which create millions of small Python
	   objects and are approaching the maximum process image size for
	   the platform.  As with the first class of programs, migrating
	   to Python 2.3 will be impossible without rewriting.
	   
	3. 64-bit Python programs which create millions of small Python
	   objects.  These programs will continue to run, but may see a
	   marked performance problem.  In my situation I have seen increases
	   in run times vary from 0% to 40%.
	   
I do not believe my experiences will prove to be unique among the Python
community.  Given this possibility, is it desirable to re-insert the
logic to make the cyclic GC optional now, heading off potential problems
that would surface only months from now, once Python 2.3 is officially rolled
out and installed worldwide?














:s Robert H. Ledwith               INET: rledwith@cas.org
:s Unless explicitly stated to the contrary, nothing in this posting should
:s be construed as representing my employer's opinions.