On Fri, Jun 20, 2008 at 10:25 AM, Antoine Pitrou <solipsis@pitrou.net> wrote:
Kevin Jacobs <jacobs <at> bioinformed.com> <bioinformed <at> gmail.com> writes:
+1 on a C API for enabling and disabling GC. I have several instances
where I create a large number of objects non-cyclic objects where I see huge GC overhead (30+ seconds with gc enabled, 0.15 seconds when disabled).
Could you try to post a stripped-down, self-contained example of such behaviour?
$ python -m timeit 'zip(*[range(1000000)]*5)' 10 loops, best of 3: 496 msec per loop $ python -m timeit -s 'import gc; gc.enable()' 'zip(*[range(1000000)]*5)' 10 loops, best of 3: 2.93 sec per loop Note that timeit cheats and disables GC by default. Attached is a less stripped down script to demonstrate the super-linear behavior for somewhat naively coded transpose operators. The output is listed below: FUNCTION ROWS COLUMNS GC ENABLED GC DISABLED -------------- ----------- ---------- ------------ ------------ transpose_comp 5 0 0.0000 0.0000 transpose_comp 5 250000 0.5535 0.3537 transpose_comp 5 500000 1.4359 0.6868 transpose_comp 5 750000 2.7148 1.0760 transpose_comp 5 1000000 3.8070 1.3936 transpose_comp 5 1250000 5.5184 1.7617 transpose_comp 5 1500000 7.8828 2.1308 transpose_comp 5 1750000 9.3279 2.5364 transpose_comp 5 2000000 11.8248 2.7399 transpose_comp 5 2250000 14.7436 3.1585 transpose_comp 5 2500000 18.4452 3.5818 transpose_comp 5 2750000 21.4856 3.8988 transpose_comp 5 3000000 24.4110 4.3148 transpose_zip 5 0 0.0000 0.0000 transpose_zip 5 250000 0.2537 0.0658 transpose_zip 5 500000 0.8380 0.1324 transpose_zip 5 750000 1.7507 0.1989 transpose_zip 5 1000000 2.6169 0.2648 transpose_zip 5 1250000 4.0760 0.3317 transpose_zip 5 1500000 5.8852 0.4145 transpose_zip 5 1750000 7.3925 0.5161 transpose_zip 5 2000000 10.0755 0.6708 transpose_zip 5 2250000 14.2698 0.7760 transpose_zip 5 2500000 16.7291 0.9022 transpose_zip 5 2750000 20.3833 1.0179 transpose_zip 5 3000000 24.5515 1.0971 Hope this helps, -Kevin