[Python-Dev] Windows Low Fragementation Heap yields speedup of
Martin.Gfeller at comit.ch
Thu Feb 17 19:34:50 CET 2005
what immediately comes to mind are Modules/cPickle.c and Modules/cStringIO.c, which (I believe) are heavily used by ZODB (which in turn is heavily used by the application).
The lists also get fairly large, although not huge - up to typically 50000 (complex) objects in the tests I've measured. As I said, I don't speak C, so I can only speculate - do the lists at some point grow beyond the upper limit of obmalloc, but are handled by the LFH (which has a higher upper limit, if I understood Tim Peters correctly)?
From: Evan Jones [mailto:ejones at uwaterloo.ca]
Sent: Thursday, 17 Feb 2005 02:26
To: Python Dev
Cc: Gfeller Martin; Martin v. Löwis
Subject: Re: [Python-Dev] Windows Low Fragementation Heap yields speedup of ~15%
On Feb 16, 2005, at 18:42, Martin v. Löwis wrote:
> I must admit that I'm surprised. I would have expected
> that most allocations in Python go through obmalloc, so
> the heap would only see "large" allocations.
> It would be interesting to find out, in your application,
> why it is still an improvement to use the low-fragmentation
Hmm... This is an excellent point. A grep through the Python source
code shows that the following files call the native system malloc (I've
excluded a few obviously platform specific files). A quick visual
inspection shows that most of these are using it to allocate some sort
of array or string, so it likely *should* go through the system malloc.
Gfeller, any idea if you are using any of the modules on this list? If
so, it would be pretty easy to try converting them to call the obmalloc
functions instead, and see how that affects the performance.
More information about the Python-Dev