[soc2008-general] Memory Allocation Improvement

[Alexey Sartakov]

     Dear Sirs!

I'm intrested in join to GSoC with idea I describe bellow. If You intrested
in
me let me know please

----
Memory Allocation Improvement

This new model of memory allocation means to increase the density or the
data per one memory block being compared
with another memory allocators. That's possible because process of memory
reallocation (realloc(2)) used  so  little.
Today there are some project like libhoard and jemalloc on FreeBSD 7 were
started to change legal methods of
memory allocation. GIL in Python language makes it available to allocate
memory for multi-threaded tasks same as for
single-threaded, without additional difficulty.

Thereis possible to think over on idea to make settings for management of
adjustment by neighbourhood memory slots
in memory allocation library. That's no a modern idea, but IMHO that
effective management of memory allocations will not
be possible without management of this parameter. If thereis not possible,
then programmers will make programs
to duplicate allocator functionality, so code will be more difficult for
future support and debug of code. For example, the
special allocator was created for int type. And now we can see various
threads about advisability of this allocator
in python-dev.

This idea is different from other opensource projects. Now I have just some
of test code, so I can't say  how much of performance
will be increase up. I think that with including of this allocator to python
core, performance will increase up to 1-2 percents for
any common task.  The maximum of performance we can increase up is about 5
percents, just because memory allocations is
important, but just a little part of interpreter work. In case,  when memory
will dump onto swap, the performance will not
be depending with memory allocator algorithm. But if objects will be placed
more compact on the memory pages it
will decrease total volume of memory used. Decrease of the memory volume
down to 10 percents may increase
system performance up to 2 or more times.  More compact memory placement
also mean that current memory page will
be swapped on disk rarely, because often use of current memory page (when
memory swapped, only more rarely pages will be
dumped to disk) .

This allocator may be used  on environments where "obmalloc.c" available. If
thereis no "obmalloc.c" available then
good idea to make a compatibility code  as a part of this project.

Second part of the project is improvement of memory reallocation by
rebuilding the GC module.  I propose to decrease  head of
GC-module (PyGC) without loss of transferability and working speed. With
decrease of memory allocation performance of
python virtual machine will be increased up to 2-3 percent ever when new
object  have been created and old are destroyed.
But in case, when CPU cache off or limited, it can decrease the system
performance. I can say only about decrease of
memory allocation for Py_TPFLAGS_HAVE_GC objects.


---
S.Y. Alexey 'Dok' Sartakov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/soc2008-general/attachments/20080325/1b43c622/attachment.htm