[Python-Dev] CPython optimization: storing reference counters outside of objects

[Artur Siekielski]

2011/5/24 Sturla Molden <sturla at molden.no>:
> Den 24.05.2011 11:55, skrev Artur Siekielski:
>>
>> PYRO/multiprocessing proxies isn't a comparable solution because of
>> ORDERS OF MAGNITUDE worser performance. You compare here direct memory
>> access vs serialization/message passing through sockets/pipes.
> The bottleneck is likely the serialization, but only if you serialize large
> objects. IPC is always very fast, at least on localhost .

It cannot be "fast" compared to direct memory access. Here is a
benchmark: summing numbers in a small list in a child process using
multiprocessing "manager": http://dpaste.org/QzKr/ , and using
implicit copy of the structure after fork(): http://dpaste.org/q3eh/.
The first is 200 TIMES SLOWER. It means if the work finishes in 20
seconds using fork(), the same work will require more than one hour
using multiprocessing manager.

> If a database is too slow, I am rather sure you need
> something else than Python as well.

Disk access is about 1000x slower than memory access in C, and Python
in a worst case is 50x slower than C, so there is still a huge win
(not to mention that in a common case Python is only a few times
slower).


Artur