extension module performance problems

[Martin von Loewis]

rwgk <rwgkNOrwSPAM at cci.lbl.gov.invalid> writes:

> Can these observations be explained? Is there a way to
> get full performance under all circumstances?

I can guess about the causes. Threading in Python is done via a global
mutex, which is help by exactly one Python thread at any time, so C
code does not need to worry about multithreading. The lock is released
when a blocking system call is made. So the performance of the
threading library is critical to the performance of Python, at least
under --enable-threads. The performance impact should be less if the
thread module hasn't been imported - you may want to check if it is
(looking at sys.modules).

Creating shared libraries always gives a performance impact. To make
the shared libraries position-independent (PIC), extra code must be
generated for inter-library calls. Specifically, when you have a
register pointing to the start of the library, that register needs to
be reloaded every time you cross a shared-library boundary. On some
systems (e.g. Alpha), you also take a performance hit if the library
calls back into the executable, as each such call will go through a
thunks layer. I don't know whether it explains all the performance
loss, but it probably explains some of it.

Regards,
Martin