[Python-ideas] POPT (Python Ob ject Provider Threads)

Antoine Pitrou solipsis at pitrou.net
Tue Jun 19 10:54:34 EDT 2018

On Tue, 19 Jun 2018 16:47:46 +0200
Martin Bammer <mrbm74 at gmail.com> wrote:
> Hello,
> because Python is a very dynamic language the memory management is heavily
> used. A lot of time is used for creating (reserve memory and fill object
> structure with data) and destroying objects.

Do you have numbers about that?  One modus operandi would be to collect
profiling data using Linux "perf" on a real Python workload you care

> And here comes the idea for POPT. With this idea the Python interpreter has
> running several threads in background (1 thread for each object type) which
> manage a set of objects as an object cache. Each object in the cache is
> already preconfigured by the object provider thread. So only the part of
> the object structure which is individual has to be initialized. This saves
> a lot of processing time for the main thread and the memory management has
> much less to do, because temporarily unused objects can be reused
> immediately.

How does the main thread (or, rather, the multiple application threads)
communicate with the background object threads?  What is the
communication and synchronization overhead in this scheme?



More information about the Python-ideas mailing list