[Python-ideas] solving multi-core Python
sturla.molden at gmail.com
Thu Jun 25 02:02:05 CEST 2015
On 25/06/15 00:19, Eric Snow wrote:
>>> Solving reference counts in this situation is a separate issue that
>>> will likely need to be resolved, regardless of which machinery we use
>>> to isolate task execution.
>> As long as we have a GIL, and we need the GIL to update a reference count,
>> it does not hurt so much as it otherwise would. The GIL hides most of the
>> scalability impact by serializing flow of execution.
> It does hurt in COW situations, e.g. forking. My expectation is that
> we'll at least need to take a serious look into the matter in the
> short term (i.e. Python 3.6).
It hurts performance after forking as reference counting will trigger a
lot of page copies. Keeping reference counts in separate pages and
replacing the field in the PyObject struct would reduce this problem by
a factor of up to 512 (64 bit) or 1024 (32 bit).
It does not hurt performance with multi-threading, as Python threads are
serialized by the GIL. But if the GIL was removed it would result in a
lot of false sharing. That is a major reason we need a tracing garbage
collector instead of reference counting if we shall be able to remove
More information about the Python-ideas