[Python-ideas] solving multi-core Python
Sturla Molden
sturla.molden at gmail.com
Sun Jun 21 17:45:05 CEST 2015
Nick Coghlan <ncoghlan at gmail.com> wrote:
> Sturla, expressing out and out contempt for entire communities of
> capable, competent developers (both the creators of Windows and Java,
> and the users of those platforms) has no place on the core Python
> mailing lists. Please refrain from casually insulting entire groups of
> people merely because you don't approve of their technical choices.
I am not sure what you mean. Using threads on Windows and Java comes from a
necessity, not because developers are incompetent. Windows does not provide
a fork and processes are heavy-weight, hence multi-threading is the obvious
choice.
> Avoiding object serialisation is indeed the main objective.
Good.
> With
> subinterpreters, we have a lot more options for that than we do with
> any form of IPC, including shared references to immutable objects, and
> the PEP 3118 buffer API.
Perhaps. One could do this with shared memory as well, but a complicating
factor is that the base address must be the same (or corrected for). But
one could probably do low-level magic with memory mapping to work around
this. Particularly on 64-bit it is not really difficult to make sure a page
is mapped to the same address in two processes.
It is certainly easier to achieve within a process. But if the plan for
Erlang-style "share nothing" threads is to pickle and memcpy objects, there
is little or nothing to gain over using multiprocessing.
Sturla
More information about the Python-ideas
mailing list