On 22 Jun 2015 01:45, "Sturla Molden" <sturla.molden@gmail.com> wrote:
Nick Coghlan <ncoghlan@gmail.com> wrote:
Sturla, expressing out and out contempt for entire communities of capable, competent developers (both the creators of Windows and Java, and the users of those platforms) has no place on the core Python mailing lists. Please refrain from casually insulting entire groups of people merely because you don't approve of their technical choices.
I am not sure what you mean. Using threads on Windows and Java comes from
a
necessity, not because developers are incompetent.
The folks *designing* Windows and Java are also people, and as creators of development platforms go, it's hard to dispute their success in helping folks solve real problems. We should be mindful of that when drawing lessons from their experience.
Windows does not provide a fork and processes are heavy-weight, hence multi-threading is the obvious choice.
With subinterpreters, we have a lot more options for that than we do with any form of IPC, including shared references to immutable objects, and the PEP 3118 buffer API.
Perhaps. One could do this with shared memory as well, but a complicating factor is that the base address must be the same (or corrected for). But one could probably do low-level magic with memory mapping to work around this. Particularly on 64-bit it is not really difficult to make sure a
is mapped to the same address in two processes.
It is certainly easier to achieve within a process. But if the plan for Erlang-style "share nothing" threads is to pickle and memcpy objects,
Windows actually has superior native parallel execution APIs to Linux in some respects, but open source programming languages tend not to support them, presumably due to a combination of Microsoft's longstanding hostile perspective on open source licencing (which seems to finally be moderating with their new CEO), and the even longer standing POSIX mindset that "fork and file descriptors ought to be enough for anyone" (even if the workload in the child processes is wildly different from that in the main process). asyncio addresses that problem for Python in regards to IOCP vs select (et al), and the configurable subprocess creation options addressed it for multiprocessing, but I'm not aware of any efforts to get greenlets to use fibres when they're available. page there
is little or nothing to gain over using multiprocessing.
The Python level *semantics* should be as if the objects were being copied (for ease of use), but the *implementation* should try to avoid actually doing that (for speed of execution). Assuming that can be done effectively *within* a process between subinterpreters, then the possibility arises of figuring out how to use shared memory to federate that approach across multiple processes. That could then provide a significant performance improvement for multiprocessing. But since we have the option of tackling the simpler problem of subinterpreters *first*, it makes sense to do that before diving into the cross-platform arcana involved in similarly improving the efficiency of multiprocessing's IPC. Regards, Nick.
Sturla
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/