[Python-ideas] Python 3000 TIOBE -3%

[Sturla Molden]

Den 11.02.2012 07:15, skrev Nick Coghlan:
> Why is that relevant? If free threading is the all-singing all dancing
> wonderment you believe:
>
> 1. Fork CPython
> 2. Make it free-threaded (while retaining backwards compatibility with
> all the C extensions out there!)
> 3. Watch the developer hordes flock to your door (after all, it's the
> lack of free-threading that has held Python's growth back for the last
> two decades, so everyone will switch in a heartbeat the second you, or
> anyone else, publishes a free-threaded alternative where all their C
> extensions work. Right?).

There are several solutions to this, I think.

One is to use one interpreter per thread, and share no data between 
them, similar to tcl and PerlFork. The drawback is developers who forget 
to duplicate file handles, so one interpreter can close a handle used by 
another.

Another solution is transactional memory. Consider a database with 
commit and rollback. Not sure how to fit this with C extensions though, 
but one could in theory build a multithreaded interpreter like that.

> If a fraction of the energy that people put into asking for free 
> threading was instead put into asking "how can we make inter-process 
> communication better?", we'd no doubt have a good shared object 
> implementation in the mmap module by now (and someone might have 
> actually responded to Jesse's request for a new multiprocessing 
> maintainer when circumstances forced him to step down).

I think I already explained why BSD mmap is a dead end. We need named 
kernel objects (System V IPC or Unix domain sockets) as they can be 
communicated between processes.

There is also reasons to prefer SysV message queues over shared memory 
(Sys V or BSD), such a thread safety. I.e. access is synchronized by the 
kernel. SysV message queues also have atomic read/write, unlike sockets, 
and they are generally faster than pipes. With sockets we have to ensure 
that the correct number of bytes were read or written, which is a PITA 
for any IPC use (or any other messaging for that matter). In the 
meanwhile, take a look at ZeroMQ (actually written ØMQ). ZeroMQ also 
have atomic read/write messages.

Sturla