[Python-ideas] Python 3000 TIOBE -3%

[Masklinn]

On 2012-02-11, at 03:24 , Matt Joiner wrote:
> Threading is a tool (the most popular, and most flexible tool) for
> concurrency and parallelism. Compared to forking, multiprocessing, shared
> memory, mmap, and dozens of other auxiliary OS concepts it's also the
> easiest.

Such a statement unqualified can only be declared wrong. Threading is
the most common due to Windows issues (historically, Unix parallelism
used multiple processes and the switch happened with the advent of
multiplatform tools, which focused on threading due to Windows's poor
performances and high overhead with processes), and it is also the
easiest tool *to start using*, because you just say "start a thread".

Which is equivalent to saying grenades are the easiest tool to handle
conversations because you just pull the pin.

Threads are by far the hardest concurrency tool to use because they throw
out the window all determinism in the whole program, and that determinism
then needs to be reclaimed through (very) careful analysis and the use of
locks or other such sub-tools.

And the flexibility claim is complete nonsense.

Oh, and so are your comparisons, "shared memory" and "mmap" are not
comparable to threading since they *are used* by and in threading. And
forking and multiprocessing are the same thing, only the initialization
call changes.

Finally, multiprocessing has a far better upgrade path (as e.g. Erlang
demonstrates): if your non-deterministic points are well delineated and
your interfaces to other concurrent execution points are well defined,
scaling from multiple cores to multiple machines becomes possible.

> Not all problems are clearly chunkable or fit some alternative
> parallelism pattern. Threading is arguably the cheapest method for
> parallelism, as we've heard throughout this thread.
> 
> Just because it can be dangerous is no reason to discourage it.

Of course it is, just as manual memory management is "discouraged".

> Many alternatives are equally as dangerous, more difficult and less portable.

The main alternative to threading is multiprocessing (via fork or via starting
new processes does not matter), it is significantly less dangerous, it is
only more difficult in that you can't take extremely dangerous shortcuts and
it is just as portable (if not more).

> Python is a very popular language.someone mentioned earlier that popularity
> shouldn't be an argument for features but here it's fair ground. If Python
> 3 had unrestrained threading, this transition plunge would not be
> happening.

Threading is a red herring, nobody fundamentally cares about threading,
what users want is a way to exploit their cores. If `multiprocessing` was
rock-solid and easier to use `threading` could just be killed and nobody
would care. And we'd probably find ourselves in far better a world.