[Python-ideas] Python 3000 TIOBE -3%

[Jesse Noller]

> 
> 
> By the time we 100 core CPUs, we'll be measuring RAM in TB, so that
> shouldn't be a problem ;-)
> 
> Many Python use cases are indeed easy to scale using multiple processes
> which then each run on a separate core, so that approach is a very
> workable way forward.
> 
> If you need to share data across processes, you can use a shared
> memory mechanism. In many cases, the data to be shared will already
> be stored in a database and those can easily be accessed from all
> processes (again using shared memory).
> 
> I often find these GIL discussion a bit theoretical. In practice
> I've so far never run into any issues with Python scalability. It's
> other components that cause a lot more trouble, like e.g. database
> query scalability, network congestion or disk access being too slow.
> 
> In cases where the GIL does cause problems, it's usually better to
> consider changing the application design and use asynchronous processing
> with a single threaded design or a multi-process design where each of
> the processes only uses a low number of threads (20-50 per process).


I think the much, much better response to the questions and comments around Python, the GIL and parallel computing in general is this:

Yes, let's have more of that!

It's like asking if people like pie, or babies. 99% of people polled are going to say "Yes, let's have more of that!" - so it goes with Python, the GIL, STM, Multiprocessing, Threads, etc.

Where all of these discussions break down - and they always do - is that we lack:

1> Someone with a working patch for Pie
2> Someone with a fleshed out proposal/PEP on how to get more Pie
3> A group of people with time to bake more Pies that could help be paid to make Pie 

Banging on the table and asking for more Pie won't get us more Pie - what we need are actual proposals, in the form of well thought out PEPs, the people to implement and maintain the thing (see: unladen swallow), or working implementations.

No one in this thread is arguing that having more Pie, or babies, would be bad. No one is arguing that more/better concurrency constructs would be good. Tools like concurrent.futures in Python 3 would be a good example of something recently added. 

The problem is people, plans and time. If we can solve the People and Time problems, instead of looking to already overworked volunteers then I'm sure we can come up with a good Pie plan.

I really like pie. 

Jesse