Python GIL Thoughts
Hi all, I am very exciting about the sub-interpreters ... but I do not like some parts ... Consider the following code: ```python import _xxsubinterpreters as interpreters import threading import textwrap as tw import marshal if __name__ == '__main__': # Create a sub-interpreter interpid = interpreters.create() # If you had a function that generated some data arry = list(range(0,100)) # Create a channel channel_id = interpreters.channel_create() # Pre-populate the interpreter with a module interpreters.run_string(interpid, "import marshal; import _xxsubinterpreters as interpreters") # Define a def run(interpid, channel_id): # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Still is run in thread which is use GIL !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! interpreters.run_string(interpid, tw.dedent(""" arry_raw = interpreters.channel_recv(channel_id) arry = marshal.loads(arry_raw) result = [1,2,3,4,5] # where you would do some calculating result_raw = marshal.dumps(result) interpreters.channel_send(channel_id, result_raw) """), shared=dict( channel_id=channel_id ), ) inp = marshal.dumps(arry) interpreters.channel_send(channel_id, inp) # Run inside a thread t = threading.Thread(target=run, args=(interpid, channel_id)) t.start() # Sub interpreter will process. Feel free to do anything else now. output = interpreters.channel_recv(channel_id) interpreters.channel_release(channel_id) output_arry = marshal.loads(output) print(output_arry) ``` Add some `async` execution of `interpreters.run_string_async` and other async functions !! Also regarding the threads ... I think it is better to make `threading.Thread` as green `Thread` !! I know, I know, `Thread` is not possible due to `sync io` ... But we can to change underling `api` from `sync io` to `async io` !! User will notice nothing, but performance will be increased mush more !! Consider the following execution of `threading.Thread` in Python: ``` gil_lock.acquire() for green_thread in green_threads: thread.execute() gil_lock.release() ``` If we implement in such way we will be able to reduce resource used in system, but performance would be the same !! Sub-Interpreter would be the something like Workers in JavaScript, but much more flexible !! What we will have at the end: 1) `threading.Thread` is a green thread that do not consume a lot of resources but still useful !! 2) Real parallelizm is done within sub-interpreters like workers in JavaScript, but much more flexible Say what do you think ?
On 5/23/2020 6:24 PM, redradist@gmail.com wrote:
Hi all,
I am very exciting about the sub-interpreters ... but I do not like some parts ... Consider the following code: ```python import _xxsubinterpreters as interpreters import threading import textwrap as tw import marshal
if __name__ == '__main__': # Create a sub-interpreter interpid = interpreters.create()
# If you had a function that generated some data arry = list(range(0,100))
# Create a channel channel_id = interpreters.channel_create()
# Pre-populate the interpreter with a module interpreters.run_string(interpid, "import marshal; import _xxsubinterpreters as interpreters")
# Define a def run(interpid, channel_id): # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Still is run in thread which is use GIL !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! interpreters.run_string(interpid, tw.dedent(""" arry_raw = interpreters.channel_recv(channel_id) arry = marshal.loads(arry_raw) result = [1,2,3,4,5] # where you would do some calculating result_raw = marshal.dumps(result) interpreters.channel_send(channel_id, result_raw) """), shared=dict( channel_id=channel_id ), )
inp = marshal.dumps(arry) interpreters.channel_send(channel_id, inp)
# Run inside a thread t = threading.Thread(target=run, args=(interpid, channel_id)) t.start()
# Sub interpreter will process. Feel free to do anything else now. output = interpreters.channel_recv(channel_id) interpreters.channel_release(channel_id) output_arry = marshal.loads(output)
print(output_arry) ```
Add some `async` execution of `interpreters.run_string_async` and other async functions !!
Also regarding the threads ... I think it is better to make `threading.Thread` as green `Thread` !!
I know, I know, `Thread` is not possible due to `sync io` ... But we can to change underling `api` from `sync io` to `async io` !! User will notice nothing, but performance will be increased mush more !! Only if your workload is CPU bound. Python optimizes IO bound workload performance by releasing the GIL while doing IO. Green threads generally do not offer this option.
--Edwin
Edwin Zimmerman wrote: Only if your workload is CPU bound. Python optimizes IO bound workload performance by releasing the GIL while doing IO. Green threads generally do not offer this option.
Real threads is not needed in Python: 1) Real threads do not work parallel 2) Real threads only consume resources of OS 3) Real threads have also as trade off context switching between threads (small but ...) I think considering this trade offs we should switch Real Thread -> Green Thread internally or provide alternative functionality ... I think asyncio already provide functionality of GreenThread ... Okay maybe we should leave Real Threads as it is ... I need to think about it ... But also in my email I have mentioned about `interpreters.run_string` @Edwin Zimmerman Do you know if it is blocking call ? And if yes than I do not understand how with such API we will have TRUE Parallel execution, because in this case only one Thread at the same time could be executed, but those threads could be executed only one at the time !! Also there are drawback that we need additional thread for run sub-interpretter !! Better would be the something like this in addition `interpreters.run_string_async`, `interpreters.run_async` in addition to `interpreters.run_string`, `interpreters.run` to remove trad offs
On Mon, May 25, 2020 at 7:58 PM <redradist@gmail.com> wrote:
Edwin Zimmerman wrote: Only if your workload is CPU bound. Python optimizes IO bound workload performance by releasing the GIL while doing IO. Green threads generally do not offer this option.
Real threads is not needed in Python: 1) Real threads do not work parallel
They do when they do any sort of blocking operation.
2) Real threads only consume resources of OS 3) Real threads have also as trade off context switching between threads (small but ...)
Not enough to be a concern.
I think considering this trade offs we should switch Real Thread -> Green Thread internally or provide alternative functionality ... I think asyncio already provide functionality of GreenThread ... Okay maybe we should leave Real Threads as it is ... I need to think about it ...
Yes, leave real threads as is, thanks. They're pretty important actually. If you don't want to use threads, that's fine, but don't change the threading module to not be threads. Not really a lot of point breaking backward compatibility for, uhh, zero gain. ChrisA
Chris Angelico wrote:
On Mon, May 25, 2020 at 7:58 PM redradist@gmail.com wrote:
Edwin Zimmerman wrote: Only if your workload is CPU bound. Python optimizes IO bound workload performance by releasing the GIL while doing IO. Green threads generally do not offer this option. Real threads is not needed in Python: 1) Real threads do not work parallel They do when they do any sort of blocking operation. 2) Real threads only consume resources of OS 3) Real threads have also as trade off context switching between threads (small but ...) Not enough to be a concern. I think considering this trade offs we should switch Real Thread -> Green Thread internally or provide alternative functionality ... I think asyncio already provide functionality of GreenThread ... Okay maybe we should leave Real Threads as it is ... I need to think about it ... Yes, leave real threads as is, thanks. They're pretty important actually.
If you don't want to use threads, that's fine, but don't change the threading module to not be threads. Not really a lot of point breaking backward compatibility for, uhh, zero gain. ChrisA
I like the topic on https://www.reddit.com/r/Python/comments/bpp9hg/has_the_python_gil_been_slai... Answer written by CSI_Tech_Dept: ``` ... What probably needs to be done is to have a compile option that causes Python to work without GIL but at the cost of breaking some of API (this would especially broke C API), then developers would have an option to chose: compatibility with GIL no GIL, full multithreading, but broken compatibility If user could decide which python version to use and there was a clear benefit, existing applications would be migrated, people would also make sure their code works with both versions. ... ``` Interesting thoughts ...
redradist＠gmail.com wrote:
On Mon, May 25, 2020 at 7:58 PM redradist@gmail.com wrote:
Edwin Zimmerman wrote: Only if your workload is CPU bound. Python optimizes IO bound workload performance by releasing the GIL while doing IO. Green threads generally do not offer this option. Real threads is not needed in Python: 1) Real threads do not work parallel They do when they do any sort of blocking operation. 2) Real threads only consume resources of OS 3) Real threads have also as trade off context switching between threads (small but ...) Not enough to be a concern. I think considering this trade offs we should switch Real Thread -> Green Thread internally or provide alternative functionality ... I think asyncio already provide functionality of GreenThread ... Okay maybe we should leave Real Threads as it is ... I need to think about it ... Yes, leave real threads as is, thanks. They're pretty important actually. If you don't want to use threads, that's fine, but don't change the threading module to not be threads. Not really a lot of point breaking backward compatibility for, uhh, zero gain. ChrisA I like the topic on https://www.reddit.com/r/Python/comments/bpp9hg/has_the_python_gil_been_slai... Answer written by CSI_Tech_Dept: ... What probably needs to be done is to have a compile option that causes Python to work without GIL but at the cost of breaking some of API (this would especially broke C API),
Chris Angelico wrote: then developers would have an option to chose:
compatibility with GIL
no GIL, full multithreading, but broken compatibility
If user could decide which python version to use and there was a clear benefit, existing applications would be migrated, people would also make sure their code works with both versions. ...
Interesting thoughts ...
Also I have tried to add in CPython adaptive reference counter ... What is it ... It is reference count that is based on knowledge that there are some some additional thread ... Take a look at this implementation: ``` inline void add_reference_couter(atomic_int * acnt) { if (num_other_threads > 0) { atomic_fetch_add_explicit(acnt, 1, memory_order_acq_rel); } else { ++(*acnt); } } inline void sub_reference_couter(atomic_int * acnt) { if (num_other_threads > 0) { atomic_fetch_sub_explicit(acnt, 1, memory_order_acq_rel); } else { --(*acnt); } } ``` Or implementation with complex RefCounter: ``` struct RefCounter { atomic_uint acnt; uint cnt; }; inline void add_reference_couter(RefCounter * ref_counter) { if (num_other_threads > 0) { atomic_fetch_add_explicit(&(*ref_counter).acnt, 1, memory_order_acq_rel); } else { ++(*ref_counter).cnt; } } inline void sub_reference_couter(RefCounter * ref_counter) { if (num_other_threads > 0) { atomic_fetch_sub_explicit(&(*ref_counter).acnt, 1, memory_order_acq_rel); } else { --(*ref_counter).cnt; if (0 == (*ref_counter).cnt) { atomic_fetch_sub_explicit(&(*ref_counter).acnt, 1, memory_order_acq_rel); } } } ``` I have also prepared test branch on my Ubuntu machine for this implementation ...
On Mon, May 25, 2020 at 11:19:06AM -0000, redradist@gmail.com wrote:
I like the topic on https://www.reddit.com/r/Python/comments/bpp9hg/has_the_python_gil_been_slai...
Answer written by CSI_Tech_Dept: ``` ... What probably needs to be done is to have a compile option that causes Python to work without GIL but at the cost of breaking some of API (this would especially broke C API), then developers would have an option to chose:
compatibility with GIL
no GIL, full multithreading, but broken compatibility
If user could decide which python version to use and there was a clear benefit, existing applications would be migrated, people would also make sure their code works with both versions. ... ```
We already have that. The version of Python with GIL is called CPython, and the version with "no GIL, full multithreading, but broken compatibililty" is called Jython, or IronPython. It is true that Jython and IronPython have fallen behind CPython[1], but even in the Python 2.5 and 2.6 days when IronPython in particular was well-funded by Microsoft, actively developed and *faster than CPython* people who wanted "full multithreading" didn't use it, preferring to complain about the GIL than migrate to another interpreter. The idea that people will make sure their code works in both versions is a pipe-dream. People in general don't even test their code with the -O switch in CPython. [1] Maybe the PSF should fund some development of Jython and IronPython to get them at least to version 3.1? I think it is healthy for the greater Python ecosystem to have active alternatives to CPython. -- Steven
On 5/25/2020 5:56 AM, redradist@gmail.com wrote:
Edwin Zimmerman wrote: Only if your workload is CPU bound. Python optimizes IO bound workload performance by releasing the GIL while doing IO. Green threads generally do not offer this option.
Real threads is not needed in Python: 1) Real threads do not work parallel 2) Real threads only consume resources of OS As I said above, threads in Python provide limited parallelization when do IO. For example, if I have a thread that reads a 1GB file into memory, that thread releases the GIL while reading the file, allowing other threads to run in parallel. 3) Real threads have also as trade off context switching between threads (small but ...)
I think considering this trade offs we should switch Real Thread -> Green Thread internally or provide alternative functionality ... I think asyncio already provide functionality of GreenThread ... Okay maybe we should leave Real Threads as it is ... I need to think about it ...
But also in my email I have mentioned about `interpreters.run_string`
@Edwin Zimmerman Do you know if it is blocking call ? And if yes than I do not understand how with such API we will have TRUE Parallel execution, because in this case only one Thread at the same time could be executed, but those threads could be executed only one at the time !! Also there are drawback that we need additional thread for run sub-interpretter !! Better would be the something like this in addition `interpreters.run_string_async`, `interpreters.run_async` in addition to `interpreters.run_string`, `interpreters.run` to remove trad offs Sub-interpreters are a work in progress. The API is not anywhere near being finalized. True parallel execution will occur only if the GIL is changed to a per interpreter lock, and that requires some rather large changes to Python that haven't happened yet.
Edwin Zimmerman wrote:
On 5/25/2020 5:56 AM, redradist@gmail.com wrote:
Edwin Zimmerman wrote: Only if your workload is CPU bound. Python optimizes IO bound workload performance by releasing the GIL while doing IO. Green threads generally do not offer this option. Real threads is not needed in Python: 1) Real threads do not work parallel 2) Real threads only consume resources of OS As I said above, threads in Python provide limited parallelization when do IO. For example, if I have a thread that reads a 1GB file into memory, that thread releases the GIL while reading the file, allowing other threads to run in parallel. 3) Real threads have also as trade off context switching between threads (small but ...) I think considering this trade offs we should switch Real Thread -> Green Thread internally or provide alternative functionality ... I think asyncio already provide functionality of GreenThread ... Okay maybe we should leave Real Threads as it is ... I need to think about it ... But also in my email I have mentioned about interpreters.run_string @Edwin Zimmerman Do you know if it is blocking call ? And if yes than I do not understand how with such API we will have TRUE Parallel execution, because in this case only one Thread at the same time could be executed, but those threads could be executed only one at the time !! Also there are drawback that we need additional thread for run sub-interpretter !! Better would be the something like this in addition interpreters.run_string_async, interpreters.run_async in addition to interpreters.run_string, interpreters.run to remove trad offs
Sub-interpreters are a work in progress. The API is not anywhere near being finalized. True parallel execution will occur only if the GIL is changed to a per interpreter lock, and that requires some rather large changes to Python that haven't happened yet.
Why ? True parallel execution is possible with sub-interpreters !! `Sub-interpretter` should be run in separate thread not controlled by GIL and `sub-interpretter` will notify `main interpreter` with `atomic variable` that it is finished (like worker) !! We just need to provide two APIs: 1) Synchronized: `run`, `run_string`, that will wait until thread notify with setting `atomic variable` in true that it finished 2) Asynchronized (based on async): `run_async`, `run_string_async`, event_loop will wait on `atomic variable` reading it periodically
On Monday, May 25, 2020 redradist@gmail.com [mailto:redradist@gmail.com] wrote
Edwin Zimmerman wrote:
On 5/25/2020 5:56 AM, redradist@gmail.com wrote:
Edwin Zimmerman wrote: Sub-interpreters are a work in progress. The API is not anywhere near being finalized. True parallel execution will occur only if the GIL is changed to a per interpreter lock, and that requires some rather large changes to Python that haven't happened yet.
Why ? True parallel execution is possible with sub-interpreters !!
`Sub-interpretter` should be run in separate thread not controlled by GIL and `sub-interpretter` will notify `main interpreter` with `atomic variable` that it is finished (like worker) !! We just need to provide two APIs: 1) Synchronized: `run`, `run_string`, that will wait until thread notify with setting `atomic variable` in true that it finished 2) Asynchronized (based on async): `run_async`, `run_string_async`, event_loop will wait on `atomic variable` reading it periodically
All sub-interpreters and all threads are subject to the GIL, that is why it is called the GLOBAL Interpreter Lock. There is no such thing as a "separate thread not controlled by GIL." This is not a problem with the sub-interpreter API. It is a design decision embedded very deeply through the entire CPython code. It is not trivial to change this. However, there is work being done on this. Search the python-dev mailing list for the "PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround" thread if you want to see more. --Edwin
Edwin Zimmerman wrote:
Edwin Zimmerman wrote: On 5/25/2020 5:56 AM, redradist@gmail.com wrote: Edwin Zimmerman wrote: Sub-interpreters are a work in progress. The API is not anywhere near being finalized. True parallel execution will occur only if the GIL is changed to a per interpreter lock, and that requires some rather large changes to Python that haven't happened yet. Why ? True parallel execution is possible with sub-interpreters !! Sub-interpretter should be run in separate thread not controlled by GIL and sub-interpretter will notify main interpreter with atomic variable that it is finished (like worker) !! We just need to provide two APIs: 1) Synchronized: run, run_string, that will wait until thread notify with setting atomic variable in true that it finished 2) Asynchronized (based on async): run_async, run_string_async, event_loop will wait on atomic variable reading it periodically All sub-interpreters and all threads are subject to the GIL, that is why it is called the GLOBAL Interpreter Lock. There is no such thing as a "separate thread not controlled by GIL." This is not a problem with the sub-interpreter API. It is a design decision embedded very deeply through the entire CPython code. It is not trivial to change this. However, there is work being done on this. Search the python-dev mailing
On Monday, May 25, 2020 redradist@gmail.com [mailto:redradist@gmail.com] wrote list for the "PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround" thread if you want to see more. --Edwin
Maybe I did not know something ... but the reason that sub-interpreter was added, because of separate GIL in each sub-interpreter ? Am I right ? We should not bind all interpreters with main interpreter ... only with atomic or condition variable ... otherwise sub-interpreters is not needed at all !!
On 5/25/2020 8:29 AM, redradist@gmail.com wrote:
Edwin Zimmerman wrote:
Edwin Zimmerman wrote: On 5/25/2020 5:56 AM, redradist@gmail.com wrote: Edwin Zimmerman wrote: Sub-interpreters are a work in progress. The API is not anywhere near being finalized. True parallel execution will occur only if the GIL is changed to a per interpreter lock, and that requires some rather large changes to Python that haven't happened yet. Why ? True parallel execution is possible with sub-interpreters !! Sub-interpretter should be run in separate thread not controlled by GIL and sub-interpretter will notify main interpreter with atomic variable that it is finished (like worker) !! We just need to provide two APIs: 1) Synchronized: run, run_string, that will wait until thread notify with setting atomic variable in true that it finished 2) Asynchronized (based on async): run_async, run_string_async, event_loop will wait on atomic variable reading it periodically All sub-interpreters and all threads are subject to the GIL, that is why it is called the GLOBAL Interpreter Lock. There is no such thing as a "separate thread not controlled by GIL." This is not a problem with the sub-interpreter API. It is a design decision embedded very deeply through the entire CPython code. It is not trivial to change this. However, there is work being done on this. Search the python-dev mailing
On Monday, May 25, 2020 redradist@gmail.com [mailto:redradist@gmail.com] wrote list for the "PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround" thread if you want to see more. --Edwin Maybe I did not know something ... but the reason that sub-interpreter was added, because of separate GIL in each sub-interpreter ? No, there isn't a separate GIL for each sub-interpreter. That would be a great feature in my mind, but it isn't there.
--Edwin
Am I right ? We should not bind all interpreters with main interpreter ... only with atomic or condition variable ... otherwise sub-interpreters is not needed at all !!
On Mon, May 25, 2020 at 06:38:22PM -0400, Edwin Zimmerman wrote:
No, there isn't a separate GIL for each sub-interpreter. That would be a great feature in my mind, but it isn't there.
I believe that it is a long-term goal to eventually move to a separate GIL for each subinterpreter: https://bugs.python.org/issue24554#msg246162 but that should be considered a very long-term goal, not part of PEP 554: https://www.python.org/dev/peps/pep-0554/#id18 -- Steven
On Mon, May 25, 2020 at 11:11:14AM -0000, redradist@gmail.com wrote:
We just need to provide two APIs: 1) Synchronized: `run`, `run_string`, that will wait until thread notify with setting `atomic variable` in true that it finished 2) Asynchronized (based on async): `run_async`, `run_string_async`, event_loop will wait on `atomic variable` reading it periodically
Whenever somebody says "we just need", the word "just" can hide a huge amount of work. In this case, it's not enough to provide two APIs. We need to provide the infrastructure and implementation to make the APIs actually possible. If it was as easy as just stating the interface, it would have been done years ago. But the implementation needs to exist as well. If you still think that this is a small job that justifies the word "just", why don't you implement it and show us how simple it is? An hour or two, and it's done... *wink* -- Steven
On 26/05/2020 02:42, Steven D'Aprano wrote:
On Mon, May 25, 2020 at 11:11:14AM -0000,redradist@gmail.com wrote:
We just need to provide two APIs: 1) Synchronized: `run`, `run_string`, that will wait until thread notify with setting `atomic variable` in true that it finished 2) Asynchronized (based on async): `run_async`, `run_string_async`, event_loop will wait on `atomic variable` reading it periodically
Whenever somebody says "we just need", the word "just" can hide a huge amount of work.
Oh dear Lord yes! As a consultant programmer, I am forever pointing out to clients that the answer to any question beginning "Can you just...?" is "No." It is never "just". Never. -- Rhodri James *-* Kynesim Ltd
Yesterday RustPython team finished threading without GIL: https://github.com/RustPython/RustPython/issues/1831
On 2020-05-29 15:19, redradist@gmail.com wrote:
Yesterday RustPython team finished threading without GIL: https://github.com/RustPython/RustPython/issues/1831
That first example is Python 2...
Hello, On Fri, 29 May 2020 14:19:40 -0000 redradist@gmail.com wrote:
Yesterday RustPython team finished threading without GIL: https://github.com/RustPython/RustPython/issues/1831
So what? As was pointed out, there're bunch of Python implementations which don't use GIL. And it's very easy to stop caring about GIL, there're roughly 3 ways to deal with the problem: 1. Don't use threads, don't need to care about GIL. This is the choice of the people of 21st century. 2. Don't share data among the threads, don't have problem. Smart among us can constrain themselves with just a power of will, everyone else needs to get a separate heap per thread (or separate roots in the same heap, but that's impl detail), so they couldn't do that even they want. Oh, they could - as long as data is constant. Constness matters! This is the way which everyone and their grandma uses in the area of programming languages. Even CPython gets that in the shape of "interpreters" module. 3. Instead of GIL, have fine-grained per-object locks. That's the Java way, which stuffs a lock field into each object, even though 99.9% of them will never need it. All these 3 ways of doing concurrency were available for Python for ages, and whoever needed them, used them. GIL, and complaining about it, is strictly a paraphernalia of users of a particular implementation, CPython. [] -- Best regards, Paul mailto:pmiscml@gmail.com
-----Original Message----- From: Paul Sokolovsky <pmiscml@gmail.com> Sent: Friday, May 29, 2020 2:26 PM To: redradist@gmail.com Cc: python-ideas@python.org Subject: [Python-ideas] Re: Python GIL Thoughts
Hello,
On Fri, 29 May 2020 14:19:40 -0000 redradist@gmail.com wrote:
Yesterday RustPython team finished threading without GIL: https://github.com/RustPython/RustPython/issues/1831
So what? As was pointed out, there're bunch of Python implementations which don't use GIL. And it's very easy to stop caring about GIL, there're roughly 3 ways to deal with the problem:
1. Don't use threads, don't need to care about GIL. This is the choice of
people of 21st century.
2. Don't share data among the threads, don't have problem. Smart among us can constrain themselves with just a power of will, everyone else needs to get a separate heap per thread (or separate roots in the same heap, but that's impl detail), so they couldn't do that even they want. Oh, they could - as long as data is constant. Constness matters! This is
the the
way which everyone and their grandma uses in the area of programming languages. Even CPython gets that in the shape of "interpreters" module.
3. Instead of GIL, have fine-grained per-object locks. That's the Java way, which stuffs a lock field into each object, even though 99.9% of them will never need it.
All these 3 ways of doing concurrency were available for Python for ages, and whoever needed them, used them. GIL, and complaining about it, is strictly a paraphernalia of users of a particular implementation, CPython.
These are things that a runtime implementer has to decide on. These are also thing that a runtime use gets to complain about. "Don't share data amongst threads" is not advice to the cpython user to get truly parallel thread execution, its gaslighting them on what the GIL does and when. Its actively harmful to people who read that and don't know any better.
[]
-- Best regards, Paul mailto:pmiscml@gmail.com _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python- ideas@python.org/message/7TST6FB7PA4YJTGBUMZ5TEUPSVF3G5CD/ Code of Conduct: http://python.org/psf/codeofconduct/
Hello, On Fri, 29 May 2020 14:44:40 -0400 <tritium-list@sdamon.com> wrote: []
These are things that a runtime implementer has to decide on. These are also thing that a runtime use gets to complain about. "Don't share data amongst threads" is not advice to the cpython user to get truly parallel thread execution, its gaslighting them on what the GIL does and when. Its actively harmful to people who read that and don't know any better.
So, the first thing they should know that Python language has multiple implementations optimized for different usecases. And CPython is not the crown of the Python evolution, in a sense, it's the opposite. (Supposedly a stable base on which other implementations can build, but if falls short in many areas even regarding that.)
"Don't share data amongst threads" is not advice
Sure, it's initial assumption an implementation can make, and follow along, which eventually may lead to the removal of GIL (which was added initially as a stopgap measure, as it's indeed the simplest way to implement concurrency). Stackless Python pioneered that approach with tasklets communicating over channels in 1998, probably when Go authors didn't yet popped out from their Plan9 plans. In all fairness, I don't know if Stackless ever fully waved GIL goodbye. If not, it shows how much the Python userbase needs to get rid of GIL. And Python users surely don't need to feel bad about it, they just need to know the language history, and see where they lead it further (hanging all the hopes on CPython will likely keep things where they are). [] -- Best regards, Paul mailto:pmiscml@gmail.com
On Fri, 29 May 2020 22:18:15 +0300 Paul Sokolovsky <pmiscml@gmail.com> wrote:
In all fairness, I don't know if Stackless ever fully waved GIL goodbye. If not, it shows how much the Python userbase needs to get rid of GIL.
Yes, nevermind that Stackless was designed at a time when multicore computers were an exotic thought. And that it was a close fork of CPython, making it not better suited to remove the GIL than CPython itself. Regards Antoine.
On Fri, May 29, 2020 at 02:44:40PM -0400, tritium-list@sdamon.com wrote:
These are things that a runtime implementer has to decide on. These are also thing that a runtime use gets to complain about. "Don't share data amongst threads" is not advice to the cpython user to get truly parallel thread execution, its gaslighting them on what the GIL does and when. Its actively harmful to people who read that and don't know any better.
Please don't accuse others of gaslighting because you don't like the advice they give. Gaslighting is an expecially serious, malicious form of pyschological abuse. It doesn't mean "I disagree with your opinion", or "you're factually wrong", or even "you're lying". As for being "actively harmful", I think that the advice not to share mutable data between threads isn't harmful at all, but pretty close to essential. If you want to avoid deadlocks, livelocks and race conditions, you should avoid mutable data as much as possible. -- Steven
I've just share information, relax ;) And also it seems like they have very low overhead with atomic variables Arc<T>
participants (9)
-
Antoine Pitrou -
Chris Angelico -
Edwin Zimmerman -
MRAB -
Paul Sokolovsky -
redradist＠gmail.com -
Rhodri James -
Steven D'Aprano -
tritium-list＠sdamon.com