Re: [Python-Dev] Threading idea -- exposing a global thread lock
Josiah Carlson writes:
It would be nice if Jython or IronPython could (and would) implement these 'critical sections'. Whether they can or not, I think that it would be a useful feature in the CPython runtime.
The issue is not whether Jython and IronPython "will", it's whether they "can". Essentially, Jython and IronPython both use locking within their fundamental data structures. This then allows them to freely allow threads to run on multiple processors. Meanwhile, CPython lacks locks in its fundamental data structures, so it uses the GIL to ensure that code which might touch Python data structures executes on only one CPU at a time. The concept of a "critical section" makes great sense when there is effectively only one CPU: just stop switching threads. But if code is using multiple CPUs, what does it mean? Shut down the other CPUs? To do such a thing cooperatively would require checking some master lock at every step... (a price which CPython pays, but which the implementations built on good thread-supporting VMs don't have to). To do such a thing non-cooperatively is not supported in either VM. Since I doubt we're going to convince Sun or Microsoft to change their approach to threading, I think it is unwise to build such a feature into the Python language. Supporting it in CPython only requires (I think) no more than a very simple C extension. I think it should stay as an extension and not become part of the language. -- Michael Chermside
[Nice analysis from Michael Chermside]
The concept of a "critical section" makes great sense when there is effectively only one CPU: just stop switching threads. But if code is using multiple CPUs, what does it mean? Shut down the other CPUs? . . . I think it is unwise to build such a feature into the Python language. Supporting it in CPython only requires (I think) no more than a very simple C extension. I think it should stay as an extension and not become part of the language.
That makes sense. Once place where we already have CPython specific support is in sys.setcheckinterval(). That suggests adapting that function or adding a new one to temporarily stop switching, almost the same as sys.setcheckinterval(sys.maxint) but continuing to perform other periodic checks for control-break and such. Raymond
On 3/14/06, Raymond Hettinger <raymond.hettinger@verizon.net> wrote:
Once place where we already have CPython specific support is in sys.setcheckinterval(). That suggests adapting that function or adding a new one to temporarily stop switching, almost the same as sys.setcheckinterval(sys.maxint) but continuing to perform other periodic checks for control-break and such.
Don't forget that sys.setcheckinterval() is more of a hint than a requirement. It's easy to wrap sys.setcheckinterval() in a try/except or an if-hasattr, and just ignore the case where it doesn't exist ('it won't be necessary'). Doing the same thing with a 'critical section' would be a lot harder. I would also assume a 'critical section' should not allow threads in extension modules, even if they explicitly allow threads. That's quite a bit different from making the check-interval infinitely high for the duration of the block. -- Thomas Wouters <thomas@python.org> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
Raymond Hettinger wrote:
Once place where we already have CPython specific support is in sys.setcheckinterval(). That suggests adapting that function or adding a new one to temporarily stop switching, almost the same as sys.setcheckinterval(sys.maxint) but continuing to perform other periodic checks for control-break and such.
I object to the notion that Python does "thread switching". It doesn't. Instead, it releases the GIL under certain circumstances, which might make the operating system switch threads. Whether the operating system does that is its own choice. I don't see a value to disabling the "release the GIL from time to time". This does *not* mean there couldn't be thread-switching, anymore. E.g. inside a PRINT_* opcode, thread-switching may still occur, as file_write releases the GIL around the fwrite() call. So if you set the checkinterval to "no check", you cannot trust that there won't be any thread switching. Regards, Martin
participants (4)
-
"Martin v. Löwis" -
Michael Chermside -
Raymond Hettinger -
Thomas Wouters