[Python-Dev] Threading idea -- exposing a global thread lock

[Raymond Hettinger]

[Guido]
> Oh, no!

Before shooting this one down, consider a simpler incarnation not involving the 
GIL.  The idea is to allow an active thread to temporarily suspend switching for 
a few steps:

threading.stop_switching()
# step1
# step2
# setp3
theading.resume_switching()

To me, this version is innocuous -- it doesn't violate any existing guarantees 
(no thread knows when it is going to be switched out or how long it is going to 
be suspended) and allows the user to easily create an atomic transaction (i.e. 
swapping two global variables or printing a debug value).

While it could be abused, it has the offsetting virtue of being much simpler 
than the alternatives.

I disagree that the need is rare.  My own use case is that I sometimes add some 
debugging print statements that need to execute atomically -- it is a PITA 
because PRINT_ITEM and PRINT_NEWLINE are two different opcodes and are not 
guaranteed to pair atomically.  The current RightWay(tm) is for me to create a 
separate daemon thread for printing and to send lines to it via the queue module 
(even that is tricky because you don't want the main thread to exit before a 
print queued item is completed).  I suggest that that is too complex for a 
simple debugging print statement.  It would be great to simply write:

    with other_threads_suspended():
        print value

I haven't worked through the IronPython and Jython points of view but think 
there is probably a way to implement the simpler incarnation.

Also, unlike the gil.aquire() version, there is no blocking of the currently 
active thread.  IOW, the above incarnation won't deadlock (because the main 
thread never goes into a wait mode).


Raymond






----- Original Message ----- 
From: "Guido van Rossum" <guido at python.org>
To: "Raymond Hettinger" <python at rcn.com>
Cc: <python-dev at python.org>
Sent: Tuesday, March 14, 2006 12:06 AM
Subject: Re: [Python-Dev] Threading idea -- exposing a global thread lock


Oh, no! Please!

I just had to dissuade someone inside Google from the same idea.

IMO it's fatally flawed for several reasons: it doesn't translate
reasonably to Jython or IronPython, it's really tricky to implement,
and it's an invitation for deadlocks. The danger of this thing in the
wrong hands is too big to warrant the (rare) use case that can only be
solved elegantly using direct GIL access.

--Guido

On 3/13/06, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:
> A user on comp.lang.python has twisted himself into knots writing 
> multi-threaded
> code that avoids locks and queues but fails when running code with non-atomic
> access to a shared resource. While his specific design is somewhat flawed, it
> does suggest that we could offer an easy way to make a block of code atomic
> without the complexity of other synchronization tools:
>
>    gil.acquire()
>    try:
>       #do some transaction that needs to be atomic
>    finally:
>       gil.release()
>
> The idea is to temporarily suspend thread switches (either using the GIL or a
> global variable in the eval-loop).  Think of it as "non-cooperative"
> multi-threading. While this is a somewhat rough approach, it is dramatically
> simpler than the alternatives (i.e. wrapping locks around every access to a
> resource or feeding all resource requests to a separate thread via a Queue).
>
> While I haven't tried it yet, I think the implementation is likely to be
> trivial.
>
> FWIW, the new with-statement makes the above fragment even more readable:
>
>     with atomic_transaction():
>         # do a series of steps without interruption
>
>
> Raymond
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


--
--Guido van Rossum (home page: http://www.python.org/~guido/)