[Python-Dev] PEP 3148 ready for pronouncement

[Brian Quinlan]

On 26 May 2010, at 22:06, Floris Bruynooghe wrote:

> Hi
>
> On Sun, May 23, 2010 at 10:47:08AM +1000, Brian Quinlan wrote:
>> Jesse, the designated pronouncer for this PEP, has decided to keep
>> discussion open for a few more days.
>>
>> So fire away!
>
> In thread.py the module automatically registers a handler with atexit.
> I don't think I'm alone in thinking libraries should not be doing this
> sort of thing unconditionally behind a user's back.  I'm also not so
> sure how comfortable I am with the module-level globals.
>
> Would it not be possible to have an exit handler on each thread pool
> which the documentation reccomends you register with atexit if it
> suits your application?  I think that would get rid of the global
> singletons and hidden atexit in a fairly elegant way.

First let me explain why I install at atexit handler.

Imagine that the you write a script like this:
t = ThreadPoolExecutor(1)
t.submit(lambda url: print(urllib.open(url).read()), 'http://www.apple.com/')

You have two semantic choices here:
1. let the interpreter exit with the future still running
2. wait until the future finishes and then exit

I chose (2) but can be convinced otherwise. The obvious way to  
accomplish this is to make the worker thread non-daemon so the  
interpreter won't exit while it is running. But since the worker  
thread is part of a pool, it won't stop while it's executor is alive.

So my approach was to make worker threads daemon and install an atexit  
handler that sets a global indicating that the interpreter is exiting  
so any workers should exit when when their work queues are empty. It  
then calls join on each worker thread so the interpreter will not exit  
until they are finished.

I think that this approach is reasonable assuming that you want (2). I  
also don't have the aversion to globals that you do :-)

> Lastly _base.py creates a LOGGER (connected to sys.stderr if I
> understand correctly) and only logs a critical message to it at the
> same time as a RuntimeError is raised.  While I don't necessarily
> dislike that it uses a logger, I don't like that it's wired up to
> sys.stderr I rather think it's the application's duty to create a
> handler if it wants one.  But given that it's only used at the same
> time as a RuntimeError it does seem redundant.

The LOGGER is only use for "impossible" exceptions (exceptions in the  
machinery of the module itself) that won't be propagated because they  
occur in worker threads.

Cheers,
Brian


> Regards
> Floris
>
> PS: I've only looked at the threading part of the implementation.
>
> -- 
> Debian GNU/Linux -- The Power of Freedom
> www.debian.org | www.gnu.org | www.kernel.org
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brian%40sweetapp.com