[Python-Dev] Dealing with import lock deadlock in Import Hooks

[Antoine Pitrou]

Le Wed, 14 Aug 2013 14:17:59 +0900,
Arnaud Fontaine <arnaud.fontaine at nexedi.com> a écrit :
> Antoine Pitrou <solipsis at pitrou.net> writes:
> 
> > Le Tue, 13 Aug 2013 17:28:42 +0900, Arnaud Fontaine
> > <arnaud.fontaine at nexedi.com> a écrit :
> >> Yes. Actually, I was thinking about implementing something similar
> >> to what has been done in Python 3.3 but for Python 2.7 with a
> >> corser-grain lock. From my understanding of import.c, it should
> >> work but I was hoping that someone with more experience in import
> >> code would confirm:
> >
> > It's probably possible, but it will be non-trivial and delicate to
> > get right.
> 
> From my understanding of import.c source code, until something is
> added to sys.modules or the code loaded, there should be no
> side-effect to releasing the lock, right? (eg there is no global
> variables/data being shared for importing modules, meaning that
> releasing the lock should be safe as long as the modules loaded
> through import hooks are protected by a lock)

Er, probably, but import.c is a nasty pile of code.
It's true the import lock is there mainly to:
- avoid incomplete modules from being seen by other threads
- avoid a module from being executed twice

But that doesn't mean it can't have had any other - unintended -
benefits ;-)

(also, some import hooks might not be thread-safe, something which they
haven't had to bother about until now)

Regards

Antoine.