[Python-Dev] Impact of Namedtuple on startup time

[Antoine Pitrou]

On Tue, 18 Jul 2017 22:13:01 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> As another example of this: while trading the global import lock for
> per-module locks eliminated most of the old import deadlocks, it turns
> out that it *also* left us with some fairly messy race conditions and
> more fragile code (I still count that particular case as a win
> overall, but it definitely raises the barrier to entry for maintaining
> that code).
> 
> Unfortunately, these are frequently cases where the benefits are
> immediately visible (e.g. faster benchmark results, removing
> longstanding limitations on user code), but the downsides can
> literally take years to make themselves felt (e.g. higher defect rates
> in the interpreter, subtle bugs in previously correct user code that
> are eventually traced back to interpreter changes).

The import deadlocks were really in the category of "subtle bugs" that
only occur in certain timing conditions (especially when combined with
PyImport_ImportModuleNoBlock and/or stdlib modules which can try to
import stuff silently, such as the codecs module). So we traded a
category of "subtle bugs" due to a core design deficiency for another
category of "subtle bugs" due to an imperfect implementation, the
latter being actually fixable incrementally :-)

Disclaimer: I wrote the initial per-module lock implementation, which
was motivated by those long-standing "subtle bugs" in multi-threaded
applications.

Regards

Antoine.