[New-bugs-announce] [issue19695] Clarify how to use various import-related locks

[Brett Cannon]

New submission from Brett Cannon:

While working on the PEP 451 code I realized that the way we are handling the global import lock along with the per-module lock is a little ad-hoc. For instance, what are we wanting to use the global lock for these days? Mutations of anything off of sys? Reading anything from sys as well? Just to create the module-level locks?

And the per-module lock expects the global lock to be held already, but then does a release implicitly. That seems like the wrong way to structure the context managers; maybe pass in the lock global lock as proof it's being held? Or if we switch to much finer granularity for dealing with sys.modules (especially if it is only for mutation) then we can lock and unlock the global lock entirely within the per-module lock context manager.

At worst I think we should clearly document in the docstrings for the global and per-module context managers what we expect the lock to be used for and then really go through the import code to make sure we are holding it where we want but no more.

To start the conversation, I say the global lock is just to get the per-module locks. The per-module locks are held when finding/loading modules to prevent threading issues of requesting an incomplete module or thinking it isn't being imported when it actually is (which implicitly means holding it when accessing/mutating sys.modules for any module, e.g. trying to fetch a parent module).

----------
components: Interpreter Core
messages: 203783
nosy: brett.cannon, eric.snow, ncoghlan, pitrou
priority: low
severity: normal
stage: needs patch
status: open
title: Clarify how to use various import-related locks
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue19695>
_______________________________________