[New-bugs-announce] [issue35839] Suggestion: Ignore sys.modules entries with no __spec__ attribute in find_spec

Nick Coghlan report at bugs.python.org
Mon Jan 28 01:54:14 EST 2019

New submission from Nick Coghlan <ncoghlan at gmail.com>:

(Alternate proposal inspired by the discussions in #35806 and #35791)

Currently, a sys.modules entry with no __spec__ attribute will prevent importlib.util.find_spec() from locating the module, requiring the following workaround:

   def find_spec_bypassing_module_cache(modname):
       _missing = object()
       module = sys.modules.pop(modname, _missing)
           spec = importlib.util.find_spec(modname)
           if module is not _missing:
               sys.modules[modname] = module

The big downside of that approach is that it requires mutation of global state in order to work.

One of the easiest ways for this situation to be encountered is with code that replaces itself in sys.modules as a side effect of import, and doesn't bind __spec__ on the new object to the original module __spec__.

While we could take the hard line that all modules doing that need to transfer the attribute in order to be properly compatible with find_spec, I think there's a more pragmatic path we can take by differentiating between "__spec__ attribute doesn't exist" and "__spec__ attribute exists, but is None".

"__spec__ attribute doesn't exist" would be handled by find_spec as "Ignore the sys.modules entry entirely, and run the same search that would be run if the cache lookup had failed". This will then implicitly handle cases where a module replaces its own sys.modules entry.

By contrast, "__spec__ attribute is set to None" would be a true negative cache entry that indicated "this is a synthetic module that cannot be directly introspected or reloaded".

components: Library (Lib)
messages: 334446
nosy: brett.cannon, eric.snow, ncoghlan, ronaldoussoren
priority: normal
severity: normal
status: open
title: Suggestion: Ignore sys.modules entries with no __spec__ attribute in find_spec
type: enhancement

Python tracker <report at bugs.python.org>

More information about the New-bugs-announce mailing list