[New-bugs-announce] [issue43819] ExtensionFileLoader Does Not Implement invalidate_caches

[Ian H]

New submission from Ian H <insertinterestingnamehere at gmail.com>:

Currently there's no easy way to get at the internal cache of module spec objects for compiled extension modules. See https://github.com/python/cpython/blob/20ac34772aa9805ccbf082e700f2b033291ff5d2/Python/import.c#L401-L415. For example, these module spec objects continue to be cached even if we call importlib.invalidate_caches. ExtensionFileLoader doesn't implement the corresponding method for this.

The comment in the C file referenced above implies this is done this way to avoid re-initializing extension modules. I'm not sure if this can be fixed, but I figured I'd ask for input. Our use-case is an academic project where we've been experimenting with building an interface for linker namespaces into Python to allow for (among other things) loading multiple copies of any module without explicit support from that module. We've been able to do this without having custom builds of Python. We've instead gone the route of overriding some of the import machinery at runtime. To make this work we need a way to prevent caching of previous import-related information about a specific extension module. We currently have to rely on an unfortunate hack to get access to the internal cache of module spec objects for extension modules and modify that dictionary manually. What we have works, but any sort of alternative would be welcome.

----------
messages: 390905
nosy: Ian.H
priority: normal
severity: normal
status: open
title: ExtensionFileLoader Does Not Implement invalidate_caches
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43819>
_______________________________________