[Python-ideas] Restitute imp.load_source or equivalent in Python 3?

[Nick Coghlan]

On 9 December 2014 at 02:10, Brett Cannon <brett at python.org> wrote:
> On Mon Dec 08 2014 at 6:13:45 AM Mart Sõmermaa <mrts.pydev at gmail.com> wrote:
>>
>> http://bugs.python.org/issue14551 discusses the deprecation and removal of
>> imp.load_source. In the discussion, there seems to be a general consensus
>> that (1) a convenience function for loading modules from an arbitrary file
>> is a good thing and (2) that many people use it.
>>
>> I can only concur, the need for loading a module from an arbitrary file
>> without having to worry about sys.path or __init__.py is useful for
>> supporting plugin-like extensibility/configurability and I personally use
>> imp.load_source in a couple of projects.
>>
>> However, the discussions in http://bugs.python.org/issue14551 seemed to
>> have died off without any clear decision for action.
>>
>> Can we consider adding such a convenience function to importlib in Python
>> 3?
>
>
> You have to realize that imp.load_source() has been fundamentally broken
> since PEP 302 and just has gotten more broken since Python 3 and importlib.
> A correct replacement for imp.load_source() in Python 3.5 would be::
>
>   # Normally a finder would get you the loader and spec.
>   loader = importlib.machinery.SourceFileLoader(module_name, path)
>   spec = importlib.machinery.ModuleSpec(module_name, loader, origin=path)
>   # Basically what import does when there is no loader.create_module().
>   module = importlib.util.module_from_spec(spec)
>   # Now is the time to put the module in sys.modules if you want.
>   # How import initializes the module.
>   loader.exec_module(module)
>
> As you can see it's not as simple as "read some bytes from a file and make
> an initialized module" which is why imp.load_source() is deprecated. Plus
> this side-steps any and all finder and loaders that may be installed which
> people want to have executed instead of this recipe. So I don't really want
> to promote people side-stepping all of this through the stdlib. Now if
> people want to do this in there own code then that's fine and I'm sure
> someone has/will create a project on PyPI to implement their own solution.
> But at least for importlib I would rather keep it as-is and lower-level and
> let the community come up with their own solutions for simplifying the
> loading of source code dynamically if they are going to side-step a huge
> chunk of the import machinery to do it.

And if the goal is "run this Python code to populate a namespace"
rather than specifically needing a module as the output, then that's
what the runpy module is intended to handle, rather than using the
import system.

runpy could use updating to be able to target an *existing* namespace
rather than a new one (http://bugs.python.org/issue19982), but that
shouldn't affect the "possible alternative to imp.load_source()" case.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia