On 9 December 2014 at 02:10, Brett Cannon <brett@python.org> wrote:

On Mon Dec 08 2014 at 6:13:45 AM Mart Sõmermaa <mrts.pydev@gmail.com> wrote:

...

http://bugs.python.org/issue14551 discusses the deprecation and removal of imp.load_source. In the discussion, there seems to be a general consensus that (1) a convenience function for loading modules from an arbitrary file is a good thing and (2) that many people use it.

I can only concur, the need for loading a module from an arbitrary file without having to worry about sys.path or __init__.py is useful for supporting plugin-like extensibility/configurability and I personally use imp.load_source in a couple of projects.

However, the discussions in http://bugs.python.org/issue14551 seemed to have died off without any clear decision for action.

Can we consider adding such a convenience function to importlib in Python 3?

You have to realize that imp.load_source() has been fundamentally broken since PEP 302 and just has gotten more broken since Python 3 and importlib. A correct replacement for imp.load_source() in Python 3.5 would be::

# Normally a finder would get you the loader and spec. loader = importlib.machinery.SourceFileLoader(module_name, path) spec = importlib.machinery.ModuleSpec(module_name, loader, origin=path) # Basically what import does when there is no loader.create_module(). module = importlib.util.module_from_spec(spec) # Now is the time to put the module in sys.modules if you want. # How import initializes the module. loader.exec_module(module)

As you can see it's not as simple as "read some bytes from a file and make an initialized module" which is why imp.load_source() is deprecated. Plus this side-steps any and all finder and loaders that may be installed which people want to have executed instead of this recipe. So I don't really want to promote people side-stepping all of this through the stdlib. Now if people want to do this in there own code then that's fine and I'm sure someone has/will create a project on PyPI to implement their own solution. But at least for importlib I would rather keep it as-is and lower-level and let the community come up with their own solutions for simplifying the loading of source code dynamically if they are going to side-step a huge chunk of the import machinery to do it.

And if the goal is "run this Python code to populate a namespace" rather than specifically needing a module as the output, then that's what the runpy module is intended to handle, rather than using the import system. runpy could use updating to be able to target an *existing* namespace rather than a new one (http://bugs.python.org/issue19982), but that shouldn't affect the "possible alternative to imp.load_source()" case. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 9 December 2014 at 02:10, Brett Cannon <brett@python.org> wrote:

On Mon Dec 08 2014 at 6:13:45 AM Mart Sõmermaa <mrts.pydev@gmail.com> wrote:

...

http://bugs.python.org/issue14551 discusses the deprecation and removal of imp.load_source. In the discussion, there seems to be a general consensus that (1) a convenience function for loading modules from an arbitrary file is a good thing and (2) that many people use it.

I can only concur, the need for loading a module from an arbitrary file without having to worry about sys.path or __init__.py is useful for supporting plugin-like extensibility/configurability and I personally use imp.load_source in a couple of projects.

However, the discussions in http://bugs.python.org/issue14551 seemed to have died off without any clear decision for action.

Can we consider adding such a convenience function to importlib in Python 3?

You have to realize that imp.load_source() has been fundamentally broken since PEP 302 and just has gotten more broken since Python 3 and importlib. A correct replacement for imp.load_source() in Python 3.5 would be::

# Normally a finder would get you the loader and spec. loader = importlib.machinery.SourceFileLoader(module_name, path) spec = importlib.machinery.ModuleSpec(module_name, loader, origin=path) # Basically what import does when there is no loader.create_module(). module = importlib.util.module_from_spec(spec) # Now is the time to put the module in sys.modules if you want. # How import initializes the module. loader.exec_module(module)

As you can see it's not as simple as "read some bytes from a file and make an initialized module" which is why imp.load_source() is deprecated. Plus this side-steps any and all finder and loaders that may be installed which people want to have executed instead of this recipe. So I don't really want to promote people side-stepping all of this through the stdlib. Now if people want to do this in there own code then that's fine and I'm sure someone has/will create a project on PyPI to implement their own solution. But at least for importlib I would rather keep it as-is and lower-level and let the community come up with their own solutions for simplifying the loading of source code dynamically if they are going to side-step a huge chunk of the import machinery to do it.

And if the goal is "run this Python code to populate a namespace" rather than specifically needing a module as the output, then that's what the runpy module is intended to handle, rather than using the import system. runpy could use updating to be able to target an *existing* namespace rather than a new one (http://bugs.python.org/issue19982), but that shouldn't affect the "possible alternative to imp.load_source()" case. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia