[Python-ideas] Idea: Importing from arbitrary filenames

[Nick Coghlan]

On 16 April 2018 at 03:45, Steve Barnes <gadgetsteve at live.co.uk> wrote:
> On 15/04/2018 08:12, Nick Coghlan wrote:
>> The discoverability of these kinds of techniques could definitely
>> stand to be improved, but the benefit of adopting them is that they
>> work on all currently supported versions of Python (even
>> importlib.import_module exists in Python 2.7 as a convenience wrapper
>> around __import__), rather than needing to wait for new language level
>> syntax for them.
>
> As you say not too discoverable at the moment - I have just reread
> PEP328 & https://docs.python.org/3/library/importlib.html but did not
> find any mention of these mechanisms or even that setting an external
> __path__ variable existed as a possibility.

Yeah, the fact that "packages are ultimately just modules with a
__path__ attribute that works like sys.path" tends to get obscured by
the close association between package hierarchies and file system
layouts in the default filesystem importer.

The docs for that are all the way back in PEP 302:
https://www.python.org/dev/peps/pep-0302/#packages-and-the-role-of-path

> Maybe a documentation enhancement proposal would be in order?

If we're not covering explicit __path__ manipulation anywhere, we
should definitely mention that possibility.
https://docs.python.org/3/library/pkgutil.html#pkgutil.extend_path
does talk about it, but only in the context of scanning sys.path for
matching names, not in the context of building a package from an
arbitrary set of directory names.

I'm not sure where we could put an explanation of some of the broader
implications of that fact, though - while __path__ manipulation is
usually fairly safe, we're always a little hesitant about encouraging
too many dynamic modifications to the import system state, since it
can sometimes have odd side effects based on whether imports happen
before or after that state is adjusted..

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia