[Python-ideas] Running scripts with relative imports directly, was: Re: proposal: "python -m foo"

[Nick Coghlan]

On 6 August 2015 at 09:47, Ben Finney <ben+python at benfinney.id.au> wrote:
> Paul Sokolovsky <pmiscml at gmail.com>
> writes:
>
>> I find it quite a sign of problem, because if one accepts that one
>> can't just run a script right away, but needs to do extra steps, then
>> well, that enterprisey niche is pretty crowded and there're more
>> choices how to make it more complicated.
>
> Python's BDFL has spoken of running modules with relative import as
> top-level scripts:
>
>     I'm -1 on this and on any other proposed twiddlings of the __main__
>     machinery. The only use case seems to be running scripts that happen
>     to be living inside a module's directory, which I've always seen as
>     an antipattern. To make me change my mind you'd have to convince me
>     that it isn't.
>
>     <URL:https://mail.python.org/pipermail/python-3000/2007-April/006793.html>
>
> He doesn't describe (that I can find) what makes him think it's an
> antipattern, so I'm not clear on how he expects to be convinced it's a
> valid pattern.

It's an anti-pattern because doing it fundamentally confuses the
import system's internal state:
https://www.python.org/dev/peps/pep-0395/#why-are-my-imports-broken

Relative imports from the main module just happen to be a situation
where the failure is an obvious one rather than subtle state
corruption.

> Nonetheless, that appears to be the hurdle you'd need to
> confront.

This came up more recently during the PEP 420 discussions, when the
requirement to include __init__.py to explicitly mark package
directories was eliminated. This means there's no longer any way for
the interpreter to reliably infer from the filesystem layout precisely
where in the module hierarchy you intended a module to live. See
https://www.python.org/dev/peps/pep-0420/#discussion for references.

However, one of the subproposals from PEP 395 still offers a potential
fix: https://www.python.org/dev/peps/pep-0395/#id24

That proposes to allow explicit relative imports at the command line,
such that Paul's example could be correctly invoked as:

    python3 -m ..pkg.foo

It would also be possible to provide a syntactic shorthand for
submodules of top level packages:

    python3 -m .foo

The key here is that the interpreter is being explicitly told that the
current directory is inside a package, as well as how far down in the
package hierarchy it lives, and can adjust the way it sets sys.path[0]
accordingly before proceeding on to import "pkg.foo" as __main__.

That should be a relatively uncomplicated addition to
runpy._run_module_as_main that could be rolled into Cameron's PEP
plans. Steps required:

* count leading dots in the supplied mod_name
* remove "leading_dots-1" trailing directory names from sys.path[0]
* strip the leading dots from mod_name before continuing with the rest
of the function
* in the special case of only 1 leading dot, remove the final
directory segment from sys.path[0] and prepend it to mod_name with a
dot separator

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia