[issue16737] Different behaviours in script run directly and via runpy.run_module

[Jason R. Coombs]

Jason R. Coombs added the comment:

I've found some other inconsistencies with the use of python -m. One is the surprising behavior when running pip:

$ touch logging.py
$ pip --help > /dev/null
$ python -m pip --help > /dev/null
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 183, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 142, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 109, in _get_module_details
    __import__(pkg_name)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/__init__.py", line 21, in <module>
    from pip._vendor.requests.packages.urllib3.exceptions import DependencyWarning
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/_vendor/requests/__init__.py", line 62, in <module>
    from .packages.urllib3.exceptions import DependencyWarning
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/_vendor/requests/packages/__init__.py", line 27, in <module>
    from . import urllib3
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/_vendor/requests/packages/urllib3/__init__.py", line 8, in <module>
    from .connectionpool import (
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/_vendor/requests/packages/urllib3/connectionpool.py", line 35, in <module>
    from .connection import (
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/_vendor/requests/packages/urllib3/connection.py", line 44, in <module>
    from .util.ssl_ import (
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/_vendor/requests/packages/urllib3/util/__init__.py", line 20, in <module>
    from .retry import Retry
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/_vendor/requests/packages/urllib3/util/retry.py", line 15, in <module>
    log = logging.getLogger(__name__)
AttributeError: module 'logging' has no attribute 'getLogger'

Obviously this is a case of don't create any files that mask stdlib or other modules that your call stack might try to import. But the takeaway is that you can't in general rely on `python -m` to launch behavior comparable to launching a script.


Additionally, this inconsistency led to a [subtle bug in pytest when launched with -m](https://github.com/pytest-dev/pytest/issues/2104). In that ticket, the recommended solution (at least thusfar) is "don't use -m". I imagine that pytest (and every other project that exposes a module for launching core behavior) could work around the issue by explicitly removing the cwd from sys.path, but that seems messy.

I imagine it could prove difficult to overcome the backward incompatibilities of changing this behavior now, so I don't have a strong recommendation, but I wanted to share these experiences and get feedback and recommendations.

----------
nosy: +jason.coombs

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16737>
_______________________________________