On 25Mar2019 03:52, Terry Reedy firstname.lastname@example.org wrote:
On 3/25/2019 12:27 AM, Cameron Simpson wrote:
I was thinking about IDLE and its tangled web of circular inports, but I am now convinced that this change will not affect it. Indeed, idlelib/pyshell.py already implements idea of the proposal, ending with
if __name__ == "__main__": sys.modules['pyshell'] = sys.modules['__main__'] main()
After more investigation, I realized that to stop having duplicate modulue:
- The alias should be 'idlelib.pyshell', not 'pyshell', at least
when imports are all absolute.
The PEP499 patch effectively uses __main__.__spec__.name for the name of the alias. Does that simplify your issue?
What about (untested):
if __name__ == '__main__': if __spec__.name not in sys.modules: sys.modules[__spec__.name] = sys.modules['__main__']
When I start pyshell in my master repository directory on windows with python -m idlelib.pyshell __spec__.name is 'idlelib.pyshell, which I currently hard-coded. When I start with what should be equivalent python f:/dev/3x/lib/idlelib/pyshell.py __spec__ is None and __spec__.name an attribute error.
Um, yes. I presume that since no "import" has been done, there's no import spec (.__spec__).
Clearly the above needs to accomodate this, possibly with a fallback guess. Is sniffing the end components of __file__ at all sane? ending in idlelib/pyshell.py or pyshell.py? Or is that just getting baroque?
I don't think these are strictly the same from some kind of purist viewpoint: the path might be anything - _is_ it reasonable to suppose that it has a module name (== importable/finding through the import path)?
If I run python f:/dev/3x/lib/idlelib/pyshell.py, the PEP patch would have to notice that pyshell is a module within idlelib and alias '__main__' to 'idlelib.pyshell', not 'pyshell'. Would the same be true if within-package import were all relative?
I think so because we're using .__spec__.name, which I though was post import name resolution.
You must be doing something different when __spec__ is None ;-). I tested the patch and it does not raise AttributeError with the command above.
Indeed. I may have fudged a bit when I said "The PEP499 patch effectively uses __main__.__spec__.name". It modifies runpy.py's _run_module_as_main function, and that is called for the "python -m module_name" invocation, so it can get the module spec because it has a module name.
So the patch doesn't have to cope with __spec__ being None.
As you say, __spec__ is None for "python path/to/file.py" so __spec__ isn't any use there. Apologies.
Test 3, with the pyshell.py sys.modules assignment commented out:
[~/src/cpython-cs@github(git:PEP499-cs)]fleet*> PYTHONPATH=$PWD/Lib ./python.exe -i -m idlelib.pyshell
>>> sys.modules['__main__'] <module 'idlelib.pyshell' from '/Users/cameron/src/cpython-cs@github/Lib/idlelib/pyshell.py'> >>> sys.modules['pyshell'] Traceback (most recent call last): File "<stdin>", line 1, in <module> KeyError: 'pyshell' >>> sys.modules['idlelib.pyshell'] <module 'idlelib.pyshell' from '/Users/cameron/src/cpython-cs@github/Lib/idlelib/pyshell.py'> >>> id(sys.modules['__main__']) 4552379336 >>> id(sys.modules['idlelib.pyshell']) 4552379336
Here we've got __main__ and idlelib.pyshell the same module and no 'pyshell' in sys.modules.
I don't think I understand your "relative import" scenario.
If files other that pyshell used relative 'import ./pyshell' instead of absolute 'import idlelib.pyshell', would the sys.modules key still be 'idlelib.pyshell' or 'pyshell'? Which is to ask, would the alias needed to avoid a second pyshell module still be 'idlelib.pyshell' or 'pyshell'?
As I understand it Python 3 imports are absolute: without a leading dot a name is absolute, so "import pyshell" should install sys.module['pyshell'] _provided_ that 'pyshell' can be found in the module search path.
Conversely, an "import .pyshell" is an import relative to the current module's package name, equivalent to an import of the absolute path "package.name.pyshell", for whatever the package name is. So (a) you can only import '.pyshell' from within a package containing a 'pyshell.py' file and (b) you can't import import '.pyshell' if you're not in a package.
I stuffed a "test2.py" into the local idlelib like this:
import sys print("running", __file__, __name__) print(repr(sorted(sys.modules))) print(repr(sys.paht)) from pyshell import idle_showwarning print(repr(sorted(sys.modules)))
and fiddled with the "from pyshell import idle_showwarning" line.
(I'm presuming this is what you have in mind, since "import ./pyshell" elicits a syntax error.)
Using "./python.exe -m idlelib.test2":
Plain "pyshell" gets an ImportError - no such module.
Using ".pyshell" imports the pyshell module as "idlelib.pyshell" in sys.modules.
Which was encouraging until I went "./python.exe Lib/idlelib/test2.py". This puts Lib/idlelib (as an absolute path) at the start of sys.path.
A plain "pyshell" import works and installs sys.modules['pyshell'].
Conversely, trying the ".pyshell" import gets:
ModuleNotFoundError: No module named '__main__.pyshell'; '__main__' is not a package
So we can get 'pyshell' or 'idlelib.pyshell' into sys.modules depending how we invoke python.
HOWEVER, if you're importing the 'pyshell' from idlelib _as found in the module search path_, whether absolutely as 'idlelib.pyshell' or relatives as '.pyshell' from within the idlelib package, you should always get 'idlelib.pyshell' in the sys.modules map.
And I don't think you should need to worry about a circular import importing some top level name 'pyshell' because that's not using the idlelib package, so I'd argue it isn't your problem.
Cheers, Cameron Simpson email@example.com