[Python-Dev] BDFL-Delegate appointments for several PEPs

[Terry Reedy]

On 3/25/2019 12:27 AM, Cameron Simpson wrote:
> On 24Mar2019 23:22, Terry Reedy <tjreedy at udel.edu> wrote:
>> On 3/24/2019 10:01 PM, Terry Reedy wrote:
>>> On 3/24/2019 7:00 PM, Cameron Simpson wrote:
>>>> Did you have a specific scenario in mind?
>>>
>>> I was thinking about IDLE and its tangled web of circular inports, 
>>> but I am now convinced that this change will not affect it.  Indeed, 
>>> idlelib/pyshell.py already implements idea of the proposal, ending with
>>>
>>> if __name__ == "__main__":
>>>     sys.modules['pyshell'] = sys.modules['__main__']
>>>     main()
>>
>> After more investigation, I realized that to stop having duplicate 
>> modulue:
>> 1. The alias should be 'idlelib.pyshell', not 'pyshell', at least when 
>> imports are all absolute.
> 
> The PEP499 patch effectively uses __main__.__spec__.name for the name of 
> the alias. Does that simplify your issue?
> 
> The current PR is here if you want to look at it:
> 
>     https://github.com/python/cpython/pull/12490

The new test passes on Win10.

>> 2. It should be done at the top of the file, before the import of 
>> modules that import pyshell.
> 
> Hmm, if PEP499 comes in you shouldn't need to do this at all. If PEP499 
> gets delayed or rejected I guess you're supporting this without it. Yes, 
> you'll want to do it before any other imports happen (well, as you say, 
> before any which import pyshell).
> 
> What about (untested):
> 
>     if __name__ == '__main__':
>         if __spec__.name not in sys.modules:

When I start pyshell in my master repository directory on windows with
   python -m idlelib.pyshell
__spec__.name is 'idlelib.pyshell, which I currently hard-coded.
When I start with what should be equivalent
   python f:/dev/3x/lib/idlelib/pyshell.py
__spec__ is None and __spec__.name an attribute error.

>             sys.modules[__spec__.name] = sys.modules['__main__']
> 
> as a forward compatible setup?
> 
>> If I run python f:/dev/3x/lib/idlelib/pyshell.py, the PEP patch would 
>> have to notice that pyshell is a module within idlelib and alias 
>> '__main__' to 'idlelib.pyshell', not 'pyshell'.  Would the same be 
>> true if within-package import were all relative?
> 
> I think so because we're using .__spec__.name, which I though was post 
> import name resolution.

You must be doing something different when __spec__ is None ;-).  I 
tested the patch and it does not raise AttributeError with the command 
above.

> Testing in my PEP499 branch:
> 
> Test 1:
> 
>     [~/src/cpython-cs at github(git:PEP499-cs)]fleet*> ./python.exe -i  
> Lib/idlelib/pyshell.py
>     Traceback (most recent call last):
>       File "<string>", line 1, in <module>
>     ModuleNotFoundError: No module named 'run'

This is because of an obsolete 'command = ...' around 420.  The if line 
is correct always and the if/then not needed.

>     >>> sys.modules['__main__']
>     <module '__main__' (<_frozen_importlib_external.SourceFileLoader 
> object at 0x1088e6040>)>
>     >>> sys.modules['pyshell']
>     <module '__main__' (<_frozen_importlib_external.SourceFileLoader 
> object at 0x1088e6040>)>
>     >>> sys.modules['idlelib.pyshell']
>     <module 'idlelib.pyshell' from 
> '/Users/cameron/src/cpython-cs at github/Lib/idlelib/pyshell.py'>
> 
> So pyshell and idlelib.pyshell are distinct here.

I verified that the module was being executed twice by putting 
print('running') at the top.

  __main__ and pyshell
> are the same module, courtesy of your sys.modules assignment at the 
> bottom of pyshell.py.

Obsolete and removed.

  Test 3 below will be with that commented out.
> 
> Test 2:
> 
>     [~/src/cpython-cs at github(git:PEP499-cs)]fleet*> PYTHONPATH=$PWD/Lib 
> ./python.exe -i -m idlelib.pyshell
>     Traceback (most recent call last):
>       File "<string>", line 1, in <module>
>     ModuleNotFoundError: No module named 'run'
>     >>> sys.modules['__main__']
>     <module 'idlelib.pyshell' from 
> '/Users/cameron/src/cpython-cs at github/Lib/idlelib/pyshell.py'>
>     >>> sys.modules['pyshell']
>     <module 'idlelib.pyshell' from 
> '/Users/cameron/src/cpython-cs at github/Lib/idlelib/pyshell.py'>
>     >>> sys.modules['idlelib.pyshell']
>     <module 'idlelib.pyshell' from 
> '/Users/cameron/src/cpython-cs at github/Lib/idlelib/pyshell.py'>
>     >>> id(sys.modules['__main__'])
>     4551072712
>     >>> id(sys.modules['pyshell'])
>     4551072712
>     >>> id(sys.modules['idlelib.pyshell'])
>     4551072712
> 
> So this has __main__ and idlelib.pyshell the same module from the PEP499 
> patch and pyshell also the same from your sys.modules assignment.
> 
> Test 3, with the pyshell.py sys.modules assignment commented out:
> 
>     [~/src/cpython-cs at github(git:PEP499-cs)]fleet*> PYTHONPATH=$PWD/Lib 
> ./python.exe -i -m idlelib.pyshell
>     Traceback (most recent call last):
>       File "<string>", line 1, in <module>
>     ModuleNotFoundError: No module named 'run'
>     >>> sys.modules['__main__']
>     <module 'idlelib.pyshell' from 
> '/Users/cameron/src/cpython-cs at github/Lib/idlelib/pyshell.py'>
>     >>> sys.modules['pyshell']
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>     KeyError: 'pyshell'
>     >>> sys.modules['idlelib.pyshell']
>     <module 'idlelib.pyshell' from 
> '/Users/cameron/src/cpython-cs at github/Lib/idlelib/pyshell.py'>
>     >>> id(sys.modules['__main__'])
>     4552379336
>     >>> id(sys.modules['idlelib.pyshell'])
>     4552379336
> 
> Here we've got __main__ and idlelib.pyshell the same module and no 
> 'pyshell' in sys.modules.

> I don't think I understand your "relative import" scenario.

If files other that pyshell used relative 'import ./pyshell' instead of 
absolute 'import idlelib.pyshell', would the sys.modules key still be 
'idlelib.pyshell' or 'pyshell'?   Which is to ask, would the alias 
needed to avoid a second pyshell module still be 'idlelib.pyshell' or 
'pyshell'?