[Python-ideas] Pseudo-package for current directory

[Steven D'Aprano]

On Tue, Mar 08, 2016 at 06:34:11AM +1100, Chris Angelico wrote:

[...]
> >> So the solution is to treat the current directory as a pseudo-package.
> >> It'd be a backward-incompatible change, so it would need to be
> >> explicitly invoked. Something like:
> >>
> >> python3 -p somefile.py
> >>
> >> which would pretend to create an __init__.py in the current directory,
> >> change to the parent, and "from dirname import somefile".
[...]

> > Even if it works, it relies on the user being knowledgeable enough to
> > run `python3 -p script` instead of `python3 script`, which means the
> > people who most need this new feature are the least likely to use it. I
> > think that's the fatal objection to this: if the user knows enough to
> > run -p, she knows enough to diagnose an accidental shadowing.
> 
> Right, this is a concern. But I don't want to break backward
> compatibility. Otherwise, the common situation of having multiple .py
> files in the same directory and having them import each other would be
> broken.

Only if they shadow something in the std lib. However we fix this, 
whatever the mechanism, it is going to break backwards compatibility for 
those shadowing the stdlib, which is of course the whole point of the 
exercise.

I don't think it's worth trying to gradually introduce this: those 
affected will either want this change or be able to work around it 
easily. The work around doesn't involve going through your entire code 
base changing (potentially) thousands of lines of code. It likely 
involves changing the name of one or two files, or explicitly adding "." 
to the front of your path. So even though it is a technical break of 
backwards compatibility, I think it is (1) minor enough and (2) 
important enough to warrant just doing it.

There are three cases where people would notice a difference:

(1) Deliberate shadowing of the std lib. In this case, if you are 
intentionally shadowing a module, I think you are probably capable of 
coming up with a work-around for the change. E.g. you might explicitly 
add '.' to your PYTHONPATH environment variable to force it to the 
front, so as to get the current behaviour.

I think that's a rare case and changing the default behaviour here is 
acceptible.


(2) Accidental shadowing of the std lib where they currently experience 
mysterious breakage. E.g.:

[steve at ando tmp]$ touch time.py
[steve at ando tmp]$ idle
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python2.7/idlelib/run.py", line 7, in <module>
    import threading
  File "/usr/local/lib/python2.7/threading.py", line 13, in <module>
    from time import time as _time, sleep as _sleep
ImportError: cannot import name time


This will be fixed, which is the whole purpose of the exercise.


(3) Accidental shadowing where they *don't* currently experience 
breakage, but will after the change.

E.g. suppose I have a main script "spam.py" and a file called "code.py" 
in the same directory, and spam.py imports code. Currently, that shadows 
the stdlib code.py, but harmlessly since few (if any) things depend on 
the code module. After the change, the stdlib code.py will now shadow 
the custom one, causing breakage.

I suspect that this will be rare in well-established or professional 
code, where the devs are more familiar with the std lib, but a bit more 
common in the case of beginners. This is unfortunate, but the fix is 
simple: rename your file and change a few imports: `import code` becomes 
`import mycode as code`.


-- 
Steve