[Python-ideas] [Python-Dev] If you shadow a module in the standard library that IDLE depends on, bad things happen
Steven D'Aprano
steve at pearwood.info
Sun Nov 1 10:44:10 EST 2015
On Sun, Nov 01, 2015 at 08:53:17AM +0100, Laura Creighton wrote:
> In a message of Sun, 01 Nov 2015 17:06:30 +1100, "Steven D'Aprano" writes:
> >Scripts which deliberately or unintentionally shadow installed packages
> >will break from this change. I don't have a problem with this. You can't
> >fix harmful behaviour without breaking code that depends on that harmful
> >behaviour.
>
> This is a bad idea, if you mean 'shadows anything in site-packages'.
> I write a perfectly good working program, which then silently breaks
> because somebody happens to install a site package with a name
> conflict with my code.
I'm willing to consider that "" should appear in the middle of sys.path:
[standard-library-directories, "", site-package-directories]
but that still allows accidental shadowing of third-party packages. I
don't think that the problem of accidently shadowing numpy is less
deserving of a solution than accidently shadowing random. They're both
problems.
The downside of using a search path, instead of explicit full pathnames,
is that you can have conflicts where two or more modules have the same
name, and only the first can be imported. That's unavoidable. We can
only try to minimize the risk of accidental shadowing, not prevent it
altogether.
> Can you imagine being in the middle of
> writing and debugging such a thing and have everything start failing
> because suddenly your program isn't the one being found? How long is
> it going to take you to stop looking at your own code, and your own
> setup for the problem and begin looking at what packages got installed
> by anybody else sharing this machine, and what they are named?
Now just a minute, I'm not proposing that this change occur at random,
in the middle of a debugging session. Not even in a point (micro)
release. So the code will only break (if it breaks) when you upgrade
from version 3.x to 3.6 or higher. Moving to a new minor version number
is a big step, and people expect that things *could* break.
(They don't necessarily expect that they *will* break, but it's
certainly a possibility that they might.)
As an experienced developer, what do you do when code that works in one
version stops working in the next? Everyone I know immediately assumes
that something has changed in the new Python version, and proceeds from
there. In this case, they'll be right. Hopefully they will read the
release notes. Or the documentation. Or ask on StackOverflow.
The same applies for when you install a new package: "Everything worked
fine yesterday." "What changed?" "Well, I installed a new package with
the same name as one of my modules." "Well there you go."
Of course we can come up with scenarios that are more confusing. Suppose
you have a program that you don't use very often, say, once a year, that
relies on "" being at the front of the path, and then somebody who isn't
you installs a site-wide third-party package that now shadows one of
your modules, and you don't find out about for six months.
But this sort of thing can already happen: any time you add a
third-party module, it may shadow *other* third party modules that are
found later in the path. So I'm not creating a brand new failure mode,
I'm just changing how it applies a little.
There are all sorts of debugging strategies available to an experienced
developer, starting from "print out module.__file__" to "use the
debugger", which a beginner who has just written "turtle.py" would never
think of. If I'm going to break anyone's code, I'd rather it be yours
than theirs, since I have every confidence you can debug any problems
fairly quickly.
I don't suggest breaking working code lightly, but the issue of
shadowing the stdlib is an ever-present millstone around every Python
developer's neck. The stdlib alone, never mind site packages, is too big
to expect everyone to memorise what names they should not use. We can't
realistically expect people to avoid shadowing. But we can shift it from
"easy to shadow, and mostly affect those who are least able to cope" to
"harder to shadow, and mostly affect those who can cope".
--
Steve
More information about the Python-ideas
mailing list