How to guard against bugs like this one?

[Dan Stromberg]

kj wrote:
> I just spent about 1-1/2 hours tracking down a bug.
>
> An innocuous little script, let's call it buggy.py, only 10 lines
> long, and whose output should have been, at most two lines, was
> quickly dumping tens of megabytes of non-printable characters to
> my screen (aka gobbledygook), and in the process was messing up my
> terminal *royally*.  Here's buggy.py:
>
>
>
> import sys
> import psycopg2
> connection_params = "dbname='%s' user='%s' password='%s'" % tuple(sys.argv[1:])
> conn = psycopg2.connect(connection_params)
> cur = conn.cursor()
> cur.execute('SELECT * FROM version;')
> print '\n'.join(x[-1] for x in cur.fetchall())
>
>
> (Of course, buggy.py is pretty useless; I reduced the original,
> more useful, script to this to help me debug it.)
>
> Through a *lot* of trial an error I finally discovered that the
> root cause of the problem was the fact that, in the same directory
> as buggy.py, there is *another* innocuous little script, totally
> unrelated, whose name happens to be numbers.py.  (This second script
> is one I wrote as part of a little Python tutorial I put together
> months ago, and is not much more of a script than hello_world.py;
> it's baby-steps for the absolute beginner.  But apparently, it has
> a killer name!  I had completely forgotten about it.)
>
> Both scripts live in a directory filled with *hundreds* little
> one-off scripts like the two of them.  I'll call this directory
> myscripts in what follows. 
>
> It turns out that buggy.py imports psycopg2, as you can see, and
> apparently psycopg2 (or something imported by psycopg2) tries to
> import some standard Python module called numbers; instead it ends
> up importing the innocent myscript/numbers.py, resulting in *absolute
> mayhem*.
>
> (This is no mere Python "wart"; this is a suppurating chancre, and
> the fact that it remains unfixed is a neverending source of puzzlement
> for me.)
>
> How can the average Python programmer guard against this sort of
> time-devouring bug in the future (while remaining a Python programmer)?
> The only solution I can think of is to avoid like the plague the
> basenames of all the 200 or so /usr/lib/pythonX.XX/xyz.py{,c} files,
> and *pray* that whatever name one chooses for one's script does
> not suddenly pop up in the appropriate /usr/lib/pythonX.XX directory
> of a future release.
>
> What else can one do?  Let's see, one should put every script in its
> own directory, thereby containing the damage.
>
> Anything else?
>
> Any suggestion would be appreciated.
>
> TIA!
>
> ~k
>   
Here's a pretty simple fix that should work in about any version of 
python available:

Put modules in ~/lib.  Put scripts in ~/bin.  Your modules end with 
.py.  Your scripts don't.  Your scripts add ~/lib to sys.path as 
needed.  Things that go in ~/lib are named carefully.  Things in ~/bin 
also need to be named carefully, but for an entirely different reason - 
if you name something "ls", you may get into trouble.

Then things in ~/lib plainly could cause issues.  Things in ~/bin don't.

Ending everything with .py seems to come from the perl tradition of 
ending everything with .pl.  This perl tradition appears to have come 
from perl advocates wanting everyone to know (by looking at a URL) that 
they are using a perl CGI.  IMO, it's language vanity, and best 
dispensed with - aside from this issue, it also keeps you from rewriting 
your program in another language with an identical interface.

This does, however, appear to be a scary issue from a security 
standpoint.  I certainly hope that scripts running as root don't search 
"." for modules.