Could Emacs be rewritten in Python?

[Alex Martelli]

Carl Banks wrote:

> Robin Munn wrote:
>> I suppose I just illustrated how a try ... finally block might be
>> considered "broken and fragile". I tried to use it, and missed some
>> subtleties. I'm still not convinced that a "with"-like statement is
>> necessary in Python, but I think I just went from -1 to -0 on it.
> 
> I wouldn't be so hasty.
> 
> First, you're basing it on a bad example.  Rebinding sys.stdout is bad
> programming, period.  Wanting to save and restore it is even worse.

I disagree very deeply with this statement, and don't want to leave
it unchallenged, lest impressionable readers be left with any
impression that such a statement represents a widely held opinion
among Pythonistas.  The Python Standard library, written by the
best Pythonistas around, shows several examples of how rebinding
sys.stdout (and also restoring it) can be a perfectly good Pythonic
programming practice, and so do sundry other packages authored by
great Pythonistas, such as Aaron Watters' preppy (part of ReportLab's
product line).  No doubt Mr Banks is and will remain convinced he 
knows what is or isn't bad programming, better than (for example)
Tim Peters (author of doctest.py), Guido van Rossum (author, I
believe, of fileinput.py), etc, etc -- but I trust most readers will
be able to judge for themselves on the basis of these facts.

Of course it's not an issue of "argument from authority" -- rather,
it's one of "practicality beats purity".  Say I have to change
the string 'Carl Banks' to the string 'Tim Peters' in a zillion
files, "in-place".  Right now, I write an absolutely trivial script:

import fileinput
for line in fileinput.input(inplace=1):
    print line.replace('Carl Banks','Tim Peters'),

and I just arrange to run this script with the filepaths as
arguments -- done.  And similarly for uncountable other tasks
dealing with "in-place rewriting" of files.  The reason I can
do it so handily is, of course, that module fileinput is quite
obviously indulging in what Mr Banks calls "bad programming"
(and elsewhere has also called "stupid").  Well, what to HIM
is stupid, and bad programming, to ME (and presumably to Tim
Peters, Guido van Rossum, Aaron Watters, and so on) is an
excellent example of Python's superb practicality -- I just
LOVE module fileinput, and it's one of those I use most often
and most happily.  doctest, which also uses just the same "bad
and stupid programming", is close to fileinput in my personal
ranking of most-loved, most-used modules.  And so on.


> Second, that the "rebinding" statement should come before the "try:"
> is *not* a subtlety.  It's the idiomatic way to do it, as familiar to
> those who use it as the "while 1: break" idiom.

I suspect I may have taught, helped and consulted a few more
Python programmers than Mr Banks has, and in my experience
it's a VERY common error indeed (and subtle enough to cause
substantial headscratching to locate it) to write:
    try:
        <initialize>
        <proceed>
        ...
where one MEANS (and should have written)
   <initialize>
   try:
       <proceed>
       ...
because it somehow appears to the brain that "everything" should
be in the try clause, while in fact a small number of early
"initialization" statements should be RIGHT OUTSIDE it.  I have
noticed a similar pattern of errors in the use of quite similar
constructs in Microsoft's proprietary "structured error handling"
C extensions on NT, and also in Java, but I have less experience
using / teaching / helping with / consulting about those similar
environments, so I cannot state with confidence that the problem
does indeed generalize -- that is just an impression I have.

I conceive of 'a try/finally statement' as a conceptual unit, so
_of course_, instinctively, I put the various components thereof
"inside" the (syntactical) unit -- the statement itself -- BUT...
the _initialization_ part must NOT be syntactically "inside", just
CONCEPTUALLY inside but syntactically *just outside*.  That IS a
bit tricky and thus requires careful attention each time one uses
it -- nothing major, of course, but it just doesn't "flow off the
keyboard" quite as naturally as most Python constructs do, which
makes it somewhat more error-prone.

When the initialization is several statements long one even
ends up having to NEST several try/finally statements to make
sure just the right amount of stuff is undone -- then the level
of bothersomeness becomes DEFINITELY major (one of the very few
places where C++ is "higher-level" than Python, thanks to the
use of automatic-lifetime variables whose dtors will automatically
execute, last-in first-out, iff the corresponding ctor has also
executed correctly).  E.g., interleaving two textfiles into one,
in the Classic Python way (using Python 2.3 for convenience):

    
    outfile = open('file12','w')
    try:
        for lines in itertools.izip(open('file1'),open('file2')):
            outfile.writelines(lines)
    finally:
        outfile.close()

is one thing, but if you need to ensure timely finalization
for the files open for reading as well as the one open for
writing, then (sigh)...:

    outfile = open('file12','w')
    try:
        infile1 = open('file1')
        try:
            infile2 = open('file2')
            try:
                for lines in itertools.izip(infile1, infile2):
                    outfile.writelines(lines)
            finally:
                infile2.close()
        finally:
            infile1.close()
    finally:
        outfile.close()

"flat is better than nested", and this rather ridiculous
level of nesting is something I find DEFINITELY bothersome --
the fact that the finalizations aren't readably "aligned"
with the corresponding initializations, e.g.:

            infile2 = open('file2')
                ...
                infile2.close()

doesn't help either!


In practice the "null object" idiom can help a lot here...:

class NullFile:
    def close(self): pass
outfile = infile1 = infile2 = NullFile()

try:
    outfile = open('file12','w')
    infile1 = open('file1')
    infile2 = open('file2')
    for lines in itertools.izip(infile1, infile2):
        outfile.writelines(lines)
finally:
    infile2.close()
    infile1.close()
    outfile.close()

this does a tiny amount of extra work in certain error
situations, but i find that consideration entirely trivial
compared to the very substantial simplification -- removing
the nesting, and letting everything go INSIDE the "try"
clause... just where "instinct" wants to put it.

At least, this is definitely MY personal preference for
coding try/finally statements that would otherwise need
any kind of complicated structure.  Of course, depending
on what kind of finalization operations you need, things
may not necessarily get to be quite THIS simple (restoring
the state of certain globals may not necessarily be quite
as compactly expressed as "closing a file", for example,
though one can sometimes encapsulate it, with some work).


Alex