Python's "only one way to do it" philosophy isn't good?

[Chris Mellon]

On 6/27/07, Douglas Alan <doug at alum.mit.edu> wrote:
> "Chris Mellon" <arkanes at gmail.com> writes:
>
> > On 6/27/07, Douglas Alan <doug at alum.mit.edu> wrote:
>
> >> The C++ folks feel so strongly about this, that they refuse to provide
> >> "finally", and insist instead that you use destructors and RAII to do
> >> resource deallocation.  Personally, I think that's taking things a bit
> >> too far, but I'd rather it be that way than lose the usefulness of
> >> destructors and have to use "when" or "finally" to explicitly
> >> deallocate resources.
>
> > This totally misrepresents the case. The with statement and the
> > context manager is a superset of the RAII functionality.
>
> No, it isn't.  C++ allows you to define smart pointers (one of many
> RAII techniques), which can use refcounting or other tracking
> techniques.  Refcounting smart pointers are part of Boost and have
> made it into TR1, which means they're on track to be included in the
> next standard library.  One need not have waited for Boost, as they can
> be implemented in about a page of code.
>
> The standard library also has auto_ptr, which is a different sort of
> smart pointer, which allows for somewhat fancier RAII than
> scope-based.
>

Obviously. But theres nothing about the with statement that's
different than using smart pointers in this regard. I take it back,
there's one case - when you need only one scope in a function, with
requires an extra block while C++ style RAII allows you to

> > It doesn't overload object lifetimes, rather it makes the intent
> > (code execution upon entrance and exit of a block) explicit.
>
> But I don't typically wish for this sort of intent to be made
> explicit.  TMI!  I used "with" for *many* years in Lisp, since this is
> how non-memory resource deallocation has been dealt with in Lisp since
> the dawn of time.  I can tell you from many years of experience that
> relying on Python's refcounter is superior.
>

I question the relevance of your experience, then. Refcounting is fine
for memory, but as you mention below, memory is only one kind of
resource and refcounting is not necessarily the best technique for all
resources. Java has the same problem, where you've got GC so you don't
have to worry about memory, but no tools for managing non-memory
resources.

> Shouldn't you be happy that there's something I like more about Python
> than Lisp?
>

I honestly don't care if anyone prefers Python over Lisp or vice
versa. If you like Lisp, you know where it is.

> > Nobody in their right mind has ever tried to get rid of explicit
> > resource management - explicit resource management is exactly what you
> > do every time you create an object, or you use RAII, or you open a
> > file.
>
> This just isn't true.  For many years I have not had to explicitly
> close files in Python.  Nor have I had to do so in C++.  They have
> been closed for me implicitly.  "With" is not implicit -- or at least
> not nearly as implicit as was previous practice in Python, or as is
> current practice in C++.
>

You still don't have to manually close files. But you cannot, and
never could, rely on them being closed at a given time unless you did
so. If you need a file to be closed in a deterministic manner, then
you must close it explicitly. The with statement is not implicit and
never has been. Implicit resource management is *insufficient* for
the general resource management case. It works fine for memory, it's
okay for files (until it isn't), it's terrible for thread locks and
network connections and database transactions. Those things require
*explicit* resource management.

> > *Manual* memory management, where the tracking of references and
> > scopes is placed upon the programmer, is what people are trying to
> > get rid of and the with statement contributes to that goal, it
> > doesn't detract from it.
>
> As far as I am concerned, memory is just one resource amongst many,
> and the programmer's life should be made easier in dealing with all
> such resources.
>

Which is exactly what the with statement is for.

> > Before the with statement, you could do the same thing but you
> > needed nested try/finally blocks
>
> No, you didn't -- you could just encapsulate the resource acquisition
> into an object and allow the destructor to deallocate the resource.
>

If you did this in Python, your code was wrong. You were coding C++ in
Python. Don't do it.

> > RAII is a good technique, but don't get caught up on the
> > implementation details.
>
> I'm not -- I'm caught up in the loss of power and elegance that will
> be caused by deprecating the use of destructors for resource
> deallocation.
>

Python has *never had this*. This never worked. It could seem to work
if you carefully, manually, inspected your code and managed your
object lifetimes. This is much more work than the with statement.

To the extent that your code ever worked when you relied on this
detail, it will continue to work. There are no plans to replace
pythons refcounting with fancier GC schemes that I am aware of.

> > The with statement does exactly the same thing, but is actually
> > superior because
> >
> > a) It doesn't tie the resource managment to object creation. This
> > means you can use, for example, with lock: instead of the C++ style
> > Locker(lock)
>
> I know all about "with".  As I mentioned above, Lisp has had it since
> the dawn of time.  And I have nothing against it, since it is at times
> quite useful.  I'm just dismayed at the idea of deprecating reliance
> on destructors in favor of "with" for the majority of cases when the
> destructor usage works well and is more elegant.
>

Nothing about Pythons memory management has changed. I know I'm
repeating myself here, but you just don't seem to grasp this concept.
Python has *never* had deterministic destruction of objects. It was
never guaranteed, and code that seemed like it benefited from it was
fragile.

> > b) You can tell whether you exited with an exception, and what that
> > exception is, so you can take different actions based on error
> > conditions vs expected exit. This is a significant benefit, it
> > allows the application of context managers to cases where RAII is
> > weak. For example, controlling transactions.
>
> Yes, for the case where you might want to do fancy handling of
> exceptions raised during resource deallocation, then "when" is
> superior, which is why it is good to have in addition to the
> traditional Python mechanism, not as a replacement for it.
>

"with". And it's not replacing anything.

> >> Right, but that doesn't mean that 99.9% of the time, the programmer
> >> can't immediately tell that cycles aren't going to be an issue.
>
> > They can occur in the most bizarre and unexpected places. To the point
> > where I suspect that the reality is simply that you never noticed your
> > cycles, not that they didn't exist.
>
> Purify tells me that I know more about the behavior of my code than
> you do: I've *never* had any memory leaks in large C++ programs that
> used refcounted smart pointers that were caused by cycles in my data
> structures that I didn't know about.
>

I'm talking about Python refcounts. For example, a subtle resource
leak that has caught me before is that tracebacks hold references to
locals in the unwound stack. If you relied on refcounting to clean up
a resource, and you needed exception handling, the resource wasn't
released until *after* the exception unwound, which could be a
problem. Also holding onto tracebacks for latter processing (not
uncommon in event based programs) would artificially extend the
lifetime of the resource. If the resource you were managing was a
thread lock this could be a real problem.

> > And if you think you won't need it because python will get "real" GC
> > you're very confused about what GC does and how.
>
> Ummm, I know all about real GC, and I'm quite aware than Python has
> had it for  quite some time now.  (Though the implementation is rather
> different last I checked than it would be for a language that didn't
> also have refcounted GC.)
>
> > A generic threadsafe smart pointer, in fact, is very nearly a GIL.
>
> And how's that?  I should think that modern architectures would have
> an efficient way of adding and subtracting from an int atomically.  If
> they don't, I have a hard time seeing how *any* multi-threaded
> applications are going to be able to make good use of multiple
> processors.
>
> > Get cracking then. You're hardly the first person to say this.
> > However, of the people who say it, hardly anyone actually produces
> > any code and the only person I know of who did dropped it when
> > performance went through the floor. Maybe you can do better.
>
> I really have no desire to code in C, thank you.  I'd rather be coding
> in Python.  (Hence my [idle] desire for macros in Python, so that I
> could do even more of my work in Python.)
>

In this particular conversation, I really don't think that theres much
to say beyond put up or shut up. The experts in the field have said
that it's not practical. If you think they're wrong, you're going to
need to prove it with code, not by waving your hand.

> > There's no particular reason why Lisp is any better for AI research
> > than anything.
>
> Yes, there is.  It's a very flexible language that can adapt to the
> needs of projects that need to push the boundaries of what computer
> programmers typically do.
>

That doesn't make Lisp any better at AI programming than it is for
writing databases or spreadsheets or anything else.

> > I'm not familiar with the TIOBE metric, but I can pretty much
> > guarantee that regardless of what it says there is far more COBOL
> > code in the wild, being actively maintained (or at least babysat)
> > than there is lisp code.
>
> I'm agree that there is cedrtainly much more Cobol code being
> maintained than there is Lisp code, but that doesn't mean that there
> are more Cobol programmers writing new code than there are Lisp
> programmers writing new code.  A project would have to be run by a
> madman to begin a new project in Cobol.
>

More than you'd think, sadly. Although depending on your definition of
"new project", it may not count. There's a great deal of new code
being written in COBOL to run on top of old COBOL systems.

> >> Re Lisp, though, there used to be a joke (which turned out to be
> >> false), which went, "I don't know what the most popular programming
> >> language will be in 20 years, but it will be called 'Fortran'".  In
> >> reality, I don't know what the most popular language will be called 20
> >> years from now, but it will *be* Lisp.
>
> > And everyone who still uses the language actually called Lisp will
> > continue to explain how it isn't a "real" lisp for a laundry list of
> > reasons that nobody who gets work done actually cares about.
>
> And where are you getting this from?  I don't know anyone who claims
> that any commonly used dialect of Lisp isn't *really* Lisp.
>

The language of the future will not be Common Lisp, and it won't be a
well known dialect of Lisp. It will have many Lisp like features, and
"true" Lispers will still claim it doesn't count, just as they do
about Ruby and Python today.