[Python-ideas] Statement local functions and classes (aka PEP 3150 is dead, say 'Hi!' to PEP 403)

[Nick Coghlan]

On Sun, Oct 16, 2011 at 9:01 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> If it's not tested, how do you know it does what you need?

Because my script does what I want. Not every Python script is a
production application. We lose sight of that point at our peril.

Systems administrators, lawyers, financial analysts, mathematicians,
scientists... they're not writing continuously integrated production
code, they're not writing reusable modules, they're writing scripts to
*get things done*.

Python's insistence that every symbol be defined before it can be
referenced gets in the way of that. Imagine how annoying Python would
be if we got rid of while loops and insisted that anyone want to
implement a while loop instead write a custom iterator to use with a
for loop instead. It's taken me a long time to realise it, but that's
exactly how the lack of multi-line lambda support can feel. PEP 403
was an attempt to address *just* that problem with an approach
inspired by Ruby's blocks. That discussion convinced me that such a
terse, unintuitive idiom could never be made "Pythonic" - but there's
still hope for the more verbose, yet also more expressive, statement
local namespace concept.

Will there be times where someone writes a piece of code using a given
statement, realises they need access to the internals in order to test
them in isolation, and refactors the code accordingly? Absolutely,
just as people take "if __name__ == '__main__'" blocks and move them
into main functions to improve testability, break up large functions
into smaller ones, add helper methods to classes, or take repeated
list comprehensions and make a named function out of them.

The problem with the status quo is that there are plenty of people
(especially mathematicians and scientists) that *do* think
declaratively, just as there are people that think in terms of
functional programming rather than more typical imperative styles.
Python includes constructs that are technically redundant with each
other in order to allow people to express concepts in a way that makes
sense to *them* (e.g. keeping functools.reduce around). Unless, that
is, you think declaratively. Then Python isn't currently the language
for you - go use Ruby or some other language with a construct that
lets you supply additional details after the operation that needs
them.

To attack the question from a different perspective, how do I know
that any loops in my code work? After all, my test code can't see my
loops - it can only see the data structures they create and the
functions that contain them. So the answer is because I tested data
structures and I tested those functions. How could I test that a
weakref callback did the right thing without direct access to the
callback function? The same way I'd have to test it *anyway*, even if
I *did* have direct access: by allocating an object, dropping all
references to it, forcing a GC cycle. If we're not in the middle of
deleting the object, testing the callback actually tells me nothing
useful, since the application state is completely wrong.

All that said... is your objection *only* to the "statement local"
part of the proposal? That part is actually less interesting to me
than the out of order execution part. However, our past experience
with list comprehensions suggests that exposing the names bound inside
the suite in the containing scope would be confusing in its own way.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia