Python syntax in Lisp and Scheme

[Alex Martelli]

Pascal Costanza wrote:
   ...
>>>Does Python allow local function definitions?
   ...
>>>Can they shadow predefined functions?
   ...
>> Yes, named objects, including functions can (locally) shadow
>> (override) builtins.  It is considered a bad habit/practice unless
>> done intentionally with a functional reason.
> 
> Well, this proves that Python has a language feature that is as
> dangerous as many people seem to think macros are.

Indeed, a chorus of "don't do that" is the typical comment each
and every time a newbie falls into that particular mis-use.  Currently,
the --shadow option of PyChecker only warns about shadowing of
_variables_, not shadowing of _functions_, but there's really no
reason why it shouldn't warn about both.  Logilab's pylint does
diagnose "redefining built-in" with a warning (I think they mean
_shadowing_, not actually _redefining_, but this may be an issue
of preferred usage of terms).

"Nailing down" built-ins (at first with a built-in warning for overriding 
them, later in stronger ways -- slowly and gradually, like always, to 
maintain backwards compatibility and allow slow, gradual migration of the 
large existing codebase) is under active consideration for the next version 
of Python, expected (roughly -- no firm plans yet) in early 2005.

So, yes, Python is not perfect today (or else, we wouldn't be planning a
2.4 release...:-).  While it never went out of its way to give the user "as 
much rope as needed to shoot oneself in the foot", neither did it ever 
spend enormous energy in trying to help the user avoid many possible errors 
and dubious usage.  Such tools as PyChecker and pylint are a start, and
some of their functionality should eventually be folded back into the
core, just as tabnanny's was in the past with the -t switch.  I don't think
the fundamental Python will ever nag you for missing comments or
docstrings, too-short names, etc, the way pylint does by default (at
least, I sure hope not...!-), but there's quite a bit I _would_ like to have
it do in terms of warnings and, eventually, error messages for
"feechurs" that only exist because it was once simple to allow than
to forbid them, not by a deliberate design decision to have them there.

Note that SOME built-ins exist SPECIFICALLY for the purpose of
letting you override them.  Consider, for example, __import__ -- this
built-in function just exposes the inner mechanics of the import
statement (and friends) to let you get modules from some other
place (e.g., when your program must run off a relational database
rather than off a filesystem).  In other word, it's a rudimentary hook
in a "Template Method" design pattern (it's also occasionally handy
to let you import a module whose name is in a string, without
going to the bother of an 'exec', so it will surely stay for that purpose
even though we now have a shiny brand-new architecture for
import hooks -- but that's another story).  Having a single hook of
global effect has all the usual downsides, of course (which is exactly
why we DO have that new architecture;-): two or more complicated
packages doing import-hooks can't necessarily coexist within the
same Python application program (the only saving grace which let
us live with that simplistic hook for so many years is that importing
from strange places is typically a need of a certain deployment of
an overall application, not of a package -- still, such packages DO
exist, so the previous solution was far from perfect).

Anyway, back to your contention: I do not think that the fact that
the user can, within his functions, choose very debatable names,
such as those which shadow built-ins, is anywhere as powerful,
and therefore as dangerous, as macros.  My own functions using
'sum' will get the built-in one even if yours do weird things with
that same name as a local variable of their own.  The downsides
of shadowing are essentially as follows...

a newbie posts some fragment of his code asking for guidance,
and among other things that fragment has
    for i in range(length(thenumbers)):
        total = total + thenumbers[i]
he will receive many suggestions on how to make it better,
including the ideal one:
    total = sum(thenumbers, total)
But then he tries it out and reports "it breaks" (newbies rarely
are clueful enough to just copy and paste error messages).  And
we all waste lots of time finding out that this is because... the
hapless newbie had named HIS OWN FUNCTION 'sum', so
this was causing runaway recursion.  Having met similar issues
over and over, one starts to warn newbies against shadowing
and get sympathetic with the idea of forbidding it:-).

That doesn't really compare to an extra feature in the language
that is deliberately designed to let reasonably-clueful users do
their thing, isn't deprecated nor warned against by anybody at
all (with a few isolated voices speaking about "abuse" of macros
in this thread, but still with an appreciation for macros when
_well_ used), and is MEANT to do what newbies _accidentally_
do with shadowing & much more besides;-).


Alex