Booleans, integer division, backwards compatibility; where is Python going?

[Alex Martelli]

Paul Boddie wrote:

> Alex Martelli <aleax at aleax.it> wrote in message
> news:<RYTs8.2219$b62.57325 at news1.tin.it>...
>> 
> 
> [Iterators]
> 
>> I've noticed they immensely help Python beginners that come to them
>> without preconcepts.  Such beginners are surprised to learn that
>> these are extremely recent additions to Python, so well they fit in.
> 
> Well, I was excited a few years ago when iterators appeared in C++, so
> I suppose I should be even more excited now. :-) However, it's
> arguably a bigger deal for the implementer of the iterator (or
> generator, even) than the "users" or "clients" of them. Many beginners
> just expect things to work in the way that iterators themselves
> promote, and it was previously up to the implementer of a framework to
> provide nice interfaces to "users"/"clients" - now, it's just easier
> for the implementer.

A beginner won't go around calling iter(), whatever.next(), and
try: ... / except StopIteration: ... , of course -- he or she will
just expect "for item in container:" to work smoothly and
seamlessly.

With the previous iterator protocol, "for item in somedict:" COULD
NOT "work smoothly and seamlessly".  Nor, at all.  Think about it.

Therefore, the previous iterator protocol did NOT allow "the
implementor" of such a fundamental container as a dictionary (or
many other a mapping type) to offer to beginners (and other users
of the container) this crucial tidbit of functionality in the
seamless manner provided by "for item in container:".

Now, it's not *JUST* "easier" -- it's finally become *POSSIBLE*
at all, where it earlier wasn't.  I guess somebody with sufficient
Jesuitic education COULD try arguing that if something that was
not possible and now is CAN be fairly said to be "easier" now,
but, fair warning: I may have had more years with the Jesuits than
you, I think that "just" gives me enough purchase to tear such
an attempted, hypothetical argument into reasonably tiny pieces,
and while that might even be fun (in the usual perverted way
semantics flamewars can be fun) I'm busy enough that I don't truly
look forwards to it (whence the fair warning:-).

In many, indeed most, cases, showing a nice, seamless iterable face to 
beginners and other users is now "just" easier; in many others, it's 
become possible where it previousty was impossible (you had at least
to give up the "seamless" part of it).

>> They're great progress over the previously murky "iteration protocol".
> 
> What protocol was that again? ;-)

for item in container:
    body(item)

translated, roughly, into:

_index = 0
while 1:
    try: item = container[_index]
    except IndexError: break
    _index += 1

    body(item)

Perhaps "murky" is not an ideal characterization of this, but it
does have the problem of requiring the container to implement
__getitem__ yielding all items progressively when called with
indices of 0, 1, ..., and raising IndexError when done -- fine
for sequences, not so fine at all for mappings (not to mention
a few auxiliary issues with sequenceoids that ONLY wanted to
allow iterations and not "random-access" indexing).

In contrast, the same statement now translates, roughly, into:

_iter = iter(container)
while 1:
    try: item = _iter.next()
    except StopIteration: break

    body(item)

The key advantage is not overloading indexing-on-container to
serve for iteration-on-container as well.  Built-in function
iter knows how to build an iterator-on-sequence object if need
be (also an iterator-to-sentinel when called with 2 args,
but that's another, albeit sort of nice, thing).


> [Applications which subclass built-in types]
> 
>> Or maybe not being used to them you just don't think of cases where
>> they'd be handy (if you need to keep compatibility with 2.1, then
>> of course you'd better NOT start thinking of such cases:-).  I.e.,
>> Sapir-Whorf hypothesis somewhere between its weakest and strongest
>> forms (though I still prefer the way Wittgenstein independently
>> put it earlier -- "The limits of my language are the limits of
>> my world").
> 
> It's probably best not to bring semi-controversial linguistic themes
> into the discussion - Whorf allegedly had less of a grasp of the
> languages he used to illustrate his hypothesis than I do of the most
> recent Python features. ;-)

The ones you're nevertheless busy putting down?-)  Maybe, but that
is part of why I prefer to use the Wittgenstein quote, independently
penned in 1918 -- Wittgenstein's grasp of German and English was
excellent, his Italian and his formal-mathematics not bad, and his
thesis is in any case not based on empirical issues but on a survey
of the analytical nature of language (Tractatus Logico-Philosophicus).

> Seriously, I can't think of cases off the top of my head where
> inheriting from an integer, floating point, complex number, string,
> tuple, list or dictionary are so compelling. An "ordered dictionary"
> can be useful, but we already had UserDict to define the interface for
> that kind of thing.

Inheriting from immutables is unusual -- but far from unheard of,
as shown by the existence of UserString too.  Inheriting from
mutables is more usual, as UserList and UserDict's existence and
popularity show -- and now we don't need them any longer, as we
can subclass list and dict directly, an obvious win.  Besides
directness (no need to teach and learn two separate types for
every interface, an inheritable and a fast one) and performance,
you stand a little bit more chance to survive type-testing (not
if the adversary type-tests with type(a)==type(b), but at least
if he uses isinstance instead).  You also get many more types
you can inherit from -- you didn't have a UserFile earlier for
example, so no direct and immediate tweaking of file objects.

Sure, to some extent you could kludge it up with delegation, but
only to some extent (there are several examples in the Cookbook,
you may want to have a look).  Delegation is often more powerful
and flexible than inheritance (and nobody is taking it away),
but inheritance, when what it offers is all you need, is handier,
faster and more direct -- often a win via "do the simplest thing
that can possibly work".  Sometimes (via the Template Method
design pattern, for example) it even does offer more power than
delegation (AFAIK no existing built-in type uses the TM DP, but
nobody stops you doing it in YOUR built-in types, now that you
know how to write them in order to be inheritable-from).


> [New features in the tutorial]
> 
>> Yes, just not in the tutorial (at least not until they're in the
>> "current" language rather than needing import from the future).
> 
> That's part of my original argument, though. I get the impression that
> some packages not only require Python 2.2 because they use new,
> standard features, but also so that the author can drag users further
> into the future by making use of not-yet-standard features, if only
> for their own entertainment.

Once you're using (e.g.) iterators in your package, or subclassing
a standard class, there's no extra cost in having the modules of
your package use (e.g.) generators or true-division as well, so
why shouldn't a package author freely do so?

That something MAY not be quite ready to explain to beginners is
one thing.  A package's author is presumably no beginner.


>> AMK's mission is not to tell you WHY you can be interested in
>> major stuff such as subclassing builtins, iterators, generators,
>> though -- just to ensure you DO know easily what IS new.
> 
> It wouldn't be a bad thing to have such a source of brief rationales
> for new features, however.

Since each new feature has its own PEP and each PEP its rationale,
I suspect such a "source" could be automated (more easily if PEPs
were in easily parsable XML, of course, but, even as is...).  Of
course, as rationales would come from a variety of authors, etc,
there is no guarantee such a compilation would be appealing or
very nice to read (also, some PEP authors are better at other
tasks than at writing appealing rationales).

Is it SO hard to go read the Rationale of each PEP yourself, btw?


Alex