[Python-Dev] PEP 246 and Protocols (Was: Sneaky 'super' instances)

[Moore, Paul]

From: Phillip J. Eby [mailto:pje@telecommunity.com]
> Open protocols solve the chicken and egg problem by allowing one
> to make declarations about third-party objects.

Um....... This isn't a criticism per se, but my eyes glaze over when I
see stuff like this. I know it's not, but it reads like marketing, and
I end up not being able to follow the real point. It's my problem, not
yours, but I mention it in case I seem to fail to get the point...

[time passes]

OK, I've tried to put together a simple example of "what I expect" and
I find I can't. I want to continue to write code which "just assumes"
that it gets the types it needs - I don't want to explicitly state that
I need specific interface types - that feels like type declaration and
Java. Your IFile example reinforces that, both in terms of its naming
convention, and in the assumption that there *is* a single, usable,
"file" protocol. I know it was my example, but I pointed out later in
the same message that I really sometimes wanted seekable readline-only
files, and other times block read/write (potentially unseekable) files.

Expecting library writers to declare interface "classes" for every
subtle variation of requirements seems impractical. Expecting the
requirements to be *documented* is fine - it's having a concrete class
which encapsulates them that I don't see happening - no-one would ever
look to see if there was already a "standard" interface which said "I
have readline, seek, and tell" - they'd just write a new one. There
goes any hope of reuse. (This may be what you see as a "chicken and egg"
problem - if so, I see it as a "hopelessly unrealistic expectations"
problem instead, because it's never going to happen...)

On the other hand, expecting callers to write stuff to adapt existing
classes to the requirements of library routines is (IMHO) a non-issue.
I think that's what PEP 246 was getting at in the statement that "The
typical Python programmer is an integrator" which you quote. It's
common to write

    class wrap_source:
        def __init__(self, source):
            self.source =3D source
        def read(self, n =3D 1024):
            return self.source.get_data(n)

    lib_fn(wrap_source(my_source))

So PEP 246 is trying to make writing that sort of boilerplate easier
(in my view). The *caller* should be calling adapt(), not the callee.

On that basis, adapt(obj, file) could be a fluid notion, depending on
what the caller has "registered" as an adapter for a file object. In
a situation where he is only calling library routines which need
readable files, adapt(obj, file) may completely omit write methods. In
another context that may be a disaster. The called routines still have
expectations, and it's the caller's responsibility to respect them. But
adapt() offers a general facility to help the caller avoid writing
boilerplate classes like the above.

In this context, __adapt__ allows class writers to build compatibility
wrappers using a common framework. For example, StringIO could be
replaced by an __adapt__ method of the file class:

    class file: # OK, it's just a demo...
        def __adapt__(self, obj):
            if isinstance(obj, basestring):
                return StringIO.StringIO(obj)

Then code which wants to pass a string to a function which expects a
file could do

    process(adapt(my_string, file))

And conversely, __conform__ does the same for the writer of the string
class:

    class string:
        def __conform__(self, type):
            if type is file:
                return StringIO.StringIO(self)

Which is appropriate is basically down to which came first, string or
file. But both suffer from the problem of putting all the knowledge
in one location (and using a type switch as well).

The third option, which is the "external registry" allows a *user* of
the string and file "libraries" (bear with me here...) to say how he
wants to make strings look like files:

    import pep246
    import StringIO

    def str_to_file(str):
        return StringIO.StringIO(str)

    # Note that this effectively uses multiple dispatch
    # somewhere in there...
    pep246.register(basestring, file, str_to_file)

    # And now adapt(my_string, file) does what we want!

So I see PEP 246 as more or less entirely a class based mechanism. It
has very little to do with constraining (or even documenting) the
types of function parameters.

Of course, a library writer can define interface classes, and the
adaptation mechanism will allow concrete classes to be made to
"conform" to those interfaces, but it's not necessary. And given
the general apathy of the Python community towards interfaces (at
least as far as I see) I don't imagine this being a very popular use
pattern. And describing a PEP 246 style facility in terms of
interfaces could be a big turn-off. (This applies strongly to your
PyProtocols code - I looked for something like the pep246.register
function I suggested above, but I couldn't manage to wade through all
the IThis and IThat stuff to find it, if it was there...)

Once again, I apologise if I've missed your point. But I hope I've
explained what I see as the point of PEP 246, and where your proposal
is going off in a different direction.

But the fact remains, that neither PEP 246 nor PyProtocols has any
need to be in the core, or even in the standard library (yet). Wide
usage could change that. Usage in something that is destined for the
standard library could, too, but that one could go the other way (the
"something" gets rejected because it depends on PEP246/PyProtocols).
I believe that dependency on PEP 246 would be less of a barrier here
than dependency on PyProtocols. But don't assume that just because
I'm writing long messages that my opinion carries any weight :-)

Paul.

PS This is *way* off-topic for python-dev by now. I suggest that we
   leave things here. I don't have anything else worth adding...