Adaptation and typecasting (was Re: [Python-Dev] replacing 'global')

[Alex Martelli]

On Tuesday 28 October 2003 07:47 pm, Phillip J. Eby wrote:
   ...
> You didn't actually give any example of why 'adapt("23",int)' shouldn't
> return 23, just why adapt("foo",file) shouldn't return a file.

Which is sufficient to show that, IN GENERAL, adapt(x, sometype)
should not just be the equivalent of sometype(x), as you seemed (and,
below, still seem) to argue.  Now, if you want to give positive reasons,
specific, compelling use-cases, to show why for SOME combinations
of type(x) and sometype that general rule should be violated, go ahead,
but the burden of proof is on you.

If you do want to try and justify such specific-cases exceptions, remember:
"adapt(x, foo)" is specified as returning "x or a wrapper around x", and
clearly a new object of type foo with no actual connection to x is neither 
of those.  That's a "formal" reasoning from the PEP's actual current text.

But perhaps informal reasoning may prove more convincing -- let's try.

adaptation is *NOT* conversion -- it's not the creation of a new object
that will thereafter live a life separate from the original one.  This part
is not relevant when the objects are immutable, but it's quite relevant
to your GENERAL idea of, e.g.:
> 'adapt(x,str)' should be equivalent to x.__str__().
Say that x is mutable.  Then, "adapting x to the string protocol", if
supported, should give a wrapper object, supporting all string-object
methods, in a way that any call to such methods relies on the
current-at-call-time value of x.

But, still on that general idea of yours that I quote above, there
is worse, MUCH worse.

Consider: an object's type often supports a __str__ that, as per its
specs in the docs, is "the ``informal'' string representation of an object ...
convenient or concise representation may be used instead".  The docs
make it AMPLY clear that the purpose of __str__ is STRICTLY for
the object's type to give a (convenient, concise, possibly quite incomplete
and inaccurate) HUMAN-READABLE representation of the object.  To
assert that this is in any way equivalent to a claim, on the object type's
part, that its instances can "adapt themselves to the string protocol",
beggars belief.

It borders, I think, on the absurd, to maintain that, for example,
"<open file '/goo/bag', mode 'r' at 0x402cbae0>" *IS* my open file
object "adapted to" the string protocol.  It's clearly a mere human
readable representation, a vague phantom of the object itself.  It
should be obvious that, just as "adapting a string to the (R/O) file
protocol" means wrapping it in cStringIO.StringIO, so the reverse
adaptation, "adapting a file to the string protocol", should utilize
a wrapper object that presents the file's data with all string object
methods, for example via mmap.


> So, there is already a defined protocol within Python for conversion to
> specific types, with well-defined meaning.  One might argue that since it's

Conversion is one thing, adaptation is a different thing.  Creating a new
object "somehow related" to an existing one -- i.e., conversion -- is a very 
different thing from "wrapping" an existing object to support a different 
protocol -- adaptation.

Consider another typical case:

>>> import array
>>> x = array.array('c', 'ciao')
>>> L = list(x)
>>> x.extend(array.array('c', 'foop'))
>>> x
array('c', 'ciaofoop')
>>> L
['c', 'i', 'a', 'o']
>>>

See the point?  CONVERSION, aka construction, aka typecasting, i.e.
list(x), has created a new object, based on what WERE the contents
of x at the time at conversion, but INDEPENDENT from it henceforwards.

Adaptation should NOT work that way: adapt(x, list) would, rather,
return a wrapper, providing listlike methods (some, like pop or remove,
would delegate to x's own methods -- others, like sort, would require
more work) and _eventually performing actual operations on x_, NOT
on a separate thing that once, a long time ago, was constructed by
copying it.

Thus, I see foo(x) and adapt(x, foo) -- even in cases where foo is a
type -- as GENERALLY very different.  If you have SPECIFIC use cases
in mind where it would be clever to make the two operations coincide,
you still haven't made them; I only heard vague generalities about how
adapt(x, y) "should" work without ANY real support for them.

If the code that requests adaptation is happy, as a fall-back, to have
(e.g.) "<open file '/goo/bag', mode 'r' at 0x402cbae0>" as the "ersatz
adaptation" of a file instance to str, for example, it can always do the
fall-back itself, e.g.
    try: z = adapt(x, y)
    except TypeError: 
        try: z = y(x)
        except (TypeError, ValueError):
            # whatever other desperation measures it wants to try

To have adapt itself imply such measures would be a disaster, and
make adaptation basically unusable in all cases where one might
have (e.g.) "y is str".
    

> I don't understand the dividing line here.  Perhaps that's because Python
> doesn't really *have* an existing notion of typecasting as such, there are
> just constructors (e.g. int) and conversion methods (e.g.
> __int__).

Yeah, that's much like C++, except C++ is more general in terms of
conversion methods -- not only can a constructor for type X accept
a Y argument (or const Y&, equivalently), but type Y can also always
choose to provide an "operator X()" to typecast its instances to the
other type [I think I recall that if BOTH types try to cooperate in such
ways you end up with an ambiguity error, though:-)].  That's in contrast
to the specific few 'conversion methods' that Python supports only
for a small set of numeric types as the destination of the conversion.

Either the single-argument constructor or the operator may get used 
when you typecast (static_cast<X>(y) where y is an instance of Y).  

There isn't all that much difference between C++'s approach and 
Python's here, except for C++'s greater generality and the fact that 
in Python you always use notation X(y) to indicate the typecasting 
request.

("typecast" is not a C++ term any more than it's Python's -- I think
it's only used in some obscure languages such as CIAO, tools like
Flex/Harpoon, Mathworks, etc -- so, sorry if my use was obscure).

One important difference: in C++, you get to define whether a
one-argument constructor gets to be evaluated "implicitly", when
an object of type X is required and one of type Y is supplied
instead, or not.  If the constructor is declared explicit, then it
ONLY gets called for EXPLICIT typecasts such as X(y).

In Python, we think EIBNI, and therefore typecasts are explicit.
We do NOT "adapt" a float f to int when an int is required, as
in somelist[f]: we raise a TypeError -- if you want COERCION,
aka CONVERSION, to an int, with possible loss of information
etc, you EXPLICITLY code somelist[int(f)].  Your proposal that
adaptation be, when possible, implemented by conversion, goes
against the grain of that good habit and principle.  Adaptation
in general is not conversion -- when you know you want, or at
least can possibly tolerate as a fallback, total conversion, ASK
for it, explicitly -- perhaps as a fallback if adaptation fails, as
above.  Having "adapt(x, y)" just basically duplicate some possible
cases of y(x) would be a serious diminution of adaptation's
potential and usefulness.


> However, conversion methods and even constructors of immutable
> types are allowed to be idempotent.  'int(x) is x' can be true, for
> example.  So, how is that different?

it's part of the PEP that, if isinstance(x, y), then normally
x is adapt(x, y) [[ with a specific exception for "non substitutable
subclasses" whose usecases I do not know -- anyway, such
subclasses would need to be _specifically_ "exempted" from the
general rule, e.g. by providing an __adapt__ that raises as needed ]].

So, calling y(x) will be wrong EXCEPT when type y is immutable
AND it's EXACTLY the case that "type(x) is y", NOT a subclass,
otherwise:

>>> class xx(int): pass
...
>>> w = xx(23)
>>> type(w)
<class '__main__.xx'>
>>> type(int(w))
<type 'int'>
>>>

... the 'is' constraint is lost, despite the fact that xx IS quite
obviously "substitutable" and has requested NO exception
to the rule, AT ALL.

Again: adaptation is not conversion -- and this is NOT about
the admitted defects in the PEP, because this case is VERY
specifically spelled out there.  Implementing adapt(x, y) as
y(x) may perhaps be of some practical use in some cases,
but I am still waiting for you to show any such use case of
practical compelling interest.  I hope I have _amply_ shown
that the implementation strategy is absolutely out of the
question as a general one, so it matters up to a point if some
very specific subcases are well served by that strategy, anyway.
The key issue is, such cases, if any, will need to be very
specifically identified and justified one by one.


Alex