[Python-Dev] PEP 246: lossless and stateless

[Armin Rigo]

Hi Clark,

On Fri, Jan 14, 2005 at 12:41:32PM -0500, Clark C. Evans wrote:
> Imagine enhancing the stack-trace with additional information about
> what adaptations were made; 
> 
>     Traceback (most recent call last):
>        File "xxx", line 1, in foo
>          Adapting x to File
>        File "yyy", line 384, in bar
>          Adapting x to FileName
>        etc.

More thoughts should be devoted to this, because it would be very precious.  
There should also be a way to know why a given call to adapt() returned an
unexpected object even if it didn't crash.  Given the nature of the problem,
it seems not only "nice" but essential to have a good way to debug it.

> How can we express your thoughts so that they fit into a narrative
> describing how adapt() should and should not be used?

I'm attaching a longer, hopefully easier reformulation...


Armin
-------------- next part --------------

A view on adaptation
====================

Adaptation is a tool to help exchange data between two pieces of code; a very powerful tool, even.  But it is easy to misunderstand its aim, and unlike other features of a programming language, misusing adaptation will quickly lead into intricate debugging nightmares.  Here is the point of view on adaptation which I defend, and which I believe should be kept in mind.


Let's take an example.  You want to call a function in the Python standard library to do something interesting, like pickling (saving) a number of instances to a file with the ``pickle`` module.  You might remember that there is a function ``pickle.dump(obj, file)``, which saves the object ``obj`` to the file ``file``, and another function ``pickle.load(file)`` which reads back the object from ``file``.  (Adaptation doesn't help you to figure this out; you have to be at least a bit familiar with the standard library to know that this feature exists.)

Let's take the example of ``pickle.load(file)``.  Even if you remember about it, you might still have to look up the documentation if you don't remember exactly what kind of object ``file`` is supposed to be.  Is it an open file object, or a file name?  All you know is that ``file`` is meant to somehow "be", or "stand for", the file.  Now there are at least two commonly used ways to "stand for" a file: the file path as a string, or the file object directly.  Actually, it might even not be a file at all, but just a string containing the already-loaded binary data.  This gives a third alternative.

The point here is that the person who wrote the ``pickle.load(x)`` function also knew that the argument was supposed to "stand for" a source of binary data to read from, and he had to make a choice for one of the three common representations: file path, file object, or raw data in a string.  The "source of binary data" is what both the author of the function and you would easily agree on; the formal choice of representation is more arbitrary.  This is where adaptation is supposed to help.  With properly setup adaptation, you can pass to ``pickle.load()`` either a file name or a file object, or possibly anything else that "reasonably stands for" an input file, and it will just work.


But to understand it more fully, we need to look a bit closer.  Imagine yourself as the author of functions like ``pickle.load()`` and ``pickle.dump()``.  You decide if you want to use adaptation or not.  Adaptation should be used in this case, and ONLY in this kind of case: there is some generally agreed concept on what a particular object -- typically an argument of function -- should represent, but not on precisely HOW it should represent it.  If your function expects a "place to write the data to", it can typically be an open file or just a file name; in this case, the function would be defined like this::

    def dump_data_into(target):
        file = adapt(target, TargetAsFile)
        file.write('hello')

with ``TargetAsFile`` being suitably defined -- i.e. having a correct ``__adapt__()`` special method -- so that the adaptation will accept either a file or a string, and in the latter case open the named file for writing.

Surely, you think that ``TargetAsFile`` is a strange name for an interface if you think about adaptation in term of interfaces.  Well, for the purpose of this argument, don't.  Forget about interfaces.  This special object ``TargetAsFile`` means not one but two things at once: that the input argument ``target`` represents the place into which data should be written; and that the result ``file`` of the adaptation, as used within function itself, must be more precisely a file object.

This two-level distinction is important to keep in mind, specially when adapting built-in objects like strings and files.  For example, the adaptation that would be used in ``pickle.load(source)`` is more difficult to get right, because there are two common ways that a string object can stand for a source of data: either as the name of a file, or as raw binary data.  It is not possible to distinguish between these two differents uses of ``str`` automatically.  In other words, strings are very versatile and low-level objects which can have various meanings in various contexts, and sometimes these meanings even conflict in the same context!  More concretely, it is not possible to use adaptation to write a function ``pickle.load(source)`` which accepts either a file, a file name, or a raw binary string.  You have to make a choice.  For symmetry with the case of ``TargetAsFile``, a ``SourceAsFile`` would probably interpret a string as a file name, and the caller still has to explicitely turn a raw string into a file-like object -- by wrapping it in a ``StringIO()``.

However, it would be possible to extend our adapters to accept URLs, say, because it's possible to distinguish between a local file name and an URL.  Similarily, various other object types could unambiguously refer to, respectively, a "source" or "target" of data.


The essential point is: the criterion to keep in mind for knowing when it is reasonable or not to add new adaptation paths is whether the object you are adapting "clearly stands" for the **high-level concept** that you are adapting to, and **not** for whatever resulting type or interface the adapted object should have.  It **makes no sense** to adapt a string to a file or a file-like object.  *Never define an adapter from the string type to the file type!!*  A string and a file are two low-level concepts that mean different things.  It only makes sense to adapt a string to a "source of data" which is then represented as a file.


This subtle distinction is essential when adapting built-in types.  In large frameworks, it is perhaps more common to adapt to interfaces or between classes specific to your framework.  These interfaces and classes merge both roles: one class is a concrete objects in the Python sense -- a type -- and a single embodied concept.  In this case, the difference between a concrete instance and the concept it stands for is not so important.  This is why we can often think about adaptation as creating an adapter object on top of an instance, to provide a different interface for the object.  If you adapt an instance to an interface ``I`` you really mean that there is a common concept behind the instance and ``I``, and you want to change from the representation given by the instance to the one given by ``I``.

I believe it is useful to keep in mind that adaptation is really about converting between different concrete representations ("str", "file") of a common abstract concept ("source of data").  You have at least to realize which abstract concept you want to adapt representations of, before you define your own adapters.  If you do, then properties like the transitivity of adaptation (i.e. automatically finding longer adaptation paths A -> B -> C when asked to adapt from A to C) become desirable, because the intermediate steps are merely changes in representation for the same abstract concept ("it's the same source of data all along").  If you don't, then transitivity becomes the Source Of All Nightmares :-)