[Python-Dev] Documentation idea

[Brett Cannon]

On Fri, Oct 10, 2008 at 1:45 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> glyph at divmod.com wrote:
>>
>> On 9 Oct, 11:12 pm, python at rcn.com wrote:
>>>
>>> Background
>>> ----------
>>> In the itertools module docs, I included pure python equivalents for each
>>> of the C functions.  Necessarily, some of those equivalents are only
>>> approximate but they seem to have greatly enhanced the docs.
>>
>> Why not go the other direction?
>>
>> Ostensibly the reason for writing a module like 'itertools' in C is purely
>> for performance.  There's nothing that I'm aware of in that module which
>> couldn't be in Python.
>>
>> Similarly, cStringIO, cPickle, etc.  Everywhere these diverge, it is (if
>> not a flat-out bug) not optimal.  External projects are encouraged by a
>> wealth of documentation to solve performance problems in a similar way:
>> implement in Python, once you've got the interface right, optimize into C.
>>
>> So rather than have a C implementation, which points to Python, why not
>> have a Python implementation that points at C?  'itertools' (and similar)
>> can actually be Python modules, and use a decorator, let's call it "C", to
>> do this:
>>
>>   @C("_c_itertools.count")
>>   class count(object):
>>       """
>>       This is the documentation for both the C version of itertools.count
>>       and the Python version - since they should be the same, right?
>>       """
>
> The ancient string module did something like this, except that the rebinding
> of function names was done at the end by 'from _string import *' where
> _string had C versions of some but not all of the functions in string.  (And
> the list of replacements could vary by version and platform and compiler
> switches.)  This was great for documenting the string module.  It was some
> of the first Python code I studied after the tutorial.
>
> The problem with that and the above (with modification, see below) is the
> creation and discarding of unused function objects and the time required to
> do so.
>
> The advantage of the decorator version is that the compiler or module loader
> could be special cased to recognize the 'C' decorator and try it first
> *before* using the Python version, which would serve as a backup.  There
> could be a standard version in builtins that people could replace to
> implement non-standard loading on a particular system.  To cater to other
> implementations, the name could be something other than 'C', or we could
> define 'C' to be the initial of "Code" (in the implementation language).
>  Either way, other implementation could start with a do-nothing "C"
> decorator and run the file as is, then gradually replace with lower-level
> code.
>

The decorator doesn't have to require any special casing at all
(changing the parameters to keep the code short)::

  def C(module_name, want):
     def choose_version(ob):
         try:
           module = __import__(module_name, fromlist=[want])
           return getattr(module, want)
          except (ImportError, AttributeError):
            return ob
      return choose_version

The cost is purely during importation of the module and does nothing
fancy at all and relies on stuff already available in all Python VMs.

-Brett