[Python-Dev] buffer interface considered harmful

[Jim Fulton]

Jack Jansen wrote:
> 
> > A while ago I asked for some documentation on the Buffer
> > interface.  I basically got silence.  At this point, I
> > don't have a good idea what buffers are for and I don't see alot
> > of evidence that there *is* a design. I assume that there was
> > a design, but I can't see it.  This whole discussion makes me
> > very queasy.
> 
> Okay, as I'm apparently not the only one who is queasy let's start from
> scratch.

Yee ha!
 
> First, there is the old buffer _interface_. This is a C interface that allows
> extension (and builtin) modules and functions a unified way to access objects
> if they want to write the object to file and similar things.

Is this serialization?  What does this achiev that, say, the pickling
protocols don't achiev? What other problems does it solve?

> It is also what
> the PyArg_ParseTuple "s#" returns. This is, in C, the
> getreadbuffer/getwritebuffer interface.

Huh? "s#" doesn't return a string? Or are you saying that you can
pass a non-string object to a C function that uses "s#" and have it
bufferized and then stringized?  In either case, this is not
consistent with the documentation (interface) of PyArg_ParseTuple.
 
> Second, there's the extension the the buffer interface as of 1.5.2. This is
> again only available in C, and it allows C programmers to get an object _as an
> ASCII string_. This is meant for things like regexp modules, to access any
> "textual" object as an ASCII string. This is the getcharbuffer interface, and
> bound to the "t#" specifier in PyArg_ParseTuple.

Hm. So this is making a little more sense. So, there is a notion that
there are "textual" objects that want to provide a method for getting
their "text". How does this text differ from what you get from __str__
or __repr__?  

> Third, there is the buffer _object_, also new in 1.5.2. This sort-of exports
> the functionality of the buffer interface to Python,

How so?  Maybe I'm at sea because I still don't get what the 
C buffer interface is for.

> but it does a bit more as
> well, because the buffer objects have a sort of copy-on-write semantics that
> means they may or may not be "attached" to a python object through the buffer
> interface.

What is this thing used for?

Where does the slot in tp_as_buffer come into all of this?

Why does this need to be a slot in the first place?
Are these "textual" objects really common? Is the presense of this
slot a flag for "textualness"?
 
It would help alot, at least for me, if there was a clearer
description of what motivates these things. What problems are
they trying to solve?  

Jim

--
Jim Fulton           mailto:jim@digicool.com   Python Powered!        
Technical Director   (888) 344-4332            http://www.python.org  
Digital Creations    http://www.digicool.com   http://www.zope.org    

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.