[Python-Dev] PySet API

[Barry Warsaw]

On Tue, 2006-03-21 at 21:31 -0500, Raymond Hettinger wrote: 
> [Barry]
> > Is it your intent to push for more use of the abstract API instead of
> > the concrete APIs for all of Python's C data structures?  Current API
> > aside, are you advocating this approach for all new built-in types?
> > Would you argue that Python 3.0's C API be stripped of everything but
> > the abstract API and the bare essentials of the concrete API?
> 
> It's not up to me.  Perhaps someone else can chime-in about the philosophy
> of how the C API is supposed to balance abstract and concrete APIs.

I think it's an important point to discuss, both for trying to resolve
this impasse and for helping to direct future API designs, especially as
we get into Python 3.0.

Maybe it will help you to understand why I want a richer concrete API.
I work on an app that is deeply integrated with Python.  It's hard to
say whether we embed or extend -- it's a lot of both.  We use Python
data structures such as lists, dicts, and sets in many places as our
fundamental tracking objects.  So we know what we have, i.e. it's
definitely a set here and another set there, and we want to merge one
into the other.  Or we know we have a set of foo's here and we need to
iterate over them quickly (yes, in a tight loop) to count things or
whatever.  

So there's no question that a concrete API is very useful to us.  And
there's no questions that snaking through the abstract API causes us
real debugging pain (a point which you mostly glossed over).  We
understand the gotchas about reference counting and the possibilities
and implications about calling back into Python.  Remember, we're all
consenting adults here.  I don't think we're unique here, as the rich
concrete API of other fundamental Python objects attests to.

Your comments lead me to think that you aren't taking this important use
case into account.  You talk about duck typing, but I don't care about
that here.  I absolutely know I have a PySet, so why cause me pain to
use it?

> I know that the more one uses the abstract API, the more likely the code
> is going to be able to accept duck typed inputs.  Also, most things that
> have tp_slots have a corresponding abstract method instead of tons
> a concrete access points; hence, I would be supportive if you proposed a
> PyObject_Clear(o) function (for calling tp_clear slots when they exist and
> returning an error code when they don't).

I wouldn't object to that, but it wouldn't change my mind about
PySet_Clear().  I'm not arguing against a rich abstract API, I'm arguing
for having a richer concrete API too.  And in this case, only slightly
richer.  

> For setobject.c, if I still have a say in the matter, my strong preference is to
> keep the API minimal, expose fine-grained functions for efficiency, use
> PyNumber methods for direct access to operator style set operations,
> and use the abstract API for everything else.

I think this is a silly stance.  You agree that PySet_Next() is easier
to use than the iterator API.  We will definitely not use the latter,
and if your position stands, then we'll just have to hack Python to add
it (or implement it in an auxiliary module).  But I don't want to have
to do that, so I really don't understand your reluctance to add three
obviously useful functions.

Another point: these don't expose internal bits of the set
implementation.  Well, except for the opaque position pointer, but
that's still enough data hiding for me because you're never supposed
to /do/ anything with that variable except pass it right back to
PySet_Next().  PySet_Clear() and PySet_Update() don't expose any
implementation details -- that's the whole point!

> P.S.  One other thought:  I don't want to crystalize the API in a way
that 
> precludes
> future development of the module.  One possibility for the future is for updates 
> to take
> multiple arguments such as s.update(t,u,v)  causing three updates to be 
> folded-in at once.

I don't see any way that my proposals preclude that.  And besides, the
three API calls I'm proposing are useful /today/.

But just so we all know what we're talking about, I've uploaded the
patch to SourceForge:

http://sourceforge.net/tracker/index.php?func=detail&aid=1458476&group_id=5470&atid=305470

As with all good patches, there's (almost) more test code than
implementation.

Cheers,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 309 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20060325/dc65d315/attachment.pgp