[Python-Dev] PEP: Adding data-type objects to Python

[Travis Oliphant]

Martin v. Löwis wrote:
> Travis E. Oliphant schrieb:
> 
>>But, there are distinct disadvantages to this approach compared to what 
>>I'm trying to allow.   Martin claims that the ctypes approach is 
>>*basically* equivalent but this is just not true.
> 
> 
> I may claim that, but primarily, my goal was to demonstrate that the
> proposed PEP cannot be used to describe ctypes object layouts (without
> checking, I can readily believe that the PEP covers everything in
> the array and struct modules).
> 

That's a fine argument.  You are right in terms of the PEP as it stands. 
  However, I want to make clear that a single Python type object *could* 
be used to describe data including all the cases you laid out.  It would 
not be difficult to extend the PEP to cover all the cases you've 
described --- I'm not sure that's desireable.  I'm not trying to replace 
what ctypes does.  I'm just trying to get something that we can use to 
exchange data-format information through the extended buffer protocol.

It really comes down to using Python type-objects as the instances 
describing data-formats (which ctypes does) or "normal" Python objects 
as the instances describing data-formats (what the PEP proposes).

> 
>>It could be made more 
>>true if the ctypes objects inherited from a "meta-type" and if Python 
>>allowed meta-types to expand their C-structures.  But, last I checked 
>>this is not possible.
> 
> 
> That I don't understand. a) what do you think is not possible?

Extending the C-structure of PyTypeObject and having Python types use 
that as their "type-object".

  b)
> why is that an important difference between a datatype and a ctype?

Because with instances of C-types you are stuck with the PyTypeObject 
structure.  If you want to add anything you have to do it in the 
dictionary.

Instances of a datatype allow adding anything after the PyObject_HEAD 
structure.

> 
> If you are suggesting that, given two Python types A and B, and
> B inheriting from A, that the memory layout of B cannot extend
> the memory layout of A, then: that is certainly possible in Python,
> and there are many examples for it.
>

I know this.  I've done it for many different objects.  I'm saying it's 
not quite the same when what you are extending is the PyTypeObject and 
trying to use it as the type object for some other object.


> 
>>A Python type object is a very particular kind of Python-type.  As far 
>>as I can tell, it's not as flexible in terms of the kinds of things you 
>>can do with the "instances" of a type object (i.e. what ctypes types 
>>are) on the C-level.
> 
> 
> Ah, you are worried that NumArray objects would have to be *instances*
> of ctypes types. That wouldn't be necessary at all. Instead, if each
> NumArray object had a method get_ctype(), which returned a ctypes type,
> then you would get the same desciptiveness that you get with the
> PEP's datatype.
> 

No, I'm not worried about that (It's not NumArray by the way, it's 
NumPy.  NumPy replaces both NumArray and Numeric).

NumPy actually interfaces with ctypes quite well.  This is how I learned 
anything I might know about ctypes.  So, I'm well aware of this.

What I am concerned about is using Python type objects (i.e. Python 
objects that can be cast in C to PyTypeObject *) outside of ctypes to 
describe data-formats when you don't need it and it just complicates 
dealing with the data-format description.

> 
>>Where is the discussion that crowned the ctypes way of doing things as 
>>"the one true way"
> 
> 
> It hasn't been crowned this way. Me, personally, I just said two things
> about this PEP and ctypes:

Thanks for clarifying, but I know you didn't say this.  Others, however, 
basically did.

> a) the PEP does not support all concepts that ctypes needs

It could be extended, but I'm not sure it *needs* to be in it's real 
context.  I'm very sorry for contributing to the distraction that ctypes 
should adopt the PEP.  My words were unclear.  But, I'm not pushing for 
that.  I really have no opinion how ctypes describes data.

> b) ctypes can express all examples in the PEP
> in response to your proposal that ctypes should adopt the PEP, and
> that ctypes is not good enough to be the one true way.
> 

I think it is "good enough" in the semantic sense.  But, I think using 
type objects in this fashion for general-purpose data-description is 
over-kill and will be much harder to extend and deal with.

-Travis