[Cython] buffer syntax vs. memory view syntax

mark florisson markflorisson88 at gmail.com
Mon May 7 19:08:52 CEST 2012

On 7 May 2012 18:00, Stefan Behnel <stefan_ml at behnel.de> wrote:
> mark florisson, 07.05.2012 18:28:
>> On 7 May 2012 17:00, Dag Sverre Seljebotn wrote:
>>> On 05/07/2012 04:16 PM, Stefan Behnel wrote:
>>>> Stefan Behnel, 07.05.2012 15:04:
>>>>> Dag Sverre Seljebotn, 07.05.2012 13:48:
>>>>>> BTW, with the coming of memoryviews, me and Mark talked about just
>>>>>> deprecating the "mytype[...]" meaning buffers, and rather treat it as
>>>>>> np.ndarray, array.array etc. being some sort of "template types". That
>>>>>> is,
>>>>>> we disallow "object[int]" and require some special declarations in the
>>>>>> relevant pxd files.
>>>>> Hmm, yes, it's unfortunate that we have two different types of syntax
>>>>> now,
>>>>> one that declares the item type before the brackets and one that declares
>>>>> it afterwards.
>>>> I actually think this merits some more discussion. Should we consider the
>>>> buffer interface syntax deprecated and focus on the memory view syntax?
>>> I think that's the very-long-term intention. Then again, it may be too early
>>> to really tell yet, we just need to see how the memory views play out in
>>> real life and whether they'll be able to replace np.ndarray[double] among
>>> real users. We don't want to shove things down users throats.
>>> But the use of the trailing-[] syntax needs some cleaning up. Me and Mark
>>> agreed we'd put this proposal forward when we got around to it:
>>>  - Deprecate the "object[double]" form, where [dtype] can be stuck on any
>>> extension type
>>>  - But, do NOT (for the next year at least) deprecate np.ndarray[double],
>>> array.array[double], etc. Basically, there should be a magic flag in
>>> extension type declarations saying "I can be a buffer".
>>> For one thing, that is sort of needed to open up things for templated cdef
>>> classes/fused types cdef classes, if that is ever implemented.
>> Deprecating is definitely a good start.
> Then the first step on that road is to rework the documentation so that it
> pushes users into going for memory views instead of the plain buffer syntax.

Well, memoryviews are not yet entirely bug free (although the next
release will aim to fix the problems pointed out by users so far), and
they also have some other problems.

>> I think at least if you only
>> allow two types as buffers it will be at least reasonably clear when
>> one is dealing with fused types or buffers.
>> Basically, I think memoryviews should live up to demands of the users,
>> which would mean there would be no reason to keep the buffer syntax.
>> One thing to do is make memoryviews coerce cheaply back to the
>> original objects if wanted (which is likely). Writting
>> np.asarray(mymemview) is kind of annoying.
> ... and also doesn't do the same thing, I believe.
>> Also, OT (sorry), but I'm kind of worried about the memoryview ABI. If
>> it changes (and I intend to do so), cython modules compiled with
>> different cython versions will become incompatible if they call each
>> other through pxds. Maybe that should be defined as UB...
> Would there be a way to only use the plain buffer interface for cross
> module memory view exchange? That could be an acceptable overhead to pay
> for ABI independence.

I want to store extra flags and pointers in there as well, so I don't
think that will be enough. It will also be rather annoying and
complicate calling code.

> Stefan
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel

More information about the cython-devel mailing list