[Cython] buffer syntax vs. memory view syntax

mark florisson markflorisson88 at gmail.com
Tue May 8 12:35:30 CEST 2012

On 8 May 2012 10:48, Stefan Behnel <stefan_ml at behnel.de> wrote:
> mark florisson, 08.05.2012 11:24:
>>>>> Dag Sverre Seljebotn, 08.05.2012 09:57:
>>>>>>  1) We NEVER deprecate "np.ndarray[double]", we commit to keeping that in
>>>>>> the language. It means exactly what you would like double[:] to mean,
>>>>>> i.e.
>>>>>> a variable that is memoryview when you need to and an object otherwise.
>>>>>> When you use this type, you bear the consequences of early-binding things
>>>>>> that could in theory be overridden.
>>>>>>  2) double[:] is for when you want to access data of *any* Python object
>>>>>> in a generic way. Raw PEP 3118. In those situations, access to the
>>>>>> underlying object is much less useful.
>>>>>>   2a) Therefore we require that you do "mview.asobject()" manually; doing
>>>>>> "mview.foo()" is a compile-time error
>>> [...]
>>> Character pointers coerce to strings. Hell, even structs coerce to and
>>> from python dicts, so disallowing the same for memoryviews would just
>>> be inconsistent and inconvenient.
> Two separate things to discuss here: the original exporter and a Python
> level wrapper.
> As long as wrapping the memoryview in a new object is can easily be done by
> users, I don't see a reason to provide compiler support for getting at the
> exporter.

Well, the support is already there :) It's basically to be consistent
with numpy's attributes.

> After all, a user may have a memory view that is backed by a
> NumPy array but wants to reinterpret it as a PIL image. Just because the
> underlying object has a specific object type doesn't mean that's the one to
> use for a given use case. If a user requires a specific object *instead* of
> a bare memory view, we have the object type buffer syntax for that.

Which is better deprecated to allow only one way to do things, and to
make fused extension types less confusing.

> It's also not necessarily more efficient to access the underlying object
> than to create a new one if the underlying exporter has to learn about the
> mapped layout first.
> Regarding the coercion to Python, I do not see a problem with providing a
> general Python view object for memory views that arbitrary Cython memory
> views can coerce to. In fact, I consider that a useful feature. The builtin
> memoryview type in Python (at least the one in CPython 3.3) should be quite
> capable of providing this, although I don't mind what exactly this becomes.

There are two ways to argue this entire problem, one is from a
theoretical standpoint, and one from a pragmatic. Theoretically your
points are sound, but in practice 99% of the uses will be numpy
arrays, and in 99% of those uses people will want one back. If one
does not allow easy, compiler-supported, conversion, then any numpy
operation will go from typed memoryview slice -> memoryview object ->
buffer interface -> some computation in numpy -> buffer interface ->
typed memoryview. The compiler can help here by maintaining cached
views aided by a user callback. In the case you're not slicing, you
can just return the original object.

I'm not sure how to register those callbacks though, as making them
global may interfere between projects. Maybe it should be a module
level thing?

>> Also, if you don't allow coercion from python, then it means they also
>> cannot be used as 'def' function arguments and be called from python.
> Coercion *from* Python is not being questioned. We have syntax for that,
> and a Python memory view wrapper can easily be unboxed (even transitively)
> through the buffer interface when entering back into Cython.
> Stefan
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel

More information about the cython-devel mailing list