[Cython] memoryview slices can't be None?

[mark florisson]

On 3 February 2012 18:07, mark florisson <markflorisson88 at gmail.com> wrote:
> On 3 February 2012 18:06, mark florisson <markflorisson88 at gmail.com> wrote:
>> On 3 February 2012 17:53, Dag Sverre Seljebotn
>> <d.s.seljebotn at astro.uio.no> wrote:
>>> On 02/03/2012 12:09 AM, mark florisson wrote:
>>>>
>>>> On 2 February 2012 21:38, Dag Sverre Seljebotn
>>>> <d.s.seljebotn at astro.uio.no>  wrote:
>>>>>
>>>>> On 02/02/2012 10:16 PM, mark florisson wrote:
>>>>>>
>>>>>>
>>>>>> On 2 February 2012 12:19, Dag Sverre Seljebotn
>>>>>> <d.s.seljebotn at astro.uio.no>    wrote:
>>>>>>>
>>>>>>>
>>>>>>> I just realized that
>>>>>>>
>>>>>>> cdef int[:] a = None
>>>>>>>
>>>>>>> raises an exception; even though I'd argue that 'a' is of the
>>>>>>> "reference"
>>>>>>> kind of type where Cython usually allow None (i.e., "cdef MyClass b =
>>>>>>> None"
>>>>>>> is allowed even if type(None) is NoneType). Is this a bug or not, and
>>>>>>> is
>>>>>>> it
>>>>>>> possible to do something about it?
>>>>>>>
>>>>>>> Dag Sverre
>>>>>>> _______________________________________________
>>>>>>> cython-devel mailing list
>>>>>>> cython-devel at python.org
>>>>>>> http://mail.python.org/mailman/listinfo/cython-devel
>>>>>>
>>>>>>
>>>>>>
>>>>>> Yeah I disabled that quite early. It was supposed to be working but
>>>>>> gave a lot of trouble in cases (segfaults, mainly). At the time I was
>>>>>> trying to get rid of all the segfaults and get the basic functionality
>>>>>> working, so I disabled it. Personally, I have never liked how things
>>>>>
>>>>>
>>>>>
>>>>> Well, you can segfault quite easily with
>>>>>
>>>>> cdef MyClass a = None
>>>>> print a.field
>>>>>
>>>>> so it doesn't make sense to slices different from cdef classes IMO.
>>>>>
>>>>>
>>>>>> can be None unchecked. I personally prefer to write
>>>>>>
>>>>>> cdef foo(obj=None):
>>>>>>     cdef int[:] a
>>>>>>     if obj is None:
>>>>>>         obj = ...
>>>>>>     a = obj
>>>>>>
>>>>>> Often you forget to write 'not None' when declaring the parameter (and
>>>>>> apparently that it only allowed for 'def' functions).
>>>>>>
>>>>>> As such, I never bothered to re-enable it. However, it does support
>>>>>> control flow with uninitialized slices, and will raise an error if it
>>>>>> is uninitialized. Do we want this behaviour (e.g. for consistency)?
>>>>>
>>>>>
>>>>>
>>>>> When in doubt, go for consistency. So +1 for that reason. I do believe
>>>>> that
>>>>> setting stuff to None is rather vital in Python.
>>>>>
>>>>> What I typically do is more like this:
>>>>>
>>>>> def f(double[:] input, double[:] out=None):
>>>>>    if out is None:
>>>>>        out = np.empty_like(input)
>>>>>    ...
>>>>>
>>>>> Having to use another variable name is a bit of a pain. (Come on -- do
>>>>> you
>>>>> use "a" in real code? What do you actually call "the other obj"? I
>>>>> sometimes
>>>>> end up with "out_" and so on, but it creates smelly code quite quickly.)
>>>>
>>>>
>>>> No, it was just a contrived example.
>>>>
>>>>> It's easy to segfault with cdef classes anyway, so decent nonechecking
>>>>> should be implemented at some point, and then memoryviews would use the
>>>>> same
>>>>> mechanisms. Java has decent null-checking...
>>>>>
>>>>
>>>> The problem with none checking is that it has to occur at every point.
>>>
>>>
>>> Well, using control flow analysis etc. it doesn't really. E.g.,
>>>
>>> for i in range(a.shape[0]):
>>>    print i
>>>    a[i] *= 3
>>>
>>> can be unrolled and none-checks inserted as
>>>
>>> print 0
>>> if a is None: raise ....
>>> a[0] *= 3
>>> for i in range(1, a.shape[0]):
>>>    print i
>>>    a[i] *= 3 # no need for none-check
>>>
>>> It's very similar to what you'd want to do to pull boundschecking out of the
>>> loop...
>>>
>>
>> Oh, definitely. Both optimizations may not always be possible to do,
>> though. The optimization (for boundschecking) is easier for prange()
>> than range(), as you can immediately raise an exception as the
>> exceptional condition may be issued at any iteration.  What do you do
>> with bounds checking when some accesses are in-bound, and some are
>> out-of-bound? Do you immediately raise the exception? Are we fine with
>> aborting (like Fortran compilers do when you ask them for bounds
>> checking)? And how do you detect that the code doesn't already raise
>> an exception or break out of the loop itself to prevent the
>> out-of-bound access? (Unless no exceptions are propagating and no
>> break/return is used, but exceptions are so very common).
>>
>>>> With initialized slices the control flow knows when the slices are
>>>> initialized, or when they might not be (and it can raise a
>>>> compile-time or runtime error, instead of a segfault if you're lucky).
>>>> I'm fine with implementing the behaviour, I just always left it at the
>>>> bottom of my todo list.
>>>
>>>
>>> Wasn't saying you should do it, just checking.
>>>
>>> I'm still not sure about this. I think what I'd really like is
>>>
>>>  a) Stop cdef classes from being None as well
>>>
>>>  b) Sort-of deprecate cdef in favor of cast/assertion type statements that
>>> help the type inferences:
>>>
>>> def f(arr):
>>>    if arr is None:
>>>        arr = ...
>>>    arr = int[:](arr) # equivalent to "cdef int[:] arr = arr", but
>>>                      # acts as statement, with a specific point
>>>                      # for the none-check
>>>    ...
>>>
>>> or even:
>>>
>>> def f(arr):
>>>    if arr is None:
>>>        return 'foo'
>>>    else:
>>>        arr = int[:](arr) # takes effect *here*, does none-check
>>>        ...
>>>    # arr still typed as int[:] here
>>>
>>> If we can make this work well enough with control flow analysis I'd never
>>> cdef declare local vars again :-)
>>
>> Hm, what about the following?
>>
>> def f(arr):
>>    if arr is None:
>>        return 'foo'
>>
>>    cdef int[:] arr # arr may not be None
>
> The above would work in general, until the declaration is lexically
> encountered, the object is typed as object.

Hurr, the *variable* is typed as object :)

>>> Dag
>>>
>>> _______________________________________________
>>> cython-devel mailing list
>>> cython-devel at python.org
>>> http://mail.python.org/mailman/listinfo/cython-devel