[Numpy-discussion] On my Cython/NumPy project

[Anne Archibald]

2008/6/21 Dag Sverre Seljebotn <dagss at student.matnat.uio.no>:
> Dag wrote:
>> General feedback is welcome; in particular, I need more opinions about
>> what syntax people would like. We seem unable to find something that we
>> really like; this is the current best candidate (cdef is the way you
>> declare types on variables in Cython):
>>
>> cdef int i = 4, j = 6
>> cdef np.ndarray[np.float64, 2] arr = np.zeros((10, 10), dtype=np.float64)
>> arr[i, j] = 1
>> ...
>>
>
> Some more important points:
> - There will likely be a compiler flag (and perhaps per-block or
> per-variable pragmas) on whether bounds-checking is done or not)
> - It doesn't look like negative indices will be supported in this mode, as
> it adds another branch in potentially very tight loops. Opinions on this
> are welcome though. In safe mode, bounds checking will catch it, while in
> unsafe mode it will at best segfault and at worst corrupt data.
>
> The negative indices thing is potentially confusing to new users. Pex uses
> a different syntax (arr{i, j} for efficient array lookups) partly to make
> this fact very explicit. Thoughts?

I am very worried about the negative numbers issue. It's the sort of
thing that will readily lead to errors, and that produces a
significant difference between cython and python. I understand the
performance issues that motivate it, but cython really needs to be
easy to use or we might as well just use C. As I understand it, the
ultimate goal should be for users to be able to compile arbitrary
python code for tiny speed improvements, then add type annotations for
key variables to obtain large speed improvements. Forbidding negative
indices now prevents that goal.

My suggestion is this: allow negative indices, accepting the cost in
tight loops. (If bounds checking is enabled, the cost will be
negligible anyway.) Provide a #pragma allowing the user to assert that
a certain piece of code uses no negative indices. Ultimately, the
cython compiler will often be able to deduce than negative indices
will not be used within a particular loop and supply the pragma
itself, but in the meantime this solution allows people to use the
extremely-convenient negative indices without causing mysterious
problems, and it sets things up for the future. (Other compiler-based
cleverness can deal with the most common case by converting explicit
negative indices.)



Anne