[Cython] Cython 0.17 beta 3 released - release candidate

mark florisson markflorisson88 at gmail.com
Sun Aug 26 23:45:51 CEST 2012


On 26 August 2012 22:45, mark florisson <markflorisson88 at gmail.com> wrote:
> On 26 August 2012 22:25, Christoph Gohlke <cgohlke at uci.edu> wrote:
>> On 8/26/2012 2:09 PM, Christoph Gohlke wrote:
>>>
>>> On 8/26/2012 4:08 AM, mark florisson wrote:
>>>>
>>>> On 25 August 2012 03:07, Christoph Gohlke <cgohlke at uci.edu> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>>
>>>>> On 8/24/2012 12:43 PM, Stefan Behnel wrote:
>>>>>>
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> thanks for testing!
>>>>>>
>>>>>> Christoph Gohlke, 24.08.2012 07:20:
>>>>>>>
>>>>>>>
>>>>>>> I tested Cython-0.17b3 on Windows 7 with Visual Studio compilers.
>>>>>>>
>>>>>>> 32 bit Python 2.7 works well, only 4 test failures.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Three of those errors are in OpenMP tests - is OpenMP supported in your
>>>>>> build environment?
>>>>>
>>>>>
>>>>>
>>>>> OpenMP is available on my system, and parallel.pyd is linked to the
>>>>> openmp
>>>>> library. The prange tests fail only sometimes. On my system, the prange
>>>>> index is sometimes left at the start (zero) of the range, while the
>>>>> tests
>>>>> expect the index to be left at the stop of the range. According to the
>>>>> Cython prange enhancements webpage "the iterations of the loop body
>>>>> can be
>>>>> executed in any order" <http://wiki.cython.org/enhancements/prange>.
>>>>> Where
>>>>> does that leave the loop index?
>>>>>
>>>>
>>>> I should be the value from the last iteration, but in my experience
>>>> many compilers have buggy OpenMP implementations. I think your
>>>> compiler doesn't correctly support the lastprivate clause in all
>>>> situations. For instance, test_prange fails (which doesn't have any
>>>> break/return/exceptions), which simply has a lastprivate clause and a
>>>> schedule clause set to 'dynamic'. The test without the schedule clause
>>>> works fine (or maybe it's just luck). Or maybe it doesn't support
>>>> multiple lastprivate() clauses? I'm not entirely sure... It also seems
>>>> the thread limit on your system is 1.
>>>>
>>>> In any case, the generated code for these tests looks correct to me,
>>>> but we've had similar problems before with different compilers...
>>>
>>>
>>> A minimal example that fails for me is:
>>>
>>> def test_parallel():
>>>      cdef int i = 0, s = 0
>>>      with nogil, cython.parallel.parallel():
>>>          for i in prange(10):
>>>              s += i
>>>      return i
>>>
>>> The returned value is often 0, otherwise 9 as expected.
>>>
>>> In the generated C code I see
>>>
>>>      #pragma omp for firstprivate(__pyx_v_i) lastprivate(__pyx_v_i)
>>>
>>> If I change this to
>>>
>>>      #pragma omp parallel for firstprivate(__pyx_v_i)
>>> lastprivate(__pyx_v_i)
>>>
>>> the function always returns the expected value. Does that make sense?
>>
>>
>>
>> Well, apparently it doesn't make sense because the value of s is not
>> correct.
>>
>> Christoph
>>
>
> That will use nested parallelism.

(Which means you need 's' to be a reduction)

>>
>>>
>>> Thanks for your help and patience.
>>>
>>> Christoph
>>>
>>>
>>>>
>>>>>
>>>>>>
>>>>>> The other one is the new "initial_file_path" test that fails with this
>>>>>> linker error:
>>>>>>
>>>>>> """
>>>>>> C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\link.exe /DLL
>>>>>> /nologo /INCREMENTAL:NO /INCREMENTAL:YES        /DEBUG
>>>>>> /LIBPATH:X:\Python27\libs
>>>>>> /LIBPATH:X:\Python27\PCbuild /EXPORT:init__init__
>>>>>> build\temp.win32-2.7\Release\my_test_package\__init__.obj
>>>>>>
>>>>>>
>>>>>> /OUT:\Cython-0.17b3\BUILD\run\initial_file_path\my_test_package\__init__.pyd
>>>>>>
>>>>>> /IMPLIB:build\temp.win32-2.7\Release\my_test_package\__init__.lib
>>>>>>
>>>>>>
>>>>>> /MANIFESTFILE:build\temp.win32-2.7\Release\my_test_package\__init__.pyd.manifest
>>>>>>
>>>>>>
>>>>>> LINK : error LNK2001: unresolved external symbol init__init__
>>>>>> """
>>>>>>
>>>>>> Maybe the Windows build of distutils is broken here - it seems to
>>>>>> assume
>>>>>> the wrong module name for the package module.
>>>>>
>>>>>
>>>>>
>>>>> I think this is an issue with the test. The extension does compile
>>>>> and link
>>>>> outside of the tests.
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> I guess compiling package modules is just an overall badly supported
>>>>>> feature in CPython...
>>>>>>
>>>>>>
>>>>>>> On 64 bit Python 2.7 and 3.2 with msvc9 compiler, python.exe crashes
>>>>>>> during
>>>>>>> `test_slice_assignment (memslice.__test__)`. I tested two
>>>>>>> computers. The
>>>>>>> Windows executive can not identify in which specific module it
>>>>>>> crashes,
>>>>>>> and
>>>>>>> neither enabling faulthandler nor building with debug symbols gives
>>>>>>> any
>>>>>>> useful information. Can anyone reproduce this? It seems compiler
>>>>>>> specific
>>>>>>> since Python 3.3, which is using msvc10, does not crash.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Hmm, yes, sounds like a problem with the compiler. Would be good to get
>>>>>> this sorted out, but it's almost impossible to debug something like
>>>>>> this
>>>>>> from a distance.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Maybe the following simple example is related. It fails (not crash) when
>>>>> compiled with 64 bit msvc9, but does work with 32 bit msvc9 and
>>>>> msvc10 (32
>>>>> and 64 bit):
>>>>>
>>>>> ```
>>>>> from cython.view cimport array as cvarray
>>>>> import numpy as np
>>>>>
>>>>> narr = np.arange(8 * 14 * 11).reshape((8, 14, 11))
>>>>>
>>>>> cdef int[:, :, ::1] a = narr
>>>>> cdef int[:, :, :] b = a[2:8:2, -4:1:-1, 1:3]
>>>>>
>>>>> print narr[2:8:2, -4:1:-1, 1:3].shape
>>>>> print b.shape[0], b.shape[1], b.shape[2]
>>>>> ```
>>>>>
>>>>> On win-amd64-py2.x the shape of b is (3, 9, 3) but it should be (3,
>>>>> 9, 2)
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>>> When disabling
>>>>>>> test_slice_assignment, runtests.py completes with many failures.
>>>>>>>
>>>>>>> The results of `runtests.py -v -v` are at
>>>>>>> <http://www.lfd.uci.edu/~gohlke/pythonlibs/tests/cython/>.
>>>>>>
>>>>>>
>>>>>>
>>>>>> The 64bit output looks so broken that I wonder what went wrong here. I
>>>>>> mean, most of the problems look like this:
>>>>>>
>>>>>> """
>>>>>> Expected:
>>>>>>       Traceback (most recent call last):
>>>>>>       TypeError: m() takes at most 2 positional arguments (3 given)
>>>>>> Got:
>>>>>>       Traceback (most recent call last):
>>>>>>       TypeError: m() takes at most %Id positional argument%s (%Id
>>>>>> given)
>>>>>> """
>>>>>>
>>>>>> I have no idea how that can happen.
>>>>>>
>>>>>> I can see two other problems, one is the linker warning about the
>>>>>> module
>>>>>> init function in Py3:
>>>>>>
>>>>>> """
>>>>>> bufaccess.obj : warning LNK4197: export 'PyInit_bufaccess' specified
>>>>>> multiple times; using first specification
>>>>>> """
>>>>>
>>>>>
>>>>>
>>>>> This is "normal".
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> The other one is about mixing Py_ssize_t and int (and some other) types
>>>>>> all
>>>>>> over the place:
>>>>>>
>>>>>> """
>>>>>> bufaccess.c(3033) : warning C4244: '=' : conversion from
>>>>>> 'Py_ssize_t' to
>>>>>> 'int', possible loss of data
>>>>>> """
>>>>>>
>>>>>> Some of them look like we'd need an explicit cast in the C code
>>>>>> somewhere,
>>>>>> others might hint at lax type usage in tests.
>>>>>>
>>>>>> There's also this:
>>>>>>
>>>>>> """
>>>>>> C:\Program Files (x86)\Microsoft Visual Studio
>>>>>> 10.0\VC\INCLUDE\xlocale(323)
>>>>>> : warning C4530: C++ exception handler used, but unwind semantics
>>>>>> are not
>>>>>> enabled. Specify /EHsc
>>>>>> """
>>>>>>
>>>>>> Stefan
>>>>>>
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Christoph
>>>>>
>>>>> _______________________________________________
>>>>> cython-devel mailing list
>>>>> cython-devel at python.org
>>>>> http://mail.python.org/mailman/listinfo/cython-devel
>>>>
>>>> _______________________________________________
>>>> cython-devel mailing list
>>>> cython-devel at python.org
>>>> http://mail.python.org/mailman/listinfo/cython-devel
>>>>
>>>>
>>>>
>>> _______________________________________________
>>> cython-devel mailing list
>>> cython-devel at python.org
>>> http://mail.python.org/mailman/listinfo/cython-devel
>>>
>>>
>>>
>> _______________________________________________
>> cython-devel mailing list
>> cython-devel at python.org
>> http://mail.python.org/mailman/listinfo/cython-devel


More information about the cython-devel mailing list