[Cython] Cython 0.17 beta 3 released - release candidate

Mon Aug 27 11:42:24 CEST 2012

On 27 August 2012 01:39, Christoph Gohlke <cgohlke at uci.edu> wrote:
> On 8/26/2012 4:08 AM, mark florisson wrote:
>>
>> On 25 August 2012 03:07, Christoph Gohlke <cgohlke at uci.edu> wrote:
>>>
>>> Hi,
>>>
>>>
>>> On 8/24/2012 12:43 PM, Stefan Behnel wrote:
>>>>
>>>>
>>>> Hi,
>>>>
>>>> thanks for testing!
>>>>
>>>> Christoph Gohlke, 24.08.2012 07:20:
>>>>>
>>>>>
>>>>> I tested Cython-0.17b3 on Windows 7 with Visual Studio compilers.
>>>>>
>>>>> 32 bit Python 2.7 works well, only 4 test failures.
>>>>
>>>>
>>>>
>>>> Three of those errors are in OpenMP tests - is OpenMP supported in your
>>>> build environment?
>>>
>>>
>>>
>>> OpenMP is available on my system, and parallel.pyd is linked to the
>>> openmp
>>> library. The prange tests fail only sometimes. On my system, the prange
>>> index is sometimes left at the start (zero) of the range, while the tests
>>> expect the index to be left at the stop of the range. According to the
>>> Cython prange enhancements webpage "the iterations of the loop body can
>>> be
>>> executed in any order" <http://wiki.cython.org/enhancements/prange>.
>>> Where
>>> does that leave the loop index?
>>>
>>
>> I should be the value from the last iteration, but in my experience
>> many compilers have buggy OpenMP implementations. I think your
>> compiler doesn't correctly support the lastprivate clause in all
>> situations. For instance, test_prange fails (which doesn't have any
>> break/return/exceptions), which simply has a lastprivate clause and a
>> schedule clause set to 'dynamic'. The test without the schedule clause
>> works fine (or maybe it's just luck). Or maybe it doesn't support
>> multiple lastprivate() clauses? I'm not entirely sure... It also seems
>> the thread limit on your system is 1.
>>
>> In any case, the generated code for these tests looks correct to me,
>> but we've had similar problems before with different compilers...
>
>
>
> On my system, the following patch fixes all the lastprivate related test
> errors and does not have any side effects on other tests. It removes the
> additional initialization of the target index after `#pragma omp parallel`:
>
> diff --git a/Cython/Compiler/Nodes.py b/Cython/Compiler/Nodes.py
> index 188de3d..5ba0eee 100644
> --- a/Cython/Compiler/Nodes.py
> +++ b/Cython/Compiler/Nodes.py
> @@ -7627,7 +7627,7 @@ class ParallelRangeNode(ParallelStatNode):
>          # target index uninitialized
>          code.putln("if (%(nsteps)s > 0)" % fmt_dict)
>          code.begin_block() # if block
> -        code.putln("%(target)s = 0;" % fmt_dict)
>          self.generate_loop(code, fmt_dict)
>          code.end_block() # end if block
>
> I'm down to one remaining test failure on win32-py2.7
>
> Christoph
>

Hey Christoph,

Thanks, that's useful. Could you make a pull request? Maybe because
the entry to the parallel loop has no barrier the assignment comes
after the lastprivate? I'm not sure how that's possible, I would
expect the lastprivate to be executed after the barrier by the highest
ranking thread, so there would be no race. Maybe the last thread is
slow, and doesn't get any iterations, and assigns the firstprivate
value? Anyway, if it fixes stuff it's great :)

I think the patch should then also remove the firstprivate() clause,
since that variable wouldn't be initialized. It was only there to make
compiler warnings to away.

Mark
>
>
>>
>>>
>>>>
>>>> The other one is the new "initial_file_path" test that fails with this
>>>> linker error:
>>>>
>>>> """
>>>> C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\link.exe /DLL
>>>> /nologo /INCREMENTAL:NO /INCREMENTAL:YES        /DEBUG
>>>> /LIBPATH:X:\Python27\libs
>>>> /LIBPATH:X:\Python27\PCbuild /EXPORT:init__init__
>>>> build\temp.win32-2.7\Release\my_test_package\__init__.obj
>>>>
>>>>
>>>> /OUT:\Cython-0.17b3\BUILD\run\initial_file_path\my_test_package\__init__.pyd
>>>> /IMPLIB:build\temp.win32-2.7\Release\my_test_package\__init__.lib
>>>>
>>>>
>>>> /MANIFESTFILE:build\temp.win32-2.7\Release\my_test_package\__init__.pyd.manifest
>>>>
>>>> LINK : error LNK2001: unresolved external symbol init__init__
>>>> """
>>>>
>>>> Maybe the Windows build of distutils is broken here - it seems to assume
>>>> the wrong module name for the package module.
>>>
>>>
>>>
>>> I think this is an issue with the test. The extension does compile and
>>> link
>>> outside of the tests.
>>>
>>>
>>>
>>>>
>>>> I guess compiling package modules is just an overall badly supported
>>>> feature in CPython...
>>>>
>>>>
>>>>> On 64 bit Python 2.7 and 3.2 with msvc9 compiler, python.exe crashes
>>>>> during
>>>>> `test_slice_assignment (memslice.__test__)`. I tested two computers.
>>>>> The
>>>>> Windows executive can not identify in which specific module it crashes,
>>>>> and
>>>>> neither enabling faulthandler nor building with debug symbols gives any
>>>>> useful information. Can anyone reproduce this? It seems compiler
>>>>> specific
>>>>> since Python 3.3, which is using msvc10, does not crash.
>>>>
>>>>
>>>>
>>>> Hmm, yes, sounds like a problem with the compiler. Would be good to get
>>>> this sorted out, but it's almost impossible to debug something like this
>>>> from a distance.
>>>
>>>
>>>
>>>
>>> Maybe the following simple example is related. It fails (not crash) when
>>> compiled with 64 bit msvc9, but does work with 32 bit msvc9 and msvc10
>>> (32
>>> and 64 bit):
>>>
>>> ```
>>> from cython.view cimport array as cvarray
>>> import numpy as np
>>>
>>> narr = np.arange(8 * 14 * 11).reshape((8, 14, 11))
>>>
>>> cdef int[:, :, ::1] a = narr
>>> cdef int[:, :, :] b = a[2:8:2, -4:1:-1, 1:3]
>>>
>>> print narr[2:8:2, -4:1:-1, 1:3].shape
>>> print b.shape[0], b.shape[1], b.shape[2]
>>> ```
>>>
>>> On win-amd64-py2.x the shape of b is (3, 9, 3) but it should be (3, 9, 2)
>>>
>>>
>>>
>>>>
>>>>
>>>>> When disabling
>>>>> test_slice_assignment, runtests.py completes with many failures.
>>>>>
>>>>> The results of `runtests.py -v -v` are at
>>>>> <http://www.lfd.uci.edu/~gohlke/pythonlibs/tests/cython/>.
>>>>
>>>>
>>>>
>>>> The 64bit output looks so broken that I wonder what went wrong here. I
>>>> mean, most of the problems look like this:
>>>>
>>>> """
>>>> Expected:
>>>>       Traceback (most recent call last):
>>>>       TypeError: m() takes at most 2 positional arguments (3 given)
>>>> Got:
>>>>       Traceback (most recent call last):
>>>>       TypeError: m() takes at most %Id positional argument%s (%Id given)
>>>> """
>>>>
>>>> I have no idea how that can happen.
>>>>
>>>> I can see two other problems, one is the linker warning about the module
>>>> init function in Py3:
>>>>
>>>> """
>>>> bufaccess.obj : warning LNK4197: export 'PyInit_bufaccess' specified
>>>> multiple times; using first specification
>>>> """
>>>
>>>
>>>
>>> This is "normal".
>>>
>>>
>>>
>>>>
>>>> The other one is about mixing Py_ssize_t and int (and some other) types
>>>> all
>>>> over the place:
>>>>
>>>> """
>>>> bufaccess.c(3033) : warning C4244: '=' : conversion from 'Py_ssize_t' to
>>>> 'int', possible loss of data
>>>> """
>>>>
>>>> Some of them look like we'd need an explicit cast in the C code
>>>> somewhere,
>>>> others might hint at lax type usage in tests.
>>>>
>>>> There's also this:
>>>>
>>>> """
>>>> C:\Program Files (x86)\Microsoft Visual Studio
>>>> 10.0\VC\INCLUDE\xlocale(323)
>>>> : warning C4530: C++ exception handler used, but unwind semantics are
>>>> not
>>>> enabled. Specify /EHsc
>>>> """
>>>>
>>>> Stefan
>>>>
>>>
>>>
>>> Thanks,
>>>
>>> Christoph
>>>
>>> _______________________________________________
>>> cython-devel mailing list
>>> cython-devel at python.org
>>> http://mail.python.org/mailman/listinfo/cython-devel
>>
>> _______________________________________________
>> cython-devel mailing list
>> cython-devel at python.org
>> http://mail.python.org/mailman/listinfo/cython-devel
>>
>>
>>
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel