[Cython] Cython 0.16 RC 1

mark florisson markflorisson88 at gmail.com
Sat Apr 14 12:46:23 CEST 2012


On 12 April 2012 22:00, Wes McKinney <wesmckinn at gmail.com> wrote:
> On Thu, Apr 12, 2012 at 10:38 AM, mark florisson
> <markflorisson88 at gmail.com> wrote:
>> Yet another release candidate, this will hopefully be the last before
>> the 0.16 release. You can grab it from here:
>> http://wiki.cython.org/ReleaseNotes-0.16
>>
>> There were several fixes for the numpy attribute rewrite, memoryviews
>> and fused types. Accessing the 'base' attribute of a typed ndarray now
>> goes through the object layer, which means direct assignment is no
>> longer supported.
>>
>> If there are any problems, please let us know.
>> _______________________________________________
>> cython-devel mailing list
>> cython-devel at python.org
>> http://mail.python.org/mailman/listinfo/cython-devel
>
> I'm unable to build pandas using git master Cython. I just released
> pandas 0.7.3 today which has no issues at all with 0.15.1:
>
> http://pypi.python.org/pypi/pandas
>
> For example:
>
> 16:57 ~/code/pandas  (master)$ python setup.py build_ext --inplace
> running build_ext
> cythoning pandas/src/tseries.pyx to pandas/src/tseries.c
>
> Error compiling Cython file:
> ------------------------------------------------------------
> ...
>        self.store = {}
>
>        ptr = <int32_t**> malloc(self.depth * sizeof(int32_t*))
>
>        for i in range(self.depth):
>            ptr[i] = <int32_t*> (<ndarray> label_arrays[i]).data
>                                                          ^
> ------------------------------------------------------------
>
> pandas/src/tseries.pyx:107:59: Compiler crash in AnalyseExpressionsTransform
>
> ModuleNode.body = StatListNode(tseries.pyx:1:0)
> StatListNode.stats[23] = StatListNode(tseries.pyx:86:5)
> StatListNode.stats[0] = CClassDefNode(tseries.pyx:86:5,
>    as_name = u'MultiMap',
>    class_name = u'MultiMap',
>    doc = u'\n    Need to come up with a better data structure for
> multi-level indexing\n    ',
>    module_name = u'',
>    visibility = u'private')
> CClassDefNode.body = StatListNode(tseries.pyx:91:4)
> StatListNode.stats[1] = StatListNode(tseries.pyx:95:4)
> StatListNode.stats[0] = DefNode(tseries.pyx:95:4,
>    modifiers = [...]/0,
>    name = u'__init__',
>    num_required_args = 2,
>    py_wrapper_required = True,
>    reqd_kw_flags_cname = '0',
>    used = True)
> File 'Nodes.py', line 342, in analyse_expressions:
> StatListNode(tseries.pyx:96:8)
> File 'Nodes.py', line 342, in analyse_expressions:
> StatListNode(tseries.pyx:106:8)
> File 'Nodes.py', line 5903, in analyse_expressions:
> ForInStatNode(tseries.pyx:106:8)
> File 'Nodes.py', line 342, in analyse_expressions:
> StatListNode(tseries.pyx:107:21)
> File 'Nodes.py', line 4767, in analyse_expressions:
> SingleAssignmentNode(tseries.pyx:107:21)
> File 'Nodes.py', line 4872, in analyse_types:
> SingleAssignmentNode(tseries.pyx:107:21)
> File 'ExprNodes.py', line 7082, in analyse_types:
> TypecastNode(tseries.pyx:107:21,
>    result_is_used = True,
>    use_managed_ref = True)
> File 'ExprNodes.py', line 4274, in analyse_types:
> AttributeNode(tseries.pyx:107:59,
>    attribute = u'data',
>    initialized_check = True,
>    is_attribute = 1,
>    member = u'data',
>    needs_none_check = True,
>    op = '->',
>    result_is_used = True,
>    use_managed_ref = True)
> File 'ExprNodes.py', line 4360, in analyse_as_ordinary_attribute:
> AttributeNode(tseries.pyx:107:59,
>    attribute = u'data',
>    initialized_check = True,
>    is_attribute = 1,
>    member = u'data',
>    needs_none_check = True,
>    op = '->',
>    result_is_used = True,
>    use_managed_ref = True)
> File 'ExprNodes.py', line 4436, in analyse_attribute:
> AttributeNode(tseries.pyx:107:59,
>    attribute = u'data',
>    initialized_check = True,
>    is_attribute = 1,
>    member = u'data',
>    needs_none_check = True,
>    op = '->',
>    result_is_used = True,
>    use_managed_ref = True)
>
> Compiler crash traceback from this point on:
>  File "/home/wesm/code/repos/cython/Cython/Compiler/ExprNodes.py",
> line 4436, in analyse_attribute
>    replacement_node = numpy_transform_attribute_node(self)
>  File "/home/wesm/code/repos/cython/Cython/Compiler/NumpySupport.py",
> line 18, in numpy_transform_attribute_node
>    numpy_pxd_scope = node.obj.entry.type.scope.parent_scope
> AttributeError: 'TypecastNode' object has no attribute 'entry'
> building 'pandas._tseries' extension
> creating build
> creating build/temp.linux-x86_64-2.7
> creating build/temp.linux-x86_64-2.7/pandas
> creating build/temp.linux-x86_64-2.7/pandas/src
> gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -O2 -fPIC
> -I/home/wesm/epd/lib/python2.7/site-packages/numpy/core/include
> -I/home/wesm/epd/include/python2.7 -c pandas/src/tseries.c -o
> build/temp.linux-x86_64-2.7/pandas/src/tseries.o
> pandas/src/tseries.c:1:2: error: #error Do not use this file, it is
> the result of a failed Cython compilation.
> error: command 'gcc' failed with exit status 1
>
>
> -----
>
> I kludged this particular line in the pandas/timeseries branch so it
> will build on git master Cython, but I was treated to dozens of
> failures, errors, and finally a segfault in the middle of the test
> suite. Suffice to say I'm not sure I would advise you to release the
> library in its current state until all of this is resolved. Happy to
> help however I can but I'm back to 0.15.1 for now.
>
> - Wes
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel

It seems that the numpy stopgap solution broke something in Pandas,
I'm not sure what or how, but it leads to segfaults where code is
trying to retrieve objects from a numpy array that are NULL. I tried
disabling the numpy rewrites which unbreaks this with the cython
release branch, so I think we should do another RC either with the
attribute rewrite disabled or fixed.

Dag, do you know what could have been broken by this fix that could
lead to these results?


More information about the cython-devel mailing list