[Cython] Cython 0.16 RC 1
Dag Sverre Seljebotn
d.s.seljebotn at astro.uio.no
Sat Apr 14 15:57:28 CEST 2012
On 04/14/2012 12:46 PM, mark florisson wrote:
> On 12 April 2012 22:00, Wes McKinney<wesmckinn at gmail.com> wrote:
>> On Thu, Apr 12, 2012 at 10:38 AM, mark florisson
>> <markflorisson88 at gmail.com> wrote:
>>> Yet another release candidate, this will hopefully be the last before
>>> the 0.16 release. You can grab it from here:
>>> http://wiki.cython.org/ReleaseNotes-0.16
>>>
>>> There were several fixes for the numpy attribute rewrite, memoryviews
>>> and fused types. Accessing the 'base' attribute of a typed ndarray now
>>> goes through the object layer, which means direct assignment is no
>>> longer supported.
>>>
>>> If there are any problems, please let us know.
>>> _______________________________________________
>>> cython-devel mailing list
>>> cython-devel at python.org
>>> http://mail.python.org/mailman/listinfo/cython-devel
>>
>> I'm unable to build pandas using git master Cython. I just released
>> pandas 0.7.3 today which has no issues at all with 0.15.1:
>>
>> http://pypi.python.org/pypi/pandas
>>
>> For example:
>>
>> 16:57 ~/code/pandas (master)$ python setup.py build_ext --inplace
>> running build_ext
>> cythoning pandas/src/tseries.pyx to pandas/src/tseries.c
>>
>> Error compiling Cython file:
>> ------------------------------------------------------------
>> ...
>> self.store = {}
>>
>> ptr =<int32_t**> malloc(self.depth * sizeof(int32_t*))
>>
>> for i in range(self.depth):
>> ptr[i] =<int32_t*> (<ndarray> label_arrays[i]).data
>> ^
>> ------------------------------------------------------------
>>
>> pandas/src/tseries.pyx:107:59: Compiler crash in AnalyseExpressionsTransform
>>
>> ModuleNode.body = StatListNode(tseries.pyx:1:0)
>> StatListNode.stats[23] = StatListNode(tseries.pyx:86:5)
>> StatListNode.stats[0] = CClassDefNode(tseries.pyx:86:5,
>> as_name = u'MultiMap',
>> class_name = u'MultiMap',
>> doc = u'\n Need to come up with a better data structure for
>> multi-level indexing\n ',
>> module_name = u'',
>> visibility = u'private')
>> CClassDefNode.body = StatListNode(tseries.pyx:91:4)
>> StatListNode.stats[1] = StatListNode(tseries.pyx:95:4)
>> StatListNode.stats[0] = DefNode(tseries.pyx:95:4,
>> modifiers = [...]/0,
>> name = u'__init__',
>> num_required_args = 2,
>> py_wrapper_required = True,
>> reqd_kw_flags_cname = '0',
>> used = True)
>> File 'Nodes.py', line 342, in analyse_expressions:
>> StatListNode(tseries.pyx:96:8)
>> File 'Nodes.py', line 342, in analyse_expressions:
>> StatListNode(tseries.pyx:106:8)
>> File 'Nodes.py', line 5903, in analyse_expressions:
>> ForInStatNode(tseries.pyx:106:8)
>> File 'Nodes.py', line 342, in analyse_expressions:
>> StatListNode(tseries.pyx:107:21)
>> File 'Nodes.py', line 4767, in analyse_expressions:
>> SingleAssignmentNode(tseries.pyx:107:21)
>> File 'Nodes.py', line 4872, in analyse_types:
>> SingleAssignmentNode(tseries.pyx:107:21)
>> File 'ExprNodes.py', line 7082, in analyse_types:
>> TypecastNode(tseries.pyx:107:21,
>> result_is_used = True,
>> use_managed_ref = True)
>> File 'ExprNodes.py', line 4274, in analyse_types:
>> AttributeNode(tseries.pyx:107:59,
>> attribute = u'data',
>> initialized_check = True,
>> is_attribute = 1,
>> member = u'data',
>> needs_none_check = True,
>> op = '->',
>> result_is_used = True,
>> use_managed_ref = True)
>> File 'ExprNodes.py', line 4360, in analyse_as_ordinary_attribute:
>> AttributeNode(tseries.pyx:107:59,
>> attribute = u'data',
>> initialized_check = True,
>> is_attribute = 1,
>> member = u'data',
>> needs_none_check = True,
>> op = '->',
>> result_is_used = True,
>> use_managed_ref = True)
>> File 'ExprNodes.py', line 4436, in analyse_attribute:
>> AttributeNode(tseries.pyx:107:59,
>> attribute = u'data',
>> initialized_check = True,
>> is_attribute = 1,
>> member = u'data',
>> needs_none_check = True,
>> op = '->',
>> result_is_used = True,
>> use_managed_ref = True)
>>
>> Compiler crash traceback from this point on:
>> File "/home/wesm/code/repos/cython/Cython/Compiler/ExprNodes.py",
>> line 4436, in analyse_attribute
>> replacement_node = numpy_transform_attribute_node(self)
>> File "/home/wesm/code/repos/cython/Cython/Compiler/NumpySupport.py",
>> line 18, in numpy_transform_attribute_node
>> numpy_pxd_scope = node.obj.entry.type.scope.parent_scope
>> AttributeError: 'TypecastNode' object has no attribute 'entry'
>> building 'pandas._tseries' extension
>> creating build
>> creating build/temp.linux-x86_64-2.7
>> creating build/temp.linux-x86_64-2.7/pandas
>> creating build/temp.linux-x86_64-2.7/pandas/src
>> gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -O2 -fPIC
>> -I/home/wesm/epd/lib/python2.7/site-packages/numpy/core/include
>> -I/home/wesm/epd/include/python2.7 -c pandas/src/tseries.c -o
>> build/temp.linux-x86_64-2.7/pandas/src/tseries.o
>> pandas/src/tseries.c:1:2: error: #error Do not use this file, it is
>> the result of a failed Cython compilation.
>> error: command 'gcc' failed with exit status 1
>>
>>
>> -----
>>
>> I kludged this particular line in the pandas/timeseries branch so it
>> will build on git master Cython, but I was treated to dozens of
>> failures, errors, and finally a segfault in the middle of the test
>> suite. Suffice to say I'm not sure I would advise you to release the
>> library in its current state until all of this is resolved. Happy to
>> help however I can but I'm back to 0.15.1 for now.
>>
>> - Wes
>> _______________________________________________
>> cython-devel mailing list
>> cython-devel at python.org
>> http://mail.python.org/mailman/listinfo/cython-devel
>
> It seems that the numpy stopgap solution broke something in Pandas,
> I'm not sure what or how, but it leads to segfaults where code is
> trying to retrieve objects from a numpy array that are NULL. I tried
> disabling the numpy rewrites which unbreaks this with the cython
> release branch, so I think we should do another RC either with the
> attribute rewrite disabled or fixed.
>
> Dag, do you know what could have been broken by this fix that could
> lead to these results?
I can't imagine what causes a change like you say... one thing that
could cause a segfault is that technically we should now call
import_array in every module using numpy.pxd; while we don't do that. If
a NumPy version is used where PyArray_DATA or similar is not a macro,
you would segfault....that should be fixed...
Dag
More information about the cython-devel
mailing list