[Cython] array expressions

[mark florisson]

On 23 October 2012 02:44, Robert Bradshaw <robertwb at gmail.com> wrote:
> On Sun, Oct 21, 2012 at 5:39 AM, mark florisson
> <markflorisson88 at gmail.com> wrote:
>> On 16 October 2012 18:48, Robert Bradshaw <robertwb at gmail.com> wrote:
>>> On Sun, Oct 14, 2012 at 6:13 AM, mark florisson
>>> <markflorisson88 at gmail.com> wrote:
>>>> On 14 October 2012 14:05, Stefan Behnel <stefan_ml at behnel.de> wrote:
>>>>> mark florisson, 14.10.2012 13:59:
>>>>>> The problem with minivect as a package is that it caters to different
>>>>>> projects, which have different requirements. Cython and minivect are
>>>>>> quite closely coupled, and any future change, or in the future any
>>>>>> older version may not have the functionality Cython needs, it's not
>>>>>> exactly a stable API at this point.
>>>>>
>>>>> Ok, understood.
>>>>>
>>>>>
>>>>>> For instance Numba needs python
>>>>>> 2.7, whereas Cython needs to be compatible with python 2.4.
>>>>>>
>>>>>> Before releasing minivect I'll verify every time that it doesn't break
>>>>>> Cython, but I currently have no real promises for backwards or forward
>>>>>> compatibility. And that is really because not all use cases have yet
>>>>>> been anticipated, and some really require a change, as I've already
>>>>>> seen with Numba.
>>>>>>
>>>>>> We could list minivect as a dependency, which works for
>>>>>> easy_install/pip users, but I just foresee numerous people running
>>>>>> into problems that didn't install with pip, and I don't think an
>>>>>> exclusion of a 300kb addition is worth any of that.
>>>>>
>>>>> Fine. In that case, I'm for not making minivect a separate package at all
>>>>> but including it directly and considering it a part of Cython (and Numba
>>>>> etc.) until there is enough of an interface to make it a reusable separate
>>>>> package, or at least to support a separate installation and independent
>>>>> update. Basically, if you can't update it separately, there's no use in
>>>>> installing it separately.
>>>>>
>>>>> As long as we handle this so, we should take care to keep the generic parts
>>>>> in their separate package directory and the Cython specific parts in
>>>>> Cython, and try to keep the interface between the two as cleanly separate
>>>>> as possible, so that we can actually reach a point where both have an
>>>>> interface. I would guess that the need to support Numba from the same
>>>>> source base will encourage this kind of separation anyway.
>>>>
>>>> Yes, definitely.
>>>>
>>>>> Note that this means that minivect will fall under the release schedules of
>>>>> Cython and Numba (independently), instead of really having its own releases.
>>>>
>>>> It can have its own releases as well, but currently there isn't much
>>>> point :) Minivect can be developed independent of the releases, since
>>>> Cython and Numba need to explicitly pull in the changes. Let's make a
>>>> habit of squashing the minivect pulls to avoid its history.
>>>>
>>>> I'll also wait for Dag and Robert to see if they have a (final)
>>>> opinion before merging the subtree.
>>>
>>> As I mentioned on the pull request, looks good to me. Given the
>>> (somewhat) tight coupling with the AST but the desire to use the
>>> codebase for multiple projects, a subtree seems to make the most sense
>>> (until/if we have some kind of a plugin system).
>>>
>>> - Robert
>>> _______________________________________________
>>> cython-devel mailing list
>>> cython-devel at python.org
>>> http://mail.python.org/mailman/listinfo/cython-devel
>>
>> Great, thanks for the review Robert. There seems to be a refcount
>> issue with python 3:
>>
>> cython at sage:/jenkins/workspaces/cython-mark-build/PYVERSION/py31/build/lib.linux-x86_64-3.1-pydebug$
>> /jenkins/workspaces/cython-mark-build/PYVERSION/py31/python/bin/python
>> Python 3.1.5+ (default:5a6fa1b8767f, Apr 11 2012, 23:32:58)
>> [GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu4)] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> from Cython.Compiler import Main
>> [98632 refs]
>>>>> ^D
>> [98632 refs]
>> python: Modules/gcmodule.c:327: visit_decref: Assertion
>> `gc->gc.gc_refs != 0' failed.
>>
>> The object being visited is a UtilityCode object. I think the error
>> means the object is being visited more often than it's refcount value,
>> which means it's not increffed properly somewhere. I'm not sure how/if
>> my changes introduced this, has this problem been encountered recently
>> in Cython's master?
>
> Not that I'm aware of. The refnanny is supposed to catch this kind of
> thing, is it enabled?
>
> - Robert
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel

Yeah, it doesn't catch it. I'll try tracing the imports to see what it
is importing that is different from master. Thanks.