[Cython] unexpected side-effect in cython utility code

[mark florisson]

On 14 October 2011 16:18, mark florisson <markflorisson88 at gmail.com> wrote:
> On 14 October 2011 14:02, Stefan Behnel <stefan_ml at behnel.de> wrote:
>> Hi,
>>
>> I started working on better malloc() support and wrote this code as a test
>> to get going:
>>
>> """
>> cimport cython
>>
>> def test_malloc(int n):
>>    with cython.malloc(n*sizeof(int)) as m:
>>        for i in range(n):
>>            m[i] = i
>>        l = [ m[i] for i in range(n) ]
>>    return l
>> """
>>
>> Now, when I compile this normally, I get a compiler error about "malloc" not
>> being a cython attribute. However, when I do the same in the test runner, it
>> compiles without errors and crashes when trying to run the test. The code it
>> generates for the 'with' statement above starts like this:
>>
>> """
>>    __pyx_t_1 = PyObject_GetAttr(((PyObject *)malloc((__pyx_v_n *
>> (sizeof(int))))), __pyx_n_s____exit__); /*...*/
>> """
>>
>> It appears that something has declared malloc(). I'm pretty sure it's this
>> code in UtilityCode.py:
>>
>> """
>>    def declare_in_scope(self, dest_scope, used=False, cython_scope=None):
>>        """
>>        Declare all entries from the utility code in dest_scope. Code will
>>        only be included for used entries. If module_name is given,
>>        declare the type entries with that name.
>>        """
>>        tree = self.get_tree(entries_only=True, cython_scope=cython_scope)
>>
>>        entries = tree.scope.entries
>>        entries.pop('__name__')
>>        entries.pop('__file__')
>>        entries.pop('__builtins__')
>>        entries.pop('__doc__')
>>
>>        for name, entry in entries.iteritems():
>>            entry.utility_code_definition = self
>>            entry.used = used
>> """
>>
>> Basically, it declares everything it finds except for an explicit blacklist.
>> Bad design. As I argued before, it should use a whitelist in the utility
>> code file instead, which specifically lists the names that should be public.
>> Everything else should just be considered implementation details.
>>
>> Stefan
>> _______________________________________________
>> cython-devel mailing list
>> cython-devel at python.org
>> http://mail.python.org/mailman/listinfo/cython-devel
>>
>
> That would indeed be better, I just never got around to it and I must
> admit that it was never my priority.
>
> About cython.malloc, wouldn't it be nicer if we had automatic (stack
> or heap allocated) arrays? e.g.
>
> def func(int n):
>    cdef int array[n]
>
> I think you usually have homogenous data anyway. When you return, it
> simply goes out of scope like normal automatic variables, I see no
> clear advantage to 'with' here.
>

Actually these whitelists are really uncomfortable to work with (which
is one of the reasons I didn't use them). I think a decorator like
'@public' or some such would be nicer here.