[Cython] unexpected side-effect in cython utility code

[mark florisson]

On 14 October 2011 14:02, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Hi,
>
> I started working on better malloc() support and wrote this code as a test
> to get going:
>
> """
> cimport cython
>
> def test_malloc(int n):
>    with cython.malloc(n*sizeof(int)) as m:
>        for i in range(n):
>            m[i] = i
>        l = [ m[i] for i in range(n) ]
>    return l
> """
>
> Now, when I compile this normally, I get a compiler error about "malloc" not
> being a cython attribute. However, when I do the same in the test runner, it
> compiles without errors and crashes when trying to run the test. The code it
> generates for the 'with' statement above starts like this:
>
> """
>    __pyx_t_1 = PyObject_GetAttr(((PyObject *)malloc((__pyx_v_n *
> (sizeof(int))))), __pyx_n_s____exit__); /*...*/
> """
>
> It appears that something has declared malloc(). I'm pretty sure it's this
> code in UtilityCode.py:
>
> """
>    def declare_in_scope(self, dest_scope, used=False, cython_scope=None):
>        """
>        Declare all entries from the utility code in dest_scope. Code will
>        only be included for used entries. If module_name is given,
>        declare the type entries with that name.
>        """
>        tree = self.get_tree(entries_only=True, cython_scope=cython_scope)
>
>        entries = tree.scope.entries
>        entries.pop('__name__')
>        entries.pop('__file__')
>        entries.pop('__builtins__')
>        entries.pop('__doc__')
>
>        for name, entry in entries.iteritems():
>            entry.utility_code_definition = self
>            entry.used = used
> """
>
> Basically, it declares everything it finds except for an explicit blacklist.
> Bad design. As I argued before, it should use a whitelist in the utility
> code file instead, which specifically lists the names that should be public.
> Everything else should just be considered implementation details.
>
> Stefan
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel
>

That would indeed be better, I just never got around to it and I must
admit that it was never my priority.

About cython.malloc, wouldn't it be nicer if we had automatic (stack
or heap allocated) arrays? e.g.

def func(int n):
    cdef int array[n]

I think you usually have homogenous data anyway. When you return, it
simply goes out of scope like normal automatic variables, I see no
clear advantage to 'with' here.