[Cython] local variable handling in generators

[Stefan Behnel]

Vitja Makarov, 23.05.2011 11:24:
> 2011/5/23 Stefan Behnel:
>> Vitja Makarov, 23.05.2011 10:50:
>>>
>>> 2011/5/23 Stefan Behnel:
>>>>
>>>> Vitja Makarov, 23.05.2011 10:13:
>>>>>
>>>>> With live variable analysis that should be easy to save/restore only
>>>>> active variables at the yield point.
>>>>
>>>> "Active" in the sense of "modified", I suppose? That's what I was
>>>> expecting.
>>>
>>> Active means that variable value will be used. In my example after
>>> 'print a' a isn't used anymore.
>>
>> That's not correct then. In a generator, a modified value must be kept alive
>> over a yield, even if it is no longer used afterwards.
>>
>> We can safely reduce the write-back code to modified values, but we cannot
>> reduce it to values to that will be used later on.
>
> I'm not sure how to get modified variables list at yield point. Now I
> only know which assignments reach yield point.

But then you already know which variables were assigned to, right? That's 
all you need.


>>> For instance:
>>>
>>> for i in args:
>>>      print i
>>>
>>> this code will be translated into:
>>>
>>> PyObject *i = NULL;
>>>
>>> for (;;)
>>> {
>>>     tmp = next();
>>>     if (!tmp) break;
>>>
>>>    Pyx_XDECREF(i);
>>>    i = tmp;
>>>    print(i);
>>> }
>>>
>>> using active variables information this could be translated into:
>>>
>>> PyObject *i = NULL;
>>>
>>> for (;;)
>>> {
>>>     tmp = next();
>>>     if (!tmp) break;
>>>
>>>    i = tmp;
>>>    print(i);
>>>    Pyx_DECREF(i);
>>> }
>>
>> That's not correct, though. Python semantics dictate that 'i' must keep its
>> value until the end of the function or until it's being reassigned to,
>> whatever comes first. Remember that objects can have deallocators in Python.
>> That must not be called at an undefined point.
>>
>> The only thing that can safely be special cased is None. It's common in
>> Python code to set a variable to None when the value is worth being
>> deallocated (e.g. a large data structure). Cython can optimise this as I
>> indicated above.
>
> Ohh, I see that variable references couldn't be simply removed.
>
> Unused result reference removal seems safe to me:
>
> a = foo() # a will be assigned to NULL here
> print

No, same thing: the reference that 'a' holds here may be important, so we 
cannot just drop it at an arbitrary point. There's (likely) a reason the 
user chose to use a variable, instead of just letting the return value fall 
off the shelve silently.

Stefan