[Cython] Utility Codes and templates

[Robert Bradshaw]

On Fri, Jul 22, 2011 at 1:39 PM, mark florisson
<markflorisson88 at gmail.com> wrote:
> On 22 July 2011 22:05, Robert Bradshaw <robertwb at math.washington.edu> wrote:
>> On Fri, Jul 22, 2011 at 3:12 AM, mark florisson
>> <markflorisson88 at gmail.com> wrote:
>>> For my work on the _memview branch (and also on fused types) I noticed
>>> that UtilityCodes started weighing heavily on me in their current
>>> form, so I wrote a little loader in the _memview branch:
>>>
>>> https://github.com/markflorisson88/cython/commit/e13debed2db78680ec0bd8c343433a2b73bd5e64#L2R110
>>>
>>> The idea is simple: you put your utility codes in Cython/Utility in
>>> .pyx, .c, .h files etc, and then load them. It works for both
>>> prototypes and implementations, for UtilityCode and CythonUtilityCode:
>>
>> This sounds like it could be a nice way to organize our UtilityCode
>> snippets. So far we haven't really needed any more templating than
>> simple substitution, but for what you're doing I can see this being
>> quite handy. This may also provide a more flexible way forward for
>> supporting multiple backends.
>>
>>> myutility.c
>>>
>>> // UtilityProto: MyUtility
>>> header code here
>>>
>>> // UtilityCode: MyUtility
>>> implementation code here
>>>
>>> You can add as many other utilities as you like to the same file. You
>>> can then load it using
>>>
>>>    UtilityCode.load_utility_from_file("myutility.c", "MyUtility")
>>
>> I agree with you that having multiple related, named snippets in same
>> file is worthwhile. What about
>>
>> ////////////////////// MyUtility.proto ///////////////////////////
>>
>> and
>>
>> ############ MyCyUtility ##############
>>
>> so the chunks are easy to see.
>
> Sounds like a good idea. How would we handle the requires list in that
> case though? Or should we leave that to the Python code that loads it
> for now?

For now, lets leave them with the Python code. Eventually we could do
something similar to how we do Cython pragmas, i.e. parse the comments
that occur at the top of the file (chunk) as having special meaning.

>>> Of course you can pass in any other arguments, like proto_block, name,
>>> etc. You can additionally pass it a dict for formatting (for both the
>>> prototypes and the implementation). It will return a UtilityCode
>>> instance ready for use.
>>> You can also simply retrieve a utility code as a string, where it
>>> returns (proto, implementation).
>>>
>>> As debated before, an actual template library would be really
>>> convenient. Dag and I had a discussion about it and he suggested
>>> Tempita (by Ian Bicking), it is compatible with Python 2 and Python 3,
>>> and is pure-python. It supports all the good things like iteration,
>>> template inheritance, etc. Now I'm not sure whether it supports python
>>> 2.3 as it doesn't compile on my system, but it does support 2.4
>>> (confirmation for 2.3 would be appreciated). On a side note, I'd be
>>> perfectly happy to drop support for 2.3, it's kind of a chore.
>>> The documentation for Tempita can be found here: http://pythonpaste.org/tempita/
>>
>> Looks like a reasonable choice to me, it's small enough to ship and
>> looks versatile enough to do everything we need. There's certainly not
>> a shortage of options, but I'm happy with your and Dag's endorsement.
>>
>> In terms of delimiters, I prefer some form of strings over comments
>> (often a value is expected, and Python's comments always extend to the
>> line break) but sticking with the {{ }} syntax appeals to me even
>> more. It's pretty standard, and decent syntax hilighters shouldn't
>> choke on it (well, the surrounding code) too bad.
>
> Ok, that's fine with me. It would be easily changeable if we revert
> later anyways.

Yep, which makes it a much easier decision than trying to figure out a
templating solution for the user-visible front-end.

> I'll add tempita to Cython.

Sounds good. (If anyone else has objections, bring them up now...)

- Robert