[Cython] Utility Codes and templates

[Stefan Behnel]

Robert Bradshaw, 22.07.2011 23:44:
> On Fri, Jul 22, 2011 at 1:39 PM, mark florisson
> <markflorisson88 at gmail.com>  wrote:
>> On 22 July 2011 22:05, Robert Bradshaw<robertwb at math.washington.edu>  wrote:
>>> On Fri, Jul 22, 2011 at 3:12 AM, mark florisson
>>> <markflorisson88 at gmail.com>  wrote:
>>>> For my work on the _memview branch (and also on fused types) I noticed
>>>> that UtilityCodes started weighing heavily on me in their current
>>>> form, so I wrote a little loader in the _memview branch:
>>>>
>>>> https://github.com/markflorisson88/cython/commit/e13debed2db78680ec0bd8c343433a2b73bd5e64#L2R110
>>>>
>>>> The idea is simple: you put your utility codes in Cython/Utility in
>>>> .pyx, .c, .h files etc, and then load them. It works for both
>>>> prototypes and implementations, for UtilityCode and CythonUtilityCode:
>>>
>>> This sounds like it could be a nice way to organize our UtilityCode
>>> snippets. So far we haven't really needed any more templating than
>>> simple substitution, but for what you're doing I can see this being
>>> quite handy. This may also provide a more flexible way forward for
>>> supporting multiple backends.

It's unlikely to be useful for the ctypes backend, but C/C++ will benefit 
from it, and so may others.


>>>> myutility.c
>>>>
>>>> // UtilityProto: MyUtility
>>>> header code here
>>>>
>>>> // UtilityCode: MyUtility
>>>> implementation code here
>>>>
>>>> You can add as many other utilities as you like to the same file. You
>>>> can then load it using
>>>>
>>>>     UtilityCode.load_utility_from_file("myutility.c", "MyUtility")
>>>
>>> I agree with you that having multiple related, named snippets in same
>>> file is worthwhile. What about
>>>
>>> ////////////////////// MyUtility.proto ///////////////////////////
>>>
>>> and
>>>
>>> ############ MyCyUtility ##############

Hmm, right. Cython utility code doesn't actually need any separate parts. I 
hadn't realised that.

+1, the above is well visible and doesn't hurt the file syntax.


>> Sounds like a good idea. How would we handle the requires list in that
>> case though? Or should we leave that to the Python code that loads it
>> for now?
>
> For now, lets leave them with the Python code.

-1, I see no reason to do it the wrong way now only to clean it up later. 
Why not just do it right? It's not like that's so much more work. I can 
help out.


> Eventually we could do
> something similar to how we do Cython pragmas, i.e. parse the comments
> that occur at the top of the file (chunk) as having special meaning.

Requiring a short declaration at the top of the file makes it easier to 
find out what's in that file from a quick look at the top, rather than 
having to skip (or grep) through it. Yes, I think it makes sense to put 
this into some kind of file header.

Coming back to my ini file proposal, what about a file syntax like this:


/*
[MyUtil1]
requires = file1:abc

[MyUtil2]
; nothing to declare ...  (I think it's ; for comments - don't care)
*/

///////////////// MyUtil1.proto /////////////////////
...


and the same for Cython code:

"""
[MyUtil1]
requires = file1:abc

[MyUtil2]
requires = MyUtil1
"""

################### MyUtil1 #####################
...

The initial section is trivial to extract and to drop into RawConfigParser, 
and we are completely free to extend the metadata section later on, without 
having to touch the code as well.


>>> In terms of delimiters, I prefer some form of strings over comments
>>> (often a value is expected, and Python's comments always extend to the
>>> line break)

Yes, it's clearly a problem in Cython code. It would work very nicely for C 
code, though. I think we should use it there right from the start.


>>> but sticking with the {{ }} syntax appeals to me even
>>> more. It's pretty standard, and decent syntax hilighters shouldn't
>>> choke on it (well, the surrounding code) too bad.
>>
>> Ok, that's fine with me. It would be easily changeable if we revert
>> later anyways.
>
> Yep, which makes it a much easier decision than trying to figure out a
> templating solution for the user-visible front-end.

I'm fine with {{ }} for Cython code. It's just as good as anything else.


>> I'll add tempita to Cython.
>
> Sounds good. (If anyone else has objections, bring them up now...)

Tempita is fine with me.

Stefan