[Cython] Utility Codes and templates

[Vitja Makarov]

2011/7/26 Robert Bradshaw <robertwb at math.washington.edu>:
> On Mon, Jul 25, 2011 at 3:07 AM, mark florisson
> <markflorisson88 at gmail.com> wrote:
>> On 25 July 2011 12:00, Stefan Behnel <stefan_ml at behnel.de> wrote:
>>> Vitja Makarov, 25.07.2011 10:25:
>>>>
>>>> 2011/7/25 Stefan Behnel<stefan_ml at behnel.de>:
>>>>>
>>>>> Vitja Makarov, 25.07.2011 08:41:
>>>>>>
>>>>>> 2011/7/23 Robert Bradshaw<robertwb at math.washington.edu>:
>>>>>>>
>>>>>>> On Fri, Jul 22, 2011 at 3:12 AM, mark florisson
>>>>>>> <markflorisson88 at gmail.com>    wrote:
>>>>>>>>
>>>>>>>> For my work on the _memview branch (and also on fused types) I noticed
>>>>>>>> that UtilityCodes started weighing heavily on me in their current
>>>>>>>> form, so I wrote a little loader in the _memview branch:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> https://github.com/markflorisson88/cython/commit/e13debed2db78680ec0bd8c343433a2b73bd5e64#L2R110
>>>>>>>>
>>>>>>>> The idea is simple: you put your utility codes in Cython/Utility in
>>>>>>>> .pyx, .c, .h files etc, and then load them. It works for both
>>>>>>>> prototypes and implementations, for UtilityCode and CythonUtilityCode:
>>>>>>>
>>>>>>> This sounds like it could be a nice way to organize our UtilityCode
>>>>>>> snippets. So far we haven't really needed any more templating than
>>>>>>> simple substitution, but for what you're doing I can see this being
>>>>>>> quite handy. This may also provide a more flexible way forward for
>>>>>>> supporting multiple backends.
>>>>>>>
>>>>>>>> myutility.c
>>>>>>>>
>>>>>>>> // UtilityProto: MyUtility
>>>>>>>> header code here
>>>>>>>>
>>>>>>>> // UtilityCode: MyUtility
>>>>>>>> implementation code here
>>>>>>>>
>>>>>>>> You can add as many other utilities as you like to the same file. You
>>>>>>>> can then load it using
>>>>>>>>
>>>>>>>>    UtilityCode.load_utility_from_file("myutility.c", "MyUtility")
>>>>>>>
>>>>>>> I agree with you that having multiple related, named snippets in same
>>>>>>> file is worthwhile. What about
>>>>>>>
>>>>>>> ////////////////////// MyUtility.proto ///////////////////////////
>>>>>>>
>>>>>>> and
>>>>>>>
>>>>>>> ############ MyCyUtility ##############
>>>>>>>
>>>>>>> so the chunks are easy to see.
>>>>>>>
>>>>>>
>>>>>> C++ comments looks ugly. May be it's better to have something like this:
>>>>>>
>>>>>> /* UtilityCode:  MyUtility.proto */
>>>>>> and
>>>>>>
>>>>>> # UtilityCode: MyCyUtility
>>>>>>
>>>>>> That's also pretty easy to parse
>>>>>
>>>>> For the parser, it makes no difference. For a human, it does. A big fat
>>>>> marker like "/////////////////////" is hard to miss, whereas a tiny one
>>>>> like
>>>>> "/* ... */" is easily overlooked within a longer piece of code.
>>>>>
>>>>
>>>> Ok, but how many slashes should be there?
>>>>
>>>> I'd prefer something like:
>>>>
>>>> ########## UtilityCode: MyUtility ################
>>>>
>>>> For both C and Cython
>>>
>>> I don't think the parser should care, and I don't think we should put much
>>> thought into this kind of detail, either. I'd suggest making the separator
>>> lines at most 78 characters long, and putting as many slashes or hashes in
>>> there as are needed to fill up the lines after writing the text content.
>>> What's important to the parser is that there's more than one hash/slash at
>>> start and end, and something parsable in between. The rest is up to those
>>> who write the file.
>>
>> Yeah, I simply mandated a minimum of 5 and a maximum of 30 on either
>> side (symmetry is not enforced).
>
> I don't even think we need a max.
>
>> It's now 'MyUtility' and
>> 'MyUtility.proto'. If there's no objection to the ini-style header
>> (with requirements and other metadata possibly), then I'll implement
>> that one of these days.
>
> One drawback with the ini style is that it detaches the metadata from
> the code block itself (and require duplicating their names).
>

Are we going to move all the existing utility code away from python code?
I think that would seriously help with  moving shared C-code into
cython library (.h and .so).

Some things like generator class implementation and cyfunction could
be move into .so file.

-- 
vitja.