git-friendly cythonize script + Tempita templating

It's getting close to the finish of my Fwrap work on SciPy. I'll get something to the list on that soon. Meanwhile here's a small start re: Cython vs. SciPy: cythonize.py ------------- I've been working with a lot of Cython files. Since Cython isn't (currently at least) invoked by the build system and is not a build-time dependency, I created a development-time utility script to scan for Cython files and invoke Cython in a way that is fast and hopefully safe with git (I had a Makefile, but that didn't work well with switching git branch). https://github.com/dagss/scipy-refactor/blob/cythonize/cythonize.py (Ignore the "-refactor", it is based off scipy/master). It: - First, checks if there's been any changes since the last time it was regenerated by you (checked by sha1 hash against a local "cythonize.dat") - Then, checks with git to see if the .c has been committed in a child commit to the .pyx (it is assumed you use git) - Finally, resort to invoking Cython. I'm sure it needs some polish/Python 3 support etc. (but keep in mind that it is a development time script only). Also, it won't currently strip comments like, e.g., generate_qhull.py did. Feedback welcome. Tempita --------- I needed a template language, and since Robert Kern recommended Tempita, I went with that for wrappers I made with Fwrap: http://pythonpaste.org/tempita/ cythonize.py above will interpret ".pyx.in" as being Tempita-templated Cython source code. Later (too late) I discovered that Mako was already in use in SciPy in interpolate/interpnd.pyx. In the "cythonize" branch linked to above I've converted it to use Tempita instead, in case you agree with that. The advantage with Tempita is that it fits in a single ~1000-line Python file, so that it should be convenient to bundle it with the build system on a later date. Dag Sverre

Mon, 31 Jan 2011 15:48:54 +0100, Dag Sverre Seljebotn wrote: [clip]
Later (too late) I discovered that Mako was already in use in SciPy in interpolate/interpnd.pyx. In the "cythonize" branch linked to above I've converted it to use Tempita instead, in case you agree with that. The advantage with Tempita is that it fits in a single ~1000-line Python file, so that it should be convenient to bundle it with the build system on a later date.
I don't have strong feelings for one way or another. Since the templates should be used only for simple stuff, whatever is used probably doesn't matter too much. (Using them for more complicated stuff would be like "Hi, I put code generation in your code generation" ;). Pauli

On 01/31/2011 07:52 PM, Pauli Virtanen wrote:
Mon, 31 Jan 2011 15:48:54 +0100, Dag Sverre Seljebotn wrote: [clip]
Later (too late) I discovered that Mako was already in use in SciPy in interpolate/interpnd.pyx. In the "cythonize" branch linked to above I've converted it to use Tempita instead, in case you agree with that. The advantage with Tempita is that it fits in a single ~1000-line Python file, so that it should be convenient to bundle it with the build system on a later date.
I don't have strong feelings for one way or another. Since the templates should be used only for simple stuff, whatever is used probably doesn't matter too much. (Using them for more complicated stuff would be like "Hi, I put code generation in your code generation" ;).
:-) Well, ideally Cython should cope with most of your code generation needs, but in practice its preprocessing/macro capabilities can be too limited sometimes. Hopefully we can meet in Munich and talk more about this... Dag Sverre
participants (2)
-
Dag Sverre Seljebotn
-
Pauli Virtanen