Cython code generation with Jinja2 templates
Hi All, I would like to use Jinja2 templates to generate the Cython pyx and pxd files necessary for the Cython Optimize Zeros API proposed in #8431, maybe in a follow up PR https://github.com/scipy/scipy/pull/8431 Does anyone have any experience with code generation in SciPy? Is it best to generate the files offline and then commit them to the repository, or is it better to add some scripting to either setup.py or to the CI yaml config files? I put something together in a branch in my scipy fork, and it recreates the existing code verbatim. See the templates folder and _generate_zeros_type.py script. https://github.com/mikofski/scipy/tree/cython_optimize_generate_code/scipy/o... Any comments would be greatly appreciated. thanks! Mark -- Mark Mikofski, PhD (2005) *Fiat Lux*
Maybe better use Tempita, which we use already. For instance, https://github.com/scipy/scipy/blob/master/scipy/linalg/_decomp_update.pyx.i... On Fri, Aug 31, 2018 at 10:58 AM Mark Alexander Mikofski <mikofski@berkeley.edu> wrote:
Hi All,
I would like to use Jinja2 templates to generate the Cython pyx and pxd files necessary for the Cython Optimize Zeros API proposed in #8431, maybe in a follow up PR
https://github.com/scipy/scipy/pull/8431
Does anyone have any experience with code generation in SciPy? Is it best to generate the files offline and then commit them to the repository, or is it better to add some scripting to either setup.py or to the CI yaml config files?
I put something together in a branch in my scipy fork, and it recreates the existing code verbatim. See the templates folder and _generate_zeros_type.py script.
https://github.com/mikofski/scipy/tree/cython_optimize_generate_code/scipy/o...
Any comments would be greatly appreciated.
thanks! Mark -- Mark Mikofski, PhD (2005) Fiat Lux _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
Thanks! Although looks like Tempita is inactive, the site link is broken, and the last release was in 2013. It looks like the most current fork which adds py3 compat is https://github.com/gjhiggins/tempita - is that what you are using? Any opposition to Jinja2? Jinja2 seems to work and it's already a requirement of SciPy via Sphinx, and it looks very similar to Tempita. On Fri, Aug 31, 2018 at 1:04 AM Evgeni Burovski <evgeny.burovskiy@gmail.com> wrote:
Maybe better use Tempita, which we use already. For instance,
https://github.com/scipy/scipy/blob/master/scipy/linalg/_decomp_update.pyx.i... On Fri, Aug 31, 2018 at 10:58 AM Mark Alexander Mikofski <mikofski@berkeley.edu> wrote:
Hi All,
I would like to use Jinja2 templates to generate the Cython pyx and pxd
files necessary for the Cython Optimize Zeros API proposed in #8431, maybe in a follow up PR
https://github.com/scipy/scipy/pull/8431
Does anyone have any experience with code generation in SciPy? Is it
best to generate the files offline and then commit them to the repository, or is it better to add some scripting to either setup.py or to the CI yaml config files?
I put something together in a branch in my scipy fork, and it recreates
the existing code verbatim. See the templates folder and _generate_zeros_type.py script.
https://github.com/mikofski/scipy/tree/cython_optimize_generate_code/scipy/o...
Any comments would be greatly appreciated.
thanks! Mark -- Mark Mikofski, PhD (2005) Fiat Lux _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
-- Mark Mikofski, PhD (2005) *Fiat Lux*
Tempita is mature, and has worked fine for us. The "official" version at https://pypi.org/project/Tempita/ is Python 3 compatible (via 2to3). Jinja2 is much more heavily geared towards HTML generation, and for the use here I don't see it having much advantages.
On Fri, Aug 31, 2018 at 12:47 PM Pauli Virtanen <pav@iki.fi> wrote:
Tempita is mature, and has worked fine for us. The "official" version at https://pypi.org/project/Tempita/ is Python 3 compatible (via 2to3). Jinja2 is much more heavily geared towards HTML generation, and for the use here I don't see it having much advantages.
While we're on the subject, we might want to vendorize it. Right now, we're preferentially using Cython's vendorized version, but they have informed us that they consider that an implementation detail that external users (like us) should not rely on. -- Robert Kern
On Fri, Aug 31, 2018 at 3:00 PM Robert Kern <robert.kern@gmail.com> wrote:
On Fri, Aug 31, 2018 at 12:47 PM Pauli Virtanen <pav@iki.fi> wrote:
Tempita is mature, and has worked fine for us. The "official" version at https://pypi.org/project/Tempita/ is Python 3 compatible (via 2to3). Jinja2 is much more heavily geared towards HTML generation, and for the use here I don't see it having much advantages.
While we're on the subject, we might want to vendorize it. Right now, we're preferentially using Cython's vendorized version, but they have informed us that they consider that an implementation detail that external users (like us) should not rely on.
It is already in vendorized in NumPy, see `tools/npy_tempita` and works for both Python 2 and Python 3. Chuck
On Fri, Aug 31, 2018 at 2:23 PM Charles R Harris <charlesr.harris@gmail.com> wrote:
On Fri, Aug 31, 2018 at 3:00 PM Robert Kern <robert.kern@gmail.com> wrote:
On Fri, Aug 31, 2018 at 12:47 PM Pauli Virtanen <pav@iki.fi> wrote:
Tempita is mature, and has worked fine for us. The "official" version at https://pypi.org/project/Tempita/ is Python 3 compatible (via 2to3). Jinja2 is much more heavily geared towards HTML generation, and for the use here I don't see it having much advantages.
While we're on the subject, we might want to vendorize it. Right now, we're preferentially using Cython's vendorized version, but they have informed us that they consider that an implementation detail that external users (like us) should not rely on.
It is already in vendorized in NumPy, see `tools/npy_tempita` and works for both Python 2 and Python 3.
Cool. Should be easy to port that over to scipy, then. https://github.com/scipy/scipy/blob/master/tools/cythonize.py#L112 -- Robert Kern
On Fri, Aug 31, 2018 at 1:47 PM Pauli Virtanen <pav@iki.fi> wrote:
Tempita is mature, and has worked fine for us. The "official" version at https://pypi.org/project/Tempita/ is Python 3 compatible (via 2to3). Jinja2 is much more heavily geared towards HTML generation, and for the use here I don't see it having much advantages.
The maintained version is now at https://github.com/gjhiggins/tempita. Chuck
Ok, I agree, I will use Tempita templates to generate the cython code for PR 8431. Thanks everyone! On Fri, Aug 31, 2018, 2:27 PM Charles R Harris <charlesr.harris@gmail.com> wrote:
On Fri, Aug 31, 2018 at 1:47 PM Pauli Virtanen <pav@iki.fi> wrote:
Tempita is mature, and has worked fine for us. The "official" version at https://pypi.org/project/Tempita/ is Python 3 compatible (via 2to3). Jinja2 is much more heavily geared towards HTML generation, and for the use here I don't see it having much advantages.
The maintained version is now at https://github.com/gjhiggins/tempita.
Chuck _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
participants (5)
-
Charles R Harris -
Evgeni Burovski -
Mark Alexander Mikofski -
Pauli Virtanen -
Robert Kern