SciPy-Dev

Download

scipy-dev@python.org

July 2012

33 participants
22 discussions

Cython as build dependency, file/dll size and current issues
by Ralf Gommers July 6, 2012

July 6, 2012

Hi all, On https://github.com/scipy/scipy/pull/261 the problem with large size of generated C files from Cython came up again, and Matthew suggested to add Cython as a build time dependency. He also pointed out that this was discussed before, with most people being in favor: http://mail.scipy.org/pipermail/scipy-dev/2009-November/013272.html http://mail.scipy.org/pipermail/scipy-dev/2009-November/013308.html We discussed the same issue on https://github.com/scipy/scipy/pull/211recently, and also the size of the binary. This is probably also the right moment to point out other recent Cython issues we've had: 1. A memoryview issue with Python 2.4, either a Cython or Numpy bug: https://github.com/numpy/numpy/pull/307 2. We had to manually patch the generated C files when using Cython 0.16, to make them work with MinGW: http://projects.scipy.org/scipy/ticket/1673 3. According to Ray, there's also an indexing bug in Cython 0.16 which requires to use 0.17-dev for https://github.com/scipy/scipy/pull/261 I think it's clear that PR's like #261 above (Ray's ndimage.label rewrite) are in principle a good thing: faster and more general code which is easier to maintain. Now the question is what to do though. Here's some options that I see: a) Keep things as is for now. Accept large file/binary sizes. Manually patch the generated C if necessary. b) Keep things as is for now. Either go back to Cython 0.15, or bump required numpy version to latest dev version to not have to manually patch the generated C files. c) Keep things as they are now, without accepting too large file/binary sizes. To be defined what too large. Means we can't get the full benefits of fused types for example. d) Move to Cython as a build dependency. Write down the required versions and incompatibilities in the docs. e) Include a Cython version in the scipy git repo, patch it to solve the above issues 2 and 3 (and any other ones that come along). f) Some combination of the above. g) Any other options? Anyone have an opinion on which way to go? Ralf

6 16

PR: ENH: rewrite ndimage.label() in Cython
by Thouis (Ray) Jones July 5, 2012

July 5, 2012

I just submitted this PR: https://github.com/scipy/scipy/pull/261 (summary below) I appreciate any feedback. Thanks, Ray Jones (note: I wrote "type templating" below when I meant to write "fused types", but didn't notice until just now). Summary: This reimplements ndimage.label() in Cython, with the following results: + easier to read code + faster (roughly 3x on my MacBook Pro) + output array can be nearly any numpy type (not just int32) + if passed in, output array is used for scratch space (minimal overhead) Downsides: - Uses type templating, so requires a dev version of Cython as of 2012-07 (Cython 0.17+ should work) - The type templating results in a very large C file (there's a small function that gets instantiated N^2 times for the N types the function takes as input/output). New tests have been added to ensure that label() returns the same values as before, and to test new functionality (in-place operation).

1 0