Caveat: I no nothing about fused types in Cython. In mahotas, I use C++ templates to achieve a similar effect. I don't however, do all combinations. For example erode(image, structuring_element) will convert structuring_element to the type of image. This way only one instance for type is generated (also, structuring_element is guaranteed to be PyArray_CARRAY). structuring_element is often really small, so the time cost is negligable. Similarly, operations on labeled objects [for example, labeled_sum, which gives you the same of pixel values inside each region] require int32 labels, which is what ``label`` returns, but can have anything on the image side (again, only one per type, not an explosion). I think that the size of resulting binaries is quite reasonable as it is most often only an inner loop that gets replicated 6 times or so. My initial variants did generate all combinations and it rapidly became very large (a few MB for morphological functions, for example). My two euro-cent, Luis PS: support for multiple types without a performance cost was one of the original reasons why I rejected Cython in favour of C++ (the other was that, when I started mahotas, Cython was still Pyrex and still a major pain for users to install). On Friday, October 12, 2012 11:05:54 PM UTC+1, Stefan van der Walt wrote:
On Fri, Oct 12, 2012 at 10:38 AM, Johannes Schönberger <hannessch...@gmail.com <javascript:>> wrote:
Are those binaries really that huge? I just tested this locally and the binaries were larger, but imho not too big. The question is who actually cares about a few extra KB disk space taken by a binary?
In SciPy, we were looking at huge files, much longer compilation times, increased memory use, etc. Maybe you can compare these factors and see how it goes? I don't know if it can be avoided, on principle, since you basically have to generate separate functions for each type combination, which makes things explode combinatorially.
Stéfan
Hi Luis On Wed, Oct 17, 2012 at 7:47 AM, Luis Pedro Coelho <luis@luispedro.org> wrote:
I don't however, do all combinations. For example
erode(image, structuring_element)
will convert structuring_element to the type of image. This way only one instance for type is generated (also, structuring_element is guaranteed to be PyArray_CARRAY). structuring_element is often really small, so the time cost is negligable.
I think that is some very sensible and pragmatic advice; thank you!
My initial variants did generate all combinations and it rapidly became very large (a few MB for morphological functions, for example).
OK, so we definitely have to be careful.
PS: support for multiple types without a performance cost was one of the original reasons why I rejected Cython in favour of C++ (the other was that, when I started mahotas, Cython was still Pyrex and still a major pain for users to install).
That was probably a good choice; we made a design decision to almost always operate on floating point images internally, but that comes with its own pros and cons (e.g., memory use with very large images). On the other hand, it also means that a lot of the overhead previous devoted to thinking about ranges can now be dropped (I find it a lot simpler to code up an algorithm, knowing I'm dealing with values in [0, 1]). Cheers Stéfan
participants (2)
-
Luis Pedro Coelho
-
Stéfan van der Walt