Has anyone considered using pyrex to implement numarray ? I don't know a lot of the details but it seems to me that pyrex could unify numarray's python/c source code mixture and smooth the transition from python (==untyped pyrex) code to c (==typed pyrex) code. It would also help clueless users like me understand, and perhaps contribute to, the codebase. Also, the pyrex people have talked about applying this technique to the standard python libraries, both for readability and speed. ciao, Simon.
A Dimarts 01 Febrer 2005 02:16, Simon Burton va escriure:
Has anyone considered using pyrex to implement numarray ?
None that I'm aware of. But this issue has been already discussed in this list at least a couple of times before. See, for example: http://sourceforge.net/mailarchive/message.php?msg_id=8080203
I don't know a lot of the details but it seems to me that pyrex could unify numarray's python/c source code mixture and smooth the transition from python (==untyped pyrex) code to c (==typed pyrex) code. It would also help clueless users like me understand, and perhaps contribute to, the codebase.
Well, after the past discussions, I've got the feeling that the main problems to do that are: 1.- Construct general Ufuncs that can handle many different combinations of arrays and types. I think that this is currently done through a use of some C preprocesor than creates specific C files from kind of template (correct me if I'm wrong there). Do the same thing with Pyrex may potentially need such a preprocessor as well. 2.- Recoding the numarray beast in Pyrex could be *major* task 3.- The lack of experience with Pyrex (at least some time ago) Perhaps I miss something still more important, but my guess is that these reasons are the real stoppers. While problem 3 can be quite easily surmounted, problem 1 probably is the big one (except if one finds some easy way to deal with it). Although now that I think, one can always take advantage of the files coming from the current numarray C preprocessor and use Pyrex as the glue with Python. Problem 2 is also worrying but a combined effort could contribute to alleviate it. However, I do think that is probably worth the effort to concentrate the efforts resolving first the remaining problems with numarray performance (mainly array creation time), than go and switch to yet another implementation that would take far more time to implement. My 2 cents, --
qo< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data ""
On Thu, 3 Feb 2005, Francesc Altet wrote:
A Dimarts 01 Febrer 2005 02:16, Simon Burton va escriure:
Has anyone considered using pyrex to implement numarray ?
None that I'm aware of. But this issue has been already discussed in this list at least a couple of times before. See, for example:
http://sourceforge.net/mailarchive/message.php?msg_id=8080203
Tim Hochberg worked on a psyco version, look at: http://sourceforge.net/mailarchive/message.php?msg_id=3845401 maybe that could be used as a start. Warren Focke
On Feb 3, 2005, at 12:05 PM, Francesc Altet wrote:
A Dimarts 01 Febrer 2005 02:16, Simon Burton va escriure:
Has anyone considered using pyrex to implement numarray ?
None that I'm aware of. But this issue has been already discussed in this list at least a couple of times before. See, for example:
http://sourceforge.net/mailarchive/message.php?msg_id=8080203
I don't know a lot of the details but it seems to me that pyrex could unify numarray's python/c source code mixture and smooth the transition from python (==untyped pyrex) code to c (==typed pyrex) code. It would also help clueless users like me understand, and perhaps contribute to, the codebase.
Well, after the past discussions, I've got the feeling that the main problems to do that are:
1.- Construct general Ufuncs that can handle many different combinations of arrays and types. I think that this is currently done through a use of some C preprocesor than creates specific C files from kind of template (correct me if I'm wrong there). Do the same thing with Pyrex may potentially need such a preprocessor as well.
I meant to respond this earlier but left it hanging due to other distractions. Francesc points out that this has been raised in the past, and he also correctly mentions the biggest issue. How do you handle all the possible numeric types? If handling all that results in large amounts of C or psuedo C code, what is pyrex buying you, really? I think there are various aspects of arrays that get confused sometime. One is thinking that pyrex would make it easy to code a C loop over an array in a fairly simple way. And it would--for a case where you have fixed the types of the arrays being operated on. But what about all possible types for arguments? That's a different story (at least I think so; if someone has a clever idea that solves these problems let's hear it). Perry
Perry Greenfield wrote:
How do you handle all the possible numeric types? If handling all that results in large amounts of C or psuedo C code, what is pyrex buying you, really?
There was a quote from Greg Ewing that went something along the lines of: "I am not re-implementing C++ templates for Pyrex!" Which I think summarized his take on this. This begs the question for me: Why not use C++ templates? I know this was brought up very early in numarray development, specifically the idea of using Blitz++ (or something similar) as the basis for the new NumPy. I'm pretty sure it was rejected primarily on the grounds of limited good support for C++ templates on a wide variety of compilers. I think that's sad, as by now template support is much better. The other idea bandied about is to make Psyco multi-array aware. That would be cool! -Chris --- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
participants (5)
-
Chris Barker -
Francesc Altet -
Perry Greenfield -
Simon Burton -
Warren Focke