Hi All,
David Cournapeau has mentioned that he would like to have a numpy math library that would supply missing functions and I'm wondering how we should organise the source code. Should we put a mathlib directory in numpy/core/src? Inside that directory would be functions for single/double/extended/quad precision. Should they be in separate directories? What about complex versions? I'm thinking that a good start would be to borrow the msun functions for doubles. We should also make a list of what functions would go into the library and what interface the complex functions present.
Thoughts?
Chuck
On Mon, Apr 5, 2010 at 10:40, Charles R Harris charlesr.harris@gmail.com wrote:
Hi All,
David Cournapeau has mentioned that he would like to have a numpy math library that would supply missing functions and I'm wondering how we should organise the source code. Should we put a mathlib directory in numpy/core/src?
David already did this: numpy/core/src/npymath/
On Mon, Apr 5, 2010 at 9:43 AM, Robert Kern robert.kern@gmail.com wrote:
On Mon, Apr 5, 2010 at 10:40, Charles R Harris charlesr.harris@gmail.com wrote:
Hi All,
David Cournapeau has mentioned that he would like to have a numpy math library that would supply missing functions and I'm wondering how we
should
organise the source code. Should we put a mathlib directory in numpy/core/src?
David already did this: numpy/core/src/npymath/
Yeah, but there isn't much low level stuff there and I don't want to toss a lot of real numerical code into it. Maybe a subdirectory?
Chuck
On Mon, Apr 5, 2010 at 10:56, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Apr 5, 2010 at 9:43 AM, Robert Kern robert.kern@gmail.com wrote:
On Mon, Apr 5, 2010 at 10:40, Charles R Harris charlesr.harris@gmail.com wrote:
Hi All,
David Cournapeau has mentioned that he would like to have a numpy math library that would supply missing functions and I'm wondering how we should organise the source code. Should we put a mathlib directory in numpy/core/src?
David already did this: numpy/core/src/npymath/
Yeah, but there isn't much low level stuff there and I don't want to toss a lot of real numerical code into it.
Who cares? I don't.
Maybe a subdirectory?
I'm guessing YAGNI. Code first. Reorganize later, if necessary.
On Mon, Apr 5, 2010 at 10:00 AM, Robert Kern robert.kern@gmail.com wrote:
On Mon, Apr 5, 2010 at 10:56, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Apr 5, 2010 at 9:43 AM, Robert Kern robert.kern@gmail.com
wrote:
On Mon, Apr 5, 2010 at 10:40, Charles R Harris charlesr.harris@gmail.com wrote:
Hi All,
David Cournapeau has mentioned that he would like to have a numpy math library that would supply missing functions and I'm wondering how we should organise the source code. Should we put a mathlib directory in numpy/core/src?
David already did this: numpy/core/src/npymath/
Yeah, but there isn't much low level stuff there and I don't want to toss
a
lot of real numerical code into it.
Who cares? I don't.
I care. I want the code to be organized.
Maybe a subdirectory?
I'm guessing YAGNI. Code first. Reorganize later, if necessary.
Would that be Are or Aren't? I'm going for the first.
Chuck
On Mon, Apr 5, 2010 at 11:11, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Apr 5, 2010 at 10:00 AM, Robert Kern robert.kern@gmail.com wrote:
On Mon, Apr 5, 2010 at 10:56, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Apr 5, 2010 at 9:43 AM, Robert Kern robert.kern@gmail.com wrote:
On Mon, Apr 5, 2010 at 10:40, Charles R Harris charlesr.harris@gmail.com wrote:
Hi All,
David Cournapeau has mentioned that he would like to have a numpy math library that would supply missing functions and I'm wondering how we should organise the source code. Should we put a mathlib directory in numpy/core/src?
David already did this: numpy/core/src/npymath/
Yeah, but there isn't much low level stuff there and I don't want to toss a lot of real numerical code into it.
Who cares? I don't.
I care. I want the code to be organized.
Then do it when there is code and we can see what needs to be organized.
Maybe a subdirectory?
I'm guessing YAGNI. Code first. Reorganize later, if necessary.
Would that be Are or Aren't? I'm going for the first.
"Ain't", actually. Show us the code first. Until you do, I can't imagine that there will be enough to add another layer of hierarchy to the four we already have.
On Mon, Apr 5, 2010 at 10:43 AM, Robert Kern robert.kern@gmail.com wrote:
On Mon, Apr 5, 2010 at 11:11, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Apr 5, 2010 at 10:00 AM, Robert Kern robert.kern@gmail.com
wrote:
On Mon, Apr 5, 2010 at 10:56, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Apr 5, 2010 at 9:43 AM, Robert Kern robert.kern@gmail.com wrote:
On Mon, Apr 5, 2010 at 10:40, Charles R Harris charlesr.harris@gmail.com wrote:
Hi All,
David Cournapeau has mentioned that he would like to have a numpy math library that would supply missing functions and I'm wondering how
we
should organise the source code. Should we put a mathlib directory in numpy/core/src?
David already did this: numpy/core/src/npymath/
Yeah, but there isn't much low level stuff there and I don't want to toss a lot of real numerical code into it.
Who cares? I don't.
I care. I want the code to be organized.
Then do it when there is code and we can see what needs to be organized.
I am writing code and I want to decide up front where to put it. I know where you stand, so you need say no more. I'm waiting to see if other folks have an opinion.
Chuck
On Mon, Apr 5, 2010 at 9:50 AM, Charles R Harris charlesr.harris@gmail.comwrote:
On Mon, Apr 5, 2010 at 10:43 AM, Robert Kern robert.kern@gmail.comwrote:
On Mon, Apr 5, 2010 at 11:11, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Apr 5, 2010 at 10:00 AM, Robert Kern robert.kern@gmail.com
wrote:
On Mon, Apr 5, 2010 at 10:56, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Apr 5, 2010 at 9:43 AM, Robert Kern robert.kern@gmail.com wrote:
On Mon, Apr 5, 2010 at 10:40, Charles R Harris charlesr.harris@gmail.com wrote: > Hi All, > > David Cournapeau has mentioned that he would like to have a numpy > math > library that would supply missing functions and I'm wondering how
we
> should > organise the source code. Should we put a mathlib directory in > numpy/core/src?
David already did this: numpy/core/src/npymath/
Yeah, but there isn't much low level stuff there and I don't want to toss a lot of real numerical code into it.
Who cares? I don't.
I care. I want the code to be organized.
Then do it when there is code and we can see what needs to be organized.
I am writing code and I want to decide up front where to put it. I know where you stand, so you need say no more. I'm waiting to see if other folks have an opinion.
Chuck
Will you be using it right away? If so, organize it locally how think it'll work best, work w/ it a little while and see if you "guessed" right or if you find yourself wanting to reorganize; then provide it to us w/ the benefit of your experience. :-)
DG
On Mon, Apr 5, 2010 at 10:56 AM, David Goldsmith d.l.goldsmith@gmail.comwrote:
On Mon, Apr 5, 2010 at 9:50 AM, Charles R Harris < charlesr.harris@gmail.com> wrote:
On Mon, Apr 5, 2010 at 10:43 AM, Robert Kern robert.kern@gmail.comwrote:
On Mon, Apr 5, 2010 at 11:11, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Apr 5, 2010 at 10:00 AM, Robert Kern robert.kern@gmail.com
wrote:
On Mon, Apr 5, 2010 at 10:56, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Apr 5, 2010 at 9:43 AM, Robert Kern robert.kern@gmail.com wrote: > > On Mon, Apr 5, 2010 at 10:40, Charles R Harris > charlesr.harris@gmail.com wrote: > > Hi All, > > > > David Cournapeau has mentioned that he would like to have a numpy > > math > > library that would supply missing functions and I'm wondering how
we
> > should > > organise the source code. Should we put a mathlib directory in > > numpy/core/src? > > David already did this: numpy/core/src/npymath/ >
Yeah, but there isn't much low level stuff there and I don't want to toss a lot of real numerical code into it.
Who cares? I don't.
I care. I want the code to be organized.
Then do it when there is code and we can see what needs to be organized.
I am writing code and I want to decide up front where to put it. I know where you stand, so you need say no more. I'm waiting to see if other folks have an opinion.
Chuck
Will you be using it right away? If so, organize it locally how think it'll work best, work w/ it a little while and see if you "guessed" right or if you find yourself wanting to reorganize; then provide it to us w/ the benefit of your experience. :-)
No, but since at some point it will involve the numpy build I would like some feedback from David C. on how he thinks it should be organized. The first routines I want to add are for log1p. Note that BSD has both single and double versions but the single version copies the approximation coefficients from the double. BSD doesn't have extended or quad precision versions.
Chuck
Scientific Python Tools not only for Scientists and Engineers =============================================================
This is the title of my three-hour tutorial at PyCon US: http://us.pycon.org/2011/schedule/sessions/164/
It is a compressed version of my much longer course about:
* NumPy * SciPy * matplotlib/IPython * extensions with C and Fortran
So if your are new to these tools and go to PyCon, you might consider taking the tutorial. Also, if you know somebody who would likely be interested in this tutorial, please spread the word.
Thanks. Mike
-- Mike Müller mmueller@python-academy.de
On Tue, Apr 6, 2010 at 12:56 AM, Charles R Harris charlesr.harris@gmail.com wrote:
Yeah, but there isn't much low level stuff there and I don't want to toss a lot of real numerical code into it.
I don't understand: there is already math code there, and you cannot be much more low level than what's there (there is already quite a bit of bit twiddling for long double). I split the code into complex and real, IEEE 754 macros/funcs in another file. I don' think we need to split into one file / function, at least not with the current size of the library.
I think it is much more worthwhile to think about reorganizing the rest of numpy.core C code, the npymath library is very low hanging fruit in comparison, if only by size.
David
On Mon, Apr 5, 2010 at 11:10 AM, David Cournapeau cournape@gmail.comwrote:
On Tue, Apr 6, 2010 at 12:56 AM, Charles R Harris charlesr.harris@gmail.com wrote:
Yeah, but there isn't much low level stuff there and I don't want to toss
a
lot of real numerical code into it.
I don't understand: there is already math code there, and you cannot be much more low level than what's there (there is already quite a bit of bit twiddling for long double). I split the code into complex and real, IEEE 754 macros/funcs in another file. I don' think we need to split into one file / function, at least not with the current size of the library.
Yeah, but the added code in four versions with documentation for log1p alone will add substantially to the current size. What I am saying is that the current code is small because it uses current functions or falls back to double versions. It doesn't really implement the low level stuff.
Chuck
On Apr 5, 2010, at 12:10 PM, David Cournapeau wrote:
On Tue, Apr 6, 2010 at 12:56 AM, Charles R Harris charlesr.harris@gmail.com wrote:
Yeah, but there isn't much low level stuff there and I don't want to toss a lot of real numerical code into it.
I don't understand: there is already math code there, and you cannot be much more low level than what's there (there is already quite a bit of bit twiddling for long double). I split the code into complex and real, IEEE 754 macros/funcs in another file. I don' think we need to split into one file / function, at least not with the current size of the library.
I think it is much more worthwhile to think about reorganizing the rest of numpy.core C code, the npymath library is very low hanging fruit in comparison, if only by size.
I should have some time over the next couple of weeks, and I am very interested in refactoring the NumPy code to separate out the Python interface layer from the "library" layer as much as possible. I had some discussions with people at PyCon about making it easier for Jython, IronPython, and perhaps even other high-level languages to utilize NumPy.
Is there a willingness to consider as part of this reorganization creating a clear boundary between the NumPy library code and the Python-specific interface to it? What other re-organization thoughts are you having David?
-Travis
On Mon, Apr 05, 2010 at 05:43:41PM -0500, Travis Oliphant wrote:
I should have some time over the next couple of weeks, and I am very interested in refactoring the NumPy code to separate out the Python interface layer from the "library" layer as much as possible. I had some discussions with people at PyCon about making it easier for Jython, IronPython, and perhaps even other high-level languages to utilize NumPy.
Is there a willingness to consider as part of this reorganization creating a clear boundary between the NumPy library code and the Python-specific interface to it? What other re-organization thoughts are you having David?
I have been following discussion too well, so please pardon me if my answer is off topic or irrelevant...
At work, we want to code in a mixture of Python and C, optimizing only the bottlenecks of the computation in C. When we want to use numerical facilities in C, we would like to benefit from the fact that numpy already went through the hard work of getting basic vectorized math compiled and running on the user's computer.
Indeed, one of the issues that we have been facing lately is that deploying a Python application with some C can increase a lot the difficulty of building and installing due to the required C libraries.
The reason I bring this up, is that your refactor could make it easier for C or Cython coders to use the numpy internal to do their own dirty work. If the corresponding functions are exposed in the numpy headers, it would be fairly easy to include them in a numpy.distutils-driven build, via a call to 'numpy.get_include()'.
My 2 cents,
Gaël
Hi Travis,
On Tue, Apr 6, 2010 at 7:43 AM, Travis Oliphant oliphant@enthought.com wrote:
I should have some time over the next couple of weeks, and I am very interested in refactoring the NumPy code to separate out the Python interface layer from the "library" layer as much as possible. I had some discussions with people at PyCon about making it easier for Jython, IronPython, and perhaps even other high-level languages to utilize NumPy.
Is there a willingness to consider as part of this reorganization creating a clear boundary between the NumPy library code and the Python-specific interface to it? What other re-organization thoughts are you having David?
This is mainly it, reorganizing the code for clearer boundaries between boilerplate (python C API) and actual compuational code. Besides helping other python implementations, I think this would benefit NumPy itself in the long run (code maintainability), as well as scipy (and other C extensions). I think the npymath effort is a good example: albeit simple in nature (the API and boundaries are obvious), it has already helped a lot to solve numerous platform specific issues in numpy and scipy, and I think the overall code quality is better.
My own goals were: - exposing core computational parts through an exported C API, so that other C extensions may use it (for example, exposing basic blas/lapack operations) - dynamic loading of the code (for example depending on the CPU capabilities - I have a git branch somewhere where I started exposing a simple C API to query cpu capabilities like cache size or SSE dynamically to that intent) - more amenable codebase: I think multiarray in particular is too big. I don't know the code well enough to know what can be split and how, but I would have hoped that the scalartypes, the type descriptor could be put out of multiarray proper. Also, exposing an API for things like fancy indexing would be very useful, but I don't know if it even makes sense - I think a pure python implementation of fancy indexing as a reference would be very useful for array-like classes (I am thinking about scipy.sparse, for example).
Unfortunately, I won't be able to help much in the near future (except maybe for the fancy indexing as this could be useful for my job),
David
On Apr 6, 2010, at 9:08 AM, David Cournapeau wrote:
Hi Travis,
On Tue, Apr 6, 2010 at 7:43 AM, Travis Oliphant <oliphant@enthought.com
wrote:
I should have some time over the next couple of weeks, and I am very interested in refactoring the NumPy code to separate out the Python interface layer from the "library" layer as much as possible. I had some discussions with people at PyCon about making it easier for Jython, IronPython, and perhaps even other high-level languages to utilize NumPy.
Is there a willingness to consider as part of this reorganization creating a clear boundary between the NumPy library code and the Python-specific interface to it? What other re-organization thoughts are you having David?
This is mainly it, reorganizing the code for clearer boundaries between boilerplate (python C API) and actual compuational code. Besides helping other python implementations, I think this would benefit NumPy itself in the long run (code maintainability), as well as scipy (and other C extensions). I think the npymath effort is a good example: albeit simple in nature (the API and boundaries are obvious), it has already helped a lot to solve numerous platform specific issues in numpy and scipy, and I think the overall code quality is better.
My own goals were:
- exposing core computational parts through an exported C API, so
that other C extensions may use it (for example, exposing basic blas/lapack operations)
- dynamic loading of the code (for example depending on the CPU
capabilities - I have a git branch somewhere where I started exposing a simple C API to query cpu capabilities like cache size or SSE dynamically to that intent)
- more amenable codebase: I think multiarray in particular is too
big. I don't know the code well enough to know what can be split and how, but I would have hoped that the scalartypes, the type descriptor could be put out of multiarray proper. Also, exposing an API for things like fancy indexing would be very useful, but I don't know if it even makes sense - I think a pure python implementation of fancy indexing as a reference would be very useful for array-like classes (I am thinking about scipy.sparse, for example).
Unfortunately, I won't be able to help much in the near future (except maybe for the fancy indexing as this could be useful for my job),
I understand. It just happens that there is some significant time for me to look at this over the next few weeks and I would really like to make progress on re-factoring. I think it's O.K. if you don't have time right now to help as long as you have time to offer criticism and suggestions.
We could even do that over Skype with whomever else wanted to join us (we could do a GotoMeeting discussion as well) if you think it would be faster to just talk in a group setting instead of email. Of course, a summary of any off-line discussion should be sent to the list.
Thanks for the input,
-Travis
On Tue, Apr 6, 2010 at 9:16 PM, Travis Oliphant oliphant@enthought.com wrote:
On Apr 6, 2010, at 9:08 AM, David Cournapeau wrote:
Hi Travis,
On Tue, Apr 6, 2010 at 7:43 AM, Travis Oliphant oliphant@enthought.com wrote:
I should have some time over the next couple of weeks, and I am very
interested in refactoring the NumPy code to separate out the Python
interface layer from the "library" layer as much as possible. I had
some discussions with people at PyCon about making it easier for
Jython, IronPython, and perhaps even other high-level languages to
utilize NumPy.
Is there a willingness to consider as part of this reorganization
creating a clear boundary between the NumPy library code and the
Python-specific interface to it? What other re-organization thoughts
are you having David?
This is mainly it, reorganizing the code for clearer boundaries between boilerplate (python C API) and actual compuational code. Besides helping other python implementations, I think this would benefit NumPy itself in the long run (code maintainability), as well as scipy (and other C extensions). I think the npymath effort is a good example: albeit simple in nature (the API and boundaries are obvious), it has already helped a lot to solve numerous platform specific issues in numpy and scipy, and I think the overall code quality is better.
My own goals were:
- exposing core computational parts through an exported C API, so
that other C extensions may use it (for example, exposing basic blas/lapack operations)
- dynamic loading of the code (for example depending on the CPU
capabilities - I have a git branch somewhere where I started exposing a simple C API to query cpu capabilities like cache size or SSE dynamically to that intent)
- more amenable codebase: I think multiarray in particular is too
big. I don't know the code well enough to know what can be split and how, but I would have hoped that the scalartypes, the type descriptor could be put out of multiarray proper. Also, exposing an API for things like fancy indexing would be very useful, but I don't know if it even makes sense - I think a pure python implementation of fancy indexing as a reference would be very useful for array-like classes (I am thinking about scipy.sparse, for example).
Unfortunately, I won't be able to help much in the near future (except maybe for the fancy indexing as this could be useful for my job),
I understand. It just happens that there is some significant time for me to look at this over the next few weeks and I would really like to make progress on re-factoring. I think it's O.K. if you don't have time right now to help as long as you have time to offer criticism and suggestions. We could even do that over Skype with whomever else wanted to join us (we could do a GotoMeeting discussion as well) if you think it would be faster to just talk in a group setting instead of email. Of course, a summary of any off-line discussion should be sent to the list.
I'm not sure how much I could contribute to the discussion since I have only quite hazy knowledge of the numpy core. However, I'm interested in the outcome of the refactoring since I'm facing a "similar" problem in http://github.com/b45ch1/taylorpoly where I've implemented core algorithms in C for formal power series over the reals. Basically the formal powerseries are a new dtype and the goal is to be able to do computations like x = numpy.zeros((2,3),dtype='name of the formal powerseries') y = numpy.sin(x)
Sebastian
Thanks for the input, -Travis
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Sun, Apr 11, 2010 at 12:59 PM, Sebastian Walter sebastian.walter@gmail.com wrote:
On Tue, Apr 6, 2010 at 9:16 PM, Travis Oliphant oliphant@enthought.com wrote:
On Apr 6, 2010, at 9:08 AM, David Cournapeau wrote:
Hi Travis,
On Tue, Apr 6, 2010 at 7:43 AM, Travis Oliphant oliphant@enthought.com wrote:
I should have some time over the next couple of weeks, and I am very
interested in refactoring the NumPy code to separate out the Python
interface layer from the "library" layer as much as possible. I had
some discussions with people at PyCon about making it easier for
Jython, IronPython, and perhaps even other high-level languages to
utilize NumPy.
Is there a willingness to consider as part of this reorganization
creating a clear boundary between the NumPy library code and the
Python-specific interface to it? What other re-organization thoughts
are you having David?
This is mainly it, reorganizing the code for clearer boundaries between boilerplate (python C API) and actual compuational code. Besides helping other python implementations, I think this would benefit NumPy itself in the long run (code maintainability), as well as scipy (and other C extensions). I think the npymath effort is a good example: albeit simple in nature (the API and boundaries are obvious), it has already helped a lot to solve numerous platform specific issues in numpy and scipy, and I think the overall code quality is better.
My own goals were:
- exposing core computational parts through an exported C API, so
that other C extensions may use it (for example, exposing basic blas/lapack operations)
- dynamic loading of the code (for example depending on the CPU
capabilities - I have a git branch somewhere where I started exposing a simple C API to query cpu capabilities like cache size or SSE dynamically to that intent)
- more amenable codebase: I think multiarray in particular is too
big. I don't know the code well enough to know what can be split and how, but I would have hoped that the scalartypes, the type descriptor could be put out of multiarray proper. Also, exposing an API for things like fancy indexing would be very useful, but I don't know if it even makes sense - I think a pure python implementation of fancy indexing as a reference would be very useful for array-like classes (I am thinking about scipy.sparse, for example).
Unfortunately, I won't be able to help much in the near future (except maybe for the fancy indexing as this could be useful for my job),
I understand. It just happens that there is some significant time for me to look at this over the next few weeks and I would really like to make progress on re-factoring. I think it's O.K. if you don't have time right now to help as long as you have time to offer criticism and suggestions. We could even do that over Skype with whomever else wanted to join us (we could do a GotoMeeting discussion as well) if you think it would be faster to just talk in a group setting instead of email. Of course, a summary of any off-line discussion should be sent to the list.
I'm not sure how much I could contribute to the discussion since I have only quite hazy knowledge of the numpy core. However, I'm interested in the outcome of the refactoring since I'm facing a "similar" problem in http://github.com/b45ch1/taylorpoly where I've implemented core algorithms in C for formal power series over the reals. Basically the formal powerseries are a new dtype and the goal is to be able to do computations like x = numpy.zeros((2,3),dtype='name of the formal powerseries') y = numpy.sin(x)
Ermm, the reply above is quite poor, sorry about that. What I meant to say is the following:
If there is going to be a discussion about creating a pure C numpy library I'd like to join ;)
Sebastian
Thanks for the input, -Travis
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Apr 11, 2010, at 4:17 PM, Sebastian Walter wrote:
Ermm, the reply above is quite poor, sorry about that. What I meant to say is the following:
If there is going to be a discussion about creating a pure C numpy library I'd like to join ;)
Great. I would really like to get the discussion going. In an ideal world we can finish any kind of significant re-factoring in time for SciPy this year. It actually feels like the kind of thing that can motivate NumPy 2.0 a bit better.
It sounds to me like nobody will be opposed as long as there is continuity to the project and current code still works without disruption (i.e. the current C-API for Python extensions is available).
I am interested in re-factoring in such a way to create minimal impact on current NumPy C-API users, but improve maintainability going forward and the ability for other projects to use NumPy.
-Travis
-- Travis Oliphant Enthought Inc. 1-512-536-1057 http://www.enthought.com oliphant@enthought.com
On Mon, Apr 12, 2010 at 4:19 PM, Travis Oliphant oliphant@enthought.comwrote:
On Apr 11, 2010, at 4:17 PM, Sebastian Walter wrote:
Ermm, the reply above is quite poor, sorry about that. What I meant to say is the following:
If there is going to be a discussion about creating a pure C numpy library I'd like to join ;)
Great. I would really like to get the discussion going. In an ideal world we can finish any kind of significant re-factoring in time for SciPy this year. It actually feels like the kind of thing that can motivate NumPy 2.0 a bit better.
It sounds to me like nobody will be opposed as long as there is continuity to the project and current code still works without disruption (i.e. the current C-API for Python extensions is available).
I am interested in re-factoring in such a way to create minimal impact on current NumPy C-API users, but improve maintainability going forward and the ability for other projects to use NumPy.
My own thoughts were to have a lowlevel 'loop' library that worked with strided memory, and an intermediate level above that for buffer objects. Numpy ufuncs would be a level above that and implement policy type things like casting, kinds, etc.
Then there is the lowlevel c-library for the functions. I don't think we should aim at duplicating commonly available functions like sin and exp, but rather that subset that are sometimes unavailable. In particular, I would like to get away from having to use double versions of functions instead of type specific versions.
Chuck
On Tue, Apr 13, 2010 at 12:29 AM, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Apr 12, 2010 at 4:19 PM, Travis Oliphant oliphant@enthought.com wrote:
On Apr 11, 2010, at 4:17 PM, Sebastian Walter wrote:
Ermm, the reply above is quite poor, sorry about that. What I meant to say is the following:
If there is going to be a discussion about creating a pure C numpy library I'd like to join ;)
Great. I would really like to get the discussion going. In an ideal world we can finish any kind of significant re-factoring in time for SciPy this year. It actually feels like the kind of thing that can motivate NumPy 2.0 a bit better.
It sounds to me like nobody will be opposed as long as there is continuity to the project and current code still works without disruption (i.e. the current C-API for Python extensions is available).
I am interested in re-factoring in such a way to create minimal impact on current NumPy C-API users, but improve maintainability going forward and the ability for other projects to use NumPy.
My own thoughts were to have a lowlevel 'loop' library that worked with strided memory, and an intermediate level above that for buffer objects. Numpy ufuncs would be a level above that and implement policy type things like casting, kinds, etc.
Then there is the lowlevel c-library for the functions. I don't think we should aim at duplicating commonly available functions like sin and exp, but rather that subset that are sometimes unavailable. In particular, I would like to get away from having to use double versions of functions instead of type specific versions.
This sounds reasonable. However, I'm not sure that I understand exactly what the consequences would be. Maybe it would be a good idea that one writes down prototypical examples that should be supported by the new code?
Sebastian
Chuck
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Tue, Apr 6, 2010 at 10:08 AM, David Cournapeau wrote:
could be put out of multiarray proper. Also, exposing an API for things like fancy indexing would be very useful, but I don't know if it even makes sense - I think a pure python implementation of fancy indexing as a reference would be very useful for array-like classes (I am thinking about scipy.sparse, for example).
Unfortunately, I won't be able to help much in the near future (except maybe for the fancy indexing as this could be useful for my job),
David
Hi, I am also interested in a pure python implementation of fancy indexing... at least the array-type fancy indexing, if not the boolean kind. If someone knows of an implementation please let me know. I'll email the list again if I make any serious progress on it.
James
On Wed, Apr 14, 2010 at 9:32 PM, James Bergstra bergstrj@iro.umontreal.ca wrote:
On Tue, Apr 6, 2010 at 10:08 AM, David Cournapeau wrote:
could be put out of multiarray proper. Also, exposing an API for things like fancy indexing would be very useful, but I don't know if it even makes sense - I think a pure python implementation of fancy indexing as a reference would be very useful for array-like classes (I am thinking about scipy.sparse, for example).
Unfortunately, I won't be able to help much in the near future (except maybe for the fancy indexing as this could be useful for my job),
David
Hi, I am also interested in a pure python implementation of fancy indexing... at least the array-type fancy indexing, if not the boolean kind. If someone knows of an implementation please let me know. I'll email the list again if I make any serious progress on it.
Until then, I'm interested in a function that takes an index as input and returns True if fancy indexing is requested, False otherwise. That might be useful for your function as well. I'd use it so that my __getitem__ method can at least say that fancy indexing is not supported.
On Mon, Apr 5, 2010 at 8:40 AM, Charles R Harris charlesr.harris@gmail.comwrote:
Hi All,
David Cournapeau has mentioned that he would like to have a numpy math library that would supply missing functions and I'm wondering how we should organise the source code. Should we put a mathlib directory in numpy/core/src? Inside that directory would be functions for single/double/extended/quad precision. Should they be in separate directories? What about complex versions? I'm thinking that a good start would be to borrow the msun functions for doubles. We should also make a list of what functions would go into the library and what interface the complex functions present.
Thoughts?
For starters: you talking things like Airy, Bessel, Gamma, stuff like that?
DG
Chuck
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
participants (10)
-
Charles R Harris -
David Cournapeau -
David Goldsmith -
Gael Varoquaux -
James Bergstra -
Keith Goodman -
Mike Müller -
Robert Kern -
Sebastian Walter -
Travis Oliphant