[Numpy-discussion] Merging the refactor.

[Charles R Harris]

On Thu, Nov 11, 2010 at 3:15 PM, Travis Oliphant <oliphant at enthought.com>wrote:

> Thanks for starting the discussion, Charles.
>
> Merging of the re-factor is a priority for me once I get back from last 9
> weeks of travel I have been on (I have been travelling for business 7 of the
> last 9 weeks).
>
> Ilan Schnell has already been looking at how to accomplish the merge (and I
> have been reading up on Git so that I understand the commit model better and
> can possibly help without being a complete neophyte with git).   Pauli's
> script will be very helpful.
>
> I'm very enthused about the bug-fixes, memory-leak closures, and new tests
> that have been added on the re-factor branch.  I'm also interested in
> getting more community feedback on the ndarray library C-API, and the other
> changes that have been made.    This feedback will be essential before the
> changes can become NumPy 2.0.   I would also like to see a few more NEPs
> become part of NumPy 2.0 over the next several months.   I have a long wish
> list of additional NEPS that I've only sketched in quick drafts at this
> point as well --- datetime finishes, geometry-information, i.e. dimension
> and index labels,  reduce-by implementation, indirect arrays, and generator
> array objects.
>
>
Let's not go overboard here. I think it would be a good idea to keep the
numpy core as unencumbered as possible. Adding things that let other people
build stuff is great, but putting too much at the core will likely make
maintenance more difficult and the barrier to entry higher. IMHO, the core
of numpy functionality is access to strided memory, topped with ufuncs.
Linear algebra, random numbers, etc are add-ons, but useful ones to combine
with the core package. I think that index labels are already pushing the
limits.

What do you want to do with datetime? We could remove it from the current
trunk and leave it to come in with the refactoring when it is ready.


> My initial guess as to how quickly we would have a NumPy 2.0 was ambitious
> partly because I have had almost zero time personally to work on it, and
> partly because we have been resource constrained which has pushed us to draw
> out the project a bit.     But, I've come up with a long list of new
> features for NumPy 2.0 that I would like to hash out on the mailing lists
> over the next months as well.   My hope is for NumPy 2.0 to come out by the
> end of Q1 sometime next year.   My hopes may have to be tempered by limited
> time resources, of course.
>
>
The rule of thumb is to multiply software time estimates by four. The
multiplication needs to be done by someone uninvolved because programmers
usually think they have already accounted for the unexpected time
requirements.


> At the same time, the work on the .NET framework has pushed us to move more
> of SciPy to a Cython-generated set.   There are additional things I would
> like to see SciPy improve on as well, but I am not sure who is going to work
> on them.   If I had my dream, there would be more modularity to the
> packages, and an improved packaging system --- and of course, porting to
> Python 3k.   I would like to see core SciPy be a smaller set containing a
> few core packages.   (linear algebra, statistics, optimization,
> interpolation, signal processing, and image processing).   Then, I would
> like to see scipy.<module> packages which are released and packaged
> separately with the whole system available on github.
>
>
The past couple of years have been very busy for me (and continue to be
> busy), but I am hoping that next year will allow me more time to spend on
> promoting sprints, and participating more in the community.  I will not have
> the time I used to have when I was a full-time academic, but I plan to be
> more involved in helping promote SciPy development.  With SciPy moved over
> to github, I think that will even be possible without my stepping on
> everybody else's hard work.
>
>
Oh, the guys in the corner offices always say that. Somehow it doesn't work
out that way, someone has to keep the business going. The best way to keep
working at the bench, so to speak, is to avoid promotion in the first place.
I'm afraid it may be too late for you ;)

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20101111/b03f7705/attachment.html>