switching to numpy and failing, a user story

[Travis E. Oliphant]

greg.landrum at gmail.com wrote:
> After using numeric for almost ten years, I decided to attempt to
> switch a large codebase (python and C++) to using numpy. Here's are
> some comments about how that went.
> 
> - The code to automatically switch python stuff over just kind of
> works. But it was a 90% solution, I could do the rest by hand. Of
> course, the problem is that then the code is still using the old
> numeric API, so it's not a long term solution. Unfortunately, to switch
> to the numpy API one needs documentation, which is a problem; see
> below.

I'm glad to hear of your experiences (good and bad).  We need feedback 
exactly from users like you in order to improve things.  Yep, the code 
converter is a 80% solution, but it does work.  Improvements are always 
welcome.

> 
> - Well, ok, the automatic switching code doesn't really work all that
> well... my uses of RandomArray still work, but they generate different
> numbers. The underlying random-number generator must have changed. I'm
> sure that it's "better" now, but it's different. This is a major pain
> for my regression tests that rely on seeding the random number
> generator and getting particular results. But that's ok, I can update
> the regressions for the new RNG.
>

You definitely can't expect the same random number generator.  The new 
one (thanks to Robert Kern) is quite good.


> - My extension modules just won't build because the new numpy stuff
> lives in a different location from where Numeric used to live.

This is an easy one and is documented in lots of places on the new 
http://www.scipy.org site.  Plus, people are always willing to help out 
if you just ask.  The numpy-discussion list is quite active.  Don't be 
shy.


> - I guess I should just buy the documentation. I don't like this idea,
> because I think it's counter-productive to the project to have payware
> docs (would Python be successful if you had to buy the documentation? I
> don't think so), but that's the way this project goes.

It's probably better to call it "complete documentation."  Normal 
open-source documentation is available from http://www.scipy.org.  There 
are lots of people who have helped it.  I had to do something to at 
least pretend to justify the time NumPy took me to the people who care 
about how I spend my time (including my family).   This was the best I 
could come up with.

  I'm doubly
> unhappy about it because they payment system is using Paypal and I
> don't like Paypal at all, but I guess that's just the way it goes. Oh,
> wait, I *can't* buy the docs because I'm not in the US and the payment
> page requires a US address.

Is that really true?  A lot of people not living in the U.S. have used 
Paypal successfully.  When difficulties arise, communicating with me 
your difficulty is usually productive.

  I give up; I guess NumPy is only for people living in the US.

Definitely not true.  People in Singapore, Japan, Ghana, South Africa, 
France, Germany, New Zealand, Australia, and many other countries are 
using NumPy successfully.  Gratefully, a few have contributed by buying 
the book, but a lot more have downloaded and are successfully using it.

I'm sorry about your experience, you definitely don't *have* to buy the 
book to use NumPy.  Just like you don't *have* to buy any Python book to 
use Python.  The amount of documentation for NumPy is growing and I 
expect that trend to continue.   There is a lot of information in the 
source file.

> 
> I guess I'll come back to NumPy in 2010, when the docs are available.
> 

Or just ask on the mailing lists, use the numpy.oldnumeric interface 
(the differences are all documented in the first few pages of my book 
which is available for free now).

Thanks,

-Travis