Mailman 3 May 2009 - NumPy-Discussion

Inconsistent error messages.
by Charles R Harris 26 May '09

26 May '09

Hi All, Currently fromfile prints a message raises a MemoryError when more items are requested than read, but fromstring raises a value error: In [8]: fromstring("", count=10) --------------------------------------------------------------------------- ValueError Traceback (most recent call last) /home/charris/<ipython console> in <module>() ValueError: string is smaller than requested size In [9]: fromfile("empty.dat", count=10) 10 items requested but only 0 read --------------------------------------------------------------------------- MemoryError Traceback (most recent call last) /home/charris/<ipython console> in <module>() MemoryError: I think fromfile should also raise a ValueError in this case. Thoughts? Chuck

5 14

List/location of consecutive integers
by Andrea Gavana 25 May '09

25 May '09

Hi All, this should be a very easy question but I am trying to make a script run as fast as possible, so please bear with me if the solution is easy and I just overlooked it. I have a list of integers, like this one: indices = [1,2,3,4,5,6,7,8,9,255,256,257,258,10001,10002,10003,10004] >From this list, I would like to find out which values are consecutive and store them in another list of tuples (begin_consecutive, end_consecutive) or a simple list: as an example, the previous list will become: new_list = [(1, 9), (255, 258), (10001, 10004)] I can do it with for loops, but I am trying to speed up a fotran-based routine which I wrap with f2py (ideally I would like to do this step in Fortran too, so if you have a suggestion on how to do it also in Fortran it would be more than welcome). Do you have any suggestions? Thank you for your time. Andrea. "Imagination Is The Only Weapon In The War Against Reality." http://xoomer.alice.it/infinity77/ http://thedoomedcity.blogspot.com/

6 12

Re: [Numpy-discussion] add xirr to numpy financial functions?
by Joe Harrington 25 May '09

25 May '09

On Sun, 24 May 2009 18:14:42 -0400 josef.pktd(a)gmail.com wrote: > On Sun, May 24, 2009 at 4:33 PM, Joe Harrington <jh(a)physics.ucf.edu> wrote: > > I hate to ask for another function in numpy, but there's an obvious > > one missing in the financial group: xirr. ?It could be done as a new > > function or as an extension to the existing np.irr. > > > > The internal rate of return (np.irr) is defined as the growth rate > > that would give you a zero balance at the end of a period of > > investment given a series of cash flows into or out of the investment > > at regular intervals (the first and last cash flows are usually an > > initial deposit and a withdrawal of the current balance). > > > > This is useful in academics, but if you're tracking a real investment, > > you don't just withdraw or add money on a perfectly annual basis, nor > > do you want a calc with thousands of days of zero entries just so you > > can handle the uneven intervals by evening them out. ?Both excel and > > openoffice define a "xirr" function that pairs each cash flow with a > > date. ?Would there be an objection to either a xirr or adding an > > optional second arg (or a keyword arg) to np.irr in numpy? ?Who writes > > the code is a different question, but that part isn't hard. > > > > > > 3 comments: > > * open office has also the other function in an x??? version, so it > might be good to add it consistently to all functions > > * date type: scikits.timeseries and the gsoc for implementing a date > type would be useful to have a clear date type, or would you want to > base it only on python standard library > > * real life accuracy: given that there are large differences in the > definition of a year for financial calculations, any simple > implementation would be only approximately accurate. for example in > the open office help, oddlyield list the following option > > Basis is chosen from a list of options and indicates how the year is > to be calculated. > Basis Calculation > 0 or missing US method (NASD), 12 months of 30 days each > 1 Exact number of days in months, exact number of days in year > 2 Exact number of days in month, year has 360 days > 3 Exact number of days in month, year has 365 days > 4 European method, 12 months of 30 days each > > So, my question: what's the purpose of the financial function in numpy? > Currently it provides convenient functions for (approximate) interest > calculations. > If they get expanded to a "serious" implementation of, for example, > the main financial functions listed in the open office help (just for > reference) then maybe numpy is not the right location for it. > > I started to do something similar in matlab, and once I tried to use > real dates instead of just counting months, the accounting rules get > quickly very messy. > > Using dates as you propose would be very convenient, but the users > shouldn't be surprised that their actual payments at the end of the > year don't fully match up with what numpy told them. > > my 3cents > > Josef First point: agreed. I wish this community had a design review process for numpy and scipy, so that these things could get properly hashed out, and not just one person (even Travis) suggesting something and everyone else saying yeah-sure-whatever. Does anyone on the list have the financial background to suggest what functions "should" be included in a basic set of financial routines? xirr is the only one I've ever used in a spreadsheet, myself. Other points: Yuk. You're right. When these first came up for discussion, I had a Han Solo moment ("I've got a baaad feeling about this...") but I couldn't put my finger on why. They seemed like simple and limited functions with high utility. Certainly anything as open-ended as financial-industry rules should go elsewhere (scikits, scipy, monpy, whatever). But, that doesn't prevent a user-supplied, floating-point time array from going into a function in numpy. The rate of return would be in units of that array. Functions that convert date/time in some format (or many) and following some rule (or one of many) to such a floating array can still go elsewhere, maintained by people who know the definitions, if they have interest (pun intended). That would make the functions in numpy much more useful without bloating them or making them a maintenance nightmare. --jh--

3 2

add xirr to numpy financial functions?
by Joe Harrington 25 May '09

25 May '09

I hate to ask for another function in numpy, but there's an obvious one missing in the financial group: xirr. It could be done as a new function or as an extension to the existing np.irr. The internal rate of return (np.irr) is defined as the growth rate that would give you a zero balance at the end of a period of investment given a series of cash flows into or out of the investment at regular intervals (the first and last cash flows are usually an initial deposit and a withdrawal of the current balance). This is useful in academics, but if you're tracking a real investment, you don't just withdraw or add money on a perfectly annual basis, nor do you want a calc with thousands of days of zero entries just so you can handle the uneven intervals by evening them out. Both excel and openoffice define a "xirr" function that pairs each cash flow with a date. Would there be an objection to either a xirr or adding an optional second arg (or a keyword arg) to np.irr in numpy? Who writes the code is a different question, but that part isn't hard. --jh--

2 1

documenting index_tricks
by Ralf Gommers 25 May '09

25 May '09

Hi all, I'm documenting the index_tricks module and am unsure how to handle r_ and related objects. r_ is an instance of RClass, therefore ipython shows the RClass docstring for "r_?". RClass in the doc editor is also marked as "needs editing", and the page for r_ is completely empty (see http://docs.scipy.org/numpy/docs/numpy.r_/#numpy-r). It seems like only one of the two should be edited and then the docstring should be copied over. In Travis Oliphant's Guide to Numpy r_ is extensively documented, should I integrate part of that with one of the docstrings? About the code itself, is there a use case for having two RClass instances at the same time? If not maybe RClass should be a private class. Also a name like RowConcatenator (it is a subclass of AxisConcatenator) would make more sense. I'm not sure this change is worth the trouble though. Cheers, Ralf

3 4

Generalised Ufunc list
by Bryan Cole 25 May '09

25 May '09

Which (if any) existing ufuncs support the new generalised looping system? I'm particularly interested in a "vectorised" matrix multiply. BC

2 1

numpy.choose question
by Mark Wendell 24 May '09

24 May '09

I have a question about the numpy.choose method. I'm working with rgb image arrays (converted using PIL's 'asarray'), and would like to combine data from multiple images. The choose method seemed just the thing for what i want to do: creating new images from multiple source images and an index mask. However, it looks like the choose method chokes on anything but single-value data arrays. As soon as I give it source arrays made up of RGB int triplets (e.g., [0,128,255]), it complains with a "ValueError: too many dimensions" error. My hope was that the choose method would return an array made up of the same kind of elements that the source arrays are made up of (in this case, RGB triplets), but it seems to only like scalars. I guess it sees my image arrays at 3D arrays, and doesn't know what to do with them. Is that right? I suppose I could pre-split my images into separate R, G, and B arrays, run them through np.choose, and then recombine the results, but that seems clunky. Any advice welcome. Thanks Mark -- -- Mark Wendell

3 2

Ticket #1113, change title?
by David Warde-Farley 24 May '09

24 May '09

Can someone with the requisite permissions change the title of ticket #1113 to reflect the fact that it affects both ppc and ppc64? Alternately, if you know why the bug is happening, you could file a patch ;) David

3 4

Join us for "Scientific Computing with Python Webinar"
by Travis Oliphant 23 May '09

23 May '09

Hello all Python users: I am pleased to announce the beginning of a free Webinar series that discusses using Python for scientific computing. Enthought will host this free series which will take place once a month for 30-45 minutes. The schedule and length may change based on participation feedback, but for now it is scheduled for the fourth Friday of every month. This free webinar should not be confused with the EPD webinar on the first Friday of each month which is open only to subscribers to the Enthought Python Distribution. I (Travis Oliphant) will be the first speaker at this continuing series. I plan to present a brief (10-15) minute talk on reading binary files with NumPy using memory mapped arrays and structured data- types. This talk will be followed by a demonstration of Chaco for interactive 2-d visualization and Mayavi for interactive 3-d visualization. Both Chaco and Mayavi are open-source tools and part of the Enthought Tool Suite. They can be conveniently installed using the Enthought Python Distribution. Topics for future webinars will be chosen later based on participant feedback. This event will take place on Friday at 3:00pm CDT and will last 30 to 45 minutes depending on questions asked. Space is limited at this event. If you would like to participate, please register by going to https://www1.gotomeeting.com/register/422340144 or by clicking on the appropriate link in the attached announcement. There will be a 10 minute technical help session prior to the on-line meeting which you should plan to use if you have never participated in a GoToWebinar previously. During this time you can test your connection and audio equipment as well as familiarize yourself with the GoTo Meeting software. I am looking forward to interacting with many of you this Friday. Best regards, Travis Oliphant Enthought, Inc. Enthought is the company that sponsored the creation of SciPy and the Enthought Tool Suite. It continues to sponsor the SciPy community by hosting the SciPy mailing list and website and participating in the development of SciPy and NumPy. Enthought creates custom scientific and technical software applications and provides training on using Python for technical computing. Enthought also provides the Enthought Python Distribution. Learn more at http://www.enthought.com Travis Oliphant's bio can be read at http://www.enthought.com/company/executive-team.php > > > > > > Scientific Computing with Python Webinar > > > > > > Each webinar in this continuing series will demonstrate the use of > some aspect of Python to assist with scientific, engineering, and > technical computing. Enthought will host each meeting and select a > specific topic based on feedback from participants > Register for a session now by clicking a date below: > Fri, May 22, 2009 3:00 PM - 3:30 PM CDT > Fri, Jun 19, 2009 1:00 PM - 1:30 PM CDT > Fri, Jul 17, 2009 1:00 PM - 1:30 PM CDT > Once registered you will receive an email confirming your registration > with information you need to join the Webinar. > System Requirements > PC-based attendees > Required: Windows® 2000, XP Home, XP Pro, 2003 Server, Vista > Macintosh®-based attendees > Required: Mac OS® X 10.4 (Tiger®) or newer > >

3 2

where are the benefits of ldexp and/or "array times 2"?
by dmitrey 22 May '09

22 May '09

Hi all, I expected to have some speedup via using ldexp or multiplying an array by a power of 2 (doesn't it have to perform a simple shift of mantissa?), but I don't see the one. Have I done something wrong? See the code below. from scipy import rand from numpy import dot, ones, zeros, array, ldexp from time import time N = 1500 A = rand(N, N) b = rand(N) b2 = 2*ones(A.shape, 'int32') I = 100 t = time() for i in xrange(I): dot(A, b) # N^2 multiplications + some sum operations #A * 2.1 # N^2 multiplications, so it should consume no greater than 1st line time #ldexp(A, b2) # it should consume no greater than prev line time, isn't it? print 'time elapsed:', time() - t # 1st case: 0.62811088562 # 2nd case: 2.00850605965 # 3rd case: 6.79027700424 # Let me also note - # 1) using b = 2 * ones(N) or b = zeros(N) doesn't yield any speedup vs b = rand() # 2) using A * 2.0 (or mere 2) instead of 2.1 doesn't yield any speedup, despite it is exact integer power of 2.

4 3