Mailman 3 May 2013 - NumPy-Discussion

Effect of deprecating non-integers for integer function arguments
by Sebastian Berg 01 Jun '13

01 Jun '13

Hi, the current numpy master has deprecated non-integers for the use of indexing (not-fancy yet). However I think this should be moved further down in the numpy machinery which means that the conversion utils provided by numpy would generally raise warnings for non-integers. This means that for most numpy functions such as reshape, etc. the use of non-integers for arguments that are naturally integers is deprecated, which may also affect third party code in principle. Testing this against current SciPy appearently causes 56 failures (with deprecation warnings being raised). Are other project similarly affect by this change? The branch implementing this can be found in the PR https://github.com/numpy/numpy/pull/3243 Regards, Sebastian

2 2

genfromtxt() skips comments
by Albert Kottke 31 May '13

31 May '13

I noticed that genfromtxt() did not skip comments if the keyword names is not True. If names is True, then genfromtxt() would take the first line as the names. I am proposing a fix to genfromtxt that skips all of the comments in a file, and potentially using the last comment line for names. This will allow reading files with and without comments and/or names. The difference is here: https://github.com/arkottke/numpy/compare/my-genfromtxt Albert p.s. insert some disclaimer about my first pull request

3 4

__getitem__ and creating an array of objects
by Aldcroft, Thomas 30 May '13

30 May '13

I'm seeing some behavior that I can't understand when creating a numpy array of Python objects. Basically it seems that np.array() is calling the object __getitem__ method for one object class but not another class, and I can't understand the difference. Here is an example, starting with a simple class where __getitem__ does NOT get called: >>> class Foo(object): ... def __getitem__(self, item): ... return 1/0 ... def __len__(self, item): ... return 1 ... >>> f = Foo() >>> x = np.array([f]) >>> x array([<__main__.Foo object at 0x11a1ff10>], dtype=object) Now move to the complicated class (astropy.time.Time) which has a lot of stuff in it (including __new__, __init__, and __len__ methods, but initially no __getitem__), but is otherwise an ordinary class derived from object. This works as expected: >>> from astropy.time import Time >>> t = Time('2001-01-01', scale='utc') >>> x = np.array([t]) >>> x array([2001-01-01 00:00:00.000], dtype=object) Now inject a __getitem__ that will fail and try again: >>> Time.__getitem__ = lambda self, item: 1 / 0 >>> x = np.array([t]) ERROR: ZeroDivisionError: integer division or modulo by zero Any ideas on what is driving this difference in behavior? BTW, the value of item in the latter case is 0. Thanks, Tom

2 4

__array_priority__ ignored if __array__ is present
by Thomas Robitaille 30 May '13

30 May '13

Hi everyone, (this was posted as part of another topic, but since it was unrelated, I'm reposting as a separate thread) I've also been having issues with __array_priority__ - the following code behaves differently for __mul__ and __rmul__: """ import numpy as np class TestClass(object): def __init__(self, input_array): self.array = input_array def __mul__(self, other): print "Called __mul__" def __rmul__(self, other): print "Called __rmul__" def __array_wrap__(self, out_arr, context=None): print "Called __array_wrap__" return TestClass(out_arr) def __array__(self): print "Called __array__" return np.array(self.array) """ with output: """ In [7]: a = TestClass([1,2,3]) In [8]: print type(np.array([1,2,3]) * a) Called __array__ Called __array_wrap__ <class '__main__.TestClass'> In [9]: print type(a * np.array([1,2,3])) Called __mul__ <type 'NoneType'> """ Is this also an oversight? I opened a ticket for it a little while ago: https://github.com/numpy/numpy/issues/3164 Any ideas? Thanks! Tom

2 3

forwarded article (embracing tensors)
by Neal Becker 30 May '13

30 May '13

I thought the topic of this article might be of interest here: https://groups.google.com/forum/?fromgroups#!topic/julia-dev/GAdcYzmibyo

2 1

Parameterised dtypes
by Richard Hattersley 29 May '13

29 May '13

Hi all, I'm in the process of defining some new dtypes to handle non-physical calendars (such as the 360-day calendar used in the climate modelling world). This is all going fine[*] so far, but I'd like to know a little bit more about how much is ultimately possible. The PyArray_Descr members `metadata` and `c_metadata` allow run-time parametrisation, but is it possible to hook into the dtype('...') parsing mechanism to supply those parameters? Or is there some other dtype mechanism for supplying parameters? As an example, would it be possible to supply month lengths? >>> a = np.zeros(n, dtype='my_date[34,33,31,30,30,29,29,30,31,32,34,35]') Or is the intended use of parametrisation more like: >>> weird = my_stuff.make_dtype([34,33,31,30,30,29,29,30,31,32,34,35]) >>> a = np.zeros(n, dtype=weird) [*] The docs could do with updating, and the examples would benefit from standardising (or at least explaining the significance of the differences). I intend to post updates where possible. Richard

4 6

Behavior or finance.npv function
by Guillaume VIRY 29 May '13

29 May '13

Good morning all I have been using the financial functions for a while now, and the npv function has always bothered me. Indeed neither the documentation nor the implementation seem to be correct, and actually are not congruent with each other either. As stated in the bug report https://github.com/numpy/numpy/issues/649 : the documentation states that the summation is from 0 to M (should be M-1, if the input data is of size M by the way) which is the correct definition, but the implementation computes a summation from 1 to M, which is known to be wrong, and a mimic of Excel's NPV behavior. So basically, we would have the following : RealNPV(data) = data[0] + CurrentNPV(data[1:]) Both behaviors are fine to me, as long as the documentation is clear about it. Ideally the implementation should be corrected, but in any case there should be a documentation fix at least. I'd be glad to have your opinion about it.

2 2

Unable to building numpy with openblas using bento or distutils
by Dinesh B Vadhia 28 May '13

28 May '13

Maybe this is for the Scipy list but followed the instructions but the command: $ sudo python setup.py install to install Scipy failed because it couldn't find Lapack. However, the export commands were set as: $ export BLAS=/usr/local/lib/libopenblas.a $ export LAPACK=/usr/local/lib/libopenblas.a $ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib/ Further, the site.cfg file in the Numpy folder was copied to the Scipy folder. Numpy built and installed successfully. Scipy built (with $ python setup.py build) successfully. Any thoughts? On Sat, Mar 23, 2013 at 8:44 PM, Skipper Seabold <jsseabold(a)gmail.com> wrote: > On Sat, Mar 23, 2013 at 7:26 PM, Ake Sandgren <ake.sandgren(a)hpc2n.umu.se> wrote: >> On Sat, 2013-03-23 at 14:19 -0400, Skipper Seabold wrote: >>> Some help on this would be greatly appreciated. It's been recommended >>> to use OpenBlas over ATLAS, so I've been trying to build numpy with >>> openblas and have run into a few problems. >> >>> >>> To truly support OpenBlas, is it maybe necessary to make some >>> additions to numpy/distutils/system_info.py? >> >> Here is how. >> >> https://github.com/akesandgren/numpy/commit/363339dd3a9826f3e3e7dc4248c258d… >> > > > Thanks that works well for numpy. Test pass. I hope that makes it into > a pull request. My site.cfg looks like this. I don't know about the > lapack_opt section. It doesn't seem to work. > > [DEFAULT] > library_dirs = /home/skipper/.local/lib > include_dirs = /home/skipper/.local/include > > [openblas] > libraries = openblas > > [lapack_opt] > libraries = openblas > > Do you have any idea how to get scipy working too. I have a similar > site.cfg, but it does not find lapack, which is rolled into > libopenblas from what I understand. I can do > > export LAPACK=~/.local/lib/libopenblas.a > python setup.py build &> build.log > sudo -E python setup.py install > > There are no obvious failures in the build.log, but scipy is still > broken because it needs lapack from numpy I guess. The answer is to export BLAS=~/.local/lib/libopenblas.a export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/.local/lib/ before building and installing. Now everything works. Whew. Thanks a lot for the help. > >>>> import numpy as np >>>> np.show_config() > lapack_info: > NOT AVAILABLE > atlas_threads_info: > NOT AVAILABLE > blas_opt_info: > libraries = ['openblas', 'openblas'] > library_dirs = ['/home/skipper/.local/lib'] > language = f77 > lapack_src_info: > NOT AVAILABLE > openblas_info: > libraries = ['openblas', 'openblas'] > library_dirs = ['/home/skipper/.local/lib'] > language = f77 > lapack_opt_info: > NOT AVAILABLE > atlas_info: > NOT AVAILABLE > lapack_mkl_info: > NOT AVAILABLE > blas_mkl_info: > NOT AVAILABLE > mkl_info: > NOT AVAILABLE >>>> from scipy import stats > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "/usr/local/lib/python2.7/dist-packages/scipy/stats/__init__.py", > line 320, in <module> > from .stats import * > File "/usr/local/lib/python2.7/dist-packages/scipy/stats/stats.py", > line 242, in <module> > import scipy.linalg as linalg > File "/usr/local/lib/python2.7/dist-packages/scipy/linalg/__init__.py", > line 147, in <module> > from .misc import * > File "/usr/local/lib/python2.7/dist-packages/scipy/linalg/misc.py", > line 5, in <module> > from . import blas > File "/usr/local/lib/python2.7/dist-packages/scipy/linalg/blas.py", > line 113, in <module> > from scipy.linalg import _fblas > ImportError: libopenblas.so.0: cannot open shared object file: No such > file or directory > > Skipper

1 0

array manupulation
by Sudheer Joseph 27 May '13

27 May '13

Dear experts, I can do below operation in matlab but I wanted to do the same in python numpy array. I tried np.vstak and np.concatenate but not getting the desired results. please help. I wanted to add a row of zeros to the beginning of a numpy array. What is the available quick way. >> cc=[1:10;1:10] cc = 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 >> cc=[cc(1,:)*0;cc] cc = 0 0 0 0 0 0 0 0 0 0 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 wit best regards, Sudheer *************************************************************** Sudheer Joseph Indian National Centre for Ocean Information Services Ministry of Earth Sciences, Govt. of India POST BOX NO: 21, IDA Jeedeemetla P.O. Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55 Tel:+91-40-23886047(O),Fax:+91-40-23895011(O), Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile) E-mail:sjo.India@gmail.com;sudheer.joseph@yahoo.com Web- http://oppamthadathil.tripod.com ***************************************************************

4 8

multivariate_normal issue with 'size' argument
by Emanuele Olivetti 26 May '13

26 May '13

Hi, I'm using NumPy v1.6.1 shipped with Ubuntu 12.04 (Python 2.7.3). I observed an odd behavior of the multivariate_normal function, which does not like int64 for the 'size' argument. Short example: """ import numpy as np print np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=1) print np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=np.int64(1)) """ Which outputs: """ $ python2.7 mvn_bug.py [[ 0.28880655 0.43289446]] Traceback (most recent call last): File "mvn_bug.py", line 3, in <module> print np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=np.int64(1)) File "mtrand.pyx", line 3990, in mtrand.RandomState.multivariate_normal (numpy/random/mtrand/mtrand.c:16663) IndexError: invalid index to scalar variable. """ I had a brief look to the tracker but haven't found any mention of this issue. It might be already solved in the current NumPy (v1.7.0)... or not. I'd like to have your feedback before submitting this issue to the bug tracking system. Best, Emanuele

9 17