Mailman 3 February 2015 - NumPy-Discussion

ANN: ffnet-0.8.0 released
by Marek Wojciechowski 16 Feb '15

16 Feb '15

ffnet-0.8.0 has been released. ffnet is a fast and easy-to-use feed-forward neural network training solution for python This version supports python 3. Look at ffnet website: http://ffnet.sourceforge.net for installation instructions and documentation. Regards, -- Marek Wojciechowski --- Politechnika Łódzka Lodz University of Technology Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata. Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez pomyłkę prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie. This email contains information intended solely for the use of the individual to whom it is addressed. If you are not the intended recipient or if you have received this message in error, please notify the sender and delete it from your system.

1 0

Matrix Class
by cjw 14 Feb '15

14 Feb '15

It seems to be agreed that there are weaknesses in the existing Numpy Matrix Class. Some problems are illustrated below. I'll try to put some suggestions over the coming weeks and would appreciate comments. Colin W. Test Script: if __name__ == '__main__': a= mat([4, 5, 6]) # Good print('a: ', a) b= mat([4, '5', 6]) # Not the expected result print('b: ', b) c= mat([[4, 5, 6], [7, 8]]) # Wrongly accepted as rectangular print('c: ', c) d= mat([[1, 2, 3]]) try: d[0, 1]= 'b' # Correctly flagged, not numeric except ValueError: print("d[0, 1]= 'b' # Correctly flagged, not numeric", ' ValueError') print('d: ', d) Result: *** Python 2.7.9 (default, Dec 10 2014, 12:28:03) [MSC v.1500 64 bit (AMD64)] on win32. *** >>> a: [[4 5 6]] b: [['4' '5' '6']] c: [[[4, 5, 6] [7, 8]]] d[0, 1]= 'b' # Correctly flagged, not numeric ValueError d: [[1 2 3]] >>> -- View this message in context: http://numpy-discussion.10968.n7.nabble.com/Matrix-Class-tp39719.html Sent from the Numpy-discussion mailing list archive at Nabble.com.

11 20

Nature says 'Pick up Python'
by Sturla Molden 13 Feb '15

13 Feb '15

A recent article in Nature advice scientists to use Python, Cython and the SciPy stack. http://www.nature.com/news/programming-pick-up-python-1.16833 Sturla

1 0

Vectorizing computation
by Francesc Alted 13 Feb '15

13 Feb '15

Hi, I would like to vectorize the next computation: nx, ny, nz = 720, 180, 3 outheight = np.arange(nz) * 3 oro = np.arange(nx * ny).reshape((nx, ny)) def compute1(outheight, oro): result = np.zeros((nx, ny, nz)) for ix in range(nx): for iz in range(nz): result[ix, :, iz] = outheight[iz] + oro[ix, :] return result I think this should be possible by using an advanced use of broadcasting in numpy. Anyone willing to post a solution? Thanks, -- Francesc Alted

2 4

unpacking data values into array of bits
by Neal Becker 12 Feb '15

12 Feb '15

I need to transmit some data values. These values will be float and long values. I need them encoded into a string of bits. The only way I found so far to do this seems rather roundabout: np.unpackbits (np.array (memoryview(struct.pack ('d', pi)))) Out[45]: array([0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 1, 1, 1, 1, 1, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0], dtype=uint8) (which I'm not certain is correct) Also, I don't know how to reverse this process -- -- Those who don't understand recursion are doomed to repeat it

4 9

Using numpy on hadoop streaming: ImportError: cannot import name multiarray
by Kartik Kumar Perisetla 11 Feb '15

11 Feb '15

Hi all, for one of my projects I am using basically using NLTK for pos tagging, which internally uses a 'english.pickle' file. I managed to package the nltk library with these pickle files to make them available to mapper and reducer for hadoop streaming job using -file option. However, when nltk library is trying to load that pickle file, it gives error for numpy- since the cluster I am running this job does not have numpy installed. Also, I don't have root access thus, can't install numpy or any other package on cluster. So the only way is to package the python modules to make it available for mapper and reducer. I successfully managed to do that. But now the problem is when numpy is imported, it imports multiarray by default( as seen in *init*.py) and this is where I am getting the error: File "/usr/lib64/python2.6/pickle.py", line 1370, in load return Unpickler(file).load() File "/usr/lib64/python2.6/pickle.py", line 858, in load dispatch[key](self) File "/usr/lib64/python2.6/pickle.py", line 1090, in load_global klass = self.find_class(module, name) File "/usr/lib64/python2.6/pickle.py", line 1124, in find_class __import__(module) File "numpy.mod/numpy/__init__.py", line 170, in <module> File "numpy.mod/numpy/add_newdocs.py", line 13, in <module> File "numpy.mod/numpy/lib/__init__.py", line 8, in <module> File "numpy.mod/numpy/lib/type_check.py", line 11, in <module> File "numpy.mod/numpy/core/__init__.py", line 6, in <module> ImportError: cannot import name multiarray I tried moving numpy directory on my local machine that contains multiarray.pyd, to the cluster to make it available to mapper and reducer but this didn't help. Any input on how to resolve this(keeping the constraint that I cannot install anything on cluster machines)? Thanks! -- Regards, Kartik Perisetla

2 4

3D array and the right hand rule
by Dieter Van Eessen 11 Feb '15

11 Feb '15

Hello, I'm a novice with respect to scientific computing using python. I've read that numpy.array isn't arranged according to the 'right-hand-rule' (right-hand-rule => thumb = +x; index finger = +y, bend middle finder = +z). This is also confirmed by an old message I dug up from the mailing list archives. (see message below) I guess this has consequences for certain algorithms (been a while, should actually revise some algebra textbooks). At least it does when I try to mentally visualize a 3D array when I'm not sure which dimensions to use... What are the consequences in using 3D arrays in for example transformation algorithms or other algorithms which expect certain shape? Should I always 'reshape' before using them or do most algorithms take this into account in the underlying algebra? Does the 'Fortran contiguous' shape always respect the 'right-hand-rule' (for 3D arrays)? And just to be sure: Is the information telling if an array is C-contiguous or Fortran-contiguous always stored within the array? kind regards, Dieter / Old message From: Anne Archibald <peridot.faceted <at> gmail.com> Subject: Re: dimension aligment <http://news.gmane.org/find-root.php?message_id=ce557a360805201104r3fd2e192y…> Newsgroups: gmane.comp.python.numeric.general <http://news.gmane.org/gmane.comp.python.numeric.general> Date: 2008-05-20 18:04:46 GMT (6 years, 35 weeks, 5 days, 4 hours and 34 minutes ago) 2008/5/20 Thomas Hrabe <thrabe <at> burnham.org>: > given a *3d* *array* > a = > numpy.*array*([[[1,2,3],[4,5,6]],[[7,8,9],[10,11,12]],[[13,14,15],[16,17,18]],[[19,20,21],[22,23,24]]]) > a.shape > returns (4,2,3) > > so I assume the first digit is the 3rd dimension, second is 2nd dim and > third is the first. > > how is the data aligned in memory now? > according to the strides it should be > 1,2,3,4,5,6,7,8,9,10,... > *right*? > > if I had an *array* of more dimensions, the first digit returned by shape > should always be the highest dim. You are basically *right*, but this is a surprisingly subtle issue for numpy. A numpy *array* is basically a block of memory and some description. One piece of that description is the type of data it contains (i.e., how to interpret each chunk of memory) for example int32, float64, etc. Another is the sizes of all the various dimensions. A third piece, which makes many of the things numpy does possible, is the "strides". The way numpy works is that basically it translates A[i,j,k] into a lookup of the item in the memory block at position i*strides[0]+j*strides[1]+k*strides[2] This means, if you have an *array* A and you want every second element (A[::2]), all numpy needs to do is *hand* you back a new *array* pointing to the same data block, but with strides[0] doubled. Similarly if you want to transpose a two-dimensional *array*, all it needs to do is exchange strides[0] and strides[1]; no data need be moved. This means, though, that if you are *hand*ed a numpy *array*, the elements can be arranged in memory in quite a complicated fashion. Sometimes this is no problem - you can always use the strides to find it all. But sometimes you need the data arranged in a particular way. numpy defines two particular ways: "C contiguous" and "FORTRAN contiguous". "C contiguous" *array*s are what you describe, and they're what numpy produces by default; they are arranged so that the *right*most index has the smallest stride. "FORTRAN contiguous" *array*s are arranged the other way around; the leftmost index has the smallest stride. (This is how FORTRAN *array*s are arranged in memory.) There is also a special case: the reshape() function changes the shape of the *array*. It has an "order" argument that describes not how the elements are arranged in memory but how you want to think of the elements as arranged in memory for the reshape operation. Anne /Old message

2 2

Silent Broadcasting considered harmful
by Stefan Reiterer 11 Feb '15

11 Feb '15

15 37

Aligned / configurable memory allocation
by Antoine Pitrou 10 Feb '15

10 Feb '15

Hello, I apologize for pinging the list, but I was wondering if there was interest in either of https://github.com/numpy/numpy/pull/5457 (make array data aligned by default) or https://github.com/numpy/numpy/pull/5470 (make the array data allocator configurable)? Regards Antoine.

3 4

How to debugging python crash in ipython notebook
by Åsmund Hjulstad 10 Feb '15

10 Feb '15

Hello all, I am struggling with a python crash of an ipython notebook (kernel) that I do not know how to debug. If I run this: (valgt is a pandas dataframe with 354 lines and 14 numerical columns) sns.pairplot(valgt) plt.savefig('parvis.png', dpi=600) in the same cell, the kernel consistently crashes. If I run in two separate cells, or exporting to a Python script and running there (changing the inline render to qt), everything is OK. This is win64 on Win7, both the free build from conda, the accelerate build from conda, and also a version I have built myself. (VS2010 and Intel Fortran XE 2015) The following is the package list from the 'vanilla' conda: # packages in environment [...]: # dateutil 2.1 py34_2 ipython 2.3.1 py34_0 jinja2 2.7.3 py34_1 markupsafe 0.23 py34_0 matplotlib 1.4.2 np19py34_0 numexpr 2.3.1 np19py34_0 numpy 1.9.1 py34_0 pandas 0.15.2 np19py34_0 pip 6.0.6 py34_0 pyparsing 2.0.1 py34_0 pyqt 4.10.4 py34_0 pyreadline 2.0 py34_0 python 3.4.2 1 pytz 2014.9 py34_0 pywin32 219 py34_0 pyzmq 14.5.0 py34_0 rpy2 2.5.5 <pip> scipy 0.15.1 np19py34_0 seaborn 0.5.1 np19py34_0 setuptools 12.0.5 py34_0 six 1.9.0 py34_0 tornado 4.0.2 py34_0 rpy2 is from the Gohlke python collection. The home-built version is a python 3.4.3rc1, with certifi (14.5.14) Cython (0.21.2) ipython (2.4.1) Jinja2 (2.7.3) jsonschema (2.4.0) MarkupSafe (0.23) matplotlib (1.4.2) mistune (0.5) nose (1.3.4) numexpr (2.4.1.dev0) numpy (1.10.0.dev0+98a8fe3) pandas (0.15.2) pip (6.0.8) pyparsing (2.0.3) pyreadline (2.0) PySide (1.2.2) python-dateutil (2.4.0) pytz (2014.10) pywin32 (219) pyzmq (14.5.0) rpy2 (2.5.6) scipy (0.15.1) seaborn (0.6.dev0) setuptools (12.0.5) six (1.9.0) tornado (4.1) wheel (0.24.0) They crash in the same way, though, the same applies to a test I did with ipython 3 I am wondering how important it is to have everything built with the same compiler and MKL library. Does all packages that use numpy need to be built against the specific numpy+mkl version? For example rpy2 (which is not used near the cells where it crashes), should I always rebuild this as well if I build differently, or...? And, how can I generate debug information from the kernel that ipython notebook starts? Any pointers are most appreciated. -- mvh, Åsmund Hjulstad

1 0