Mailman 3 January 2011 - NumPy-Discussion

OpenMP-ficated loops
by Vladimir Voznesensky Jan. 19, 2011

Jan. 19, 2011

Hello. I've hacked numpy/core/src/umath/loops.c.src to use OMP acceleration. I's a little bit hairy, some things are very suboptimal, but it helps me, so I've decided to share my work as is. Note, that with errstate(...) does not work with my stuff. Use -O2 or less for Intel compiler. Always run test suite after re-compilation. Please, found the hacked file in the attachement. Cheers, VV

1 0

numpy.max(0, 1e-6) == 0 (!?)
by James Bergstra Jan. 18, 2011

Jan. 18, 2011

I find that "numpy.max(0, 1e-6) == 0" is confusing, because it makes bugs hard to spot. The doc says that the second argument to max is an optional integer. My bad. But could the function raise an error if it is passed an invalid 'axis' argument? That would have helped. James -- http://www-etud.iro.umontreal.ca/~bergstrj<http://www-etud.iro.umontreal.ca/%7Ebergstrj>

1 0

compatibility for supporting more than 2 versions of numpy
by josef.pktd＠gmail.com Jan. 18, 2011

Jan. 18, 2011

After upgrading to numpy 1.5.1 I got caught by some depreciated features. Given the depreciation policy of numpy, if we want to support more than two versions of numpy, then we need some conditional execution. Does anyone have any compatibility functions? I haven't looked at it carefully yet, but statsmodels might need things like the following if we want to support numpy 1.3 if np.__version__ < '1.5': freq,hsupp = np.histogram(rvs, histsupp, new=True) else: freq,hsupp = np.histogram(rvs,histsupp) matplotlib says it supports numpy >=1.1 but I didn't see any compatibility code that I could "borrow". Or do I worry for nothing? The compatibility.py in statsmodels is still almost empty. Josef

3 6

Strange behaviour with for loops + numpy arrays
by Stefan Reiterer Jan. 17, 2011

Jan. 17, 2011

Hi all! I made some "performance" tests with numpy to compare numpy on one cpu with mpi on 4 processesors, and something appears quite strange to me: I have the following code: N = 2**10*4 K = 16000 x = numpy.random.randn(N).astype(numpy.float32) x *= 10**10 print "x:", x t1 = time.time() #do something... for k in xrange(K): x *= 0.99 print "altered x:", x t = time.time() - t1 print "# loops:", K, "time needed:", t, " s " # loops: 1000 time needed: 0.0134310722351 s # loops: 2000 time needed: 0.028107881546 s # loops: 4000 time needed: 0.0367569923401 s # loops: 8000 time needed: 0.075756072998 s # loops: 16000 time needed: 2.11396384239 s So for K = 16000 it didn't need twice the amount of time as expected, it took 20 x more time! After that jump it seem to "normalize" # loops: 32000 time needed: 8.25508499146 s # loops: 64000 time needed: 20.5365290642 s First I suspected xrange was the culprit, but if I tried k = 0 while k < K: x *= 0.99 it changed anything. When I tried simply a=0 for k in xrange(K): a = a+1 none of the effects above triggered, so I suspect that numpy has to be involved. My Hardware is 2.3 GHz Intel Dual Core, 2 GB Ram and Ubuntu 10.04. For my tests I tried it with Python 2.6, and Sage 4.6. (which uses 2.6 too) Also changing the size of arrays or changing the computer didn't help. Has anyone an Idea what had could happen? Kind regards, maldun -- Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de

2 2

Math Library
by Charles R Harris Jan. 17, 2011

Jan. 17, 2011

Hi All, David Cournapeau has mentioned that he would like to have a numpy math library that would supply missing functions and I'm wondering how we should organise the source code. Should we put a mathlib directory in numpy/core/src? Inside that directory would be functions for single/double/extended/quad precision. Should they be in separate directories? What about complex versions? I'm thinking that a good start would be to borrow the msun functions for doubles. We should also make a list of what functions would go into the library and what interface the complex functions present. Thoughts? Chuck

10 23

defmatrix 1.3 versus 1.5
by zb Jan. 17, 2011

Jan. 17, 2011

Hi I am trying to compile a py2exe app. It works with numpy 1.3 but if I try numpy 1.5 I get an error when loading the app.exe Traceback (most recent call last): File "artisan.pyw", line 109, in <module> File "numpy\__init__.pyo", line 136, in <module> File "numpy\add_newdocs.pyo", line 9, in <module> File "numpy\lib\__init__.pyo", line 5, in <module> File "numpy\lib\index_tricks.pyo", line 15, in <module> ImportError: No module named defmatrix I've noticed that there is a change in the location of defmatrix from 1.3 to the newer 1.5. 1.3 numpy/core/defmatrix 1.5 numpy/matrixlib/defmatrix How could I make py2exe work? Any tips? Thanks

1 1

MDP release 3.0
by Tiziano Zito Jan. 17, 2011

Jan. 17, 2011

We are glad to announce release 3.0 of the Modular toolkit for Data Processing (MDP). MDP is a Python library of widely used data processing algorithms that can be combined according to a pipeline analogy to build more complex data processing software. The base of available algorithms includes signal processing methods (Principal Component Analysis, Independent Component Analysis, Slow Feature Analysis), manifold learning methods ([Hessian] Locally Linear Embedding), several classifiers, probabilistic methods (Factor Analysis, RBM), data pre-processing methods, and many others. What's new in version 3.0? -------------------------- - Python 3 support - New extensions: caching and gradient - Automatically generated wrappers for scikits.learn algorithms - Shogun and libsvm wrappers - New algorithms: convolution, several classifiers and several user-contributed nodes - Several new examples on the homepage - Improved and expanded tutorial - Several improvements and bug fixes - New license: MDP goes BSD! Resources --------- Download: http://sourceforge.net/projects/mdp-toolkit/files Homepage: http://mdp-toolkit.sourceforge.net Mailing list: http://lists.sourceforge.net/mailman/listinfo/mdp-toolkit-users Acknowledgments --------------- We thank the contributors to this release: Sven Dähne, Alberto Escalante, Valentin Haenel, Yaroslav Halchenko, Sebastian Höfer, Michael Hull, Samuel John, José Quesada, Ariel Rokem, Benjamin Schrauwen, David Verstraeten, Katharina Maria Zeiner. The MDP developers, Pietro Berkes Zbigniew Jędrzejewski-Szmek Rike-Benjamin Schuppner Niko Wilbert Tiziano Zito

1 0

Re: [Numpy-discussion] find machine maximum numpy array size
by Tom Holderness Jan. 17, 2011

Jan. 17, 2011

Hi, How do I find the maximum possible array size for a given data type on a given architecture? For example if I do the following on a 32-bit Windows machine: matrix = np.zeros((8873,9400),np.dtype('f8')) I get, Traceback (most recent call last): File "<pyshell#115>", line 1, in <module> matrix = np.zeros((8873,9400),np.dtype('f8')) MemoryError If I reduce the matrix size then it works. However, if I run the original command on an equivalent 32-bit Linux machine this works fine (presumably some limit of memory allocation in the Windows kernel? I tested increasing the available RAM and it doesn't solve the problem). Is there a way I can find this limit? When distributing software to users (who all run different architectures) it would be great if we could check this before running the process and catch the error before the user hits "run". Many thanks in advance, Tom

2 1

[Numpy-discussion] http://mail.scipy.org/pipermail/numpy-discussion/2011-January/054512.html
by Alex Ter-Sarkissov Jan. 17, 2011

Jan. 17, 2011

hi thanks I've sorted ou the issue Alex

1 0

NOOB Alert: Looping Through Text Files...
by dstaley Jan. 16, 2011

Jan. 16, 2011

Warning, I am a python noob. Not only do I not know python, I really don't know anything about programming outside of ArcInfo and the ancient AML language. Regardless, here is my problem.... Let's say I have three text files (test1.txt, test2.txt and test3.txt). Each text file has 1 line of text in it "This is my text file", to which I want to add (append) a new line of text saying "I suck at Python and need help", and then save the file with a suffix attached (eg test1_modified.txt, test2_modified.txt, test3_modified.txt). I guess this is the equivalent of making a change in MS Word and using the "Save As..." command to maintain the integrity of the original file, and save the changes in a new file. But, I want to also do that in a loop (this is a simplified example of something I want to do with hundreds of text files). Now, I understand how to add this line to an existing text file: text_file = open("test1.txt", "a") text_file.write("\nI suck at Python and need help") text_file.close() While this adds a line of text, it saves the change to the original file (does not add the _modified.txt suffix to the file name), nor does it allow me to loop through all three of the files. I'm sure this is an easy thing to do, and is online in a million places. Unfortunately, I just cannot seem to find the answer. Here is my thought process: First i would define the list from which I would loop: textlist = ["test1.txt", "test2.txt", "test3.txt"] for i in textlist: text_file = open(textlist, "a") text_file.write("\nI suck at Python and need help") text_file.close() But, this doesn't work. It gives me the error: coercing to Unicode: need string or buffer, list found SO, I guess I need to do this from something other than a list? Even if it did work, it does not suit my needs as it does not create a new file and does not allow me to add the _modified.txt suffix, which will allow me to keep the original file intact. >From a responses to a previous post, this seems as if it may have something to do with a python dictionary, but I'm not sure. I'm probably totally off on how this should even be written, so any advice or suggestions would be greatly appreciated. Thanks in advance for your help! -DS -- View this message in context: http://old.nabble.com/NOOB-Alert%3A-Looping-Through-Text-Files...-tp3067609… Sent from the Numpy-discussion mailing list archive at Nabble.com.

4 3