Mailman 3 April 2009 - NumPy-Discussion

documentation suggestion
by Neal Becker 13 Apr '09

13 Apr '09

numpy.ndarray says: buffer : object exposing buffer interface, optional Used to fill the array with data. I found this misleading. It isn't just used to _fill_ the array - it is the actual storage for the array. If set to an object exporting the buffer interface, any updates to the ndarray will change the value of this object. As an example: from numpy import * import eos import mmap import os fd = os.open ('test.dat', os.O_RDWR|os.O_CREAT) size = 128 os.ftruncate (fd, size) m = mmap.mmap (fd, size, prot=mmap.PROT_READ|mmap.PROT_WRITE, flags=mmap.MAP_SHARED) u = ndarray (shape=(size,), dtype=uint8, buffer=m) u[:] = 24 mmap exports buffer interface (Also not mentioned in the documentation for mmap!). Updating the value of u (last line above) will change the file on disk.

3 3

numpy.test() errors r6862
by Nils Wagner 13 Apr '09

13 Apr '09

====================================================================== ERROR: test suite ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/nwagner/local/lib64/python2.6/site-packages/nose-0.10.4-py2.6.egg/nose/suite.py", line 154, in run self.setUp() File "/home/nwagner/local/lib64/python2.6/site-packages/nose-0.10.4-py2.6.egg/nose/suite.py", line 180, in setUp if not self: File "/home/nwagner/local/lib64/python2.6/site-packages/nose-0.10.4-py2.6.egg/nose/suite.py", line 65, in __nonzero__ test = self.test_generator.next() File "/home/nwagner/local/lib64/python2.6/site-packages/nose-0.10.4-py2.6.egg/nose/loader.py", line 221, in generate for test in g(): File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/lib/tests/test_format.py", line 440, in test_memmap_roundtrip shape=arr.shape, fortran_order=fortran_order) File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/lib/format.py", line 484, in open_memmap mode=mode, offset=offset) File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/core/memmap.py", line 231, in __new__ mm = mmap.mmap(fid.fileno(), bytes, access=acc, offset=offset) error: [Errno 22] Invalid argument ====================================================================== ERROR: test_mmap (test_io.TestSaveLoad) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/testing/decorators.py", line 169, in knownfailer return f(*args, **kwargs) File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/lib/tests/test_io.py", line 92, in test_mmap self.roundtrip(a, file_on_disk=True, load_kwds={'mmap_mode': 'r'}) File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/lib/tests/test_io.py", line 100, in roundtrip RoundtripTest.roundtrip(self, np.save, *args, **kwargs) File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/lib/tests/test_io.py", line 67, in roundtrip arr_reloaded = np.load(load_file, **load_kwds) File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/lib/io.py", line 193, in load return format.open_memmap(file, mode=mmap_mode) File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/lib/format.py", line 484, in open_memmap mode=mode, offset=offset) File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/core/memmap.py", line 231, in __new__ mm = mmap.mmap(fid.fileno(), bytes, access=acc, offset=offset) error: [Errno 22] Invalid argument ---------------------------------------------------------------------- Ran 1889 tests in 12.656s FAILED (KNOWNFAIL=1, errors=2)

2 4

converting from scipy.io.fread
by Christoph T. Weidemann 12 Apr '09

12 Apr '09

I noticed that scipy.io.fread is deprecated and wanted to change some code I got from somebody else accordingly. This code contains statements like: fread(fid, 1, 'h') fread(fid, 1, 'l') fread(fid, 1, 'd') fread(fid, 6, 'B') and variations of those with different number and letters. The docstring suggested that numpy.fromfile may be a good alternative, but I'm having some trouble mapping the fread commands to appropriate fromfile commands. Particularly, I could not figure out how to map the read_type values (which the fread docstring suggests are PyArray types -- 'h', however, is not listed as a possible value in the docstring) to the dtypes in fromfile and how to translate the fread Num variable into the counts required for fromfile. I would appreciate any help converting these fread commands to non-deprecated commands (using fromfile or any other method that may be more appropriate). Unfortunately I know very little about how exactly the data is stored in the binary file, so it would be great if the conversion could make do with the information from the call to fread.

1 0

[Annoucement] Europython 2009
by Ciarán Mooney 12 Apr '09

12 Apr '09

Hi, There has been a inkling of interest in a Scipy/Numpy tutorial on the Europython 2009 lists. But we need someone to lead it! So even though the deadline for papers has passed, they may still be open to an offer. If not, please come along anyway to mingle, meet and network with other python-ista's. The following is from the Europython Commitee. ====== On behalf of the EuroPython 2009 organisation it is my privilege and honour to announce the 'Call for Participation' for EuroPython 2009! EuroPython is the conference for the communities around Python, including the Django, Zope and Plone communities. This years conference will be held in Birmingham, UK from Monday 29th June to Saturday 4th July 2009. Talk & Themes Do you have something you wish to present at EuroPython? Go to http://www.europython.eu/talks/cfp/ for this years themes and submissions criteria, the deadline is on 5th April 2009. Other Talks, Activities and Events Have you got something which does not fit the above? Visit http://www.europython.eu/talks/ Help Us Out We could use a hand - any contribution is welcome. Please take a look at http://www.europython.eu/contact/ . Sponsors An unique opportunity to affiliate with the prestigious EuroPython conference! http://www.europython.eu/sponsors/ Spread the Word Improve our publicity by distributing this announcement in your corner of the community, please coordinate this with the organizers: http://www.europython.eu/contact/ General Information For more information about the conference, please visit http://www.europython.eu/ Looking forward to seeing you! The EuroPython Team =========== Regards, Ciarán

1 0

Preserving trailing zeros when printing an array
by Minjae Kim 12 Apr '09

12 Apr '09

Hello, When I work with arrays, I would like to print trailing zeros also. Somehow, Numpy refuses to print trailing zeros of a number within an array. For example, if I set the print precision to 5 decimal digits (set_printoptions(5)) and try to print array([0.10000001], it gives me 0.1 instead of 0.10000 . I would like to see all trailing zeros when I print it on the shell (up to a predefined number of digits). (This is confirmed with Numpy functions like array2string and array_str) Any help is welcome. Best, Minjae

1 0

numpy.ma.arg{min,max}
by Adam Oliner 12 Apr '09

12 Apr '09

Hi there, The documentation for numpy.ma.argmin says: "Returns array of indices of the maximum values along the given axis." Aside from probably meaning to say the 'minimum' values, it also doesn't seem to return an array: >>> a array([1, 1, 1, 5, 5]) >>> numpy.ma.argmax(a) 3 >>> numpy.ma.argmin(a) 0 I was expecting to get more than one index for both calls. Am I doing something wrong or misreading the documentation? - Adam J. Oliner oliner(a)cs.stanford.edu

3 3

Using where with tuples
by Frank Peacock 12 Apr '09

12 Apr '09

Hello I am trying to use the where function on a numpy array which was obtained from an image using the asarray function. The code is as follows: from numpy import * from Image import * im=open("a.gif") im2=im.convert("RGBA") a2 = asarray(im2,uint8) c = zeros((140,90,4),uint8) c[:,:] = [0,0,0,255] b=where(a2[:,:]==(255,0,0,255),c,a2) c=fromarray(b,"RGBA") c.save("c.gif") I am trying to convert only red colours to black but although this works it also converts other tuples as well. How do I ensure that the tuple condition is enforced exactly, i.e. only on red colours? Thanks

2 1

Python version dependency
by Charles R Harris 11 Apr '09

11 Apr '09

Hi All, Sturla Molden has a patch adding offset support to memmap. However... the feature requires Python 2.5 and above. The Python version is checked for in the code but I am wondering if we want to do that sort of thing. Using advanced features here and there doesn't bother me too much as long as they are marked with a grepable signature like "REMOVE_IN_FUTURE" so the code doesn't accumulate too much crud. On the other hand, maybe we should just wait until python 2.5 becomes a requirement of numpy. Thoughts? Chuck

2 1

Bug or feature?
by Charles R Harris 11 Apr '09

11 Apr '09

Ticket #1083 <http://projects.scipy.org/numpy/ticket/1083>, In [3]: np.array([324938], dtype=np.uint8) Out[3]: array([74], dtype=uint8) i.e., 324938 is silently downcast. This is common numpy behavior, but I wonder if this case shouldn't be an exception. Or in general, if conversion from out of range python numbers should raise an error. It does for the int32 case on 32 bit machines: In [27]: np.array([2**32 - 1], dtype=np.uint32) Out[27]: array([4294967295], dtype=uint32) In [28]: np.array([2**32], dtype=np.uint32) --------------------------------------------------------------------------- ValueError Traceback (most recent call last) /home/charris/<ipython console> in <module>() ValueError: setting an array element with a sequence. Although the error message seems singularly inappropriate. Chuck

1 0

Constant array of tuples
by Frank Peacock 11 Apr '09

11 Apr '09

Hello I would like to know whether there is a simple way to construct a constant array of tuples: How do I construct an array of size (width*height) where each element is (w,x,y,z) Frank -----Original Message----- From: numpy-discussion-bounces(a)scipy.org [mailto:numpy-discussion-bounces@scipy.org] On Behalf Of numpy-discussion-request(a)scipy.org Sent: 10 April 2009 15:19 To: numpy-discussion(a)scipy.org Subject: Numpy-discussion Digest, Vol 31, Issue 26 Send Numpy-discussion mailing list submissions to numpy-discussion(a)scipy.org To subscribe or unsubscribe via the World Wide Web, visit http://mail.scipy.org/mailman/listinfo/numpy-discussion or, via email, send a message with subject or body 'help' to numpy-discussion-request(a)scipy.org You can reach the person managing the list at numpy-discussion-owner(a)scipy.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Numpy-discussion digest..." Today's Topics: 1. Re: frombuffer alignment for ctypes.Structure array (David Cournapeau) 2. Re: DVCS at PyCon (Ondrej Certik) 3. Re: DVCS at PyCon (David Cournapeau) 4. Re: DVCS at PyCon (Matthieu Brucher) 5. numpy import on x86_64 arch (Vincent Thierion) 6. Re: numpy import on x86_64 arch (David Cournapeau) 7. Replacing colours in numpy array (Frank Peacock) ---------------------------------------------------------------------- Message: 1 Date: Fri, 10 Apr 2009 17:28:23 +0900 From: David Cournapeau <david(a)ar.media.kyoto-u.ac.jp> Subject: Re: [Numpy-discussion] frombuffer alignment for ctypes.Structure array To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> Message-ID: <49DF0327.5010209(a)ar.media.kyoto-u.ac.jp> Content-Type: text/plain; charset=ISO-8859-1 Roland Schulz wrote: > The is no align or aligned option to frombuffer. What is the best way > to tell numpy to align the data as the C-struct/ctypes.Stucture array is? You could add a 'fake' field in between to get the right alignment, maybe ? import numpy as N from ctypes import * class C(Structure): _fields_=[("a",c_int),("b",c_short), ('', c_short)] c=(C*2)() _ctypes_to_numpy = {c_short : N.int16,c_int : N.int32} ty = N.dtype([(x,_ctypes_to_numpy[y]) for x,y in C._fields_]) x=N.frombuffer(c,ty) You may use a different type if you need another alignment, or using something like 'VN' in the dtype, e.g.: a = N.dtype([('a', np.int), ('b', np.short), ('', 'V2')]) David ------------------------------ Message: 2 Date: Fri, 10 Apr 2009 01:54:15 -0700 From: Ondrej Certik <ondrej(a)certik.cz> Subject: Re: [Numpy-discussion] DVCS at PyCon To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> Message-ID: <85b5c3130904100154l797d062fn90ea9389bc2da0b(a)mail.gmail.com> Content-Type: text/plain; charset=UTF-8 On Fri, Apr 10, 2009 at 1:07 AM, David Cournapeau <david(a)ar.media.kyoto-u.ac.jp> wrote: > Eric Firing wrote: >> >> This is simply wrong. ?Mercurial uses hard links for cloning a repo that >> is on the same disk, so it is faster and much more space-efficient than >> copying the files. > > Yes, but maybe Ondrej talks about an older hg version ? Hg could not > handle NTFS hardlink for some time, but it does now if you have pywin32. > > And still, switching between branches is faster in git than in hg, for > some technical reasons I can't really explain (but can be found on the > ML archives). But as I said previously, speed is not really an argument > for me. If hg is fast enough for python, it is obviously fast enough for > numpy and scipy. As long as it does not takes minutes to merge/review > the 5 lines difference between two branches as is the case in svn right > now, I am happier :) > >> ?But if you do want named branches in a given repo, >> you can have that also with hg. ?Granted, it has not always been part of >> hg, but it is now. ?Same with rebasing and transplanting. >> > > As I understand, and correct me if I am wrong, the problems with named > branches are: > ? ?- you can't remove them later, it is in the repository 'forever' > ? ?- it is not easy to make them publicly available Plus with git, you can fetch the remote repository with all the branches and browse them locally in your remote branches, when you are offline. And merge them with your own branches. In mercurial, it seems the only way to see what changes are there and which branch and which commits I want to merge is to use "hg in", but that requires the internet connection, so it's basically like svn. I don't know if mercurial has improved in this lately, but at least for me, that's a major problem with mercurial. But since python now moved to mercurial too, maybe they will help fix this. :) It seems to me like the python distutils vs cmake discussion. I also prefer the "unpythonic" cmake. Ondrej ------------------------------ Message: 3 Date: Fri, 10 Apr 2009 17:44:23 +0900 From: David Cournapeau <david(a)ar.media.kyoto-u.ac.jp> Subject: Re: [Numpy-discussion] DVCS at PyCon To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> Message-ID: <49DF06E7.9040401(a)ar.media.kyoto-u.ac.jp> Content-Type: text/plain; charset=ISO-8859-1 Eric Firing wrote: > Speaking to Josef: does tortoise-hg provide a satisfactory windows gui, > from your standpoint? > Another solution may be eclipse integration. I don't know if that would work for Josef, but there is a git plugin for eclipse, and I can at least clone branches from a remote repository, and work with it. Is there a hg eclipse plugin ? I am not very knowledgeable about IDE, David ------------------------------ Message: 4 Date: Fri, 10 Apr 2009 11:13:59 +0200 From: Matthieu Brucher <matthieu.brucher(a)gmail.com> Subject: Re: [Numpy-discussion] DVCS at PyCon To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> Message-ID: <e76aa17f0904100213q48a7d4aaq8e476cf95bcb6f34(a)mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1 2009/4/10 David Cournapeau <david(a)ar.media.kyoto-u.ac.jp>: > Eric Firing wrote: >> Speaking to Josef: does tortoise-hg provide a satisfactory windows gui, >> from your standpoint? >> > > Another solution may be eclipse integration. I don't know if that would > work for Josef, but there is a git plugin for eclipse, and I can at > least clone branches from a remote repository, and work with it. > > Is there a hg eclipse plugin ? I am not very knowledgeable about IDE, Yes, there is MercurialEclipse. I don't know how it handles branches. I use BzrEclipse for my work, and it doesn't handle branches at all, you have to fall back to the command line. Matthieu -- Information System Engineer, Ph.D. Website: http://matthieu-brucher.developpez.com/ Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn: http://www.linkedin.com/in/matthieubrucher ------------------------------ Message: 5 Date: Fri, 10 Apr 2009 13:14:32 +0200 From: Vincent Thierion <vincent.thierion(a)ema.fr> Subject: [Numpy-discussion] numpy import on x86_64 arch To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> Message-ID: <fe7702cd0904100414g59e97bd4pebac3fad0a6e110f(a)mail.gmail.com> Content-Type: text/plain; charset="iso-8859-1" Hello, I import numpy module (numpy-1.0.4) on a x86_64 machine (on which I don't have any root privileges) after having install it thanks to python "setup.py install --prefix=../numpy". In this manner, I obtain a 64 bits compatible numpy library. (the "numpy" folder used for install is created just before install process) I had the following error : * import numpy File "../numpy/lib64/python2.3/site-packages/numpy/__init__.py", line 43, in ? File "../numpy/lib64/python2.3/site-packages/numpy/linalg/__init__.py", line 4, in ? File "../numpy/lib64/python2.3/site-packages/numpy/linalg/linalg.py", line 25, in ? ImportError: liblapack.so.3: cannot open shared object file: No such file or directory *I looked for liblapack.so.3 and find it in /usr/*lib*/liblapack.so.3. It seems it doesn't work because numpy 64 bits version seems to need /usr/* lib64*/liblapack.so.3. That I tested in other machine and it works when numpy can find /usr/*lib64*/liblapack.so.3. So is there a solution to use numpy 64 bits version with a /usr/*lib*/liblapack.so.3 or is it absolutly necessary to install lapack 64 bits version ? Thank you Vincent

2 1