Mailman 3 March 2010 - NumPy-Discussion

Re: [Numpy-discussion] combinatorics
by Sam Tygier March 4, 2010

March 4, 2010

itertools in the python standard library has what you need >>> import itertools >>> list( itertools.product([4,5], [7,8,9]) ) [(4, 7), (4, 8), (4, 9), (5, 7), (5, 8), (5, 9)] (all the itertools functions return generators, so the list() is to convert it to a list) Sam

1 0

setting decimal accuracy in array operations (scikits.timeseries)
by Marco Tuckner March 3, 2010

March 3, 2010

Hello, am using the scikit.timeseries to convert a hourly timeseries to a lower frequency unsing the appropriate function [1]. When I compare the result to the values calculated with a Pivot table in Excel there is a difference in the values which reaches quite high values in the total sum of all monthly values. I found out that the differnec arises from different decimal settings: In Python the numbers show: 12.88888888 whereas in Excel I see: 12.8888888888888 The difference due to the different decimals is small for single values and accumulates to a 2-digit number for the total of all values. * Why do these differences arise? * What can I do to achive comparable values? Thanks in advance for any hint, Marco [1] http://pytseries.sourceforge.net/generated/scikits.timeseries.convert.html P.S.: Sorry if this is a numpy question but as I was using the scikit I though this is the right forum.

4 7

how to work with numpy.int8 in c
by James Bergstra March 3, 2010

March 3, 2010

Could someone point me to documentation (or even numpy src) that shows how to allocate a numpy.int8 in C, or check to see if a PyObject is a numpy.int8? Thanks, James -- http://www-etud.iro.umontreal.ca/~bergstrj

8 16

EPD 6.1 & Upcoming Webinars
by Amenity Applewhite March 2, 2010

March 2, 2010

March is shaping up to be as busy as ever: planning SciPy 2010 (http://conference.scipy.org/scipy2010), two great webinars...and a new release of EPD! * Enthought Python Distribution 6.1 * In EPD 6.1, NumPy and SciPy are dynamically linked against the MKL linear algebra routines. This allows EPD users to seamlessly benefit from the highly optimized BLAS and LAPACK routines in the MKL. We were certainly expecting to observe performance improvements, but we were surprised at just how dramatic the optimizations were for applications run on dual and multi-core Intel(R) processors. Refer to our benchmarking tests for more info: http://www.enthought.com/epd/mkl/ Then try it yourself! http://www.enthought.com/products/getepd.php * Enthought March Webinars * This Friday, Travis Oliphant will lead a webinar on optimization methods in EPD. Then, on the 19th, we'll host a webinar on Python libraries for integrating C and C++ code, namely Weave, Cython, and ctypes. Enthought Python Distribution Webinar How do I... optimize? Friday, March 5: 1pm CST/7pm UTC Wait-list (for non-subscribers): email amenity(a)enthought.com Scientific Computing with Python Webinar Weave, Cython, and ctypes Friday, March 19: 1pm CST/7pm UTC Register: https://www.gotomeeting.com/register/335697152 Enjoy! ~ The Enthought Team ~ Open Course Austin, TX: http://www.enthought.com/training/open-austin-sci.php Python for Scientists and Engineers * May 17- 19 Interfacing with C / C++ and Fortran * May 20 Introduction to UIs and Visualization * May 21 Financial Open Course, London, UK: http://www.enthought.com/training/open-london-fin.php Python for Quants * March 8-10 Software Craftsmanship * March 11 Introduction to UIs and Visualization * March 12 * Pricing, licensing, and training inquiries * Didrik and Matt are dedicated to answering your questions and getting you the support you need. US : Matt Harward mharward(a)enthought.com Europe: Didrik Pinte dpinte(a)enthought.com

1 0

take not respecting masked arrays?
by Peter Shinners March 2, 2010

March 2, 2010

I have a 2D masked array that has indices into a 1D array. I want to use some form of "take" to fetch the values into the 2D array. I've tried both numpy.take and numpy.ma.take, but they both return a new unmasked array. I can get it working by converting the take results into a masked array and applying the original mask. But the values that are masked are actually illegal indices. This means I need to switch the take mode away from "raise", but I actually want raise. I'm still new to numpy, so it's likely I've overlooked something. Is there a masked take?

3 6

Snow Leopard Py-2.7a3 _init_posix issue; IO test segfault
by Tom Loredo March 2, 2010

March 2, 2010

Bruce Southey wrote: On Fri, Feb 26, 2010 at 6:59 PM, David Warde-Farley <dwf(a)cs.toronto.edu> wrote: > On 26-Feb-10, at 7:43 PM, Charles سمير Doutriaux wrote: > >> Any idea on how to build a pure 32bit numpy on snow leopard? > > If I'm not mistaken you'll probably want to build against the > Python.org Python rather than the wacky version that comes installed > on the system. The Python.org installer is a 32-bit Python that > installs itself in /Library. > > David > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion(a)scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > If you remain with 2.7 then you should also view the thread started 3 days ago: 'distutils problem with NumPy-1.4 & Py-2.7a3 (Snow Leopard)' http://mail.scipy.org/pipermail/numpy-discussion/2010-February/048882.html In particular: Ticket 1355 - that should be resolved with r8260 (thanks Stefan): http://projects.scipy.org/numpy/ticket/1355 Ticket 1409 http://projects.scipy.org/numpy/ticket/1409 Ticket 1345: http://projects.scipy.org/numpy/ticket/1345 ~~~~~~~~~~~~~~~~~~~~ Ticket 1409 indicates the _init_posix issue was fixed 5 days ago, but as of today the unnecessary _init_posix reference is still in the version available on SVN (r8270): numpy-r8270:145$ time python setup.py build --fcompiler=gnu95 Running from numpy source directory.Traceback (most recent call last): File "setup.py", line 210, in <module> setup_package() File "setup.py", line 187, in setup_package from numpy.distutils.core import setup File "/Volumes/Tracking/Temp-work/Sandbox-installs/numpy-r8270/numpy/distutils/__init__.py", line 7, in <module> import ccompiler File "/Volumes/Tracking/Temp-work/Sandbox-installs/numpy-r8270/numpy/distutils/ccompiler.py", line 22, in <module> _old_init_posix = distutils.sysconfig._init_posix AttributeError: 'module' object has no attribute '_init_posix' Per Robert's suggestion, I commented out the offending line (22) in ccompiler.py, and the build proceeded. I am using a 64-bit universal build. The resulting numpy gives a segfault on test: >>> numpy.test(verbose=10) .. test_ip_basic (test_multiarray.TestFromBuffer) ... ok test_multiarray.TestIO.test_ascii ... Segmentation fault At this point I don't know if this is just a 64-bit issue; I'm trying to look into it but 32-bit building for 10.6 was unintentionally crippled in the 2.6.x series and 2.7a3; it should be fixed in the next 2.7 release and in 2.6.5. I can also verify that this segfault bug remains: http://projects.scipy.org/numpy/ticket/1345 Dealing with it has been postponed since it affects 2.7 which isn't due out for a few months. However, 2.6.5rc1 is scheduled for today (I don't see it yet!), with final in 2 weeks; it may be worth seeing if these issues will appear in 2.6.5. BTW: The current NumPy and SciPy SVN (r8270, r6250) install successfully with 64-bit Py-2.6.4 on Snow Leopard; the NumPy tests are fine; the SciPy tests have many errors but at least they no longer segfault. I'll report further in another thread; here it's just to point out that the issues above do not affect 64-bit 2.6.4. -Tom ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/

2 1

todos before 1.4.1 RC1
by Ralf Gommers March 2, 2010

March 2, 2010

Hi all, Here are some requests / things I think need to be done before a 1.4.1 RC1 can be put out. 1. Bump up the version to 1.4.1 2. Update the release notes, including an explanation of why 1.4.0 was pulled. 3. Patrick and I need info on how to upload to Sourceforge. David or Jarrod, can you tell us how this works (offlist)? I guess I can do 1 and 2, but I don't have commit rights. I'm happy to get them, if you'd all prefer that I submit patches for a while first that's fine too. Then, I'm also going to try this once more: in my opinion we should not release numpy 1.4.1 before scipy 0.7.2. Consider this: numpy 1.3 + scipy 0.7.1 = OK numpy 1.4.1 + scipy 0.7.2 = OK numpy 1.3 + scipy 0.7.2 = OK numpy 1.4.1 + scipy 0.7.1 = NOT OK (Cython issue) So either release 1.4.1 and 0.7.2 at the same time, or scipy 0.7.2 first. Even having an incompatible combination out there for a couple of days/weeks is a bad idea imho. Cheers, Ralf

2 1

C-api and masked arrays
by Martin Raspaud March 1, 2010

March 1, 2010

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi all, We are using at the moment a c extension which should manipulate masked arrays. What we do is to fill the masked array with a given value (say 65535 if we run uint16 arrays), do the manipulation, and convert back to masked arrays when we go back to python. This seems like a naive way to do, is there another cleverer way to do it ? Thanks, Martin -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJLi9dyAAoJEBdvyODiyJI4VmEIALRM/a1YXj0WBhHd3aGwEZgK TDakjA+vxlbTksWzZdLs28sYToc1fj0hAeh+VFLPwgOLbORgAeKg4F040DQIV8ea UXY+HZ3sI6W0c4V+6rhqTnKReGH8bxNl/2H3/s3gnAC/feULijCwJhfRo4yXxRzg tjKBejMFsjVWB/QvVHq1FHUOPaz8wWmF5ju9kc28RisiG5hsaAyFfLE4RtA0oaJn Z7YImX1D+NcdI7BrelUippL4gyLzHjAY34FfjCuJIuZ5AkwP/ezZKGBLHDpK3CTx JwJ83vsRjQCwjGUruFNPzhm/VPM39EH8Q7NiM1N6ikQaeG3LdywcNFpxLobeSt4= =8M1y -----END PGP SIGNATURE-----

3 4

NumPy & SciPy with Snow Leopard 64-bit Py-2.6.4
by Tom Loredo March 1, 2010

March 1, 2010

Just wanted to report qualified success installing NumPy & SciPy under a 64-bit build of Python-2.6.4 (universal framework) on OS X 10.6.2 (current Snow Leopard). I am using the current SVN checkouts (numpy r8270, scipy r6250). NumPy has installed successfully for some time now and the current SVN maintains this: >>> numpy.test() .. Ran 2521 tests in 8.518s OK (KNOWNFAIL=4, SKIP=1) SciPy has been causing me problems for weeks with segfaults with IFFT tests, as reported on scipy-dev (no one ever responded to this so I made no progress in diagnosing it): http://mail.scipy.org/pipermail/scipy-dev/2010-February/013921.html However, r6250 now runs scipy.test() without segfault, though with 22 errors and 2 failures. The full tests give: >>> scipy.test('full') .. Ran 4982 tests in 652.478s FAILED (KNOWNFAIL=13, SKIP=27, errors=22, failures=6) I am not familiar with what many of the tests are covering, so I cannot assess the severity of all the errors and failures. Some of them seem to be bugs in the tests (e.g., use of an unexpected keyword "new" in several histogram tests); others are innocuous (e.g., missing PIL, which I haven't installed yet). I've posted the report here: http://www.pastie.org/848651 I'd appreciate comments on which issues are nontrivial and deserve attention to as a SciPy user. E.g., the first error is in an lapack test and involves a ValueError where infs or NaNs appear where they shouldn't. Is this a bug in the test, or does it indicate a 64-bit issue that is making inf/NaN appear where it shouldn't? E.g., there are arpack errors, but I don't know what the "Error info=-8" message signifies. Other arpack errors are due to large solution mismatches, which I presume are serious and deserve attention. Thanks, Tom Loredo ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/

3 2

Snow Leopard
by Charles سمير Doutriaux March 1, 2010

March 1, 2010

Hi there, I'm having a lot of trouble on Snow Leopard. I'm trying to build a 32bit only version of our system.... Turns out I had to use python 2.7a3 in order to get Python --universalsdk to work.... But now numpy fails to build as well... First because of the: >> _old_init_posix = distutils.sysconfig._init_posix error I got it to use sysconfig instead But now when running python setup.py build install I get: creating build/src.macosx-10.3-fat-2.7 creating build/src.macosx-10.3-fat-2.7/numpy creating build/src.macosx-10.3-fat-2.7/numpy/distutils building library "npymath" sources customize NAGFCompiler Could not locate executable f95 customize AbsoftFCompiler Could not locate executable f90 Could not locate executable f77 customize IBMFCompiler Could not locate executable xlf90 Could not locate executable xlf customize IntelFCompiler Could not locate executable ifort Could not locate executable ifc customize GnuFCompiler Could not locate executable g77 customize Gnu95FCompiler Could not locate executable gfortran customize G95FCompiler Could not locate executable g95 don't know how to compile Fortran code on platform 'posix' C compiler: gcc -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -m32 compile options: '-Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/include -I/lgm/cdat/trunk/include/python2.7 -c' gcc: _configtest.c gcc _configtest.o -o _configtest ld: warning: in _configtest.o, file is not of required architecture Undefined symbols: "_main", referenced from: __start in crt1.o ld: symbol(s) not found collect2: ld returned 1 exit status ld: warning: in _configtest.o, file is not of required architecture Undefined symbols: "_main", referenced from: __start in crt1.o ld: symbol(s) not found collect2: ld returned 1 exit status failure. removing: _configtest.c _configtest.o Traceback (most recent call last): File "setup.py", line 210, in <module> setup_package() File "setup.py", line 203, in setup_package configuration=configuration ) File "/svn/cdat/trunk/build/numpy/numpy/distutils/core.py", line 186, in setup return old_setup(**new_attr) File "/lgm/cdat/trunk/lib/python2.7/distutils/core.py", line 152, in setup dist.run_commands() File "/lgm/cdat/trunk/lib/python2.7/distutils/dist.py", line 953, in run_commands self.run_command(cmd) File "/lgm/cdat/trunk/lib/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "/svn/cdat/trunk/build/numpy/numpy/distutils/command/build.py", line 37, in run old_build.run(self) File "/lgm/cdat/trunk/lib/python2.7/distutils/command/build.py", line 127, in run self.run_command(cmd_name) File "/lgm/cdat/trunk/lib/python2.7/distutils/cmd.py", line 326, in run_command self.distribution.run_command(command) File "/lgm/cdat/trunk/lib/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "/svn/cdat/trunk/build/numpy/numpy/distutils/command/build_src.py", line 152, in run self.build_sources() File "/svn/cdat/trunk/build/numpy/numpy/distutils/command/build_src.py", line 163, in build_sources self.build_library_sources(*libname_info) File "/svn/cdat/trunk/build/numpy/numpy/distutils/command/build_src.py", line 298, in build_library_sources sources = self.generate_sources(sources, (lib_name, build_info)) File "/svn/cdat/trunk/build/numpy/numpy/distutils/command/build_src.py", line 385, in generate_sources source = func(extension, build_dir) File "numpy/core/setup.py", line 670, in get_mathlib_info raise RuntimeError("Broken toolchain: cannot link a simple C program") RuntimeError: Broken toolchain: cannot link a simple C program Note: it's saying soemtihng about mac 10.3.... Any idea on how to build a pure 32bit numpy on snow leopard? Thx, C.

3 3