Mailman 3 July 2009 - NumPy-Discussion

Internal Math Library of Numpy
by Nanime Puloski 30 Jul '09

30 Jul '09

Does Numpy use Python's standard math library when calculating elementary functions such as exp(x) and acos(x)? Also, what is the internal library of Numpy and Python's standard math library? Are they platform independent?

2 1

question about a data structure
by Ernest Adrogué 30 Jul '09

30 Jul '09

Hello, Suppose I want to store some data (a non-negative integer) related to an "event" involving two entities. It occurs to me that the way to do it is using a 2-d array, like a table: For example: a b c a 1 b 2 0 c 5 in the event 'b-a' the data is 2, and in the event 'c-a' is 5, etc. Now, I need to be able to differentiate between 0 and 'no data'. I know I could use a number like -1, but this would affect arithmetic operations on the array, and also I need that indexing one such location with no data returns an empty array. Is it possible to do this with the standard array class? Thanks. Ernest

3 3

Reductions and binary ops on recarrays...
by Fernando Perez 30 Jul '09

30 Jul '09

Howdy, we recently had a discussion about being able to do some common things like reductions and binary operations on recarrays, and there didn't seem to be much consensus on it being needed in the core of numpy. Since we do actually need this quite pressingly for everyday tasks, we wrote a very simple version of this today, I'm attaching it here in case it proves useful to others. Basically it lets you do reductions and binary operations on record arrays whose dtype is a simple composite of native ones (example at the end). For our needs it's quite useful, so it may also be to others. Cheers, f Example: >>> import recarrutil as ru >>> dt = np.dtype(dict(names=['x','y'],formats=[float,float])) >>> x = np.arange(6,dtype=float).reshape(2,3) >>> y = np.arange(10,16,dtype=float).reshape(2,3) >>> z = np.empty( (2,3), dt).view(np.recarray) >>> z.x = x >>> z.y = y >>> z rec.array([[(0.0, 10.0), (1.0, 11.0), (2.0, 12.0)], [(3.0, 13.0), (4.0, 14.0), (5.0, 15.0)]], dtype=[('x', '<f8'), ('y', '<f8')]) >>> ru.mean(z) rec.array((2.5, 12.5), dtype=[('x', '<f8'), ('y', '<f8')]) >>> ru.mean(z,0) rec.array([(1.5, 11.5), (2.5, 12.5), (3.5, 13.5)], dtype=[('x', '<f8'), ('y', '<f8')]) >>> ru.mean(z,1) rec.array([(1.0, 11.0), (4.0, 14.0)], dtype=[('x', '<f8'), ('y', '<f8')]) >>> ru.add(z,z) rec.array([[(0.0, 20.0), (2.0, 22.0), (4.0, 24.0)], [(6.0, 26.0), (8.0, 28.0), (10.0, 30.0)]], dtype=[('x', '<f8'), ('y', '<f8')]) >>> ru.subtract(z,z) rec.array([[(0.0, 0.0), (0.0, 0.0), (0.0, 0.0)], [(0.0, 0.0), (0.0, 0.0), (0.0, 0.0)]], dtype=[('x', '<f8'), ('y', '<f8')])

3 5

Agenda items for steering committee meeting, 2009/08
by Stéfan van der Walt 30 Jul '09

30 Jul '09

Hi all, We are soliciting agenda items for the SciPy Steering Committee meeting taking place during the SciPy09 conference. Please view the current list at http://www.scipy.org/SteeringCommittee_08_2009 and post new items to this thread. Kind regards Stéfan

1 0

how to configure f2py on windows XP?
by ian 30 Jul '09

30 Jul '09

Hi,everyone！ I installed MSVS2005 + Intel visual fortran 9.1 + Python 2.5 + numpy-1.3.0-win32-superpack-python2.5.exe on my computer. The numpy works well: Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] on win32 Type "copyright", "credits" or "license()" for more information. **************************************************************** Personal firewall software may warn about the connection IDLE makes to its subprocess using this computer's internal loopback interface. This connection is not visible on any external interface and no data is sent to or received from the Internet. **************************************************************** IDLE 1.2 >>> import numpy >>> but when I use : python c:\python25\scripts\f2py.py --fcompiler=intel -c hello.f -m hello , the result is: running build running config_cc unifing config_cc, config, build_clib, build_ext, build commands --compiler options running config_fc unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options running build_src building extension "hello" sources f2py options: [] f2py:> c:\docume~1\admini~1\locals~1\temp\tmpwktrfc\src.win32-2.5\hellomodule.c creating c:\docume~1\admini~1\locals~1\temp\tmpwktrfc creating c:\docume~1\admini~1\locals~1\temp\tmpwktrfc\src.win32-2.5 Reading fortran codes... Reading file 'hello.f' (format:fix,strict) Post-processing... Block: hello Block: foo Post-processing (stage 2)... Building modules... Building module "hello"... Constructing wrapper function "foo"... foo(a) Wrote C/API module "hello" to file "c:\docume~1\admini~1\locals~1\temp\tmpwktrfc\src.win32-2.5/hellomodule.c" adding 'c:\docume~1\admini~1\locals~1\temp\tmpwktrfc\src.win32-2.5\fortranobject.c' to sources. adding 'c:\docume~1\admini~1\locals~1\temp\tmpwktrfc\src.win32-2.5' to include_dirs. copying C:\Python25\lib\site-packages\numpy\f2py\src\fortranobject.c -> c:\docume~1\admini~1\locals~1\temp\tmpwktrfc\src.win32-2.5 copying C:\Python25\lib\site-packages\numpy\f2py\src\fortranobject.h -> c:\docume~1\admini~1\locals~1\temp\tmpwktrfc\src.win32-2.5 running build_ext No module named msvccompiler in numpy.distutils; trying from distutils error: python was built with Visula Studio 2003; extensions must be built with a compiler than can generate compatible binaries. Visual Studio 2003 was not found on this system. If you have Cygwin installed, you can try compiling with MingW32, by passing "-c mingw32" to setup.py. who could give me some suggestions on how to configure f2py? Thanks in advance. ian liu.ian0571(a)gmail.com 2009-07-30

3 3

installation issue with a non-standard location of a python build
by Daniel Wheeler 29 Jul '09

29 Jul '09

Hi, I'm having an installation issue building numpy 1.3.0 using python 2.6.2 which has been built in my local area in a non-standard place and was configured with --prefix=/users/wd15/usr/i686/4.0. When I run numpy's config ("python setup.py config") everything seems to be fine. However, when I run "python setup.py build", it gives the following error: $ python --version Python 2.6.2 $ uname -a Linux xxx.xxx.xxx 2.6.18-4-k7 #1 SMP Mon Mar 26 17:57:15 UTC 2007 i686 GNU/Linux $ python setup.py build Running from numpy source directory. non-existing path in 'numpy/distutils': 'site.cfg' F2PY Version 2 ... File "numpy/core/setup.py", line 186, in check_types "Cannot compiler 'Python.h'. Perhaps you need to "\ SystemError: Cannot compiler 'Python.h'. Perhaps you need to install python-dev|python-devel. $ Now, as I compiled python from source, "Python.h" is sitting neatly in "/users/wd15/usr/i686/4.0/include/python2.6". The output from build (not included above) even claims that the compile options point there: C compiler: gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC compile options: '-Inumpy/core/src -Inumpy/core/include -I/users/wd15/usr/i686/4.0/include/python2.6 -c' so the build seems to be aware of the include location. Any ideas? Cheers -- Daniel Wheeler

2 4

distutils -- compiling extension twice, two sets of compiler settings
by Sebastian Haase 28 Jul '09

28 Jul '09

Hi, maybe someone here could help me by using distutils: my problem is that I have some C++ files that I need to get compiled twice, first to make a "singleFloat"-module using gcc -DUSE_DOUBLE=0 and then to make the "doubleFloat"-module using gcc -DUSE_DOUBLE=1. Apparently distutils recognises (like "make") that the object files are already there and skips the gcc for the second run. My setup.py so far is something like this: ext_ccg_s = Extension('_CCG_smodule', [ 'Priithon_25_C/ccg/CCG_caller.cpp', 'Priithon_25_C/ccg/ccg_funct.cpp', 'Priithon_25_C/ccg/ccg.cpp', 'Priithon_25_C/ccg/CCG_s_wrap.cxx', ], define_macros=[('USE_DOUBLE', '0'),], ) ext_ccg_d = Extension('_CCG_dmodule', [ 'Priithon_25_C/ccg/CCG_caller.cpp', 'Priithon_25_C/ccg/ccg_funct.cpp', 'Priithon_25_C/ccg/ccg.cpp', 'Priithon_25_C/ccg/CCG_d_wrap.cxx', ], define_macros=[('USE_DOUBLE', '1'),], ) setup(... ext_package='Priithon_bin', ext_modules=[ext_ccg_s, ext_ccg_d], ) Thanks for any help, -- Sebastian Haase

2 1

nans and sort orders.
by Charles R Harris 28 Jul '09

28 Jul '09

Hi All, I've changed the sort routines so that they sort nans to the end. The sort order is now Real types: [R, nan] Complex types: [R + Rj, R + nanj, nan + Rj, nan + nanj] Where R stands for the usual extended range of reals, i.e. [-inf, finite, +inf] and when two complex numbers have the same pattern of nans then they are sorted on the non-nan part, if any. So the sorts now work with nans but that implies some other fixups on which I would like some feedback. 1) Search sorted -- Search sorted depends on the compare function of the ndarray, I think that compare function should be changed to correspond to the sort order. 2) Methods argmax/argmin/ -- should be compatible with max/min 3) max/min ufuncs -- Problematic, min would fail to propagate nans, should probably stay the same. 4) fmax/fmin ufuncs -- Should stay with c definition 5) Logical comparison ufuncs -- I think these should remain compatible with the c operators. Note that there is a mix of implementations, max/min on void types use the type comparison functions for instance. In other words, changing the type comparisons could mean rewriting some of the other functions. Thoughts? Chuck

1 0

Numpy/Scipy and the Python African Tour
by Emmanuelle Gouillart 28 Jul '09

28 Jul '09

Dear users of Numpy and Scipy, here is an informal report on the last event of the Python African Tour, which took place in Dakar (Senegal) on July 6-10th. It might interest only a fraction of the lists, so I apologize for the spamming. What is the Python African Tour? ---------------------------------- It is a recent initiative to promote the use of Python in African countries, that we owe to Kamon Ayeva (http://kayeva.wordpress.com/). Born a Togolese, Kamon now lives in Paris (France) where he works as a web developer and a trainer in web technologies. It occurred to him that but a few African developers (Hi, Stefan! :D) participate in Python-related development or conferences. IT-related technologies and web development can contribute to economic growth in developing countries, and, because of the clarity and flexibility of the language, Python-trained developers may have a competitive advantage to develop successful businesses. The Python African Tour sends a few volunteers to places in Africa where a small core of Python users or developers already exists. These deliver a training over a few days, about Python and some of its applications. The goal is to create a small but dynamical community of users that will keep on growing after the volunteers have left. A first event was held in Morocco last December. Whereas the emphasis had been put on an introduction to the Python language and to the Django web framework, I proposed to give this time an additional course about the use of Python in scientific computing, which was nicknamed "Scientific Python". Numpy/Scipy is an opportunity to present the language as attractive for not only geeks, but also academic staff -- the people who may precisely teach the language in the universities and engineering departments. Organization of the Dakar event ---------------------------------- This year, three European trainers flew to Dakar for the event: Kamon Ayeva, Daniel Nouri and myself. For all courses we worked in pairs with three local organizers and Python experts: Thomas Noël, Ousmane Wilane and Sergiu Mihai. The course was organized as follows: * 1.5 day for a general introduction to Python * 1 day of specialization, either on the Django web framework (Kamon and Daniel) or on Python for scientific computing (Emmanuelle) * 1 day of barcamp, or informal talks showing how we would use Python in our daily work. * 1.5 day of "sprint", I should rather say practical work. The hosting organizations were the AUF (Agence universitaire de la Francophonie), the Dakar Linux User Group (DakarLUG) and the ESP (Ecole Supérieure Polytechnique, a master-level engineer school). The local team did a great job on announcing the event, rounding up potential attendees (and sponsors!), and selecting people. The course was free of charge, but there were only a limited number of positions. Finally, about 50 people showed up at the course (20 having chosen the Scientific Python course). Facilities at AUF were excellent: three rooms with a total of 70 PCs running the latest Ubuntu. The course on "Scientific Python" ---------------------------------- Course material ............... Whereas we reused the slides of the Moroccan event on the introduction to Python, I wrote the slides for the 3-4 hour course about Scientific Python. The slides were written in French, English being an issue for a part of the students. They can be found on http://www.dakarlug.org/pat/scientifique/html/ for French-reading people (my apologies to the others! I'm considering translating the slides to English but I will do it more rapidly if I'm given the incentive to do so, so tell me if you may be interested by an English version!). Given the duration of the course and the fact that the trainees had already been introduced to Python, I chose to concentrate mostly on Numpy, and on Scipy to a lesser extent. However, I tried to explain as soon as possible how to use numpy and scipy in a "real life" scientific workflow: plotting data with matplotlib or mayavi, opening data files, finding documentation, etc. How it went ............ Together with Thomas Noël, we followed the same group of 20 people during the training week. The group was composed of both students (master and PhD), and a few senior academic staff, professors and researchers. I'm glad to say that the course was a real success, insofar as everybody was convinced he or she could use Python with some benefit for his or her research. Most people were already using either Matlab (with some license issues...), R or Java for scientific computing. They were attracted by the following features of Python: * Python is a free software, so there are no license problems * with its wide range of scientific modules, almost *everything* can be done with Python Also, people were encouraged when they saw at the end of the week that they could work on their own data using their usual algorithms in Python, without too many difficulties. Speaking about difficulties, one may say that there was some disparity in the level of experience inside the group, some being very well at ease with array programming while others were obviously discovering programming. It was therefore difficult to meet everybody's needs, but well, I hope everybody learned something! Some pictures of the event are on http://dakarlug.org/pat/galerie/. What's next: outlook on the PAT -------------------------------- The Dakar event was a very encouraging one, and some other countries (Togo or Zambia) are already considered for the next stop of the PAT. Of course, before keeping moving towards the next event, we first would like to see what happens in Dakar after the PAT has left. Some pedagogical staff decided there would be a course on Python next academic year, and this is a very positive decision: the one-week course was definitely too short for the students. Now the big question is who is going to teach the course, as all professors were beginners in Python. A new mailing-list has also been created to gather the trainees together after the event. If you are interested by the PAT and you want to know more about past and future events, you may write to the dedicated mailing-list python-african-tour-discussion(a)lists.openplans.org, or take a look at the website http://www.coactivate.org/projects/python-african-tour. Thanks a lot for your patience if you have read so far! Cheers, Emmanuelle

2 1

Naming of OS-X binaries
by Christopher Barker 28 Jul '09

28 Jul '09

Hi folks, I just went to get the latest stable build, and found that there was this for Python 2.5: numpy-1.3.0-py2.5-macosx10.5.dmg However, as far as I can tell, that is really built for the python.org framework build of python, which works on 10.3.9 and above, not just 10.5. I do know that it works on a 10.4 PPC machine. In fact, if I see "10.5" in the name, I'm likely to assume that it is built for the Apple-supplied python2.5, which came with 10.5 It really should be re-named. Maybe something like: numpy-1.3.0-py2.5-macosx10-py.org-univ.dmg or numpy-1.3.0-py2.5-macosx10.3.dmg If you want to be compact. As long as someone is taking the trouble to build a binary, people might as well be able to know they can use it! I know, the world of multiple builds for OS-X is still a mess. SIGH. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

1 0

Commit privileges for Darren Dale
by Charles R Harris 28 Jul '09

28 Jul '09

Hi All, In case it got buried in the thread, Darren is asking for commit privileges. I think it's a good idea. Chuck

4 5

New test failures
by Nils Wagner 28 Jul '09

28 Jul '09

>>> numpy.__version__ '1.4.0.dev7270' Python 2.5.1 on 64-bit box ====================================================================== FAIL: test_umath.TestComplexFunctions.test_precisions_consistent ---------------------------------------------------------------------- Traceback (most recent call last): File "/data/home/nwagner/local/lib/python2.5/site-packages/nose-0.10.4-py2.5.egg/nose/case.py", line 182, in runTest self.test(*self.arg) File "/data/home/nwagner/local/lib/python2.5/site-packages/numpy/core/tests/test_umath.py", line 545, in test_precisions_consistent assert_almost_equal(fcf, fcd, decimal=6, err_msg='fch-fcd %s'%f) File "/data/home/nwagner/local/lib/python2.5/site-packages/numpy/testing/utils.py", line 383, in assert_almost_equal "DESIRED: %s\n" % (str(actual), str(desired))) AssertionError: Items are not equal: ACTUAL: (0.666239+1.06128j) DESIRED: (0.666239432493+1.06127506191j) ====================================================================== FAIL: test_csingle (test_linalg.TestEig) ---------------------------------------------------------------------- Traceback (most recent call last): File "/data/home/nwagner/local/lib/python2.5/site-packages/numpy/linalg/tests/test_linalg.py", line 39, in test_csingle self.do(a, b) File "/data/home/nwagner/local/lib/python2.5/site-packages/numpy/linalg/tests/test_linalg.py", line 94, in do assert_almost_equal(dot(a, evectors), multiply(evectors, evalues)) File "/data/home/nwagner/local/lib/python2.5/site-packages/numpy/linalg/tests/test_linalg.py", line 23, in assert_almost_equal old_assert_almost_equal(a, b, decimal=decimal, **kw) File "/data/home/nwagner/local/lib/python2.5/site-packages/numpy/testing/utils.py", line 383, in assert_almost_equal "DESIRED: %s\n" % (str(actual), str(desired))) AssertionError: Items are not equal: ACTUAL: [[-0.28785625-0.21230127j 2.13664055+3.62986112j] [ 0.20296739+0.16528448j 4.73750353+6.42351294j]] DESIRED: [[-0.28785625-0.21230127j 2.13664031+3.62986112j] [ 0.20296741+0.16528448j 4.73750353+6.42351341j]] ====================================================================== FAIL: test_csingle (test_linalg.TestPinv) ---------------------------------------------------------------------- Traceback (most recent call last): File "/data/home/nwagner/local/lib/python2.5/site-packages/numpy/linalg/tests/test_linalg.py", line 39, in test_csingle self.do(a, b) File "/data/home/nwagner/local/lib/python2.5/site-packages/numpy/linalg/tests/test_linalg.py", line 124, in do assert_almost_equal(dot(a, a_ginv), identity(asarray(a).shape[0])) File "/data/home/nwagner/local/lib/python2.5/site-packages/numpy/linalg/tests/test_linalg.py", line 23, in assert_almost_equal old_assert_almost_equal(a, b, decimal=decimal, **kw) File "/data/home/nwagner/local/lib/python2.5/site-packages/numpy/testing/utils.py", line 383, in assert_almost_equal "DESIRED: %s\n" % (str(actual), str(desired))) AssertionError: Items are not equal: ACTUAL: [[ 1.00000024e+00 +2.38418579e-07j -5.96046448e-08 +0.00000000e+00j] [ 5.36441803e-07 +3.57627869e-07j 9.99999821e-01 +0.00000000e+00j]] DESIRED: [[ 1. 0.] [ 0. 1.]] ====================================================================== FAIL: test_csingle (test_linalg.TestSVD) ---------------------------------------------------------------------- Traceback (most recent call last): File "/data/home/nwagner/local/lib/python2.5/site-packages/numpy/linalg/tests/test_linalg.py", line 39, in test_csingle self.do(a, b) File "/data/home/nwagner/local/lib/python2.5/site-packages/numpy/linalg/tests/test_linalg.py", line 100, in do assert_almost_equal(a, dot(multiply(u, s), vt)) File "/data/home/nwagner/local/lib/python2.5/site-packages/numpy/linalg/tests/test_linalg.py", line 23, in assert_almost_equal old_assert_almost_equal(a, b, decimal=decimal, **kw) File "/data/home/nwagner/local/lib/python2.5/site-packages/numpy/testing/utils.py", line 383, in assert_almost_equal "DESIRED: %s\n" % (str(actual), str(desired))) AssertionError: Items are not equal: ACTUAL: [[ 1.+2.j 2.+3.j] [ 3.+4.j 4.+5.j]] DESIRED: [[ 1.00000012+2.00000024j 2.00000024+3.00000024j] [ 3.00000048+4.00000048j 4.00000048+5.j ]] ---------------------------------------------------------------------- Ran 2179 tests in 67.754s FAILED (KNOWNFAIL=1, failures=4) <nose.result.TextTestResult run=2179 errors=0 failures=4>

2 1

DumbArray, LoadArray, Numeric -- numpy
by Nils Wagner 28 Jul '09

28 Jul '09

Hi all, how can I import arrays in numpy which are stored by DumpArray in the old Numeric package ? Nils

1 0

Test failures on FreeBSD buildbot
by Charles R Harris 28 Jul '09

28 Jul '09

====================================================================== ERROR: test_nan_array (test_utils.TestApproxEqual) ---------------------------------------------------------------------- Traceback (most recent call last): File "/tmp/numpy-buildbot/b12/numpy-install/lib/python2.5/site-packages/numpy/testing/tests/test_utils.py", line 236, in test_nan_array self._assert_func(anan, anan) File "../numpy-install/lib/python2.5/site-packages/numpy/testing/utils.py", line 433, in assert_approx_equal ValueError: math domain error ====================================================================== ERROR: test_nan_items (test_utils.TestApproxEqual) ---------------------------------------------------------------------- Traceback (most recent call last): File "/tmp/numpy-buildbot/b12/numpy-install/lib/python2.5/site-packages/numpy/testing/tests/test_utils.py", line 248, in test_nan_items self._assert_func(anan, anan) File "../numpy-install/lib/python2.5/site-packages/numpy/testing/utils.py", line 433, in assert_approx_equal ValueError: math domain error ---------------------------------------------------------------------- Ran 2177 tests in 30.179s I expect there is a problem with the FreeBSD math libraries.

3 6

datetime Parsing Functions
by Marty Fuhry 27 Jul '09

27 Jul '09

I'm a Summer of Code student working on the datetime implementation. The past few weeks I've been writing code to parse between dates and longs with a frequency. I'd like some feedback on my code before it gets integrated into the NumPy datetime branch: Clone URL: git://github.com/martyfuhry/npy_datetime.git Blog: http://fuhrysoc.blogspot.com The main file in question is in the parsing directory called c_parse.c. I have been working on the setitem and getitem routines for datetime64 objects (current functions in c_parse.c are long_to_date(time / string) and date(time / string)_to_long). The frequencies in the NEP are all supported, but I haven't yet implemented derived units for custom frequencies. The tests (located in the parsing/tests directory) are thorough, but I'd appreciate feedback on those, too. Is there anything I should do differently? Are there sections that need more documentation? Am I forgetting anything? Thanks in advance for any help. -Marty Fuhry

1 0

Re: [Numpy-discussion] String manipulation
by David Goldsmith 27 Jul '09

27 Jul '09

Hi, Chris. Look at this, _I'm_ answering one of _your_ questions (correctly, I hope): --- On Tue, 7/21/09, Christopher Barker <Chris.Barker(a)noaa.gov> wrote: > I don't see why: > > np.array('a string', dtype='S1') > > results in a length (1,) array, for instance. > > Actually, I think I do -- numpy is treating the string as a > single > scalar, rather than a sequence of characters, and doing its > best to > convert that scaler to a length one string. However, I Well, as for why "[it's doing] its best to convert that scalar to a length one string," that's because you used dtype='S1' instead of dtype='S8': >>> np.array('a string', dtype='S1') array('a', dtype='|S1') >>> np.array('a string', dtype='S8') array('a string', dtype='|S8') but as for shape, I can't reproduce your result at all: >>> np.array('a string', dtype='S1').shape () >>> np.array('a string', dtype='S8').shape () and as for size, yes, in both cases it is treating the entire string as a single "element": >>> np.array('a string', dtype='S1').size 1 >>> np.array('a string', dtype='S8').size 1 > don't know if > there is a compelling reason why it should do that -- in > other contexts, > Python generally treats strings as a sequence of > characters. I see your point (global consistency) but personally, IMO, if you want that kind of string behavior, work in the Python namesapace: I like (though I've yet to actually use it) options (if you want to process a single string array element in the "standard" way, extract it from the array first w/ an index - it's only a few more characters of code). :-) DG > -Chris > > > > > > > > > > -- > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R > (206) 526-6959 voice > 7600 Sand Point Way NE (206) > 526-6329 fax > Seattle, WA 98115 > (206) 526-6317 main > reception > > Chris.Barker(a)noaa.gov > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion(a)scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion >

4 6

Installed C libraries and using core math library
by David Cournapeau 27 Jul '09

27 Jul '09

Hi, I have recently integrated my work on numpy.distutils to build so called "installable" C libraries, that is pure C libraries which can be installed and reused by 3rd parties. The general documentation is in the distutils section: http://docs.scipy.org/doc/numpy/reference/distutils.html#building-installab… Example to reuse the math library is given in the core math library section in the C api chapter: http://docs.scipy.org/doc/numpy/reference/c-api.coremath.html#linking-again… cheers, David

1 0

Not enough storage for memmap on 32 bit Win XP for accumulated file size above approx. 1 GB
by Kim Hansen 27 Jul '09

27 Jul '09

OS. Win XP SP3, 32 bits Python: 2.5.4 Numpy: 1.3.0 I have am having some major problems converting a 750 MB recarray into a 850 MB recarray To save RAM I would like to use a read-only and a writeable memap for the two recarrays during the conversion. So I do something like: import os from stat import ST_SIZE import numpy as np ... records = os.stat(toconvert_path)[ST_SIZE] / toconvert_dtype.itemsize toconvert = np.memmap(toconvert_path, dtype=toconvert_dtype, mode="r").view(np.recarray) result = np.memmap(result_path, dtype = result_dtype, mode = "w+", shape=(records,)) The code manages to create the toconvert memmap (750 MB), but when trying to create the second memap object I get File "C:\Python25\Lib\site-packages\numpy\core\memmap.py", line 226, in __new__ mm = mmap.mmap(fid.fileno(), bytes, access=acc) WindowsError: [Error 8] Not enough storage is available to process this command By tracing before and after, I can see the file size is zero before calling mmap.mmap and has the expected 850 MB size after the WindowsError has been thrown somewhere inside mmap.mmap. There is 26 GB of free disc space, so the error message seems wrong? If I comment out the creation of the first memmap, I can successfully create the result memmap, so the error seems to be related to the accumulated size of all mmap.mmaps generated. I have other cases with somewhat smaller files to convert where the transition is OK. It seem like I begin to get these problems when the accumulated size of memmaps exceeds 1GB I am surprised by this, as http://docs.scipy.org/doc/numpy/reference/arrays.classes.html#memory-mapped… mentions there are upper bounds to the size when using versions of python before 2.5. >From this I had the impression that there was no size limit as long as you were using ver. >=2.5 (as I am) Is it due to the 32 bit OS I am using? Is there anything I can do to resolve the problem? Best wishes, Kim

7 18

Why does assert_array_almost_equal sometimes raise ValueError instead of AssertionError ?
by David Cournapeau 27 Jul '09

27 Jul '09

Hi, In some cases, some of the testing functions assert_array_* raise a ValueError instead of AssertionError: >>> np.testing.assert_array_almost_equal(np.array([1, 2, np.nan]), np.array([1, 2, 3])) # raises ValueError >>> np.testing.assert_array_almost_equal(np.array([1, 2, np.inf]), np.array([1, 2, 3])) # raises AssertionError This seems at least inconsistent - is there a rationale or should this be considered as a bug ? cheers, David

2 1

Re: [Numpy-discussion] fft issues
by David Goldsmith 26 Jul '09

26 Jul '09

--- On Sun, 7/26/09, Ralf Gommers <ralf.gommers(a)googlemail.com> wrote: > 3. For the hfft/ihfft functions, I am unsure what this > means: "a signal whose spectrum has Hermitian > symmetry". Can someone give a one-liner for checking > this condition, and give an example where signals like this > may occur? I think I can handle this one too. DG

1 0

fft issues
by Ralf Gommers 26 Jul '09

26 Jul '09

Hi, I came across a few issues in the fft module: 1. fftshift and ifftshift do not accept an integer for the `axes` argument. I opened a ticket, #1182, and attached a patch with test that fixes this. 2. The functions refft, irefft, refft2, irefft2, refftn and irefftn have been deprecated for at least three years. Isn't it time to remove them? 3. For the hfft/ihfft functions, I am unsure what this means: "a signal whose spectrum has Hermitian symmetry". Can someone give a one-liner for checking this condition, and give an example where signals like this may occur? Cheers, Ralf

1 0

ValueError: cannot convert float to NaN to integer
by Keith Goodman 25 Jul '09

25 Jul '09

On Ubuntu 9.04, python 2.6.2, numpy 1.2.1 this gives a ValueError: x = np.array([1,2,3]) x[0] = np.nan ValueError: cannot convert float to NaN to integer But on Debian squeeze, python 2.5.4, numpy 1.2.1 the assignment works (well, the float nan is convert to the int 0): x[0] = np.nan x array([0,1,2]) The numpy version numbers are the same. Is this a python issue? BTW, is there a way to determine the svn revision number of numpy when installed from binary?

2 2

Re: [Numpy-discussion] String to integer array of ASCII values
by David Goldsmith 24 Jul '09

24 Jul '09

--- On Thu, 7/23/09, Peter <numpy-discussion(a)maubp.freeserve.co.uk> wrote: > I should have guessed that one. Why isn't numpy.fromstring > listed with the > other entries in the "From existing data" section here? Maybe because it got listed in the I/O category and wasn't to be double listed? > http://docs.scipy.org/doc/numpy/reference/routines.array-creation.html > > This looks like a simple improvement to the > documentation... Yup, that it is; corrected here: http://docs.scipy.org/numpy/docs/numpy-docs/reference/routines.array-creati… But not yet showing up at your link. :-( DG

2 1

pdf for multivariate normal function?
by per freem 23 Jul '09

23 Jul '09

hi all, i'm trying to find the function for the pdf of a multivariate normal pdf. i know that the function "multivariate_normal" can be used to sample from the multivariate normal distribution, but i just want to get the pdf for a given vector of means and a covariance matrix. is there a function to do this? thank you.

4 3

Odd nosetest error in test_umath.py
by Matthew Brett 23 Jul '09

23 Jul '09

Hi, I just got an email about a broken buildbot build. Looking at the logs, I see this: ====================================================================== FAIL: Test bug in reduceat with structured arrays copied for speed. ---------------------------------------------------------------------- Traceback (most recent call last): File "/var/lib/python-support/python2.5/nose/case.py", line 182, in runTest self.test(*self.arg) File "/home/numpybb/Buildbot/numpy/b14/numpy-install/lib/python2.5/site-packages/numpy/core/tests/test_umath.py", line 818, in test_reduceat assert_array_almost_equal(h1, h2) File "../numpy-install/lib/python2.5/site-packages/numpy/testing/utils.py", line 537, in assert_array_almost_equal File "../numpy-install/lib/python2.5/site-packages/numpy/testing/utils.py", line 395, in assert_array_compare AssertionError: Arrays are not almost equal (mismatch 100.0%) x: array([ 701.1171875, 801.1171875, 1001.1171875, 7501.1171875], dtype=float32) y: array([ 700., 800., 1000., 7500.], dtype=float32) I'm posting because I saw this bug before on one of my own machines. It is an extremely strange error that seemed to occur in the following circumstances: 1) Running via nosetests 2) With one or more other tests in the same file as the test above, where the test was a TestCase class 3) With nose 0.10.3, 0.10.4, but not 0.11 The error given seemed to be reliable, but differed in the numbers given for different versions of the test file (but stayed the same for any given version when the test was repeated) Consider this example on my own machine: mb312@fedora11:~$ nosetests sp/numpy/core/tests/test_umath.py ... (mismatch 100.0%) x: array([ 800., 900., 1100., 7600.], dtype=float32) y: array([ 700., 800., 1000., 7500.], dtype=float32) ---------------------------------------------------------------------- Ran 71 tests in 0.064s FAILED (errors=1, failures=1) (where the error is a SkipTest). Now, just running the failing test via nose: mb312@fedora11:~$ nosetests sp/numpy/core/tests/test_umath.py:test_reduceat . ---------------------------------------------------------------------- Ran 1 test in 0.001s OK I'm flagging it in case y'all run into it, and in case someone knows what it means, and whether it is a problem. It does not occur, as I mentioned, for nose 0.11 Matthew

2 2