Mailman 3 August 2010 - NumPy-Discussion

18 Nov '10

Hi, I'm getting a couple of test failures with Python 2.6, Numpy 1.2.0, Nose 0.10.4: nose version 0.10.4 ..........................................................................................................................................................................................................................................................................................................................................................................................................................................F................K........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................./share/home/me/usr/lib/python2.6/site-packages/numpy/lib/tests/test_io.py:68: SyntaxWarning: assertion is always true, perhaps remove parentheses? assert(c.readlines(), ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................./share/home/me/usr/lib/python2.6/site-packages/numpy/ma/tests/test_core.py:1315: SyntaxWarning: assertion is always true, perhaps remove parentheses? assert(store._mask, True) /home/me/usr/lib/python2.6/site-packages/numpy/ma/tests/test_core.py:1322: SyntaxWarning: assertion is always true, perhaps remove parentheses? assert(store._mask, True) /home/me/usr/lib/python2.6/site-packages/numpy/ma/tests/test_core.py:1989: SyntaxWarning: assertion is always true, perhaps remove parentheses? assert(test.mask, [0,1,0,0,0,0,0,0,0,0]) ...............................................E................................................................................................................................................................................ ====================================================================== ERROR: Tests the min/max functions with explicit outputs ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/me/usr/lib/python2.6/site-packages/numpy/ma/tests/test_core.py", line 653, in test_minmax_funcs_with_output result = npfunc(xm,axis=0,out=nout) File "/home/me/usr/lib/python2.6/site-packages/numpy/core/fromnumeric.py", line 1525, in amin return amin(axis, out) File "/home/me/usr/lib/python2.6/site-packages/numpy/ma/core.py", line 2978, in min np.putmask(out, newmask, np.nan) ValueError: cannot convert float NaN to integer ====================================================================== FAIL: test_umath.TestComplexFunctions.test_against_cmath ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/me/usr/lib/python2.6/site-packages/nose-0.10.4-py2.6.egg/nose/case.py", line 182, in runTest self.test(*self.arg) File "/home/me/usr/lib/python2.6/site-packages/numpy/core/tests/test_umath.py", line 268, in test_against_cmath assert abs(a - b) < atol, "%s %s: %s; cmath: %s"%(fname,p,a,b) AssertionError: arcsin 2: (1.57079632679-1.31695789692j); cmath: (1.57079632679+1.31695789692j) ---------------------------------------------------------------------- Ran 1726 tests in 8.856s FAILED (KNOWNFAIL=1, errors=1, failures=1) <nose.result.TextTestResult run=1726 errors=1 failures=1>

2 1

inversion of large matrices
by Daniel Elliott 12 Oct '10

12 Oct '10

Hello, I am new to Python (coming from R and Matlab/Octave). I was preparing to write my usual compute pdf of a really high dimensional (e.g. 10000 dimensions) Gaussian code in Python but I noticed that numpy had a function for computing the log determinant in these situations. Is there a function for performing the inverse or even the pdf of a multinomial normal in these situations as well? Thank you. Numpy and scipy look pretty awesome. - dan

10 17

lexsort() of datetime objects doesn't work
by Ernest Adrogué 11 Sep '10

11 Sep '10

Hi, I was trying to use lexsort with an array of datetime.date objects, but it doesn't seem to work. In [86]: date = np.array([datetime.date(2000, 9, 17), datetime.date(2000, 10, 1), datetime.date(2000, 10, 22), datetime.date(2000, 11, 1)]) In [90]: date Out[90]: array([2000-09-17, 2000-10-01, 2000-10-22, 2000-11-01], dtype=object) In [91]: np.lexsort((date,)) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) /home/ernest/src/footy/dataset.py in <module>() ----> 1 2 3 4 5 TypeError: merge sort not available for item 0 The normal argsort works, however this is in a method that uses lexsort and I can't change it without breaking things. Ernest

3 3

Boolean arrays
by Brett Olsen 02 Sep '10

02 Sep '10

Hello, I have an array of non-numeric data, and I want to create a boolean array denoting whether each element in this array is a "valid" value or not. This is straightforward if there's only one possible valid value: >>> import numpy as N >>> ar = N.array(("a", "b", "c", "b", "b", "a", "d", "c", "a")) >>> ar == "a" array([ True, False, False, False, False, True, False, False, True], dtype=bool) If there's multiple possible valid values, I've come up with a couple possible methods, but they all seem to be inefficient or kludges: >>> valid = N.array(("a", "c")) >>> (ar == valid[0]) | (ar == valid[1]) array([ True, False, True, False, False, True, False, True, True], dtype=bool) >>> N.array(map(lambda x: x in valid, ar)) array([ True, False, True, False, False, True, False, True, True], dtype=bool) Is there a numpy-appropriate way to do this? Thanks, Brett Olsen

9 11

Github migration?
by Travis Oliphant 02 Sep '10

02 Sep '10

Hi all, I'm curious as to the status of the Github migration and if there is anything I can do to help. I have a couple of weeks right now and I would love to see us make the transition of both NumPy and SciPy to GIT. On a slightly related note, it would really help the numpy-refactor project if it received more input from others in the community. Right now, the numpy-refactor is happening on a public github branch (named numpy-refactor) awaiting numpy itself to be on github. It would be more useful if the numpy-refactor branch were a regular branch of the github NumPy project. The numpy-refactor project is making great progress and we have a working core library that can be built on Windows, Mac, and Linux. The goal is for this numpy-refactor to become the basis for NumPy 2.0 which should come out sometime this Fall. Already, a lot of unit tests have been written and code coverage has increased on the core NumPy code which I think we all agree is "a good thing" In addition, some lingering reference count bugs (particularly in object arrays) have been found and squashed. There is also some good progress on the Cython backend for .NET which would allow and also put pressure on migration of most of SciPy to Cython-or-Fwrap generated extension modules. I am looking forward to working on NumPy a little more over the coming months. All the best, -Travis

6 23

comparison between arrays of strings and numerical types
by Ernest Adrogué 31 Aug '10

31 Aug '10

Hi, I find this a bit odd: In [18]: np.array(['a','b','c','d']) > 'a' Out[18]: array([False, True, True, True], dtype=bool) In [19]: np.array(['a','b','c','d']) > 4 Out[19]: True In [20]: np.array(['a','b','c','d']) > 4.5 Out[20]: True Is that right? I was expecting an element-wise comparison, but it returns a single truth value. Cheers, Ernest

2 2

Python3 and intp error (related to ticket 99)
by Bruce Southey 31 Aug '10

31 Aug '10

Hi, I was curious why there was a difference in number of known failures between Python2.6 and Python3.1 which is associated a test due to ticket 99: http://projects.scipy.org/numpy/ticket/99 While this ticket was closed, it fails with Python 3.1 as indicated by the message of the test output is: 'Ticket #99 ... KNOWNFAIL: numpy.intp('0xff', 16) not supported on Py3, as it does not inherit from Python int' I do understand that np.intp is integer size of a pointer. But it appears to be mainly used for access to C programs. The only Python numpy usage I saw was with the delete and insert function in 'numpy/lib/function_base.py'. Does this really need to be exposed in Python? If it does not, then could be removed for Numpy 2? Otherwise, at the very least np.intp must have the same behavior across Python versions. As per the ticket, since it is an integer type, should it have the same properties as a regular integer? Bruce $ python3.1 Python 3.1.2 (r312:79147, Mar 24 2010, 10:44:23) [GCC 4.4.3 20100127 (Red Hat 4.4.3-4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy.__version__ '1.5.0' >>> numpy.intp('0xff', 16) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: function takes at most 1 argument (2 given) >>> numpy.intp('0xff') Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: setting an array element with a sequence. >>> Yet there is no problem with Python 2.6: $ python Python 2.6.4 (r264:75706, Jun 4 2010, 18:20:31) [GCC 4.4.4 20100503 (Red Hat 4.4.4-2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy.__version__ '1.5.0' >>> numpy.intp('0xff', 16) 255 >>>

2 1

ANN: NumPy 1.5.0 release
by Ralf Gommers 31 Aug '10

31 Aug '10

I am pleased to announce the availability of NumPy 1.5.0. This is the first NumPy release to include support for Python 3, as well as for Python 2.7. Binaries, sources, documentation and release notes can be found at https://sourceforge.net/projects/numpy/files/. Thank you to everyone who contributed to this release. Enjoy, the Numpy developers ========================= NumPy 1.5.0 Release Notes ========================= Highlights ========== Python 3 compatibility ---------------------- This is the first NumPy release which is compatible with Python 3. Support for Python 3 and Python 2 is done from a single code base. Extensive notes on changes can be found at `<http://projects.scipy.org/numpy/browser/trunk/doc/Py3K.txt>`_. Note that the Numpy testing framework relies on nose, which does not have a Python 3 compatible release yet. A working Python 3 branch of nose can be found at `<http://bitbucket.org/jpellerin/nose3/>`_ however. Porting of SciPy to Python 3 is expected to be completed soon. :pep:`3118` compatibility ------------------------- The new buffer protocol described by PEP 3118 is fully supported in this version of Numpy. On Python versions >= 2.6 Numpy arrays expose the buffer interface, and array(), asarray() and other functions accept new-style buffers as input. New features ============ Warning on casting complex to real ---------------------------------- Numpy now emits a `numpy.ComplexWarning` when a complex number is cast into a real number. For example: >>> x = np.array([1,2,3]) >>> x[:2] = np.array([1+2j, 1-2j]) ComplexWarning: Casting complex values to real discards the imaginary part The cast indeed discards the imaginary part, and this may not be the intended behavior in all cases, hence the warning. This warning can be turned off in the standard way: >>> import warnings >>> warnings.simplefilter("ignore", np.ComplexWarning) Dot method for ndarrays ----------------------- Ndarrays now have the dot product also as a method, which allows writing chains of matrix products as >>> a.dot(b).dot(c) instead of the longer alternative >>> np.dot(a, np.dot(b, c)) linalg.slogdet function ----------------------- The slogdet function returns the sign and logarithm of the determinant of a matrix. Because the determinant may involve the product of many small/large values, the result is often more accurate than that obtained by simple multiplication. new header ---------- The new header file ndarraytypes.h contains the symbols from ndarrayobject.h that do not depend on the PY_ARRAY_UNIQUE_SYMBOL and NO_IMPORT/_ARRAY macros. Broadly, these symbols are types, typedefs, and enumerations; the array function calls are left in ndarrayobject.h. This allows users to include array-related types and enumerations without needing to concern themselves with the macro expansions and their side- effects. Changes ======= polynomial.polynomial --------------------- * The polyint and polyder functions now check that the specified number integrations or derivations is a non-negative integer. The number 0 is a valid value for both functions. * A degree method has been added to the Polynomial class. * A trimdeg method has been added to the Polynomial class. It operates like truncate except that the argument is the desired degree of the result, not the number of coefficients. * Polynomial.fit now uses None as the default domain for the fit. The default Polynomial domain can be specified by using [] as the domain value. * Weights can be used in both polyfit and Polynomial.fit * A linspace method has been added to the Polynomial class to ease plotting. * The polymulx function was added. polynomial.chebyshev -------------------- * The chebint and chebder functions now check that the specified number integrations or derivations is a non-negative integer. The number 0 is a valid value for both functions. * A degree method has been added to the Chebyshev class. * A trimdeg method has been added to the Chebyshev class. It operates like truncate except that the argument is the desired degree of the result, not the number of coefficients. * Chebyshev.fit now uses None as the default domain for the fit. The default Chebyshev domain can be specified by using [] as the domain value. * Weights can be used in both chebfit and Chebyshev.fit * A linspace method has been added to the Chebyshev class to ease plotting. * The chebmulx function was added. * Added functions for the Chebyshev points of the first and second kind. histogram --------- After a two years transition period, the old behavior of the histogram function has been phased out, and the "new" keyword has been removed. correlate --------- The old behavior of correlate was deprecated in 1.4.0, the new behavior (the usual definition for cross-correlation) is now the default. Checksums ========= 738572dd3e5d4cd89e98c76cc3f162a9 release/installers/numpy-1.5.0-py2.5-python.org.dmg f58ebc1840974cf2ef8b4f41516dc288 release/installers/numpy-1.5.0-py2.6-python.org.dmg d7232048392ede8d8d8fb57839cb4b91 release/installers/numpy-1.5.0-py2.7-python.org.dmg c5130a11f920706cdc665ef1e07772e2 release/installers/numpy-1.5.0-win32-superpack-python2.5.exe b46a52f82126ace1eb7cb627623c64dc release/installers/numpy-1.5.0-win32-superpack-python2.6.exe 8a93c004a104f6231de4c398e2b3c48f release/installers/numpy-1.5.0-win32-superpack-python2.7.exe 1d255b58764d465e64b7b3eee832aa9e release/installers/numpy-1.5.0-win32-superpack-python3.1.exe 3a8bfdc434df782d647161c48943ee09 release/installers/numpy-1.5.0.tar.gz 11354c0ca15ca6f37df9589bd4f25943 release/installers/numpy-1.5.0.zip

2 1

Re: [Numpy-discussion] numpy histogram normed=True (bug / confusing behavior)
by Nils Becker 31 Aug '10

31 Aug '10

> On Sat, Aug 28, 2010 at 04:12, Zbyszek Szmek <zbyszek(a)in.waw.pl> wrote: >> Hi, >> >> On Fri, Aug 27, 2010 at 06:43:26PM -0600, Charles R Harris wrote: >>> ? ?On Fri, Aug 27, 2010 at 2:47 PM, Robert Kern <robert.kern(a)gmail.com> >>> ? ?wrote: >>> >>> ? ? ?On Fri, Aug 27, 2010 at 15:32, David Huard <david.huard(a)gmail.com> >>> ? ? ?wrote: >>> ? ? ?> Nils and Joseph, >>> ? ? ?> Thanks for the bug report, this is now fixed in SVN (r8672). >>> >>> ? ? ?While we're at it, can we change the name of the argument? "normed" >>> ? ? ?has caused so much confusion over the years. We could deprecate >>> ? ? ?normed=True in favor of pdf=True or density=True. >> I think it might be a good moment to also include a different type of normalization: >> ? ? ? n = n / n.sum() >> i.e. the frequency of counts in each bin. This one is of course very simple to calculate >> by hand, but very common. I think it would be useful to have this normalization >> available too. [http://www.itl.nist.gov/div898/handbook/eda/section3/histogra.htm] > > My feeling is that this is trivial to do "by hand". I do not see a > reason to add an option to histogram() to do this. > Hi, +1 for not silently changing the behavior of normed=True. (I'm one of the people who have worked around it). One argument in favor of putting both normalizing styles 'frequency' and 'density' may be that the documentation will automatically become very clear. A user sees all options and there is little chance of a misunderstanding. Of course, a sentence like "If you want frequency normalization, use histogram(data, normalized=False)/sum(data)" would also make things clear, without adding the frequency option. Nils

7 13

conversion to 1D array of specified dtype
by Mark Bakker 31 Aug '10

31 Aug '10

Hello list, What is the easiest way to convert a function argument to at least a 1D array of specified dtype? atleast_1d(3,dtype='d') doesn't work (numpy 1.3.0) array(atleast_1d(3),dtype='d') works but seems cumbersome Thanks, Mark

2 1