Mailman 3 January 2013 - NumPy-Discussion

Merging structured arrays with mixed dtypes including '|O4'
by Alex Jan. 28, 2013

Jan. 28, 2013

Let's say I have two structured arrays with dtypes as per below >>> getdat.dtype dtype([('Tstamp', '|O4'), ('Vf', '<f4'), ('Vq', '<f4'), ('Sf', '<f4'), ('Sq', '<f4'), ('Bt', '<f4'), ('SPL', '<f4')]) >>> out.dtype dtype([('Viscosity_cSt', '<f4'), ('Density_kgm3', '<f4'), ('GVF', '<f4')]) Then merging this per the below gives an error. Why? >>> rfn.merge_arrays((getdat, out), flatten = True, usemask = False, asrecarray=False) Traceback (most recent call last): File "<pyshell#43>", line 1, in <module> rfn.merge_arrays((getdat, out), flatten = True, usemask = False, asrecarray=False) File "C:\Python27\lib\site-packages\numpy\lib\recfunctions.py", line 458, in merge_arrays dtype=newdtype, count=maxlength) ValueError: cannot create object arrays from iterator The issue seems to be object field 'Tstamp' which contains python datetime objects. I can merge structured arrays with numeric formats. Any help much appreciated. Alex van der Spek

1 0

Installing numpy-mkl binary on top of Python(x, y)
by olli.wallin＠elisanet.fi Jan. 27, 2013

Jan. 27, 2013

Hi, if I want to have a painless Python installation build against Intel MKL on Windows, one obvious choice is to just buy the EPD package. However, as I already do have a C++ licence of the MKL library I was wondering if I could just install the Python(x,y) -distribution and then take one of the NumPy-MKL binaries provided by Christoph Gohlke. Is it simple as that? Any downsides, will SciPy work as well? On the plus side, I would get Spyder2 without hassle and it looks nice to a former Matlab user. I apologize for such a simple question, I would have tried it myself but this is for my work where only IT support has the admin rights and I have mac at home. I want it to be as clearcut for them as possible so I get things up and running. I did try to search the internet and the list but did not find a conclusive answer. Many thanks in advance for any help. All the best, Olli --

3 2

NumPy int32 array to Excel through COM server is failing
by Raul Cota Jan. 26, 2013

Jan. 26, 2013

Hello, We came across a problem trying to get an array across COM when wrapped as a COM server using the win32com extension. What caught our attention is that arrays of type float64 work fine but do not work for any other array. Does anyone know if there is something we could do at the NumPy level to make it work ? Arrays are mapped onto Variant/SafeArray and the conversion to a Variant seems to be failing for anything that is not a float64. It is not a huge deal for us to workaround the problem but it is kind of ugly and I just wanted to make sure there was not something simple that could be done (particularly if something like this would be considered a bug). I include working sample code below that reproduces the problem where Excel instantiates a com server and requests an array of size and dtype. On the Python side, the code sets up the com server and exposes the function that just returns an array of ones. Code in Excel to get an array: ======================================== Public Sub NumPyVariantTest() Dim npcom As Object Dim arr '... instantiate com object Set npcom = CreateObject("NPTest.COM") size = 7 '... test float64. Works ! arr = npcom.GetNPArray(size, "float64") '... test int32. Fails ! arr = npcom.GetNPArray(size, "int32") End Sub ======================================== Code in Python to set up com server and expose the GetNPArray function ======================================== import win32com.server.util import win32com.client from pythoncom import CLSCTX_LOCAL_SERVER, CLSCTX_INPROC import sys, os import numpy from numpy import zeros, ones, array class NPTestCOM(object): """COM accessible version of CommandInterface""" _reg_clsid_ = "{A0E551F5-2F22-4FB4-B28E-FF1B6809D21C}" _reg_desc_ = "NumPy COM Test" _reg_progid_ = "NPTest.COM" _reg_clsctx_ = CLSCTX_INPROC _public_methods_ = ['GetNPArray'] _public_attrs_ = [] _readonly_attrs_ = [] def GetNPArray(self, size, dtype): """ Return an arbitrary NumPy array of type dtype to check conversion to Variant""" return ones(size, dtype=dtype) if __name__ == '__main__': import win32com.server.register import _winreg dllkey = 'nptestdll' if len(sys.argv) > 1 and sys.argv[1] == 'unregister': win32com.server.register.UnregisterClasses(NPTestCOM) software_key = _winreg.OpenKey(_winreg.HKEY_LOCAL_MACHINE, 'SOFTWARE') vmg_key = _winreg.OpenKey(software_key, 'VMG') _winreg.DeleteKey(vmg_key, dllkey) _winreg.CloseKey(vmg_key) _winreg.CloseKey(software_key) else: win32com.server.register.UseCommandLine(NPTestCOM) software_key = _winreg.OpenKey(_winreg.HKEY_LOCAL_MACHINE, 'SOFTWARE') vmg_key = _winreg.CreateKey(software_key, 'VMG') _winreg.SetValue(vmg_key, dllkey, _winreg.REG_SZ, os.path.abspath(os.curdir)) _winreg.CloseKey(vmg_key) _winreg.CloseKey(software_key) ======================================== Regards, Raul -- Raul Cota (P.Eng., Ph.D. Chemical Engineering) Research & Development Manager Phone: (403) 457 4598 Fax: (403) 457 4637 Virtual Materials Group - Canada www.virtualmaterials.com

1 0

[SCIPY2013] Feedback on mini-symposia themes
by Jonathan Rocher Jan. 23, 2013

Jan. 23, 2013

Dear community members, We are working hard to organize the SciPy2013 conference (Scientific Computing with Python) <http://conference.scipy.org/scipy2013/>, this June 24th-29th in Austin, TX. We would like to probe the community about the themes you would be interested in contributing to or participating in for the mini-symposia at SciPy2013. These mini-symposia are held to discuss scientific computing applied to a specific *scientific domain/industry* during a half afternoon after the general conference. Their goal is to promote industry specific libraries and tools, and gather people with similar interests for discussions. For example, the SciPy2012<http://conference.scipy.org/scipy2012/schedule/conf_schedule_1.php> edition successfully hosted 4 mini-symposia on Astronomy/Astrophysics, Bio-informatics, Meteorology, and Geophysics. Please join us and voice your opinion to shape the next SciPy conference at: http://www.surveygizmo.com/s3/1114631/SciPy-2013-Themes Thanks, The Scipy2013 organizers -- Jonathan Rocher, PhD Scientific software developer Enthought, Inc. jrocher(a)enthought.com 1-512-536-1057 http://www.enthought.com

1 1

ANN: scikit-learn 0.13 released!
by Andreas Mueller Jan. 21, 2013

Jan. 21, 2013

Hi all. I am very happy to announce the release of scikit-learn 0.13. New features in this release include feature hashing for text processing, passive-agressive classifiers, faster random forests and many more. There have also been countless improvements in stability, consistency and usability. Details can be found on the what's new <http://scikit-learn.org/stable/whats_new.html>page. Sources and windows binaries are available on sourceforge, through pypi (http://pypi.python.org/pypi/scikit-learn/0.13) or can be installed directly using pip: pip install -U scikit-learn A big "thank you" to all the contributors who made this release possible! In parallel to the release, we started a small survey <https://docs.google.com/spreadsheet/viewform?formkey=dFdyeGNhMzlCRWZUdldpME…> to get to know our user base a bit more. If you are using scikit-learn, it would be great if you could give us your input. Best, Andy

2 1

Do we want scalar casting to behave as it does at the moment?
by Matthew Brett Jan. 21, 2013

Jan. 21, 2013

Hi, I wanted to check that everyone knows about and is happy with the scalar casting changes from 1.6.0. Specifically, the rules for (array, scalar) casting have changed such that the resulting dtype depends on the _value_ of the scalar. Mark W has documented these changes here: http://docs.scipy.org/doc/numpy/reference/ufuncs.html#casting-rules http://docs.scipy.org/doc/numpy/reference/generated/numpy.result_type.html http://docs.scipy.org/doc/numpy/reference/generated/numpy.promote_types.html Specifically, as of 1.6.0: In [19]: arr = np.array([1.], dtype=np.float32) In [20]: (arr + (2**16-1)).dtype Out[20]: dtype('float32') In [21]: (arr + (2**16)).dtype Out[21]: dtype('float64') In [25]: arr = np.array([1.], dtype=np.int8) In [26]: (arr + 127).dtype Out[26]: dtype('int8') In [27]: (arr + 128).dtype Out[27]: dtype('int16') There's discussion about the changes here: http://mail.scipy.org/pipermail/numpy-discussion/2011-September/058563.html http://mail.scipy.org/pipermail/numpy-discussion/2011-March/055156.html http://mail.scipy.org/pipermail/numpy-discussion/2012-February/060381.html It seems to me that this change is hard to explain, and does what you want only some of the time, making it a false friend. Is it the right behavior for numpy 2.0? Cheers, Matthew

12 87

another little index puzzle
by Neal Becker Jan. 21, 2013

Jan. 21, 2013

I have an array to be used for indexing. It is 2d, where the rows are all the permutations of some numbers. So: array([[-2, -2, -2], [-2, -2, -1], [-2, -2, 0], [-2, -2, 1], [-2, -2, 2], ... [ 2, 1, 2], [ 2, 2, -2], [ 2, 2, -1], [ 2, 2, 0], [ 2, 2, 1], [ 2, 2, 2]]) Here the array is 125x3 I want to select all the rows of the array in which all the 3 elements are equal, so I can remove them. So for example, the 1st and last row.

3 2

New numpy functions: filled, filled_like
by Nathaniel Smith Jan. 21, 2013

Jan. 21, 2013

Hi all, PR 2875 adds two new functions, that generalize zeros(), ones(), zeros_like(), ones_like(), by simply taking an arbitrary fill value: https://github.com/numpy/numpy/pull/2875 So np.ones((10, 10)) is the same as np.filled((10, 10), 1) The implementations are trivial, but the API seems useful because it provides an idiomatic way of efficiently creating an array full of inf, or nan, or None, whatever funny value you need. All the alternatives are either inefficient (np.ones(...) * np.inf) or cumbersome (a = np.empty(...); a.fill(...)). Or so it seems to me. But there's a question of taste here; one could argue instead that these just add more clutter to the numpy namespace. So, before we merge, anyone want to chime in? (Bonus, extra bike-sheddy survey: do people prefer np.filled((10, 10), np.nan) np.filled_like(my_arr, np.nan) or np.filled(np.nan, (10, 10)) np.filled_like(np.nan, my_arr) ?) -n

22 50

Shouldn't all in-place operations simply return self?
by eat Jan. 19, 2013

Jan. 19, 2013

Hi, In a recent thread http://article.gmane.org/gmane.comp.python.numeric.general/52772 it was proposed that .fill(.) should return self as an alternative for a trivial two-liner. I'm raising now the question: what if all in-place operations indeed could return self? How bad this would be? A 'strong' counter argument may be found at http://mail.python.org/pipermail/python-dev/2003-October/038855.html. But anyway, at least for me. it would be much more straightforward to implement simple mini dsl's ( http://en.wikipedia.org/wiki/Domain-specific_language) a much more straightforward manner. What do you think? -eat P.S. FWIW, if this idea really gains momentum obviously I'm volunteering to create a PR of it.

10 14

Casting Bug or a "Feature"?
by Patrick Marsh Jan. 18, 2013

Jan. 18, 2013

Greetings, I spent a couple hours today tracking down a bug in one of my programs. I was getting different answers depending on whether I passed in a numpy array or a single number. Ultimately, I tracked it down to something I would consider a bug, but I'm not sure if others do. The case comes from taking a numpy integer array and adding a float to it. When doing var = np.array(ints) + float, var is cast to an array of floats, which is what I would expect. However, if I do np.array(ints) += float, the result is an array of integers. I can understand why this happens -- you are shoving the sum back into an integer array -- but without thinking through that I would expect the behavior of the two additions to be equal...or at least be consistent with what occurs with numbers, instead of arrays. Here's a trivial example demonstrating this import numpy as np a = np.arange(10) print a.dtype b = a + 0.5 print b.dtype a += 0.5 print a.dtype >> int64 >> float64 >> int64 >> <type 'int'> >> <type 'float'> >> <type 'float'> An implication of this arrises from a simple function that "does math". The function returns different values depending on whether a number or array was passed in. def add_n_multiply(var): var += 0.5 var *= 10 return var aaa = np.arange(5) print aaa print add_n_multiply(aaa.copy()) print [add_n_multiply(x) for x in aaa.copy()] >> [0 1 2 3 4] >> [ 0 10 20 30 40] >> [5.0, 15.0, 25.0, 35.0, 45.0] Am I alone in thinking this is a bug? Or is this the behavior that others would have expected? Cheers, Patrick --- Patrick Marsh Ph.D. Candidate / Liaison to the HWT School of Meteorology / University of Oklahoma Cooperative Institute for Mesoscale Meteorological Studies National Severe Storms Laboratory http://www.patricktmarsh.com

10 14