Mailman 3 July 2009 - NumPy-Discussion

find_common_type broken?
by Ralf Gommers 20 Jul '09

20 Jul '09

Hi, While documenting find_common_type I found two problems. As I understand it, "common type" is a type to which all input types can be cast without loss of precision. 1. using any array types always returns "dtype('object')": >>> np.find_common_type([np.ndarray], []) dtype('object') >>> np.find_common_type([np.ndarray, np.ma.MaskedArray, np.recarray], []) dtype('object') 2. the second example below seems to be wrong, it should return dtype('float64'): >>> np.find_common_type([], [np.int64, np.float64]) dtype('float64') >>> np.find_common_type([], [np.int64, np.float32]) dtype('int64') One other question, why do type comparisons for numpy types and python built-ins do the opposite: >>> np.int32 > np.float32 False >>> np.int64 > np.float64 False >>> int > float True The numpy result makes more sense to me, what's going on with the builtins? Ralf

3 7

Update numpy.lib.ufunclike.log2
by Brian Lewis 20 Jul '09

20 Jul '09

Should numpy.lib.ufunclike.log2 be updated to: x = nx.asanyarray(x) if y is None: y = nx.log2(x) else: nx.log2(x, y) return y

1 1

Repeated Dot Operations
by T J 20 Jul '09

20 Jul '09

Hi, Is there a good way to perform dot on an arbitrary list of arrays which avoids using a loop? Here is what I'd like to avoid: # m1, m2, m3 are arrays >>> out = np.(m1.shape[0]) >>> prod = [m1, m2, m3, m1, m2, m3, m3, m2] >>> for m in prod: ... out = np.dot(out, m) ... I was hoping for something like np.dot.reduce(prod).

2 1

ANN: line_profiler 1.0b2 released
by Robert Kern 20 Jul '09

20 Jul '09

http://pypi.python.org/pypi/line_profiler/ http://packages.python.org/line_profiler/ This release fixes the "negative timings" issue on Windows. Future announcements will occur on python-announce. I just wanted to make sure my earliest users here who ran into this bug are aware of the fix. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

4 6

indexing recarrays
by John [H2O] 20 Jul '09

20 Jul '09

I have a file containing mixed data types: strings, floats, datetime output(i.e. strings), and ints. Something like: #ID, name, date, value 1,sample,2008-07-10 12:34:20,344.56 Presuming I get them nicely into a recarray (see my other http://www.nabble.com/recarray-and-datetime-objects-td24568340.html post ) then I would like to work with them using the convention that I have become used to: D[:,0] #means D for all rows, column 0 But it seems now I have to use: D['columnID'] Am I missing something, or is there a way to keep referencing columns by the column integer index?? Thanks! On another note... are recarrays and structured arrays the same?? -- View this message in context: http://www.nabble.com/indexing-recarrays-tp24568432p24568432.html Sent from the Numpy-discussion mailing list archive at Nabble.com.

2 1

re carray and datetime objects
by John [H2O] 20 Jul '09

20 Jul '09

Hello, I have a file containing mixed data types: strings, floats, datetime output(i.e. strings), and ints. Something like: #ID, name, date, value 1,sample,2008-07-10 12:34:20,344.56 And so forth. It seems using recarrays is efficient and a prefered habit to get into wrg to numpy, so I am trying the following: dtype=[('ID','int'),('name','|S10'),('date','????'),('value','float')] DATA = np.loadtxt(infile,dtype=dtype) This works if I set 'date' type to 'S20', but wouldn't it be more appropriate to use object? Also, what is the next step to then use the values as datetime objects? Do I have to take the data out of the array, convert to datetime, then put back in the array? dates = DATA['date'] dates = [dt.datetime.strptime(d,'%Y-%m-%d %H:%M:%S') for d in dates] DATA['dates'] = np.array(dates) ???? Thanks! -- View this message in context: http://www.nabble.com/recarray-and-datetime-objects-tp24568340p24568340.html Sent from the Numpy-discussion mailing list archive at Nabble.com.

1 0

Re: [Numpy-discussion] list dtype?
by David Goldsmith 20 Jul '09

20 Jul '09

99 times out of 100, the way to do things in Numpy (as in Python) is simpler than what you thought - that's why we love 'em. ;-) DG --- On Sun, 7/19/09, Mark Wendell <mark.wendell(a)gmail.com> wrote: > From: Mark Wendell <mark.wendell(a)gmail.com> > Subject: Re: [Numpy-discussion] list dtype? > To: "Discussion of Numerical Python" <numpy-discussion(a)scipy.org> > Date: Sunday, July 19, 2009, 3:59 PM > Excellent! I guess I was > over-thinking it. > > Many thanks. > Mark > > On Sun, Jul 19, 2009 at 4:50 PM, David Goldsmith<d_l_goldsmith(a)yahoo.com> > wrote: > > > > Robert'll correct me if I'm wrong, but I think what he > meant is that all you have to do is: > > > > array_of_lists = numpy.empty(<shape tuple>, > dtype=object) > > > > and you'll be able to fill array_of_lists with > whatever you want: > > > >>>> aol = np.empty((2,2),dtype=object) > >>>> aol[:,:] = [[[1, 2, 3], ['a','b','c']], > [[(), (), ()], [{}, {}, {}]]] > >>>> aol > > array([[[1, 2, 3], ['a', 'b', 'c']], > > [[(), (), ()], [{}, {}, {}]]], dtype=object) > > > > However, you do have to specify the shape ahead of > time: > > > >>>> aol = np.empty((),dtype=object) > >>>> aol[:,:] = [list([5, 6, 7]),list(['a', > 'b', 'c'])] > > Traceback (most recent call last): > > File "<stdin>", line 1, in <module> > > IndexError: 0-d arrays can't be indexed. > >>>> aol.shape = (2,2) > > Traceback (most recent call last): > > File "<stdin>", line 1, in <module> > > ValueError: total size of new array must be unchanged > > > > (Of course, if you need to change its size and shape, > you can always instantiate a new array, initializing the > "old" portion of it w/ the existing array and filling in the > rest w/ the new values.) > > > > DG > > > > --- On Sun, 7/19/09, Mark Wendell <mark.wendell(a)gmail.com> > wrote: > > > >> From: Mark Wendell <mark.wendell(a)gmail.com> > >> Subject: Re: [Numpy-discussion] list dtype? > >> To: "Discussion of Numerical Python" <numpy-discussion(a)scipy.org> > >> Date: Sunday, July 19, 2009, 3:19 PM > >> Sorry for being a newbie about it, > >> but could you provide an example? > >> > >> I was able to do this: > >> > >> d = np.dtype( {'names':['mylist'], > 'formats':[np.object]}) > >> but I'm not sure if that's the correct way to > produce a > >> python list-type dtype. > >> > >> Further, how do I then define a new array with > that as a > >> dtype? > >> > >> When I do this: > >> a = np.array( (5,5), dtype = d) > >> I get the error: > >> ValueError: size of tuple must match number of > fields. > >> > >> > >> Any help appreciated. Thanks! > >> Mark > >> > >> On Sun, Jul 19, 2009 at 2:37 PM, Robert Kern<robert.kern(a)gmail.com> > >> wrote: > >> > On Sun, Jul 19, 2009 at 11:22, Mark > Wendell<mark.wendell(a)gmail.com> > >> wrote: > >> >> Is it possible to create a dtype that is > a list of > >> arbitrary length? > >> > > >> > Use the object dtype. > >> > > >> > -- > >> > Robert Kern > >> > > >> > "I have come to believe that the whole world > is an > >> enigma, a harmless > >> > enigma that is made terrible by our own mad > attempt to > >> interpret it as > >> > though it had an underlying truth." > >> > -- Umberto Eco > >> > > _______________________________________________ > >> > NumPy-Discussion mailing list > >> > NumPy-Discussion(a)scipy.org > >> > http://mail.scipy.org/mailman/listinfo/numpy-discussion > >> > > >> > >> > >> > >> -- > >> -- > >> Mark Wendell > >> "I guess songs are just interesting things to do > with the > >> air." Tom Waits > >> _______________________________________________ > >> NumPy-Discussion mailing list > >> NumPy-Discussion(a)scipy.org > >> http://mail.scipy.org/mailman/listinfo/numpy-discussion > >> > > > > > > > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion(a)scipy.org > > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > > > > > -- > -- > Mark Wendell > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion(a)scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion >

1 0

Re: [Numpy-discussion] list dtype?
by David Goldsmith 20 Jul '09

20 Jul '09

Robert'll correct me if I'm wrong, but I think what he meant is that all you have to do is: array_of_lists = numpy.empty(<shape tuple>, dtype=object) and you'll be able to fill array_of_lists with whatever you want: >>> aol = np.empty((2,2),dtype=object) >>> aol[:,:] = [[[1, 2, 3], ['a','b','c']], [[(), (), ()], [{}, {}, {}]]] >>> aol array([[[1, 2, 3], ['a', 'b', 'c']], [[(), (), ()], [{}, {}, {}]]], dtype=object) However, you do have to specify the shape ahead of time: >>> aol = np.empty((),dtype=object) >>> aol[:,:] = [list([5, 6, 7]),list(['a', 'b', 'c'])] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: 0-d arrays can't be indexed. >>> aol.shape = (2,2) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: total size of new array must be unchanged (Of course, if you need to change its size and shape, you can always instantiate a new array, initializing the "old" portion of it w/ the existing array and filling in the rest w/ the new values.) DG --- On Sun, 7/19/09, Mark Wendell <mark.wendell(a)gmail.com> wrote: > From: Mark Wendell <mark.wendell(a)gmail.com> > Subject: Re: [Numpy-discussion] list dtype? > To: "Discussion of Numerical Python" <numpy-discussion(a)scipy.org> > Date: Sunday, July 19, 2009, 3:19 PM > Sorry for being a newbie about it, > but could you provide an example? > > I was able to do this: > > d = np.dtype( {'names':['mylist'], 'formats':[np.object]}) > but I'm not sure if that's the correct way to produce a > python list-type dtype. > > Further, how do I then define a new array with that as a > dtype? > > When I do this: > a = np.array( (5,5), dtype = d) > I get the error: > ValueError: size of tuple must match number of fields. > > > Any help appreciated. Thanks! > Mark > > On Sun, Jul 19, 2009 at 2:37 PM, Robert Kern<robert.kern(a)gmail.com> > wrote: > > On Sun, Jul 19, 2009 at 11:22, Mark Wendell<mark.wendell(a)gmail.com> > wrote: > >> Is it possible to create a dtype that is a list of > arbitrary length? > > > > Use the object dtype. > > > > -- > > Robert Kern > > > > "I have come to believe that the whole world is an > enigma, a harmless > > enigma that is made terrible by our own mad attempt to > interpret it as > > though it had an underlying truth." > > -- Umberto Eco > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion(a)scipy.org > > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > > > > > -- > -- > Mark Wendell > "I guess songs are just interesting things to do with the > air." Tom Waits > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion(a)scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion >

2 1

list dtype?
by Mark Wendell 20 Jul '09

20 Jul '09

Is it possible to create a dtype that is a list of arbitrary length? More specifically, let's say I have a custom dtype like this: d = dtype( {'names':['index','xcoord','ycoord','weight'], 'formats': [ uint16, uint16, uint16, float ] } ) Now I'd like to create a 2D array where each entry is a LIST of items of type d. Each list may contain only one instance, or many, and I intend to write routines that will access and extend that list. So I guess the more general question is... how do I create a dtype that is a list, and can the length of that list vary from entry to entry in a 2D array? I hope I've explained this well enough. Thanks in advance. Mark -- -- Mark Wendell "I guess songs are just interesting things to do with the air." Tom Waits

2 2

Re: [Numpy-discussion] [Jepp-users] win64 numpy hangs during tests using embedded interpreter
by Jon Wright 19 Jul '09

19 Jul '09

Ah, bugger, found the answer and it is bad news, from http://docs.python.org/c-api/init.html "Note that the PyGILState_*() functions assume there is only one global interpreter (created automatically by Py_Initialize() <http://docs.python.org/c-api/init.html#Py_Initialize>). Python still supports the creation of additional interpreters (using Py_NewInterpreter() <http://docs.python.org/c-api/init.html#Py_NewInterpreter>), but mixing multiple interpreters and the PyGILState_*() API is unsupported." Since jepp is using Py_NewInterpreter() to allow many interpreters then it means it won't play nicely with extension modules who use PyGILState, like numpy. I can get around it by disabling threads in numpy altogether, but I'd rather have them working in these days of multi-core CPUs. Any ideas? Thanks, Jon Jon Wright wrote: > Hi There, > > I am trying to use numpy with jepp (from jepp.sourceforge.net) which is > an embedded python interpreter on win64 (vista). With the sourceforge > binaries I get intermittent crashes on import which went away when I > compiled svn trunk with msvc9. Suggests there is something incompatible > about them? > > Jepp has been compiled with Py_OptimizeFlag = 0 in pyembed_startup and > also removing PyRun_SimpleString("__builtins__.__import__ = > jep.jimport\n"); > from pyembed_thread_init (otherwise numpy is not found). > > Using the python.org win64 interpreter it seems to work fine, so I'm > looking for advice on how to debug the problem. Compiling with debugging > on (edit distutils/msvc9compiler.py as described here > http://mail.python.org/pipermail/python-list/2003-October/229543.html ) > I found it is blocking here: > > > > umath.pyd!_error_handler(int method=2, _object * > errobj=0x000000004d774d88, char * errtype=0x000000004c06c280, int > retstatus=1, int * first=0x00000000024f8de0) Line 69 + > > ALLOW_C_API; > 000000004C04A783 call qword ptr [__imp_PyGILState_Ensure > (4C0684C8h)] > -> 000000004C04A789 mov dword ptr [__save__],eax > switch(method) { > > Way up the call stack it seems jepp has done this to call numpy: > > PyEval_AcquireLock(); > prevThread = PyThreadState_Swap(jepThread->tstate); > ... > result = PyRun_String(str, /* = numpy.test() */ > Py_single_input, > jepThread->globals, > jepThread->globals); > > So it seems there is some sort of a threading problem. Does anyone have > any ideas? > > Thanks for any tips, > > Jon > > > ------------------------------------------------------------------------------ > Enter the BlackBerry Developer Challenge > This is your chance to win up to $100,000 in prizes! For a limited time, > vendors submitting new applications to BlackBerry App World(TM) will have > the opportunity to enter the BlackBerry Developer Challenge. See full prize > details at: http://p.sf.net/sfu/Challenge > _______________________________________________ > Jepp-users mailing list > Jepp-users(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/jepp-users >

1 0