Mailman 3 September 2004 - NumPy-Discussion

numarray-->Numeric conversion?
by Francesc Alted 13 Sep '04

13 Sep '04

Hi, I've been thinking in ways to convert from/to Numeric to numarray objects in a non-expensive way. For the Numeric --> numarray there is an very easy way to do that: In [45]: num=Numeric.arange(10, typecode="i") In [46]: na=numarray.array(buffer(num), typecode=num.typecode(), shape=num.shape) In [47]: na Out[47]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) that creates a new numarray object without a memory copy In [48]: num[2]=3 In [49]: num Out[49]: array([0, 1, 3, 3, 4, 5, 6, 7, 8, 9],'i') In [50]: na Out[50]: array([0, 1, 3, 3, 4, 5, 6, 7, 8, 9]) i.e. both num (Numeric) and na (numarray) arrays shares the same memory. If you delete one reference: In [51]: del num the other is still accessible In [52]: na Out[52]: array([0, 1, 3, 3, 4, 5, 6, 7, 8, 9]) However, it seems that it is not so easy to go the other way, that is convert a numarray object to a Numeric without a memory copy. It seems that Numeric.array constructor get the shape from the sequence object that is passed, while numarray approach seems more sofisticated in the sense that if it detects that the first argument is a buffer sequence, you must specify the shape on the constructor. Anyone knows if building a Numeric from a buffer (specifying both type and shape) is possible (I mean, at Python level) at all?. If that would be the case, adapting libraries that use Numeric to numarray objects and vice-versa would be very easy and cheap (in terms of memory access). -- Francesc Alted

2 1

extracting a random subset of a vector
by Curzio Basso 08 Sep '04

08 Sep '04

Hi all, I have an optimization problem. I currently use the following code to select a random subset of a rank-1 array: ---------------------------------------------------------- import numarray as NA import numarray.random_array as RA N = 1000 M = 100 full = NA.arange(N) subset = full[RA.permutation(N)][:M] --------------------------------------------------------- However, it's quite slow (at least with N~40k), and from the hotshot output is looks like it's the indexing, not the permutation, which takes time. Does anyone have a suggestion on a faster pure-python implementation? thanks

6 10

Compare strings.array with None.
by Shin 07 Sep '04

07 Sep '04

I got the following strange behavior in numarray.strings. Is it intended or a bug? Thanks. >>> from numarray import * >>> x = [1,2] >>> x == None 0 >>> from numarray import strings >>> x = strings.array(['a','b']) >>> x == None TypeError (And exit from mainloop.) -- Daehyok Shin

2 1

possible bug with character arrays and shuffle
by Faheem Mitha 06 Sep '04

06 Sep '04

Hi, Consider ******************************************************************** In [1]: import random In [2]: import numarray.strings as numstr In [3]: foo = numstr.array(['a', 'c', 'a', 'g', 'g', 'g', 'g', ...: 'g','a','a','a','a'],shape=(12,1)) In [4]: foo Out[4]: CharArray([['a'], ['c'], ['a'], ['g'], ['g'], ['g'], ['g'], ['g'], ['a'], ['a'], ['a'], ['a']]) In [5]: for i in range(50): ...: random.shuffle(foo) ...: In [6]: foo Out[6]: CharArray([['a'], ['a'], ['a'], ['a'], ['a'], ['a'], ['a'], ['a'], ['a'], ['a'], ['a'], ['a']]) ****************************************************************** Either I am doing something horribly wrong, or shuffle is badly broken, since it is supposed to "shuffle list x in place". I haven't checked shuffle carefully, but it seems to randomize for a bit Ok, and then after a while, it suddenly starts returning all the same characters. It took me a while to track this one down. This is with Debian sarge and package versions ii python2.3-numarray 1.0-2 An array processing package modelled after Python-Numeric ii python 2.3.4-1 An interactive high-level object-oriented language (default version) Please advise of a possible workaround. Or should I simply use a different randomizing function, like sample? Please cc me, I'm not subscribed. Thanks. Faheem.

1 2

possible bug in concatenating character arrays
by Faheem Mitha 05 Sep '04

05 Sep '04

The following recipe gives a segmentation fault. ************************************************************************ In [1]: import numarray.strings as numstr In [2]: foo = numstr.array("acgttatcgt", shape=(3,3)) In [3]: foo[0] + foo[1] Segmentation fault ************************************************************************* It works if instead one does: ************************************************************************* In [4]: a = foo[0] In [5]: b = foo[1] In [6]: a Out[6]: CharArray(['a', 'c', 'g']) In [7]: b Out[7]: CharArray(['t', 't', 'a']) In [8]: a + b Out[8]: CharArray(['at', 'ct', 'ga']) *************************************************************************** In the case of numerical arrays, either method works. In [9]: bar = numarray.array(range(9), shape=(3,3)) In [10]: bar Out[10]: array([[0, 1, 2], [3, 4, 5], [6, 7, 8]]) In [11]: a = bar[0] In [12]: b = bar[1] In [13]: a + b Out[13]: array([3, 5, 7]) In [14]: bar[0] + bar[1] Out[14]: array([3, 5, 7]) ************************************************************ If this is a bug (I'd appreciate confirmation of this), should I report to the bug tracking system? Please cc me, I'm not subscribed. Faheem.

2 4

RE: [Numpy-discussion] possible bug in concatenating character arrays
by Nadav Horesh 05 Sep '04

05 Sep '04

It works without any problem under numarray 1.1 (from CVS) and python 2.3.4 and 2.4.a2 under RH9 Nadav. -----Original Message----- From: Faheem Mitha [mailto:faheem@email.unc.edu] Sent: Sun 05-Sep-04 04:45 To: numpy-discussion(a)lists.sourceforge.net Cc: Subject: [Numpy-discussion] possible bug in concatenating character arrays The following recipe gives a segmentation fault. ************************************************************************ In [1]: import numarray.strings as numstr In [2]: foo = numstr.array("acgttatcgt", shape=(3,3)) In [3]: foo[0] + foo[1] Segmentation fault ************************************************************************* It works if instead one does: ************************************************************************* In [4]: a = foo[0] In [5]: b = foo[1] In [6]: a Out[6]: CharArray(['a', 'c', 'g']) In [7]: b Out[7]: CharArray(['t', 't', 'a']) In [8]: a + b Out[8]: CharArray(['at', 'ct', 'ga']) *************************************************************************** In the case of numerical arrays, either method works. In [9]: bar = numarray.array(range(9), shape=(3,3)) In [10]: bar Out[10]: array([[0, 1, 2], [3, 4, 5], [6, 7, 8]]) In [11]: a = bar[0] In [12]: b = bar[1] In [13]: a + b Out[13]: array([3, 5, 7]) In [14]: bar[0] + bar[1] Out[14]: array([3, 5, 7]) ************************************************************ If this is a bug (I'd appreciate confirmation of this), should I report to the bug tracking system? Please cc me, I'm not subscribed. Faheem. ------------------------------------------------------- This SF.Net email is sponsored by BEA Weblogic Workshop FREE Java Enterprise J2EE developer tools! Get your free copy of BEA WebLogic Workshop 8.1 today. http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click _______________________________________________ Numpy-discussion mailing list Numpy-discussion(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion

1 0

RE: [Numpy-discussion] possible bug in concatenating character arrays
by Nadav Horesh 05 Sep '04

05 Sep '04

I dont get this result with numarray 1.1 (from CVS) with python versions 2.3.4 and 2.4.a2: >>> import numarray.strings as numstr >>> foo = numstr.array("acgttatcgt", shape=(3,3)) >>> foo[0] + foo[1] CharArray(['at', 'ct', 'ga']) Nadav. -----Original Message----- From: Faheem Mitha [mailto:faheem@email.unc.edu] Sent: Sun 05-Sep-04 04:45 To: numpy-discussion(a)lists.sourceforge.net Cc: Subject: [Numpy-discussion] possible bug in concatenating character arrays The following recipe gives a segmentation fault. ************************************************************************ In [1]: import numarray.strings as numstr In [2]: foo = numstr.array("acgttatcgt", shape=(3,3)) In [3]: foo[0] + foo[1] Segmentation fault ************************************************************************* It works if instead one does: ************************************************************************* In [4]: a = foo[0] In [5]: b = foo[1] In [6]: a Out[6]: CharArray(['a', 'c', 'g']) In [7]: b Out[7]: CharArray(['t', 't', 'a']) In [8]: a + b Out[8]: CharArray(['at', 'ct', 'ga']) *************************************************************************** In the case of numerical arrays, either method works. In [9]: bar = numarray.array(range(9), shape=(3,3)) In [10]: bar Out[10]: array([[0, 1, 2], [3, 4, 5], [6, 7, 8]]) In [11]: a = bar[0] In [12]: b = bar[1] In [13]: a + b Out[13]: array([3, 5, 7]) In [14]: bar[0] + bar[1] Out[14]: array([3, 5, 7]) ************************************************************ If this is a bug (I'd appreciate confirmation of this), should I report to the bug tracking system? Please cc me, I'm not subscribed. Faheem. ------------------------------------------------------- This SF.Net email is sponsored by BEA Weblogic Workshop FREE Java Enterprise J2EE developer tools! Get your free copy of BEA WebLogic Workshop 8.1 today. http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click _______________________________________________ Numpy-discussion mailing list Numpy-discussion(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion

1 0

Reshaping along an axis
by Karthikesh Raju 03 Sep '04

03 Sep '04

Hi all, Suppose i have a tuple or a 1D array as a = (1,2,3,4,5,6,7,8,9) presently reshape(a,(3,3)) gives me 1 2 3 4 5 6 7 8 9 i.e reshaping is done column wise. How do one specifiy reshape to work row wise as: reshape(a,(3,3)) = 1 4 7 2 5 8 3 6 9 With warm regards karthik ----------------------------------------------------------------------- Karthikesh Raju, email: karthik(a)james.hut.fi karthikesh.raju(a)gmail.com Researcher, http://www.cis.hut.fi/karthik Helsinki University of Technology, Tel: +358-9-451 5389 Laboratory of Comp. & Info. Sc., Fax: +358-9-451 3277 Department of Computer Sc., P.O Box 5400, FIN 02015 HUT, Espoo, FINLAND -----------------------------------------------------------------------

2 1

efficient summation
by Darren Dale 01 Sep '04

01 Sep '04

I am trying to effieciently sum over a subset of the elements of a matrix. In Matlab, this could be done like: a=[1,2,3,4,5,6,7,8,9,10] b = [1,0,0,0,0,0,0,0,0,1] res=sum(a(b)) %this sums the elements of a which have corresponding elements in b that are true Is there anything similar in numarray (or numeric)? I thought masked arrays looked promising, but I find that masking 90% of the elements results in marginal speedups (~5%, instead of 90%) over the unmasked array. Thanks! Darren

3 3

Re: [Numpy-discussion] sum of a masked array
by smpitts＠ou.edu 01 Sep '04

01 Sep '04

Darren, I tried your code on my system: Python 2.2 and numarray 1.0. The type error looks like a bug in the array display code. >>> Rx = ones((2500,2500)) >>> N = make_mask_none((2500,2500)) >>> Rx = array(Rx,mask=N) >>> print average(Rx) ## works [ 1. 1. 1. ..., 1. 1. 1.] >>> s = sum(Rx) >>> print s Traceback (most recent call last): File "<stdin>", line 1, in ? File "/usr/lib/python2.2/site-packages/numarray/ma/MA.py", line 742, in __str__ return str(filled(self, f)) File "/usr/lib/python2.2/site-packages/numarray/generic.py", line 499, in __str__ return arrayprint.array2string(self, separator=" ", style=str) File "/usr/lib/python2.2/site-packages/numarray/arrayprint.py", line 188, in array2string separator, prefix) File "/usr/lib/python2.2/site-packages/numarray/arrayprint.py", line 137, in _array2string data = _leading_trailing(a) File "/usr/lib/python2.2/site-packages/numarray/arrayprint.py", line 105, in _leading_trailing b = _gen.concatenate((a[:_summaryEdgeItems], File "/usr/lib/python2.2/site-packages/numarray/generic.py", line 1028, in concatenate return _concat(arrs) File "/usr/lib/python2.2/site-packages/numarray/generic.py", line 1012, in _concat dest = arrs[0].__class__(shape=destShape, type=convType) TypeError: __init__() got an unexpected keyword argument 'type' but the array is fine >>> for i in range(2500): ... assert s[i] == 2500 >>> Hopefully someone with more knowledge can help you out. I still use MA with Numeric for the most part. -- Stephen Pitts smpitts(a)ou.edu

2 1

NumPy-Discussion September 2004