add.reduce for character arrays
Hi, The following does not work. I am trying to do something similar to add.reduce, except that I want strings concatenated across the appropriate axes. I was hoping that Python would "magically" recognise that addition in strings translated to concatenation. Incidentally, the error message is not terribly imformative. Is there any way to make this work, or any other method people can suggest, or do I need to write my own function? Thanks. Regards, Faheem. ********************************************************************** In [43]: s Out[43]: CharArray([['a', 'b', 'c'], ['d', 'e', 'f'], ['g', 'h', 'i']]) In [44]: numarray.add.reduce(s,1) --------------------------------------------------------------------------- error Traceback (most recent call last) /home/faheem/wc/corrmodel/boost/<console> error: getShape: sequence object nested more than MAXDIM
On Sat, 2004-04-17 at 19:06, Faheem Mitha wrote:
Hi,
The following does not work. I am trying to do something similar to add.reduce, except that I want strings concatenated across the appropriate axes. I was hoping that Python would "magically" recognise that addition in strings translated to concatenation.
Use object arrays (numarray.objects) instead. While CharArrays do have an overloaded add operator they don't support reduction. Object arrays have more general ufuncs which apply object oriented operators and also support reduction. In general, character arrays (numarray.strings) do not support the numerical ufuncs found in numarray
Incidentally, the error message is not terribly imformative.
True enough. The numerical getShape code should recognize strings and raise an exception pointing out that strings can't be passed into numerical code. This is fixed in CVS.
Is there any way to make this work, or any other method people can suggest, or do I need to write my own function?
Again, use numarray.objects.
Thanks. Regards, Faheem.
********************************************************************** In [43]: s Out[43]: CharArray([['a', 'b', 'c'], ['d', 'e', 'f'], ['g', 'h', 'i']])
In [44]: numarray.add.reduce(s,1) --------------------------------------------------------------------------- error Traceback (most recent call last)
/home/faheem/wc/corrmodel/boost/<console>
error: getShape: sequence object nested more than MAXDIM
------------------------------------------------------- This SF.Net email is sponsored by: IBM Linux Tutorials Free Linux tutorial presented by Daniel Robbins, President and CEO of GenToo technologies. Learn everything from fundamentals to system administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- Todd Miller <jmiller@stsci.edu>
On Mon, 19 Apr 2004, Todd Miller wrote:
On Sat, 2004-04-17 at 19:06, Faheem Mitha wrote:
Hi,
The following does not work. I am trying to do something similar to add.reduce, except that I want strings concatenated across the appropriate axes. I was hoping that Python would "magically" recognise that addition in strings translated to concatenation.
Use object arrays (numarray.objects) instead. While CharArrays do have an overloaded add operator they don't support reduction. Object arrays have more general ufuncs which apply object oriented operators and also support reduction. In general, character arrays (numarray.strings) do not support the numerical ufuncs found in numarray
Incidentally, the error message is not terribly imformative.
True enough. The numerical getShape code should recognize strings and raise an exception pointing out that strings can't be passed into numerical code. This is fixed in CVS.
Is there any way to make this work, or any other method people can suggest, or do I need to write my own function?
Again, use numarray.objects.
Yes, that works. Thanks. In [13]: numarray.objects.add.reduce(s,dim=1) Out[13]: ObjectArray(['abc', 'def', 'ghi']) BTW, that makes me wonder what the rules are for the order in which this reduction proceeds for arrays bigger than dimension two. Since there is no total order in this case, the answer is not obvious. For string concatentation (since it is not commutative) at least this matters. I didn't see this documented in the manual but I may have missed it. Faheem.
On Mon, 2004-04-19 at 23:20, Faheem Mitha wrote:
Yes, that works. Thanks.
In [13]: numarray.objects.add.reduce(s,dim=1) Out[13]: ObjectArray(['abc', 'def', 'ghi'])
BTW, that makes me wonder what the rules are for the order in which this reduction proceeds for arrays bigger than dimension two. Since there is no total order in this case, the answer is not obvious.
(I think) the answer is that for numarray, reductions ultimately occur along the innermost axis. Off-axis reductions are performed by a swapaxis between the off-axis and the innermost axis, followed by the reduction, followed by a swapaxis back between the new innermost axis and the off-axis. I'm not sure there's an especially good reason for the swap back (since the original swapped axis no longer exists), but it seems to be what Numeric does and not doing it in numarray broke extension (linear_algebra?) self-tests.
For string concatentation (since it is not commutative) at least this matters. I didn't see this documented in the manual but I may have missed it.
Reductions always occur in order of increasing indexes with the "reduction-so-far" as the left operand and the next index as the right operand. Reductions occur in a sort of depth-first manner with the outermost dimensions varying most slowly. Regards, Todd -- Todd Miller <jmiller@stsci.edu>
On Tue, 20 Apr 2004, Todd Miller wrote:
On Mon, 2004-04-19 at 23:20, Faheem Mitha wrote:
Yes, that works. Thanks.
In [13]: numarray.objects.add.reduce(s,dim=1) Out[13]: ObjectArray(['abc', 'def', 'ghi'])
BTW, that makes me wonder what the rules are for the order in which this reduction proceeds for arrays bigger than dimension two. Since there is no total order in this case, the answer is not obvious.
(I think) the answer is that for numarray, reductions ultimately occur along the innermost axis. Off-axis reductions are performed by a swapaxis between the off-axis and the innermost axis, followed by the reduction, followed by a swapaxis back between the new innermost axis and the off-axis. I'm not sure there's an especially good reason for the swap back (since the original swapped axis no longer exists), but it seems to be what Numeric does and not doing it in numarray broke extension (linear_algebra?) self-tests.
For string concatentation (since it is not commutative) at least this matters. I didn't see this documented in the manual but I may have missed it.
Reductions always occur in order of increasing indexes with the "reduction-so-far" as the left operand and the next index as the right operand. Reductions occur in a sort of depth-first manner with the outermost dimensions varying most slowly.
Sorry about the slow reply. This message got lost in the mess in my inbox, which I have recently being cleaning out. I just wanted to say that it would be useful to have the above documented in some appropriate place in the numarray manual. 99% of the time the exact procedure for a reduction won't matter, but when someone is using a non-commutative operator it will. If this is documented in the manual I could not find it. Thanks. Faheem.
participants (2)
-
Faheem Mitha -
Todd Miller