[Numpy-discussion] Who uses matrix?

[Tim Hochberg]

Bill Baxter wrote:

> On 5/13/06, *Tim Hochberg* <tim.hochberg at cox.net 
> <mailto:tim.hochberg at cox.net>> wrote:
>
>     Bill Baxter wrote:
>
>     >
>     > +1 on a .T shortcut for arrays.
>     > +1 on a .H shortcut for arrays, too.  (Instead of
>     .conj().transpose())
>
>     -1.
>
>     These are really only meaningul for 2D arrays. Without the axis
>     keyword
>     they generally do the wrong things for arrays of other dimensionality
>     (there I usually, but not always, want to transpose the first two or
>     last two axes). In addition, ndarray is already overstuffed with
>     methods
>     and attributes, let's not add any more without careful consideration.
>
>
> What am I missing here?  There's already a .transpose() method on array.
> From my quick little test it doesn't seem like the argument is useful:
> >>> a = num.rand(2,2)
> >>> a
> array([[ 0.96685836,  0.55643033],
>        [ 0.86387107,  0.39331451]])
> >>> a.transpose(1)
> array([ 0.96685836,  0.55643033])
> >>> a
> array([[ 0.96685836,  0.55643033],
>        [ 0.86387107,  0.39331451]])
> >>> a.transpose(0)
> array([ 0.96685836,  0.86387107])
> >>> a.transpose()
> array([[ 0.96685836,  0.86387107],
>        [ 0.55643033 ,  0.39331451]])
>
> (Python 2.4 / Windows XP)
>
> But maybe I'm using it wrong.  The docstring isn't much help:
> ---------------------------
> >>> help(a.transpose)
> Help on built-in function transpose:
>
> transpose(...)
>     m.transpose(<None>)
> ---------------------------

No, the docstring does not appear to be useful. I'm not sure what's 
happening to your argument there, something bogus I assume; you are 
supposed to pass a sequence, which becomes the new axis order. For example.

 >>> import numpy
 >>> a = numpy.arange(6).reshape([3,2,1])
 >>> a.shape
(3, 2, 1)
 >>> a.copy().transpose().shape # By default, the axes are just reversed
(1, 2, 3)
 >>> a.copy().transpose([0,2,1]).shape  # Transpose last two axes
(3, 1, 2)
 >>> a.copy().transpose([1,0,2]).shape # Transpose first two axes
(2, 3, 1)


Most of the time, when I'm using 3D arrays, I use one of the last two 
versions of transpose.

Back to passing a single number. Looking at array_transponse, it appears 
that a.transpose(1) and a.transpose([1]) are equivalent. The problem is 
that transpose is accepting sequences of the lengths other than 
len(a.shape). The results appear bogus: chunks of the array actually 
disappear as you illustrate. It looks like this can be fixed by just 
adding a check in PyArray_Transpose that permute->len is the same as 
ap->nd. I won't have time to get to this today; if possible, could you 
enter a ticket on this so it doesn't fall through the cracks?

>
> Assuming I'm missing something about transpose(), that still doesn't 
> rule out a shorter functional form, like a.T() or a.t().  
>
> The T(a) that you suggest is short, but it's just not the order people 
> are used to seeing things in their math.  Besides, having a global 
> function with a name like T is just asking for trouble.  And defining 
> it in every function that uses it would get tedious.
>
> Even if the .T shortcut were added as an attribute and not a function, 
> you could still use .transpose() for N-d arrays when the default two 
> axes weren't the ones you wanted to swap.  Yes, 2-D *is* a special 
> case of array, but -- correct me if I'm wrong -- it's a very common 
> special case.

Not in my experience. I can't claim that this is typical for everyone, 
but my usage is most often of 3D arrays that represent arrays of 
matrices. Also common is representing an array of 2x2 (or 4x4) matrices 
as [[X00, X01], [X10, X11]], where the various Xs are (big) 1D arrays or 
scalars. This is because I often know that one of the Xs is either all 
ones or all zeros and by storing them as a scalar values I can save 
space, and by eliding subsequent operations by special casing one and 
zero, time. My usage of vanilla 2D arrays is relatively limited and 
occurs mostly where I interface with the outside world.

> Matlab's transpose operator, for instance, just raises an error if 
> it's used on anything other than a 1D- or 2D- array.  There's some 
> other way to shuffle axes around for N-d arrays in matlab, but I 
> forget what.  Not saying that the way Matlab does it is right, but my 
> own experience reading and writing code of various sorts is that 2D 
> stuff is far more common than arbitrary N-d.  But for some reason it 
> seems like most people active on this mailing list see things as just 
> the opposite.  Perhaps it's just my little niche of the world that is 
> mostly 2D. The other thing is that I suspect you don't get so many 
> transposes when doing arbitrary N-d array stuff so there's not as much 
> need for a shortcut.  But again, not much experience with typical N-d 
> array usages here.

It's hard to say. People coming from Matlab etc tend to see things in 
terms of 2D. Some of that may be just that they work in different 
problem domains than, for instance, I do. Image processing for example 
seems to be mostly 2D. Although in some cases you might actually want to 
work on stacks of images, in which case you're back to the 3D regime. 
Part of it may also be that once you work an ND array language for a 
while you see ND solutions to some problems that previously you only saw 
2D solutions for.

>
>  
>
>     > And some briefer way to make a new axis than 'numpy.newaxis'
>     would be
>     > nice too (I'm trying to keep my global namespace clean these days).
>     > --- Whoa I just noticed that a[None,:] has the same effect as
>     > 'newaxis'.  Is that a blessed way to do things?
>
>     numpy.newaxis *is* None. I don't think that I'd spell it that way
>     since
>     None doesn't really have the same conotations as newaxis, so I think
>     I'll stick with np.newaxis, which is how I spell these things.
>
>
> Well, to a newbie, none of the notation used for indexing has much of 
> any connotation.  It's all just arbitrary symbols.  ":" means the 
> whole range? 2:10 means a range from 2 to 9?  Sure, most of those are 
> familiar to Python users (even '...'?) but even a heavy python user 
> would be puzzled by something like r_[1:2:3j].  Or reshape(-1,2).  The 
> j is a convention, the -1 is a convention.   NewAxis seems common 
> enough to me that it's worth just learning None as another numpy 
> convention. 
>
> As long as "numpy.newaxis *is* None", as you say, and not "is 
> *currently* None, but subject to change without notice" ,

It's been None since the beginning, so I don't think it's likely to 
change. Not being in charge, I can't guarantee that, but it would likely 
be a huge pain to do so I don't see it happening.

> then I think I'd rather use None.  It has the feel of being something 
> that's a fundamental part of the language.  The first few times I saw 
> indexing with numpy.newaxis it made no sense to me.  How can you index 
> with a new axis?  What's the new axis'th element of an array?  "None" 
> says to me the index we're selecting is "None" as in "None of the 
> above", i.e. we're not taking from the elements that are there, or not 
> using up any of our current axes on this index.  Also when you see 
> None, it's clear to a Python user that there's some trickery going on, 
> but when you see  a[NewAxis,:] (as it was the first time I saw it) 
> it's not clear if NewAxis is some numeric value defined in the code 
> you're reading or by Numpy or what.  For whatever reason None seems 
> more logical and symmetrical to me than numpy.newaxis.   Plus it seems 
> that documentation can't be attached to numpy.newaxis because of it 
> being None, which I recall also confused me at first.  
> "help(numpy.newaxis)" gives you a welcome to Python rather than 
> information about using numpy.newaxis.

In abstract, I don't see any problem with that, but assuming that your 
sharing code with others, everyone's likely to be happier if you spell 
newaxis as newaxis, just for consistencies sake. On a related note, I 
think newaxis is probably overused. at least by me, I just grepped my 
code for newaxis, and I'd say at least half the uses of newaxis would 
have been clearer as reshape.

-tim